DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claims 1-10 are presented for examination.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on July 11, 2023 and February 12, 2025 are in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statements are being considered by the examiner.
Drawings
The drawings are objected to because (a) “determination” in reference character 203 is written on two different lines; (b) reference character 509 appears in the drawings but not the specification; (c) in reference character S802, “parition” should be “partition”; (d) in reference character 925, “partioning” should be “partitioning”; and (e) in reference character S1305, “partion” should be “partition”. Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. The figure or figure number of an amended drawing should not be labeled as “amended.” If a drawing figure is to be canceled, the appropriate figure must be removed from the replacement sheet, and where necessary, the remaining figures must be renumbered and appropriate changes made to the brief description of the several views of the drawings for consistency. Additional replacement sheets may be necessary to show the renumbering of the remaining figures. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
Specification
The lengthy specification has not been checked to the extent necessary to determine the presence of all possible minor errors. Applicant’s cooperation is requested in correcting any errors of which applicant may become aware in the specification.
The title of the invention is not descriptive. A new title is required that is clearly indicative of the invention to which the claims are directed.
Examiner would also note that two copies of each of the claims, specification, and drawings were submitted. While the specifications and claims appear to be identical to each other, the drawings appear to have minor cosmetic differences from each other. Examiner requests that Applicant indicate which of the two drawing sets is intended to be published.
Claim Objections
Claim 8 is objected to because of the following informalities: “different than” should be “different from”. Claims 9-10 are objected to for dependency on claim 8. Appropriate correction is required.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are: “feature determination unit”, “data selection unit”, “machine learning model management unit”, and “knowledge transfer unit” in claim 1; “partition unit” in claim 2; “machine learning model management unit” in claim 3; “private device”, “public device”, “weighted knowledge transfer device”, “partitioning optimization unit”, “data selection unit”, and “machine learning unit” in claim 8; and “evaluation unit” in claims 9-10.
Regarding the “private device”, paragraph 53 of the specification indicates that the private device may include hard disk drives, solid state drives, or cloud-based storage repositories. Any one of these will be deemed to read on the claim. For an analysis of the other claimed limitations, see rejection under 35 USC § 112(b) infra.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claims 1-6 and 8-10 are rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention.
The claim limitations identified above invoke 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. However, with the exception of the “private device,” the written description fails to disclose the corresponding structure, material, or acts for performing the entire claimed functions and to clearly link the structure, material, or acts to the functions. Therefore, it is unclear whether Applicant had possession of the claimed invention as of the effective filing date. See analysis in rejection under 35 USC § 112(b) infra for further analysis.
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-6 and 8-10 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
The claim limitations identified above invoke 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. However, with the exception of the “private device,” the written description fails to disclose the corresponding structure, material, or acts for performing the entire claimed functions and to clearly link the structure, material, or acts to the functions.
Regarding the “feature determination unit”, at most paragraphs 70-71 of the published specification indicate that the feature determination unit may use statistical analysis methods or NLP methods to perform the claimed dataset generation. Paragraphs 102-06 and Fig. 6 also delineate the functions of this unit, but do not delineate how the set of target features is determined other than to state that statistical analysis may be used.
Regarding the “data selection unit”, paragraph 80 states that the data selection unit may use GANs to generate the public training set or the similarity weight vector, but does not disclose how the GANs create that specific dataset other than to delineate the general functions of a GAN. Paragraphs 109-118 and Fig. 7 also indicate the functions of the unit, but do not sufficiently disclose how functions such as “indicat[ing] weights of a set of public features” are accomplished.
Regarding the “machine learning model management unit”, paragraphs 94-96 of the specification detail the functions of this unit, but there is no algorithm disclosed for performing the claimed function of generating a public label vector other than to say that a set of trained private machine learning models are used to generate the vector.
Regarding the “knowledge transfer unit”, Fig. 8 and paragraphs 121-31 describe the functions of the knowledge transfer unit (Examiner will assume for purposes of examination that the claimed “knowledge transfer unit” and the disclosed “weighted knowledge transfer unit” are the same). However, the specification does not delineate how the public machine learning model is generated other than to say that it is generated by training models on partitions of data and trained using the claimed data. However, no details of the training process are disclosed, so it is unclear how the generation takes place.
Regarding the “partition unit”, Fig. 11 and paragraphs 152-156 of the specification delineate the functions of the partition unit, but no disclosure of the specific steps taken by the partition unit to ensure that “each partition includes a mutually exclusive set of private features with respect to other partitions” is provided.
Regarding the “public device”, paragraphs 56-57 disclose the functions of the public device, but no algorithm for providing machine learning model services to users is disclosed.
Regarding the “weighted knowledge transfer device”, the claims indicate that the “weighted knowledge transfer device” comprises other units and devices that perform functions for which the disclosure fails to provide the structure, material, or acts for performing, as shown above and below. Therefore, a fortiori, there is insufficient structure, material, or acts for performing all the claimed functions of the “weighted knowledge transfer device.”
Regarding the “partitioning optimization unit”, paragraphs 140-45 disclose the functions of the partitioning optimization unit, but do not disclose an algorithm sufficient for performing the claimed functions of dividing the private dataset and generating a set of trained private machine learning models.
Regarding the “machine learning unit”, paragraphs 97-99 describe the functions of this unit, but do not sufficiently describe how the training and deployment of the models are accomplished.
Regarding the “evaluation unit”, paragraphs 160-62 describe the functions of this unit, but do not do so in sufficient detail to provide an algorithm for performing the claimed functions of evaluating performance and determining model configuration parameters.
Therefore, the claims are indefinite and are rejected under 35 U.S.C. 112(b) or pre-AIA 35 U.S.C. 112, second paragraph. For purposes of examination, any software that performs the claimed functions will be deemed to read on the claims.
Applicant may:
(a) Amend the claims so that the claim limitations will no longer be interpreted as limitations under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph;
(b) Amend the written description of the specification such that it expressly recites what structure, material, or acts perform the entire claimed functions, without introducing any new matter (35 U.S.C. 132(a)); or
(c) Amend the written description of the specification such that it clearly links the structure, material, or acts disclosed therein to the functions recited in the claims, without introducing any new matter (35 U.S.C. 132(a)).
If applicant is of the opinion that the written description of the specification already implicitly or inherently discloses the corresponding structure, material, or acts and clearly links them to the functions so that one of ordinary skill in the art would recognize what structure, material, or acts perform the claimed functions, applicant should clarify the record by either:
(a) Amending the written description of the specification such that it expressly recites the corresponding structure, material, or acts for performing the claimed functions and clearly links or associates the structure, material, or acts to the claimed functions, without introducing any new matter (35 U.S.C. 132(a)); or
(b) Stating on the record what the corresponding structure, material, or acts, which are implicitly or inherently set forth in the written description of the specification, perform the claimed functions. For more information, see 37 CFR 1.75(d) and MPEP §§ 608.01(o) and 2181.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-10 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. The analysis of the claims will follow the 2019 Revised Patent Subject Matter Eligibility Guidance, 84 Fed. Reg. 50 (“2019 PEG”).
Claim 1
Step 1: The claim recites a device comprising multiple “units” that invoke 35 USC § 112(f), as noted above. Since paragraphs 39-40 of the instant specification as published indicate that these units are executed on one or more processors, these claims will be construed as belonging to the statutory category of machines.
Step 2A Prong 1: The claim recites, inter alia:
[G]enerat[ing] a public knowledge transfer dataset and a private knowledge transfer dataset by determining a set of target features shared between a public dataset and a private dataset: This limitation could encompass mentally determining the shared target features and mentally generating the datasets based thereon.
[G]enerat[ing], based on a similarity calculation of the public knowledge transfer dataset and the private knowledge transfer dataset: a public training dataset which is a subset of the public knowledge transfer dataset that achieves a similarity threshold with respect to the private knowledge transfer dataset, and a similarity weight vector that indicates weights of a set of public features included in the public training dataset: This limitation could encompass mentally generating the public training dataset and the similarity weight vector by mentally calculating the similarity between the two datasets.
[G]enerat[ing] … a public label vector that indicates labels for the set of public features: This limitation could encompass mentally generating the public label vector.
Step 2A Prong 2: This judicial exception is not integrated into a practical application. The claim further recites that the relevant functions are performed by a “feature determination unit,” a “data selection unit,” a “machine learning model management unit,” and a “knowledge transfer unit” and that the generation of the public label vector is performed “by processing the public training dataset with a set of trained private machine learning models trained based on the private knowledge transfer dataset”. However, these amount to mere instructions to apply the judicial exception using a generic computer programmed with a generic class of computer algorithms. MPEP § 2106.05(f). The claim further recites “generat[ing] a public machine learning model based on the weight vector, the public training dataset, and the public label vector”, which amounts to a mere restriction of the judicial exception to the field of use of machine learning model generation. MPEP § 2106.05(h).
Step 2B: The claim does not contain significantly more than the judicial exception. The analysis at this step mirrors that of step 2A, prong 2. As an ordered whole, the claim is directed to a mentally performable process of generating datasets with features and labels. Nothing in the claim provides significantly more than this. As such, the claim is not patent eligible.
Claim 2
Step 1: A machine, as above.
Step 2A Prong 1: The claim recites, inter alia, “divid[ing] the private knowledge transfer dataset into a plurality of partitions; … wherein each partition includes a mutually exclusive set of private features with respect to other partitions.” This limitation could encompass mentally dividing the dataset into partitions.
Step 2A Prong 2: This judicial exception is not integrated into a practical application. The claim further recites that the division is performed by a “partition unit”. However, this amounts to a mere instruction to apply the judicial exception using a generic computer programmed with generic classes of computer algorithm. MPEP § 2106.05(f).
Step 2B: The claim does not contain significantly more than the judicial exception. The claim further recites that the division is performed by a “partition unit”. However, this amounts to a mere instruction to apply the judicial exception using a generic computer programmed with generic classes of computer algorithm. MPEP § 2106.05(f).
Claim 3
Step 1: A machine, as above.
Step 2A Prong 1: The claim recites the same judicial exceptions as in claim 2.
Step 2A Prong 2: This judicial exception is not integrated into a practical application. The claim further recites that “the machine learning model management unit generates the set of trained private machine learning models by training each of a set of machine learning models based on a separate partition of the plurality of partitions.” However, this amounts to a mere restriction of the judicial exception to the field of use of machine learning model training. MPEP § 2106.05(h).
Step 2B: The claim does not contain significantly more than the judicial exception. The claim further recites that “the machine learning model management unit generates the set of trained private machine learning models by training each of a set of machine learning models based on a separate partition of the plurality of partitions.” However, this amounts to a mere restriction of the judicial exception to the field of use of machine learning model training. MPEP § 2106.05(h).
Claim 4
Step 1: A machine, as above.
Step 2A Prong 1: The claim recites, inter alia, “determining … a probability of the first set of generated features belonging to the private knowledge transfer dataset; and selecting, as the public training dataset, a subset of the set of generated features associated with a probability of belonging to the private knowledge transfer dataset that exceeds a first probability threshold.” These limitations could encompass mentally determining the probability of the set of generated features and mentally selecting the subset of generated features.
Step 2A Prong 2: This judicial exception is not integrated into a practical application. The claim further recites “generating a trained generator network by training a generator network with the private knowledge transfer dataset to generate a first set of generated features; generating a trained discriminator network by training a discriminator network to distinguish between the first set of generated features and a set of private features included in the private knowledge transfer dataset” and then determining the probability “using the trained discriminator network”. However, this amounts to a mere instruction to apply the judicial exception using a generic computer programmed with generic classes of computer algorithm. MPEP § 2106.05(f).
Step 2B: The claim does not contain significantly more than the judicial exception. The claim further recites “generating a trained generator network by training a generator network with the private knowledge transfer dataset to generate a first set of generated features; generating a trained discriminator network by training a discriminator network to distinguish between the first set of generated features and a set of private features included in the private knowledge transfer dataset” and then determining the probability “using the trained discriminator network”. However, this amounts to a mere instruction to apply the judicial exception using a generic computer programmed with generic classes of computer algorithm. MPEP § 2106.05(f).
Claim 5
Step 1: A machine, as above.
Step 2A Prong 1: The claim recites, inter alia:
[M]erging the private knowledge transfer dataset and the public knowledge transfer dataset into a merged dataset: This limitation could encompass mentally merging the datasets.
[D]etermining, by … process[ing] the public knowledge transfer dataset, a probability indicating a likelihood that the set of public features belongs to the private knowledge transfer dataset: This limitation could encompass mentally determining the probability that the features belong to the dataset.
[S]electing, as the public training dataset, a subset of the set of public features associated with a probability of belonging to the private knowledge transfer dataset that exceeds a first probability threshold: This limitation could encompass mentally selecting the subset of features exceeding the probability threshold as the public training set.
Step 2A Prong 2: This judicial exception is not integrated into a practical application. The claim further recites that determining the probability is performed “by using the trained discriminator network” that is “generat[ed] … by training a discriminator network to classify a first set of features of the merged dataset as belonging to the private knowledge transfer dataset or the public knowledge transfer dataset”. However, this amounts to a mere instruction to apply the judicial exception using a generic computer programmed with generic classes of computer algorithm. MPEP § 2106.05(f).
Step 2B: The claim does not contain significantly more than the judicial exception. The claim further recites that determining the probability is performed “by using the trained discriminator network” that is “generat[ed] … by training a discriminator network to classify a first set of features of the merged dataset as belonging to the private knowledge transfer dataset or the public knowledge transfer dataset”. However, this amounts to a mere instruction to apply the judicial exception using a generic computer programmed with generic classes of computer algorithm. MPEP § 2106.05(f).
Claim 6
Step 1: A machine, as above.
Step 2A Prong 1: The claim recites that “the similarity calculation is selected from the group consisting of a Euclidean distance calculation method, a Manhattan distance calculation method, a Chebyshev distance calculation method, and a Mahalanobis distance calculation method.” The generation of the training datasets using this similarity calculation remains mentally performable under these additional limitations.
Step 2A Prong 2: This judicial exception is not integrated into a practical application. See claim 1 analysis.
Step 2B: The claim does not contain significantly more than the judicial exception. See claim 1 analysis.
Claim 7
Step 1: The claim recites a method; therefore, it is directed to the statutory category of processes.
Step 2A Prong 1: The claim recites, inter alia:
[G]enerating a public knowledge transfer dataset and a private knowledge transfer dataset by determining a set of target features shared between the public dataset and the private dataset: This limitation could encompass mentally generating the datasets by mentally determining the shared target features.
[G]enerating, based on a similarity calculation of the public knowledge transfer dataset and the private knowledge transfer dataset: a public training dataset which is a subset of the public knowledge transfer dataset that achieves a similarity threshold with respect to the private knowledge transfer dataset, and a similarity weight vector that indicates weights of a set of public features included in the public training dataset: This limitation could encompass mentally generating the public training set and similarity weight vector by mentally calculating a similarity between the public and private knowledge transfer datasets.
[G]enerating, by processing the public training dataset …, a public label vector that indicates labels for the set of public features: This limitation could encompass mentally generating a public label vector.
Step 2A Prong 2: This judicial exception is not integrated into a practical application. The claim further recites that the generation of the public label vector is performed with a “set of trained private machine learning models” that are generated “by training a set of machine learning models with the private knowledge transfer dataset”. However, this amounts to a mere instruction to apply the judicial exception using a generic computer programmed with generic classes of computer algorithm. MPEP § 2106.05(f). The claim further recites “generating a public machine learning model based on the weight vector, the public training dataset, and the public label vector.” However, this merely limits the judicial exception to the field of use of generating machine learning models. MPEP § 2106.05(h).
Step 2B: The claim does not contain significantly more than the judicial exception. The analysis at this step mirrors that of step 2A, prong 2. As an ordered whole, the claim is directed to a mentally performable method of generating datasets and label vectors. Nothing in the claim provides significantly more than this. As such, the claim is not patent eligible.
Claim 8
Step 1: The claim recites a device comprising multiple “units” and “devices” that invoke 35 USC § 112(f), as noted above. Since paragraphs 39-40 of the instant specification as published indicate that these units are executed on one or more processors, these claims will be construed as belonging to the statutory category of machines.
Step 2A Prong 1: The claim recites, inter alia:
[D]ivid[ing] the private dataset into a first plurality of partitions: This limitation could encompass mentally dividing the dataset.
[G]enerat[ing], as a processed public dataset, a group of labels and weights created by processing the public dataset …, divide the processed public dataset into a second plurality of partitions based on a set of thresholds determined based on the group of labels and weights: This limitation could encompass mentally generating the labels and weights and mentally partitioning the dataset based on the thresholds.
Step 2A Prong 2: This judicial exception is not integrated into a practical application. The claim further recites that the method is performed using a “weighted knowledge transfer system comprising: a private device that stores a private dataset; a public device configured to include a public dataset and provide machine learning model based services to users; and a weighted knowledge transfer device configured to generate a set of trained public machine learning models using the private dataset and the public dataset, wherein: the weighted knowledge transfer device is communicably connected to the private device via a first network connection; the weighted knowledge transfer device is communicably connected to the public device via a second network connection different [from] the first network connection; [and] the private device is inaccessible from the public device” that further includes a “partitioning optimization unit,” “data selection unit,” and a “machine learning unit.” However, these limitations amount to mere instructions to apply the judicial exception using a generic computer programmed with generic classes of computer algorithm. MPEP § 2106.05(f).
The claim further recites “generat[ing] a set of trained private machine learning models by training each of a set of machine learning models based on a separate partition of the first plurality of partitions” and “generat[ing] a set of trained public machine learning models by training each of a set of machine learning models based on a separate partition of the second plurality of partitions”. These limitations merely restrict the field of use of the judicial exception to generating trained machine learning models. MPEP § 2106.05(h).
Finally, the claims recite “deploy[ing] the set of trained public machine learning models to the public device for provision to users.” This limitation recites the insignificant extra-solution activity of mere data gathering and output. MPEP § 2106.05(g).
Step 2B: The claim does not contain significantly more than the judicial exception. The analysis at this step mirrors that of step 2A, prong 2, with the exception that the deploying limitation, in addition to reciting insignificant extra-solution activity, also recites the well-understood, routine, and conventional activity of receiving or transmitting data over a network. MPEP § 2106.05(d)(II); OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015) (sending messages over a network). As an ordered whole, the claim is directed to a mentally performable process of dividing datasets into partitions and assigning labels and weights to them. Nothing in the claim provides significantly more than this. As such, the claim is not patent eligible.
Claim 9
Step 1: A machine, as above.
Step 2A Prong 1: The claim recites, inter alia, “evaluat[ing] a performance of the set of trained private machine learning models with respect to a set of external test data selected from the private dataset; and determin[ing], based on the performance of the set of trained private machine learning models with respect to the set of external test data selected from the private dataset, a first set of model configuration parameters for the set of trained private machine learning models that achieves a predetermined performance criterion.” These limitations could encompass mentally evaluating the performance of the models by visually observing their outputs and mentally determining what the model configuration parameters should be based thereon.
Step 2A Prong 2: This judicial exception is not integrated into a practical application. The claim further recites that the above functions are performed by an “evaluation unit.” However, this amounts to a mere instruction to apply the judicial exception using a generic computer programmed with generic classes of computer algorithm. MPEP § 2106.05(f).
Step 2B: The claim does not contain significantly more than the judicial exception. The claim further recites that the above functions are performed by an “evaluation unit.” However, this amounts to a mere instruction to apply the judicial exception using a generic computer programmed with generic classes of computer algorithm. MPEP § 2106.05(f).
Claim 10
Step 1: A machine, as above.
Step 2A Prong 1: The claim recites, inter alia, “evaluat[ing] a performance of the set of trained public machine learning models with respect to a set of external test data selected from the private dataset; and determin[ing], based on the performance of the set of trained private machine learning models with respect to the set of external test data selected from the private dataset, a second set of model configuration parameters for the set of trained public machine learning models that achieves a predetermined performance criterion.” These limitations could encompass mentally evaluating the performance of the models by observing their outputs when fed the external test data and mentally determining the second set of model configuration parameters based thereon.
Step 2A Prong 2: This judicial exception is not integrated into a practical application. The claim further recites that the above functions are performed by an “evaluation unit.” However, this amounts to a mere instruction to apply the judicial exception using a generic computer programmed with generic classes of computer algorithm. MPEP § 2106.05(f).
Step 2B: The claim does not contain significantly more than the judicial exception. The claim further recites that the above functions are performed by an “evaluation unit.” However, this amounts to a mere instruction to apply the judicial exception using a generic computer programmed with generic classes of computer algorithm. MPEP § 2106.05(f).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1-3 and 7 are rejected under 35 U.S.C. 103 as being unpatentable over Papernot et al., “Semi-Supervised Knowledge Transfer for Deep Learning from Private Training Data,” in arXiv preprint arXiv:1610.05755 (2017) (“Papernot”) in view of Sidorov et al., “Soft Similarity and Soft Cosine Measure: Similarity of Features in Vector Space Model,” in 18.3 Computación y Sistemas 491-504 (2014) (“Sidorov”).
Regarding claim 1, Papernot discloses “[a] weighted knowledge transfer device comprising:
a feature determination unit configured to generate a public knowledge transfer dataset and a private knowledge transfer dataset (Papernot Figure 1 discloses a set of sensitive data [private knowledge transfer dataset] and a set of public data [public knowledge transfer dataset]) …;
a data selection unit configured to generate …:
a public training dataset which is a subset of the public knowledge transfer dataset (Papernot Fig. 1 discloses a set of incomplete public data [subset of the public knowledge transfer dataset]) …, …;
a machine learning model management unit configured to generate, by processing the public training dataset with a set of trained private machine learning models trained based on the private knowledge transfer dataset, a public label [set] that indicates labels for the set of public features (Papernot Fig. 1 shows that the dataset used to train the student model is labeled [to create a public label set indicating labels for the set of public features] with an aggregate teacher model aggregated from a set of models [private machine learning models] trained based on the sensitive data [private knowledge transfer dataset] and that are not accessible by an adversary); and
a knowledge transfer unit configured to generate a public machine learning model based on … the public training dataset[] and the public label [set] (Papernot Fig. 1 discloses that a student model accessible to an adversary [i.e., a public machine learning model] is generated based on the incomplete public data [public training dataset] and the labels produced by the aggregate teacher model [public label set]).”
Papernot appears not to disclose explicitly the further limitations of the claim. However, Sidorov discloses “determining a set of target features shared between a [first] dataset and a [second] dataset (Hamming distance counts how many features two vectors do not share [i.e., it indirectly counts how many they do share] – Sidorov, paragraph spanning pp. 492-93); …
generat[ing], based on a similarity calculation of the … [first] dataset and the … [second] dataset:
a … dataset that achieves a similarity threshold with respect to the [second] dataset (there is almost always some similarity between words which can be identified independently from the vector space model; a natural assumption can be made that this similarity can be modeled as a cosine between these objects – Sidorov, p. 494, paragraphs immediately preceding and containing Equation (8); a small number of highly similar pairs of features [feature sets = datasets] is identified whose similarity measure is greater than a threshold – id. at sec. 2.3, second paragraph), and
a similarity weight vector that indicates weights of a set of … features included in the … dataset (if the similarity between features is given by some list, automatically only a small number of elements in the matrix sij of similarities will not equal zero; the matrix sij can be operated upon as a sparse matrix – Sidorov, sec. 2.3, third-fourth paragraphs [i.e., sij is a series of weight vectors showing the similarity weights between feature i and each feature j = 1, …, n]); … [and]
generat[ing] a … model based on the weight vector (sij is used to construct data vectors a and b, and a and b are used to form a soft cosine value [model] between pairs of features that represents a weight or importance that is a similarity of the two features – Sidorov, sec. 2.2, paragraph containing Equations (15)-(16)) ….”
Sidorov further discloses that the data are organized as vectors (traditional cosine measure is commonly used to determine the similarity between two objects represented as vectors – Sidorov, p. 49, first full paragraph) ….”
Sidorov and the instant application both relate to similarity metrics in artificial intelligence and are analogous. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Papernot to generate a similarity vector indicating the similarities between features, as disclosed by Sidorov, and an ordinary artisan could reasonably expect to have done so successfully. Doing so would allow the system to account for similarity between data points, thereby creating a more robust model. See Sidorov, abstract.
Regarding claim 2, Papernot, as modified by Sidorov, discloses “a partition unit configured to divide the private knowledge transfer dataset into a plurality of partitions (see mapping of limitation below); …
wherein each partition includes a mutually exclusive set of private features with respect to other partitions (instead of training a single model to solve the task associated with a dataset, the data are partitioned in n disjoint [mutually exclusive] sets and a model is trained separately on each set – Papernot, sec. 2.1, subsection entitled “Training the Ensemble of Teachers”; see also Fig. 1 [showing that the training data are sensitive and not accessible by an adversary, i.e., private]).”
Regarding claim 3, Papernot, as modified by Sidorov, discloses that “the machine learning model management unit generates the set of trained private machine learning models by training each of a set of machine learning models based on a separate partition of the plurality of partitions (instead of training a single model to solve the task associated with a dataset, the data are partitioned in n disjoint [mutually exclusive] sets and a model is trained separately on each set – Papernot, sec. 2.1, subsection entitled “Training the Ensemble of Teachers”).”
Claim 7 is a method claim corresponding to device claim 1 and is rejected for the same reasons as given in the rejection of that claim, with the exception that claim 7 additionally recites the following limitations, taught by Papernot: “receiving a public dataset and a private dataset (Papernot Fig. 1 shows the receipt of sensitive data [public dataset] and incomplete public data [private dataset]); … [and]
generating, by training a set of machine learning models with the private knowledge transfer dataset, a set of trained private machine learning models (Papernot Fig. 1 shows that the sensitive data are partitioned into n disjoint sets [private knowledge transfer dataset], each of which is used to trained a teacher model that is not accessible by an adversary [i.e., private]) ….”
Claim 6 is rejected under 35 U.S.C. 103 as being unpatentable over Papernot in view of Sidorov and further in view of Afshar et al. (US 20210224582) (“Afshar”).
Regarding claim 6, neither Papernot nor Sidorov appears to disclose explicitly the further limitations of the claim. However, Afshar discloses that “the similarity calculation is selected from the group consisting of a Euclidean distance calculation method, a Manhattan distance calculation method, a Chebyshev distance calculation method, and a Mahalanobis distance calculation method (similarity value may be a distance metric between a first vector space distance and a second vector space distance in vector space (e.g., Euclidean distance or Chebyshev distance) – Afshar, paragraph 68).”
Afshar and the instant application both relate to distance metrics in machine learning and are analogous. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the combination of Papernot and Sidorov to calculate a Chebyshev distance as the similarity metric, as disclosed by Afshar, and an ordinary artisan could reasonably expect to have done so successfully. Doing so would give the system an objective measure of similarity between two data points, thereby increasing the information with which the system may work. See Afshar, paragraph 68.
Claims 8-10 are rejected under 35 U.S.C. 103 as being unpatentable over Papernot in view of Tetko et al., “Efficient Partition of Learning Data Sets for Neural Network Training,” in 10.8 Neural Networks 1361-74 (1997) (“Tetko”).
Regarding claim 8, Papernot discloses “[a] weighted knowledge transfer system comprising:
a private device that stores a private dataset (Papernot Fig. 1 shows sensitive data [stored on a private device]);
a public device configured to include a public dataset and provide machine learning model[-]based services to users (Papernot Fig. 1 shows a public device that comprises a student model [machine learning model-based services] accessible by an adversary [user] and incomplete public data [public dataset]); and
a weighted knowledge transfer device configured to generate a set of trained public machine learning models using the private dataset and the public dataset (Papernot Fig. 1 shows a student model accessible by an adversary [set of trained public machine learning models] that is trained using incomplete public data [public dataset] and predictions given by the aggregated teacher models [themselves trained on sensitive data, i.e., a private dataset; note that the incomplete public data and their private labels may be considered to be the weighted knowledge transfer device]),
wherein:
the weighted knowledge transfer device is communicably connected to the private device via a first network connection (Papernot Fig. 1 shows that the aggregate teacher model outputs labels for the incomplete data, i.e., is connected to the system that provides the data for the student model);
the weighted knowledge transfer device is communicably connected to the public device via a second network connection different [from] the first network connection (Papernot Fig. 1 shows that the public data [i.e., the system that provides the data for the student model, or the knowledge transfer device] are fed to the student model via a different connection from that used to transfer the outputs of the aggregate teacher);
the private device is inaccessible from the public device (Papernot Fig. 1 shows that the student and the incomplete public data [public device] are accessible by an adversary and that the remainder of the system [private device] is not);
the weighted knowledge transfer device includes:
a partitioning optimization unit configured to:
divide the private dataset into a first plurality of partitions (see mapping of limitation below), and
generate a set of trained private machine learning models by training each of a set of machine learning models based on a separate partition of the first plurality of partitions (instead of training a single model to solve the task associated with a dataset, the data are partitioned in n disjoint sets and a model is trained separately on each set – Papernot, sec. 2.1, subsection entitled “Training the Ensemble of Teachers”; see also Fig. 1 [showing that the ensemble of teachers is not accessible to the adversary, i.e., they are private]);
a data selection unit configured to:
generate, as a processed public dataset, a group of labels … created by processing the public dataset using the set of trained private machine learning models (Papernot Fig. 1 and associated caption show that the incomplete public data are processed using the aggregated teacher model [trained private machine learning models] to predict labels and complete the public data), …; and
a machine learning unit configured to:
generate a set of trained public machine learning models …, and deploy the set of trained public machine learning models to the public device for provision to users (Papernot Fig. 1 shows that the student model [set of trained public machine learning models] is generated using the completed public data and is accessible to an adversary [i.e., deployed to a public device] for querying [i.e., provision to users]).”
Papernot appears not to disclose explicitly the further limitations of the claim. However, Tetko discloses “generat[ing] … a group of … weights (unsupervised learning algorithm uses arrays of predicted values to subdivide the initial training set into a number of clusters Q; for each cluster q1, a normalized mean squared error between the target values and ensemble predictions was calculated; a weight proportional to the mean squared error [weight] is attributed to each cluster – Tetko, p. 1366, second full paragraph) …;
divid[ing] the processed … dataset into a second plurality of partitions based on a set of thresholds determined based on the group of labels and weights (unsupervised learning algorithm uses arrays of predicted values to subdivide [partition] the initial training set into a number of clusters Q; for each cluster q1, a normalized mean squared error between the target values [labels] and ensemble predictions was calculated; a weight proportional to the mean square error [i.e., based on the labels] is attributed to each cluster, and a case is randomly selected given a probability [threshold] of selecting a case xi belonging to q proportional to the weight – Tetko, p. 1366, second full paragraph); and …
training each of a set of machine learning models based on a … partition of the second plurality of partitions (for each ANN [set of machine learning models] computation begins with two cases in the learning data set selected at random; at the end of the first iteration, four additional cases are selected from the input data set and added to the corresponding data sets; thus, the second iteration of the training procedure for each ANN is based upon four cases in the learning data set – Tetko, p. 1366, third full paragraph) ….”
Tetko and the instant application both relate to dataset partitioning for machine learning and are analogous. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Papernot to partition the dataset based on thresholds, as disclosed by Tetko, and an ordinary artisan could reasonably expect to have done so successfully. Doing so would speed up the training of each network and reduce overfitting by ensuring that each network is only trained on a portion of the data. See Tetko, sec. 3, first paragraph.
Regarding claim 9, Papernot, as modified by Tetko, discloses “an evaluation unit configured to:
evaluate a performance of the set of trained private machine learning models with respect to a set of external test data selected from the private dataset (number n of partitions of private data for training the teacher models [private models] is tested by empirically finding appropriate values of n by measuring a test set [external test data] accuracy [performance] of each teacher trained on one of the n partitions of training data – Papernot, sec. 4.1, subsection entitled “Prediction accuracy”; see also sec. 4.1, first paragraph (indicating that the ensemble of teachers was evaluated on respective test sets, i.e., sets culled from the private data)); and
determine, based on the performance of the set of trained private machine learning models with respect to the set of external test data selected from the private dataset, a first set of model configuration parameters for the set of trained private machine learning models that achieves a predetermined performance criterion (number n of teachers [model configuration parameter] is limited by a tradeoff between the classification task’s complexity and the available data; larger values of n lead to larger absolute gaps, but imply a smaller training dataset for each teacher; potentially reducing the accuracy [accuracy/complexity = predetermined performance criteria]; number n of partitions of private data for training the teacher models [private models] is tested [determined] by empirically finding appropriate values of n by measuring a test set [external test data] accuracy [performance] of each teacher trained on one of the n partitions of training data – Papernot, sec. 4.1, subsection entitled “Prediction accuracy”).”
Regarding claim 10, the rejection of claim 8 is incorporated. Papernot further discloses that “the evaluation unit is further configured to:
evaluate a performance of the set of trained public machine learning models with respect to a set of external test data selected from the private dataset (number n of partitions of private data for training the teacher models [private models] is tested by empirically finding appropriate values of n by measuring a test set [external test data] accuracy [performance] of each teacher trained on one of the n partitions of training data – Papernot, sec. 4.1, subsection entitled “Prediction accuracy”; see also sec. 4.1, first paragraph (indicating that the ensemble of teachers was evaluated on respective test sets, i.e., sets culled from the private data)); and
determine, based on the performance of the set of trained private machine learning models with respect to the set of external test data selected from the private dataset, a … set of model configuration parameters for the set of trained public machine learning models that achieves a predetermined performance criterion (number n of teachers [model configuration parameter] is limited by a tradeoff between the classification task’s complexity and the available data; larger values of n lead to larger absolute gaps, but imply a smaller training dataset for each teacher; potentially reducing the accuracy [accuracy/complexity = predetermined performance criteria]; number n of partitions of private data for training the teacher models [private models] is tested [determined] by empirically finding appropriate values of n by measuring a test set [external test data] accuracy [performance] of each teacher trained on one of the n partitions of training data – Papernot, sec. 4.1, subsection entitled “Prediction accuracy”).”
Papernot appears not to disclose explicitly the further limitations of the claim. However, Tetko discloses “determin[ing] … a second set of model configuration parameters (methods aimed to decrease the number of adjustable parameters in an ANN include pruning algorithms, penalized regularization algorithms, and specialized architexture and weight sharing – Tetko, p. 1362, second full paragraph) ….” It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Papernot to determine a set of model configuration parameters, as disclosed by Tetko, and an ordinary artisan could reasonably expect to have done so successfully. Doing so would speed up the training of each network and reduce overfitting by ensuring that each network is only trained on a portion of the data. See Tetko, sec. 3, first paragraph.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to RYAN C VAUGHN whose telephone number is (571)272-4849. The examiner can normally be reached M-R 7:00a-5:00p ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kamran Afshar, can be reached at 571-272-7796. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/RYAN C VAUGHN/ Primary Examiner, Art Unit 2125