DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of Claims
The present application is being examined under the claims filed 10/16/2025.
Claims 1-6 are pending.
Response to Amendment
This Office Action is in response to Applicant’s communication filed 10/16/2025 in response to office action mailed 07/16/2025. The Applicant’s remarks and any amendments to the claims or specification have been considered with the results that follow.
Response to Arguments
Regarding Claim Objections
In Remarks page 5-6
(Examiner summarizes Applicant’s arguments) Applicant argues that claim amendments obviate the objections.
Examiner’s response to Argument 1
Examiner agrees that the amendments overcome all claim objections.
Regarding 35 U.S.C. 112
In Remarks page 6-7, Argument 2
(Examiner summarizes Applicant’s arguments) Applicant argues that claim amendments obviate 35 U.S.C. 112(b) rejections.
Examiner’s response to Argument 2
Most of the rejections under 35 U.S.C. 112(b). However, some remain. See rejections under 35 U.S.C. 112(b) for further details.
Regarding 35 U.S.C. 101
In Remarks page 7-8, Argument 3
(Examiner summarizes Applicant’s arguments) Applicant argues that claim 1 is not directed to a mental process because the limitations perform computer operations in a federated learning environment. Particularly, Applicant states that performing data lookups, adding noise to local histograms, generating a global histogram, and training a decision tree model cannot be performed in the human mind, but specific data operations implemented on a computer system.
Examiner’s response to Argument 3
Examiner disagrees. Histograms are a mathematical object and the claims explicitly recite operations of adding noise to local histograms using differential privacy which is a known kind of mathematical operation. Adding noise to a histogram and generating a histogram are mathematical calculations. The claim is not merely based on/involves math but actually recites a particular mathematical calculation. The specification even explicitly states that (paragraph [0039]) “the local histogram consists of the first-order derivative and the second-order derivative of each sample. By calculating the first-order derivatives and second-order derivatives of all samples locally, and using the quantile sketch to construct the histogram, the leakage of data features can be avoided.” The operations specified by Applicant are well known statistical techniques performed using mathematics, not specific computer operations.
In Remarks page 8, Argument 4
Applicant respectfully submits that the present application applies decision trees to federated learning, which is not a conventional combination, and thus enabling efficient implementation of federated learning in horizontal data distribution scenarios.
Examiner’s response to Argument 4
Examiner disagrees. The limitations as claimed do not provide any unconventional details about the way in which federated learning and decision trees are combined. For example, as shown in the prior office action, adding noise to a histogram through secure aggregation is well-understood, routine, and conventional activity. The other limitations are recited at a high level of generality and do not substantially limit the judicial exception to anything more than generic machine learning and well-known computer operations. Even if Applicant’s specification discloses a non-generic combination of federated learning and decision trees, the requisite inventive details are not present in the claims.
In Remarks page 8, Argument 5
In addition, by combining decision trees, differential privacy, and secure aggregation, the transmission efficiency of data is greatly improved, while ensuring data security and reducing the required running time.
The above are specific improvements to the operation method of the computer system and practical applications of machine learning, rather than just improvements to abstract ideas. In summary, the present application has the advantages of easy use and efficient training, which can protect data privacy and provide quantitative support for the level of data protection. These improvements are directly achieved by the technical features in the claims.
Thus, independent claim 1 and dependent claims 2-6 provide an inventive concept in Step 2B.
Examiner’s response to Argument 5
Examiner disagrees. MPEP 2106.05(a) recites
After the examiner has consulted the specification and determined that the disclosed invention improves technology, the claim must be evaluated to ensure the claim itself reflects the disclosed improvement in technology. […] That is, the claim must include the components or steps of the invention that provide the improvement described in the specification
Applicant merely states that the combination of different technologies results in improvements in efficiency, data privacy, and other areas. However, the claim does not recite all of the necessary steps to provide an improvement. That is, a person having ordinary skill in the art would not recognize that the claim provides any improvement beyond what is already well-known, mere instructions to apply, and merely limiting a judicial exception to a field of use.
Regarding 35 U.S.C. 103
In Remarks page 8, Argument 6
First, the application scenario of Liu' s technical solution is different from that of the present application. On the one hand, in the application scenario of the present application, a participant can be an institution, and each institution possesses data from a plurality of users. Therefore, it is necessary to first aggregate local information, i.e., to update the local histograms. In contrast, FedXGB in Liu's technical solution is applied in a model scenario where each participant corresponds to a single user and only holds its own data. Accordingly, Liu' s solution does not involve any local information aggregation process. On the other hand, in the technical solution of Liu, FedXGB only computes local gradients for each candidate and reports them, without performing any local information statistics. Thus, a person of ordinary skill in the art, based on the teachings from Liu, would have no motivation to update the local histograms.
Examiner’s response to Argument 6
Applicant’s argument is unconvincing. The claims do not recite each participant being an institution comprising a plurality of users. Applicant does not argue on the claims as filed, but instead on alleged features of the invention that do not appear in the claims. Applicant may amend the claims with features present in the specification, however features from the specification are not to be read into the claims (see MPEP 2111.01 II).
In Remarks page 9, Argument 7
Second, in the present application, the local histograms are constructed based on quantile sketch, the local histograms are further merged into a global histogram, and the root node of the decision tree is trained according to the global histogram. After the coordinator sends the node information to the participants, the participants update the local histograms based on the node information. However, in the technical solution of Liu, each user computes local gradient for each candidate split and employs SecAgg to calculate split scores, after which the server selects the candidate with the highest score as the optimal split and returns it. Thus, it can be seen that the source of the split value is the candidate split, which is not the same as the quantile sketch of the present application.
Examiner’s response to Argument 7
Applicant’s arguments are not convincing. Applicant argues that Liu does not teach features for which Ong is relied upon (constructing local histograms based on quantile sketch, see NF office action page 23). One cannot show nonobviousness by attacking references individually where the rejections are based on combinations of references. See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981); In re Merck & Co., 800 F.2d 1091, 231 USPQ 375 (Fed. Cir. 1986).
In Remarks page 9, Argument 8
Third, in the Office Action, it is believed that Liu does not disclose the technical feature "adding a noise to the local histograms". Accordingly, Liu cannot achieve the same improvements as the present application, namely, significantly improving data transmission efficiency, reducing runtime, and enabling industrial deployment while still protecting data privacy. The technical solution of Liu is limited to secure training of XGBoost in a secret-sharing environment, with improvements directed to different communication semantics, rather than to improvements in data volume or transmission efficiency.
Examiner’s response to Argument 8
As with the argument above, Applicant argues that Liu teaches features for which another reference (Bonawitz) is relied upon. Thus the arguments are unconvincing.
In Remarks page 10, Argument 9
Furthermore, Bonawitz does not teach how to add or calibrate noise into histograms. Bonawitz merely refers to secure aggregation and uses random masking values to achieve "noise addition". Based on the teachings from Bonawitz, a person of ordinary skill in the art would not be motivated to adopt the principle of differential privacy to add noise. Therefore, claim 1 is patentable over Ong, in view of Li and Bonawitz.
Examiner’s response to Argument 9
Examiner disagrees. While Ong teaches histogram representations, Bonawitz teaches the concepts of differential privacy and particularly describes adding noise for machine learning models. In addition to the cited portion in the specification, see page 20 column 2 paragraph 2 recites “In the local privacy setting, users distrust the aggregator, and so before any user submits her value to the aggregator, she adds noise”. This whole column describes in detail a process to add noise during a federated learning process. Furthermore, as noted in the office action, the methods disclosed by Bonawitz improve efficiency and privacy, thus a person having ordinary skill in the art would be motivated to combine Bonawitz with a federated learning systems such as the Ong-Liu combination. Applicant provides mere allegation that Bonawitz does not teach adding noise to histograms while ignoring that Ong teaches histograms and Bonawitz’ clear disclosure of adding noise in a federated learning context. Therefore, the 35 U.S.C. 103 rejection of claim 1 is maintained.
In Remarks page 10, Argument 10
Claims 2-6 depend from claim 1 Therefore, claims 2-6 are also patentable over Ong, in view of Li and Bonawitz, at least for their dependency from the patentable base claim, and also because of additional features of these claims. Therefore, Applicant respectfully requests withdrawal of the Sec. 103 rejections.
Examiner’s response to Argument 10
Claims 2-6 are unpatentable for the reasons given above, and in the rejections under 35 U.S.C. 103 below.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 3 and 5 are rejected under 35 U.S.C. 112(b) as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor regards as the invention.
Regarding Claim 3
Claim 3 recites the limitation "the coordinator calculating a data percentage of a quantile candidate value according to the number of samples that counted in step (a) and statistics got in step (c)". There is insufficient antecedent basis for this limitation in the claim. In particular, there is insufficient antecedent basis for “statistics got in step (c)”. It is unclear what is meant by “statistics”, and step (c) does not clarify what is meant by “statistics”. Therefore, the claim is rendered indefinite.
Regarding Claim 5
Claim 5 recites the limitation " wherein the samples with feature values smaller than a node value selected in the step (5) into the left sub-node". There is insufficient antecedent basis for this limitation in the claim. In particular, there is insufficient antecedent basis for “a node value selected in step (5)”. There is no “node value” selected in step 5, so the limitation is unclear rendering the claim indefinite.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-6 are rejected under 35 U.S.C. 101 for containing an abstract idea without significantly more.
Regarding Claim 1:
Step 1 – Is the claim to a process, machine, manufacture, or composition of matter?
Yes, the claim is to a process.
Step 2A – Prong 1 – Does the claim recite an abstract idea, law of nature, or natural phenomenon?
Yes, the claim recites the abstract ideas of:
A federated learning method for decision tree-oriented horizontal, wherein the decision tree is Gradient Boosting Decision Trees, and the method comprises the following steps: (1) a plurality of participants searching for a quantile sketch of data of features in a data feature set by dichotomy, and publishing the quantile sketch to the participants — This limitation is directed to the abstract idea of a mental process (including an observation, evaluation, judgement, opinion) which can be performed by the human mind, or by a human using pen and paper (see MPEP 2106.04(a)(2) III. C.). The limitation is directed to a mental process because it amounts to performing an evaluation of data to approximate a distribution of features into categories.
(2) the participants respectively constructing local histograms of the features in the data feature set according to the quantile sketch searched in step (1), and adding a noise to the local histograms;— This limitation is directed to the abstract idea of a mathematical process, and mathematical calculations in particular (MPEP 2106.04(a)(2) I. C.). The claim describes the mathematical operations of addition and differential privacy.
Step 2A – Prong 2 – Does the claim recite additional elements that integrate the judicial exception into a practical application?
No, the claim does not recite additional elements that integrate the judicial exception into a practical application. The additional elements:
(3) subsequently, the participants except a coordinator sending the local histograms added with the noise to the coordinator through secure aggregation, wherein the coordinator is one of the participants — This limitation is directed to insignificant application of data, which has been recognized by the courts (as per Ameranth, 842 F.3d at 1241-42, 120 USPQ2d at 1854-55.) as insignificant extra-solution activity (see MPEP 2106.05(g)).
(4) the coordinator merging the local histograms of the features into a global histogram, and training a root node of a first decision tree according to the global histogram — This limitation is directed to mere instructions to apply a judicial exception. Using machine learning training to apply a judicial exception (see MPEP 2106.05(f)) is insufficient to integrate the judicial exception into a practical application. Even if the machine learning training is implemented on a generic computer (see MPEP 2106.05(f)(2), 2106.04(d)), the limitation does not integrate the judicial exception into a practical application.
(5) the coordinator sending node information to other participants, wherein the node information comprises a selected data feature and separation methods corresponding to the selected data feature in the global histogram — This limitation is directed to mere data gathering and outputting which has been recognized by the courts (as per Ultramercial, 772 F.3d at 715, 112 USPQ2d at 1754) as insignificant extra-solution activity (see MPEP 2106.05(g)).
(6) the participants updating the local histogram according to the node information — This limitation is directed to mere data gathering and outputting which has been recognized by the courts (as per Ultramercial, 772 F.3d at 715, 112 USPQ2d at 1754) as insignificant extra-solution activity (see MPEP 2106.05(g)).
(7) repeating steps (2)-(6) according to the updated local histograms until the training of remaining child nodes in the first decision tree is completed; (8) repeating step (7) until the training of all decision trees is completed to obtain a final Gradient Boosting Decision Trees model — This limitation is directed to mere instructions to apply a judicial exception. Using repetition of steps to apply a judicial exception (see MPEP 2106.05(f)) is insufficient to integrate the judicial exception into a practical application. Even if the repeating is implemented on a generic computer (see MPEP 2106.05(f)(2), 2106.04(d)), the limitation does not integrate the judicial exception into a practical application.
Step 2B – Does the claim recite additional elements that amount to significantly more than the abstract idea itself?
No, the claim does not recite additional elements which amount to significantly more than the abstract idea itself. The additional elements as identified in step 2A prong 2:
(3) subsequently, the participants except a coordinator sending the local histograms added with the noise to the coordinator through secure aggregation, wherein the coordinator is one of the participants — This limitation is recited in a merely generic manner and amounts to performing secure aggregation, which is well-understood, routine, and conventional activity. A factual determination that this element is well-understood, routine, and conventional activity (see MPEP 2106.05(d) I.) is supported by, Bonawitz et al. “Practical Secure aggregation for Privacy-Preserving Machine Learning”, which recites that (page 1 column 1 paragraph 4) “The secure aggregation problem has been a rich area of research: different approaches include works based on generic secure multi-party computation protocols”. Therefore, the additional element cannot amount to significantly more than the judicial exception under step 2B.
(4) the coordinator merging the local histograms of the features into a global histogram, and training a root node of a first decision tree according to the global histogram — Mere instructions to apply a judicial exception (see MPEP 2106.05(f)) and using a generic computer as a tool (see MPEP 2106.05(f)(2), 2106.05(d)) cannot amount to significantly more than the judicial exception itself.
(5) the coordinator sending node information to other participants, wherein the node information comprises a selected data feature and separation methods corresponding to the selected data feature in the global histogram — This limitation is recited at a high level of generality and amounts to mere data gathering of transmitting and receiving data over a network, which is well-understood, routine, and conventional activity (see MPEP 2106.05(d) II.), which cannot amount to significantly more than the judicial exception.
(6) the participants updating the local histogram according to the node information — This limitation is recited at a high level of generality and amounts to mere data gathering of storing and retrieving information in memory, which is well-understood, routine, and conventional activity (see MPEP 2106.05(d) II.), which cannot amount to significantly more than the judicial exception.
(7) repeating steps (2)-(6) according to the updated local histograms until the training of remaining child nodes in the first decision tree is completed; (8) repeating step (7) until the training of all decision trees is completed to obtain a final Gradient Boosting Decision Trees model — Mere instructions to apply a judicial exception (see MPEP 2106.05(f)) and using a generic computer as a tool (see MPEP 2106.05(f)(2), 2106.05(d)) cannot amount to significantly more than the judicial exception itself.
Regarding Claim 2
Claim 2 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. The claim is dependent on claim 1 which included an abstract idea (see rejection for claim 1). The claim recites the additional limitations:
Step 2A Prong 2:
wherein the data feature set is personal privacy information — This limitation is directed to merely limiting a judicial exception to a particular field of use (see MPEP 2106.05(h)) as it merely limits the field of the data feature set.
Thus, the judicial exception is not integrated into a practical application (see MPEP 2106.04(d) I.), failing step 2A prong 2.
Step 2B:
The additional elements as identified in step 2A prong 2:
wherein the data feature set is personal privacy information — Merely limiting a judicial exception to a particular field of use (see MPEP 2106.05(h)) cannot amount to significantly more than the judicial exception.
Thus, the claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception under step 2B.
Regarding Claim 3
Claim 3 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. The claim is dependent on claim 1 which included an abstract idea (see rejection for claim 1). The claim recites the additional limitations:
Step 2A Prong 1:
(b) the coordinator setting a maximum value and a minimum value the features, and taking an average value of the maximum value and the minimum value of feature values as a quantile candidate value for the data feature— This limitation is directed to the abstract idea of a mathematical process, and mathematical calculations in particular (MPEP 2106.04(a)(2) I. C.). The claim describes the mathematical calculations of computing an average.
(d) the coordinator calculating a data percentage of a quantile candidate value according to the number of samples that counted in step (a) and statistics got in step (c), if the data percentage is less than a data percentage of a target quantile, taking the quantile candidate value as the minimum value, and if the data percentage is greater than the data percentage of the target quantile, taking the quantile candidate value as the maximum value, a mean value is recalculated to obtain an updated mean value, and the updated mean value is taken as the quantile candidate value — This limitation is directed to the abstract idea of a mathematical process, and mathematical calculations in particular (MPEP 2106.04(a)(2) I. C.). The claim describes the mathematical operation of calculating percentages and averages.
Step 2A Prong 2:
wherein the dichotomy in step (1) comprises: (a) the coordinator obtaining a total number of samples held by the plurality of participants through a secure aggregation method — This limitation is directed to mere data gathering and outputting which has been recognized by the courts (as per Ultramercial, 772 F.3d at 715, 112 USPQ2d at 1754) as insignificant extra-solution activity (see MPEP 2106.05(g)).
(c) the coordinator sending the quantile candidate values of all feature to other participants, the participants count a number of samples smaller than the quantile candidate value of the features in the data feature set, respectively, and sending the results to the coordinator via the secure aggregation method — This limitation is directed to insignificant application of data, which has been recognized by the courts (as per Ameranth, 842 F.3d at 1241-42, 120 USPQ2d at 1854-55.) as insignificant extra-solution activity (see MPEP 2106.05(g)).
(e) repeating the processes (c)-(d) until the data percentage of a quantile is equal to the data percentage of the target quantile; (f) repeating the processes (b)-(d) to search for remaining quantiles, wherein quantiles constitute the quantile sketch. — This limitation is directed to mere instructions to apply a judicial exception. Using repetition of steps to apply a judicial exception (see MPEP 2106.05(f)) is insufficient to integrate the judicial exception into a practical application. Even if the repeating is implemented on a generic computer (see MPEP 2106.05(f)(2), 2106.04(d)), the limitation does not integrate the judicial exception into a practical application.
Thus, the judicial exception is not integrated into a practical application (see MPEP 2106.04(d) I.), failing step 2A prong 2.
Step 2B:
The additional elements as identified in step 2A prong 2:
wherein the dichotomy in step (1) comprises: (a) the coordinator obtaining a total number of samples held by the plurality of participants through a secure aggregation method — This limitation is recited in a merely generic manner and amounts to performing secure aggregation, which is well-understood, routine, and conventional activity. A factual determination that this element is well-understood, routine, and conventional activity (see MPEP 2106.05(d) I.) is supported by, Bonawitz et al. “Practical Secure aggregation for Privacy-Preserving Machine Learning”, which recites that (page 1 column 1 paragraph 4) “The secure aggregation problem has been a rich area of research: different approaches include works based on generic secure multi-party computation protocols”. Therefore, the additional element cannot amount to significantly more than the judicial exception under step 2B.
(c) the coordinator sending the quantile candidate values of all feature to other participants, the participants count a number of samples smaller than the quantile candidate value of the features in the data feature set, respectively, and sending the results to the coordinator via the secure aggregation method — This limitation is recited in a merely generic manner and amounts to performing secure aggregation, which is well-understood, routine, and conventional activity. A factual determination that this element is well-understood, routine, and conventional activity (see MPEP 2106.05(d) I.) is supported by the evidence provided in the limitation above.
(e) repeating the processes (c)-(d) until the data percentage of a quantile is equal to the data percentage of the target quantile; (f) repeating the processes (b)-(d) to search for remaining quantiles, wherein quantiles constitute the quantile sketch. — Mere instructions to apply a judicial exception (see MPEP 2106.05(f)) and using a generic computer as a tool (see MPEP 2106.05(f)(2), 2106.05(d)) cannot amount to significantly more than the judicial exception itself.
Thus, the claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception under step 2B.
Regarding Claim 4
Claim 4 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. The claim is dependent on claim 1 which included an abstract idea (see rejection for claim 1). The claim merely recites the additional abstract idea:
Step 2A Prong 1:
wherein the local histograms are composed of first-order derivatives and second-order derivatives of all sample, respectively — This limitation is directed to the abstract idea of a mathematical process, and a mathematical relationship in particular (MPEP 2106.04(a)(2) I. A.). The claim describes the mathematical operations of first and second derivatives in words.
Thus, the judicial exception is not integrated into a practical application (see MPEP 2106.04(d) I.), failing step 2A prong 2. The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception under step 2B.
Regarding Claim 5
Claim 5 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. The claim is dependent on claim 1 which included an abstract idea (see rejection for claim 1). The claim recites the additional limitations:
Step 2A Prong 1:
obtaining an optimal separation method by calculation — This limitation is directed to the abstract idea of a mathematical process, and mathematical calculations in particular (MPEP 2106.04(a)(2) I. C.). The claim describes performing calculations in words.
and vertically divides the global histogram into two parts according to the separation method — This limitation is directed to the abstract idea of a mental process (including an observation, evaluation, judgement, opinion) which can be performed by the human mind, or by a human using pen and paper (see MPEP 2106.04(a)(2) III. C.). The limitation is directed to a mental process because it amounts to performing an evaluation of given data to separate it into two groups based on a known rule.
Step 2A Prong 2:
wherein the method of training the root node of the first decision tree according to the global histogram specifically comprises: the coordinator traversing the features in the data feature set and simultaneously traversing separation methods of the global histogram of the features— This limitation is directed to mere data gathering and outputting which has been recognized by the courts (as per Ultramercial, 772 F.3d at 715, 112 USPQ2d at 1754) as insignificant extra-solution activity (see MPEP 2106.05(g)).
Thus, the judicial exception is not integrated into a practical application (see MPEP 2106.04(d) I.), failing step 2A prong 2.
Step 2B:
The additional elements as identified in step 2A prong 2:
wherein the method of training the root node of the first decision tree according to the global histogram specifically comprises: the coordinator traversing the features in the data feature set and simultaneously traversing separation methods of the global histogram of the features — This limitation is recited at a high level of generality and amounts to mere data gathering of storing and retrieving information in memory, which is well-understood, routine, and conventional activity (see MPEP 2106.05(d) II.), which cannot amount to significantly more than the judicial exception.
Thus, the claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception under step 2B.
Regarding Claim 6
Claim 6 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. The claim is dependent on claim 1 which included an abstract idea (see rejection for claim 1). The claim recites the additional limitations:
Step 2A Prong 1:
wherein the step (6) comprises the following sub- steps:
(6.1) the plurality of participants selecting a corresponding quantile as a value of a node according to the node information returned by the coordinator and referring to the quantile sketch; — This limitation is directed to the abstract idea of a mental process (including an observation, evaluation, judgement, opinion) which can be performed by the human mind, or by a human using pen and paper (see MPEP 2106.04(a)(2) III. C.). The limitation is directed to a mental process because it amounts to performing an evaluation of given data to select a value based on a known rule.
(6.2) according to the value of the node, the participants splitting the samples they own to left and tight sub-nodes of the node, wherein the samples with feature values smaller than a node value selected in the step (5) into the left sub-node, and the samples with feature values larger than the node value into the right sub-node — This limitation is directed to the abstract idea of a mental process (including an observation, evaluation, judgement, opinion) which can be performed by the human mind, or by a human using pen and paper (see MPEP 2106.04(a)(2) III. C.). The limitation is directed to a mental process because it amounts to performing an evaluation of given data to separate it into two groups based on a known rule.
Step 2A Prong 2:
and updating the local histograms — This limitation is directed to mere data gathering and outputting which has been recognized by the courts (as per Ultramercial, 772 F.3d at 715, 112 USPQ2d at 1754) as insignificant extra-solution activity (see MPEP 2106.05(g)).
Thus, the judicial exception is not integrated into a practical application (see MPEP 2106.04(d) I.), failing step 2A prong 2.
Step 2B:
The additional elements as identified in step 2A prong 2:
and updating the local histograms — This limitation is recited at a high level of generality and amounts to mere data gathering of storing and retrieving information in memory, which is well-understood, routine, and conventional activity (see MPEP 2106.05(d) II.), which cannot amount to significantly more than the judicial exception.
Thus, the claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception under step 2B.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-2 and 4-6 are rejected under 35 U.S.C. 103 as being unpatentable over Ong et al. (PGPUB no. US 20220083906 A1) herein referred to as Ong in view of NPL reference Liu et al. “Boosting Privately: Privacy-Preserving Federated Extreme Boosting for Mobile Crowdsensing” and NPL reference Bonawitz et al. “Practical Secure Aggregation for Privacy-Preserving Machine Learning” herein referred to as Bonawitz.
Regarding Claim 1:
Ong teaches:
A federated learning method for decision tree-oriented horizontal, wherein the decision tree is Gradient Boosting Decision Trees
(paragraph [0025]) “The XGBoost performs a federated quantile sketch method to fuse the histograms received from each party resulting in a global histogram.”
and the method comprises the following steps: (1) a plurality of participants searching for a quantile sketch of data of features in a data feature set by dichotomy, and publishing the quantile sketch to the participants;
(paragraph [0043]) “The parties 150 can also exploit the epsilon hyperparameter using a quantile sketch process to generate histograms reflecting their respective training datasets 155.”; (paragraph [0025]) “The XGBoost performs a federated quantile sketch method to fuse the histograms received from each party[*Examiner notes: publishing quantile sketch to all participants] resulting in a global histogram.”; (paragraph [0026]) “the tree boosting aggregator transmits the model and the epsilon hyperparameter to each of the parties and receives back new model updates and new histograms from the parties.”
(2) the participants respectively constructing local histograms of the features in the data feature set according to the quantile sketch searched in step (1)
(paragraph [0043]) “The parties 150 can also exploit the epsilon hyperparameter using a quantile sketch process to generate histograms reflecting their respective training datasets 155.”;
(4) the coordinator merging the local histograms of the features into a global histogram, and training a root node of a first decision tree according to the global histogram
(paragraph [0025]) “The XGBoost performs a federated quantile sketch method to fuse the histograms received from each party[*Examiner notes: merging the local histograms] resulting in a global histogram.”; (paragraph [0006]) “The method further includes determining at least one split candidate in a decision tree used by the machine learning model using the global histogram and rebuilding the machine learning model by adding the split candidate to the decision tree.”
(7) repeating steps (2)-(6) according to the updated local histograms until the training of remaining child nodes in the first decision tree is completed; (8) repeating step (7) until the training of all decision trees is completed to obtain a final Gradient Boosting Decision Trees model
(paragraph [0053]) “If the machine learning model 120 has not reached a stopping criterion, the process 200 can return to step 230 and repeat the steps in a recursive manner until the stopping criterion is achieved. However, if the stopping criterion is achieved, the process 200 is complete, and the training process stops.”
Ong does not teach:
and adding a noise to the local histograms;
(3) subsequently, the participants except a coordinator sending the local histograms added with the noise to the coordinator through secure aggregation, wherein the coordinator is one of the participants;
(5) the coordinator sending node information to other participants, wherein the node information comprises a selected data feature and separation methods corresponding to the selected data feature in the global histogram
(6) the participants updating the local histogram according to the node information
However, Liu teaches:
(5) the coordinator sending node information to other participants, wherein the node information comprises a selected data feature and separation methods corresponding to the selected data feature in the global histogram
(page 4 column 1 paragraph 4) “For boosting, S[*Examiner notes: mapped to coordinator] randomly selects a sub-sample Q’ ⊂ Q, and invokes SecFind(Q’, U’; E{(u,Ru)}u ∈U0 ) to find the optimal split[*Examiner notes: node information]. […] Moreover, each user updates the value of ^y for the new round of training after receiving the newly constructed CART[*Examiner notes: coordinator sending node information to other participants]”
(6) the participants updating the local histogram according to the node information
(page 4 column 1 last paragraph) “Moreover, each user updates the value of ^y[*Examiner notes: updates local histogram] for the new round of training after receiving the newly constructed CART.”
Ong, Liu, and the instant application are analogous because they are all directed to machine learning.
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the present invention to modify the federated gradient boosting as taught by Ong with the sending and updating of data as taught by Liu because (Liu page 1 abstract) “The results show that FedXGB is secure in the honest-but-curious model, and attains approximate accuracy and convergence rate with the original model in low runtime.”
Bonawitz teaches:
and adding a noise to the local histograms;
(3) subsequently, the participants except a coordinator sending the local histograms added with the noise to the coordinator through secure aggregation, wherein the coordinator is one of the participants;
(page 20 column 2 paragraph 2) “In such cases, secure aggregation composes well with differential privacy [22]. […] For example, when computing averages, partial averages over subgroups of users may be computed and privacy-preserving noise may be incorporated [23, 32] before revealing the results to the data aggregator.”
Ong, Liu, Bonawitz, and the instant application are analogous because they are all directed to machine learning.
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the present invention to modify the gradient boosting of Ong in view of Liu with the secure aggregation taught by Bonawitz because (Bonawitz page 2 column 2 end of paragraph 2) “Thus, secure aggregation over just 1024-user subgroups holds the promise of a 32× improvement in differentially private estimate precision. We anticipate that these utility gains will be crucial as methods for differentially private deep learning in the trusted-aggregator setting [1] are adapted to support untrusted aggregators”
Regarding Claim 2
Ong in view of Liu and Bonawitz teaches:
The federated learning method for decision tree-oriented horizontal according to claim 1
(see rejection of claim 1)
And Ong further teaches:
wherein the data feature set is personal privacy information
(paragraph [0020]) “During the training process, the data is held by each party and does not leave the party. The model can then be transferred in part from one party to another under an encryption scheme, such that other parties cannot re-engineer the data at any given party. The resulting model is approximate to an ideal model with all data transferred to a single party.”
Regarding Claim 4
Ong in view of Liu and Bonawitz teaches:
The federated learning method for decision tree-oriented horizontal according to claim 1
(see rejection of claim 1)
And Ong further teaches:
wherein the local histograms are composed of first-order derivatives and second-order derivatives of all sample, respectively
(paragraph [0020]) “Extreme gradient boosting (“XGBoost”) is a scalable implementation of the gradient boosting framework that combines a linear model with a boosting tree model. It can use a first derivative and a second derivative of a loss function for second-order derivation. This allows the algorithm to converge faster than a typical gradient boosting algorithm while also improving efficiency of the optimal solution of the model.”
Regarding Claim 5
Ong in view of Liu and Bonawitz teaches:
The federated learning method for decision tree-oriented horizontal according to claim 1
(see rejection of claim 1)
Liu further teaches:
wherein the method of training the root node of the first decision tree according to the global histogram specifically comprises: the coordinator traversing the features in the data feature set and simultaneously traversing separation methods of the global histogram of the features, obtaining an optimal separation method by calculation, and vertically divides the global histogram into two parts according to the separation method
(page 4 column 2 paragraph 1) “The most important operation in XGBoost is to optimize the tree structure by finding the optimal split for each node of each CART. For the centralized training method, XGBoost achieves the split finding by simply sorting the feature values of training sample, and then, traversing the values to find the optimal split [16].”
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the present invention to combine Ong and Bonawitz with Liu for the same reasons given in claim 1 above.
Regarding Claim 6
Ong in view of Liu and Bonawitz teaches:
The federated learning method for decision tree-oriented horizontal according to claim 1,
(see rejection of claim 1)
And Ong further teaches:
wherein the step (6) comprises the following sub-steps: (6.1) the plurality of participants selecting a corresponding quantile as a value of a node according to the node information returned by the coordinator and referring to the quantile sketch; (paragraph [0051]) “The split finding component 140 utilizes an approximation algorithm to determine split candidates for the machine learning model 120. This is illustrated at step 250. The split finding component 140 can determine the split candidates[*Examiner notes: selecting corresponding quantile as value of node] based on points according to percentiles of feature distribution[*Examiner notes: according to quantile sketch] represented in the histograms provided by the parties 150 after training.”
(6.2) according to the value of the node, the participants splitting the samples they own to left and tight sub-nodes of the node, wherein the samples with feature values smaller than a node value selected in the step (5) into the left sub-node, and the samples with feature values larger than the node value into the right sub-node
(paragraph [0023]) “In decision tree learning, a tree is built by splitting the source set, constituting the root node of the tree, into subsets constituting the successor children. The splitting can be based on a set of splitting rules based on classification features”
and updating the local histograms
(paragraph [0024]) “Each party can transmit back corresponding model updates[*Examiner notes: updating local histograms] to the tree boosting aggregator, including local histograms of the training data used.”
Claim 3 is rejected under 35 U.S.C. 103 as being unpatentable over Ong in view of Liu and Bonawitz, and further in view of Mohammad et al. (PGPUB no. US 20210035693 A1) and Dutt et al. (PGPUB no. US 20210406744 A1).
Regarding Claim 3
Ong in view of Liu and Bonawitz teaches:
The federated learning method for decision tree-oriented horizontal according to claim 1
(see rejection of claim 1)
Bonawitz further teaches:
wherein the dichotomy in step (1) comprises: (a) the coordinator obtaining a total number of samples held by the plurality of participants through a secure aggregation method;
(page 1 column 1 paragraph 3) “As described in Section 2, the secure aggregation primitive can be used to privately combine the outputs of local machine learning on user devices, in order to update a global model.”
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the present invention to modify the gradient boosting of Ong in view of Liu and Bonawitz to further include the coordinator obtaining the total number of samples as taught by Bonawitz because (Bonawitz page 1 column 1 paragraph 3) “Training models in this way offers tangible benefits—a user’s device can share an update knowing that the service provider will only see that update after it has been averaged with those of other users.”
Liu further teaches:
(c) the coordinator sending the quantile candidate values of all feature to other participants, the participants count a number of samples smaller than the quantile candidate value of the features in the data feature set, respectively, and sending the results to the coordinator via the secure aggregation method;
(page 21 paragraph 3) “To add a node to the tree, each enclave in the cluster first proposes candidate splitting points according to percentiles of feature distribution. The algorithm then maps the continuous features into buckets split by these candidate points, aggregates the statistics, and finds the best solution among proposals based on the aggregated statistics”
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the present invention to combine Ong and Bonawitz with Liu for the same reasons given in claim 1 above.
Ong in view of Liu and Bonawitz does not explicitly teach:
(b) the coordinator setting a maximum value and a minimum value the features, and taking an average value of the maximum value and the minimum value of feature values as a quantile candidate value for the data feature;
(d) the coordinator calculating a data percentage of a quantile candidate value according to the number of samples that counted in step (a) and statistics got in step (c), if the data percentage is less than a data percentage of a target quantile, taking the quantile candidate value as the minimum value, and if the data percentage is greater than the data percentage of the target quantile, taking the quantile candidate value as the maximum value, a mean value is recalculated to obtain an updated mean value, and the updated mean value is taken as the quantile candidate value
(e) repeating the processes (c)-(d) until the data percentage of a quantile is equal to the data percentage of the target quantile; (f) repeating the processes (b)-(d) to search for remaining quantiles, wherein quantiles constitute the quantile sketch.
Mohammad teaches:
(b) the coordinator setting a maximum value and a minimum value the features, and taking an average value of the maximum value and the minimum value of feature values as a quantile candidate value for the data feature;
(paragraph [0045]) “In order to determine which features are candidates for elimination, the data cleaning and feature engineering subsystem 208 may generate boxplots for every variable. Example boxplots are shown in FIGS. 4A-4C. A boxplot may contain a mean value, a maximum value, a minimum value, a 75-percentile, a 25-percentile, outlier values, and/or the like.”
Ong, Liu, Bonawitz, Mohammad and the instant application are analogous because they are all directed to machine learning.
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the present invention to modify the gradient boosting of Ong in view of Liu and Bonawitz with the maximum, minimum, and mean as taught by Mohammad because (Mohammad paragraph [0030]) “The present methods and systems leverage the ubiquity of EHR data and machine learning models to improve predictive analytics with respect to, for example, patient scoring and hospitalization prediction. […] The present methods and systems may thus assist in building awareness, improving patient care, aiding doctors for intervention, and reducing overall costs of treatment.”
Dutt teaches:
(d) the coordinator calculating a data percentage of a quantile candidate value according to the number of samples that counted in step (a) and statistics got in step (c), if the data percentage is less than a data percentage of a target quantile, taking the quantile candidate value as the minimum value, and if the data percentage is greater than the data percentage of the target quantile, taking the quantile candidate value as the maximum value, a mean value is recalculated to obtain an updated mean value, and the updated mean value is taken as the quantile candidate value;
(paragraph [0049]) “In determining whether a model satisfies a target accuracy, the training size calculator may compute a confidence interval for a percentage of example predictions that are below an error threshold and determine whether the target percentile is equal to or below a lower bound of the confidence interval. If the target percentile is equal to or below the lower bound, the unknown actual percentage of example predictions below an error threshold is, with a probability of a defined confidence level, larger than the target percentile. Therefore, the training size calculator may stop adding additional training examples to the training data and return the model. If the target percentile is above the lower bound of the confidence interval, the training size calculator may determine whether the target percentile is equal to or above the upper bound of the confidence interval. The training size calculator may also determine whether the target percentile is contained in the confidence interval.”
(e) repeating the processes (c)-(d) until the data percentage of a quantile is equal to the data percentage of the target quantile; (f) repeating the processes (b)-(d) to search for remaining quantiles, wherein quantiles constitute the quantile sketch. (paragraph [0066]) “The training size calculator 114 may repeat this process until the model 138 satisfies the target accuracy 120.”
Ong, Liu, Bonawitz, Mohammad, Dutt, and the instant application are analogous because they are all directed to machine learning.
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the present invention to modify the gradient boosting of Ong in view of Liu, Bonawitz, and Mohammad with the repeating as taught by Dutt because (Dutt paragraph [0053]) “The model construction improvements bring 10× or more savings for a large fraction of expressions compared to existing training methods. Finally, injected estimates may bring improvement similar to injecting true selectivities.”
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Ezra J Baker whose telephone number is (703)756-1087. The examiner can normally be reached Monday - Friday 10:00 am - 8:00 pm ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, David Yi can be reached at (571) 270-7519. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/E.J.B./Examiner, Art Unit 2126
/DAVID YI/Supervisory Patent Examiner, Art Unit 2126
Prosecution Insights