Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
This action is response to the communication filed on February 03, 2026. Claims 1-20 are pending.
Response to Arguments
Applicant’s arguments regarding art rejection filed on February 03, 2026 have been considered but are moot in the view of new ground of rejection. The argument regarding 101 rejection is not persuasive.
Regarding 101 rejection applicant argues claims are directed to an improvement in computing technology (e.g., effective shared model development by participants with different private data) and is patent eligible.
In response examiner respectfully disagree. The claims as drafted is nothing more than mental process. As describe in the rejection the limitations identifying, training, determining as drafted, is a process that, under its broadest reasonable interpretation, covers performance of the limitation in the mind. The additional limitation mixing the proxy parameters considered as insignificant extra-solution activity. Accordingly, even in combination, these additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea. With respect to integration of the abstract idea into a practical application, the additional element of mixing step amounts to no more than mere instructions to apply the exception using a generic computer component. The courts have recognized these functions as well‐understood, routine, and conventional as they are claimed in a merely generic manner (e.g., at a high level of generality) or as insignificant extra-solution activity (see MPEP 2106.05(d) II, Receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information)). Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. The claim is not patent eligible.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. Regarding the claim 1, it recites a processor; and a computer-readable medium having instructions executable by the processor for: identifying a set of proxy parameters for a proxy model and a set of private parameters for a private model; training the proxy parameters and private parameters for a training iteration by: identifying a training batch from a private training data set; determining a set of proxy predictions from the proxy model applied to the training batch with the set of proxy parameters; determining a set of private predictions from the private model applied to the training batch with the set of private parameters; training the proxy parameters to reduce a proxy loss based the set of proxy predictions evaluated with respect to labels for the training batch and a first distillation loss based on the set of proxy predictions evaluated with respect to the set of private predictions; training the private parameters to reduce a private loss based on the set of private predictions evaluated with respect to labels for the training batch and a second distillation loss based on the set of private predictions evaluated with respect to the set of proxy predictions; and mixing the proxy parameters with one or more sets of other proxy model parameters trained with different private data.
The claim recited the limitation of identifying, training, determining as drafted, is a process that, under its broadest reasonable interpretation, covers performance of the limitation in the mind. User can mentally identify a parameter or train own self by reading batch data as recited and if necessary, user can use physical aid (pen and paper). Hence, the limitations are a mental process. See MPEP 2106.04(a)(2) III, B, If a claim recites a limitation that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper, the limitation falls within the mental processes grouping, and the claim recites an abstract idea. See, e.g., Benson, 409 U.S. at 67, 65, 175 USPQ at 674-75, 674 (noting that the claimed "conversion of [binary-coded decimal] numerals to pure binary numerals can be done mentally," i.e., "as a person would do it by head and hand."). Therefore, the identifying and determining limitation are a mental process.
The claim recites one additional element: mixing the proxy parameters with one or more sets of other proxy model parameters trained with different private data. The mixing step as recited is nothing but putting data together which is a mere data gathering in a form of insignificant extra-solution activity, (see Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information)). Accordingly, even in combination, these additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea. The claim is directed to the abstract idea.
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional element of mixing step amounts to no more than mere instructions to apply the exception using a generic computer component. The courts have recognized these functions as well‐understood, routine, and conventional as they are claimed in a merely generic manner (e.g., at a high level of generality) or as insignificant extra-solution activity (see MPEP 2106.05(d) II, Receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information)). Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. The claim is not patent eligible.
Claim 2 is dependent on claim 1 and includes all the limitations of claim 1. Therefore, claim 2 recites the same abstract idea of model training system. The claim recites the limitations of wherein mixing the proxy parameters, including replacing the proxy parameters with proxy parameters based on the one or more other proxy model parameters, which can be done mentally with or without the use of a physical aid (e.g., pen and paper) or with a generic computer and is not an inventive concept that meaningfully limits the abstract idea. Therefore, the limitation is a mental process.
Claim 3 is dependent on claim 1 and includes all the limitations of claim 1. Therefore, claim 3 recites the same abstract idea of model training system. The claim recites the limitations of wherein mixing the proxy parameters includes sending the proxy parameters and a bias matrix to another system training another proxy model, which can be done mentally with or without the use of a physical aid (e.g., pen and paper) or with a generic computer and is not an inventive concept that meaningfully limits the abstract idea. Therefore, the limitation is a mental process.
Claim 4 is dependent on claim 1 and includes all the limitations of claim 1. Therefore, claim 4 recites the same abstract idea of model training system. The claim recites the limitations of wherein mixing the proxy parameters includes receiving a bias matrix for each set of other proxy model parameters and applying the received bias matrix to debias the proxy parameters, which can be done mentally with or without the use of a physical aid (e.g., pen and paper) or with a generic computer and is not an inventive concept that meaningfully limits the abstract idea. Therefore, the limitation is a mental process.
Claim 5 is dependent on claim 1 and includes all the limitations of claim 1. Therefore, claim 5 recites the same abstract idea of model training system. The claim recites the limitations of wherein mixing the proxy model parameters with the one or more other proxy model parameters is based on an adjacency matrix, which can be done mentally with or without the use of a physical aid (e.g., pen and paper) or with a generic computer and is not an inventive concept that meaningfully limits the abstract idea. Therefore, the limitation is a mental process.
Claim 6 is dependent on claim 5 and includes all the limitations of claim 5. Therefore, claim 6 recites the same abstract idea of model training system. The claim recites the limitations of wherein the adjacency matrix is modified in different training iterations, which can be done mentally with or without the use of a physical aid (e.g., pen and paper) or with a generic computer and is not an inventive concept that meaningfully limits the abstract idea. Therefore, the limitation is a mental process.
Claim 7 is dependent on claim 6 and includes all the limitations of claim 6. Therefore, claim 7 recites the same abstract idea of model training system. The claim recites the limitations of wherein the adjacency matrix is determined for the training iteration by an exponential communication protocol, which can be done mentally with or without the use of a physical aid (e.g., pen and paper) or with a generic computer and is not an inventive concept that meaningfully limits the abstract idea. Therefore, the limitation is a mental process.
Claim 8 is dependent on claim 1 and includes all the limitations of claim 1. Therefore, claim 8 recites the same abstract idea of model training system. The claim recites the limitations of wherein the proxy model is trained with a differentially private algorithm, which can be done mentally with or without the use of a physical aid (e.g., pen and paper) or with a generic computer and is not an inventive concept that meaningfully limits the abstract idea. Therefore, the limitation is a mental process.
Claim 9 is dependent on claim 1 and includes all the limitations of claim 1. Therefore, claim 9 recites the same abstract idea of model training system. The claim recites the limitations of wherein the differentially private algorithm measures a privacy cost of training the proxy model, which can be done mentally with or without the use of a physical aid (e.g., pen and paper) or with a generic computer and is not an inventive concept that meaningfully limits the abstract idea. Therefore, the limitation is a mental process.
Claim 10 is dependent on claim 9 and includes all the limitations of claim 9. Therefore, claim 10 recites the same abstract idea of model training system. The claim recites the limitations of wherein the privacy cost is measured for a plurality of training iterations and the model training ends when a total privacy cost reaches a threshold, which can be done mentally with or without the use of a physical aid (e.g., pen and paper) or with a generic computer and is not an inventive concept that meaningfully limits the abstract idea. Therefore, the limitation is a mental process.
Claim 11 is dependent on claim 1 and includes all the limitations of claim 1. Therefore, claim 11 recites the same abstract idea of model training system. The claim recites the limitations of wherein the proxy model and private model have different model architectures, which can be done mentally with or without the use of a physical aid (e.g., pen and paper) or with a generic computer and is not an inventive concept that meaningfully limits the abstract idea. Therefore, the limitation is a mental process.
As to claims 12-20, they have similar limitations as of claims 1-11 above. Hence, they are rejected under the same rational as of claims 1-11 above.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-20 are rejected under 35 U.S.C. 103 as being unpatentable over Tarumi et al. (Pub. No. : US 20220138603 A1) in the view of Jiang et al. (Federated Learning Algorithm Based on Knowledge Distillation) and Sun et al. (Pub. No. : US 20200272940 A1).
As to claim 1 Tarumi teaches a system for shared model training with private data protection, comprising:
a processor and a computer-readable medium having instructions executable by the processor (paragraph [0104]: The integration device 101 includes the processor 201 that executes a program and the storage device 202 that stores the program) for:
training the private parameters to reduce a private loss based on the set of private predictions evaluated with respect to labels for the training batch and the set of proxy predictions (paragraph [0092]: The private data analysis device PSi creates the machine learning model using the parameter of the machine learning model. The private data analysis device PSi inputs the private data 220 corresponding to the selected feature amount to the machine learning model and outputs a prediction result).
Tarumi does not explicitly disclose but Jiang teaches identifying a set of proxy parameters for a proxy model (fig.1, student network) and a set of private parameters for a private model (fig.1, teacher network);
training the proxy parameters and private parameters for a training iteration by: identifying a training batch from a private training data set (fig.1, input x);
determining a set of proxy predictions from the proxy model applied to the training batch with the set of proxy parameters (fig.1, output total loss);
determining a set of private predictions from the private model applied to the training batch with the set of private parameters (fig.1, output soft target);
training the proxy parameters to reduce a proxy loss based the set of proxy predictions evaluated with respect to labels for the training batch and the set of private predictions (fig.1, the output soft target of the teacher network is input into the student network at the cross entropy stage); and
mixing the proxy parameters with one or more sets of other proxy model parameters trained with different private data (fig.2, FedDistill Algorithm).
It would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to modify Tarumi by adding above limitation as taught by Jiang to improve the whole model learning ability (Jiang, page 166 section V).
Tarumi and Jiang do not explicitly disclose but Sun teaches a first distillation loss based on the set of proxy predictions evaluated with respect to the set of private predictions and a second distillation loss based on the set of private predictions evaluated with respect to the set of proxy predictions (paragraphs [0033]-[0038], [0070]-[0073]: FIGS. 3A-3B, the private data sources 303a-n and teacher modules 230a-n may be hosted outside the model provider 120, e.g., at the data provider(s) 110. To learn the aggregated knowledge, the student module 240 is trained to minimize the difference between its own classification output and the aggregated classification output P.sup.t from the teacher modules, e.g., the knowledge distillation loss. At the knowledge distillation loss module 320, the knowledge distillation loss Lk is calculated according to Equation (3)). It would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to modify Tarumi and Jiang by adding above limitation as taught by Sun to protect data privacy and honor the trust between cooperating parties (Sun, paragraph [0004]).
As to clam 2 Tarumi together with Jiang and Sun teaches the system of claim 1. Jiang teaches wherein mixing the proxy parameters, including replacing the proxy parameters with proxy parameters based on the one or more other proxy model parameters (Page 164, column 1, 4th paragraph).
As to clam 3 Tarumi together with Jiang and Sun teaches the system of claim 1. Jiang teaches wherein mixing the proxy parameters includes sending the proxy parameters and a bias matrix to another system training another proxy model (Page 164, column 1, 4th paragraph).
As to clam 4 Tarumi together with Jiang and Sun teaches the system of claim 1. Jiang teaches wherein mixing the proxy parameters includes receiving a bias matrix for each set of other proxy model parameters and applying the received bias matrix to debias the proxy parameters (page 166, 1st column).
As to clam 5 Tarumi together with Jiang and Sun teaches the system of claim 1. Tarumi teaches wherein mixing the proxy model parameters with the one or more other proxy model parameters is based on an adjacency matrix (paragraph [0076]).
As to clam 6 Tarumi together with Jiang and Sun teaches the system of claim 5. Tarumi teaches wherein the adjacency matrix is modified in different training iterations (paragraph [0079]).
As to clam 7 Tarumi together with Jiang and Sun teaches the system of claim 6. Tarumi teaches wherein the adjacency matrix is determined for the training iteration by an exponential communication protocol (paragraph [0077]).
As to clam 8 Tarumi together with Jiang and Sun teaches the system of claim 1. Tarumi teaches wherein the proxy model is trained with a differentially private algorithm (paragraph [0070]).
As to clam 9 Tarumi together with Jiang and Sun teaches the system of claim 1. Tarumi teaches wherein the differentially private algorithm measures a privacy cost of training the proxy model (paragraph [0125]).
As to clam 10 Tarumi together with Jiang and Sun teaches the system of claim 1. Jiang teaches wherein the privacy cost is measured for a plurality of training iterations and the model training ends when a total privacy cost reaches a threshold (page 164, 1st column 3rd paragraph).
As to clam 11 Tarumi together with Jiang and Sun teaches the system of claim 1. Tarumi teaches wherein the proxy model and private model have different model architectures (paragraphs [0005], [0125]).
As to claims 12-20, they have similar limitations as of claims 1-11 above. Hence, they are rejected under the same rational as of claims 1-11 above.
Examiner's Note: Examiner has cited particular columns and line numbers or paragraphs in the references as applied to the claims above for the convenience of the applicant. Although the specified citations are representative of the teachings of the art and are applied to the specific limitations within the individual claim, other passages and figures may apply as well. It is respectfully requested from the applicant in preparing responses, to fully consider the references in its entirety as potentially teaching of all or part of the claimed invention, as well as the context.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
The prior art made of record, listed on form PTO-892, and not relied upon, if any, is considered pertinent to applicant's disclosure.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MD I UDDIN whose telephone number is (571)270-3559. The examiner can normally be reached M-F, 8:00 am to 5:00 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Sherief Badawi can be reached at 571-272-9782. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MD I UDDIN/Primary Examiner, Art Unit 2169
Prosecution Insights