Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
This action is response to the communication filed on July 2, 2026. Claims 1-16, 18-20 are pending.
Response to Arguments
Applicant's arguments filed on July 2, 2026 have been fully considered but they are not persuasive.
Response to 101 rejection:
Applicant argues the claims recite practical, applied technological process, integrate the judicial exception into a practical application and the additional elements amounting to significantly more.
In response examiner respectfully disagree. In the view of applicant specification paragraph [0086]-[0094], the claimed limitations generate an approximate polynomial, approximating a neural network operation, a mean of the input data, and a standard deviation of the input data are nothing but mathematical algorithm. Therefore, the claim falls within the “mathematical concepts” grouping of abstract ideas. Even though, the claim limitation are tied to one or more processor and neural network, but these are nothing but a generic computer component and well‐understood, routine, and conventional. The courts have recognized these functions as well‐understood, routine, and conventional as they are claimed in a merely generic manner (e.g., at a high level of generality) or as insignificant extra-solution activity (see MPEP 2106.05(d) II, Receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information)). Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. Hence, applicant argument is not persuasive.
Response to Art rejection:
(1) Applicant argues cited arts fail to teach generate an approximate polynomial, approximating a neural network operation comprising a rectified linear unit (ReLU), of a portion of a deep neural network model that is configured to receive input data, by using weighted least squares based on parameters corresponding to the generation of the approximate polynomial, a mean of the input data, and a standard deviation of the input data.
In response examine respectfully disagrees. Claim limitations are broadly interpreted during examiner consistent with the specification. It appears to examiner that applicant believe office cannot lift Lee’s least squares teaching out of its exponential function context and apply it to the claimed ReLU approximation. Applicant characterization is not right. Lee does teache as claimed generate an approximate polynomial, approximating a neural network operation comprising a rectified linear unit (ReLU), of a portion of a deep neural network model that is configured to receive input data, by using weighted least squares based on parameters corresponding to the generation of the approximate polynomial, a mean of the input data, and a standard deviation of the input data (page 11, page 7 section Exponential function, page 7 second column second paragraph from last: Algorithm 6 shows the homomorphic evaluation method for the ReLU function using the composite polynomials generated by Algorithm 5 as the input. After homomorphically evaluating the p;'s in order, we homomorphically evaluate x(l + sign(x))/2.This composition of polynomials ensures that the average approximation precision is approximately 16-bit precision and approximate the exponential function in [ -1, 1] using the least-squares method, and we find that the approximate polynomial, wherein the key-switching operation includes the decomposing, multi-sum, and mod-down operations. While the mod-down operation is linear with the level of the input ciphertext).
(2) The argument regarding dependent claims are not persuasive as those arguments are not specific. Applicant just stated Neither Lee nor Zhang disclose dependent claims limitation but did not provide any explanation.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. Regarding the claim 1, it recites one or more processors configured to execute instructions; and one or more memories storing the instructions; wherein the execution of the instructions by the one or more processors configures the one or more processors to: generate an approximate polynomial, approximating a neural network operation comprising a rectified linear unit (ReLU), of a portion of a deep neural network model that is configured to receive input data, by using weighted least squares based on parameters corresponding to the generation of the approximate polynomial, a mean of the input data, and a standard deviation of the input data; and generate a homomorphic encrypted data operation result based on the input data and the approximate polynomial that approximates the neural network operation.
The claimed limitations as drafted, is a process that, under its broadest reasonable interpretation in view of specification, covers performance of the limitation in the mathematical concepts grouping. In the view of applicant specification paragraph [0086]-[0094], the claimed limitations generate an approximate polynomial, approximating a neural network operation, a mean of the input data, and a standard deviation of the input data are nothing but mathematical algorithm. Therefore, the claim falls within the “mathematical concepts” grouping of abstract ideas. Accordingly, the claim recites an abstract idea.
This judicial exception is not integrated into a practical application. In particular, the claim only recites two elements which do not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. The claim is directed to an abstract idea.
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. With respect to integration of the abstract idea into a practical application, the claim steps amounts to no more than mere instructions to apply the exception using a generic computer component as recited in the claim processor and memory. The courts have recognized these functions as well‐understood, routine, and conventional as they are claimed in a merely generic manner (e.g., at a high level of generality) or as insignificant extra-solution activity (see MPEP 2106.05(d) II, Receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information)). Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. The claim is not patent eligible.
Alternatively, the claims also fall under the grouping of mental process. The claim recited the limitation of “generate an approximate polynomial, approximating a neural network operation comprising a rectified linear unit (ReLU), of a portion of a deep neural network model that is configured to receive input data, by using weighted least squares based on parameters corresponding to the generation of the approximate polynomial, a mean of the input data, and a standard deviation of the input data” as drafted, is a process that, under its broadest reasonable interpretation, covers performance of the limitation in the mind. User can mentally generate encrypted data from received (input) data for the neural network which is a mental process.
The limitation “generate an approximate polynomial, approximating a neural network operation, of a portion of a deep neural network model that is configured to receive input data, by using weighted least squares based on parameters corresponding to the generation of the approximate polynomial, a mean of the input data, and a standard deviation of the input data” can be interpreted as additional limitation. The receiving input data as recited amounts to mere data gathering for generate an approximate polynomial, approximating a neural network operation, a mean of the input data, and a standard deviation, which is a form of insignificant extra-solution activity, (see Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information)). Accordingly, even in combination, these additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea. The claim is directed to the abstract idea.
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional element of generate an approximate polynomial, approximating a neural network operation, of a portion of a deep neural network model that is configured to receive input data steps amounts to no more than mere instructions to apply the exception using a generic computer component such as processor and memory as recited in the claim. The courts have recognized these functions as well‐understood, routine, and conventional as they are claimed in a merely generic manner (e.g., at a high level of generality) or as insignificant extra-solution activity (see MPEP 2106.05(d) II, Receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information)). Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. The claim is not patent eligible.
Claim 2 is dependent on claim 1 and includes all the limitations of claim 1. Therefore, claim 2 recites the same abstract idea of data encryption of neural network. The claim recites the limitations of wherein the execution of the instructions by the one or more processors configures the one or more processors to: implement the deep neural network model, including a generation of the input data by implementing another portion of the deep neural network model, the generation of the approximate polynomial, and the generation of the homomorphic encrypted data operation result, which can be done mentally with or without the use of a physical aid (e.g., pen and paper) or with a generic computer in the form of insignificant extra-solution activity which is not an inventive concept that meaningfully limits the abstract idea. Therefore, the limitation is a mental process.
Claim 3 is dependent on claim 2 and includes all the limitations of claim 2. Therefore, claim 3 recites the same abstract idea of data encryption of neural network. The claim recites the limitations of wherein the execution of the instructions by the one or more processors configures the one or more processors to: perform the generation of an approximate polynomial and the generation of respective homomorphic encrypted data operation results for plural portions of the deep neural network model that have respective neural network operations that are each configured to receive corresponding input data respectively generated by plural other portions of the deep neural network model; and generate a result of the deep neural network model dependent on the corresponding input data respectively generated by the plural other portions of the deep neural network model and the respective homomorphic encrypted data operation results, which can be done mentally with or without the use of a physical aid (e.g., pen and paper) or with a generic computer in the form of insignificant extra-solution activity which is not an inventive concept that meaningfully limits the abstract idea. Therefore, the limitation is a mental process.
Claim 4 is dependent on claim 1 and includes all the limitations of claim 1. Therefore, claim 4 recites the same abstract idea of data encryption of neural network. The claim recites the limitations of wherein the parameters comprise a correction constant for correcting a degree of the approximate polynomial and the standard deviation, wherein the weighted least squares is based on a corrected standard deviation based on the correction constant, and wherein the generation of the homomorphic encrypted data operation result is based on the approximate polynomial with a corrected degree based on the correction constant, which can be done mentally with or without the use of a physical aid (e.g., pen and paper) or with a generic computer in the form of insignificant extra-solution activity which is not an inventive concept that meaningfully limits the abstract idea. Therefore, the limitation is a mental process.
Claim 5 is dependent on claim 4 and includes all the limitations of claim 4. Therefore, claim 5 recites the same abstract idea of data encryption of neural network. The claim recites the limitations of wherein, for the generation of the approximate polynomial, the one or more processors are configured to: calculate the standard deviation; and generate the corrected standard deviation by multiplying the standard deviation by the correction constant, which can be done mentally with or without the use of a physical aid (e.g., pen and paper) or with a generic computer in the form of insignificant extra-solution activity which is not an inventive concept that meaningfully limits the abstract idea. Therefore, the limitation is a mental process.
Claim 6 is dependent on claim 4 and includes all the limitations of claim 4. Therefore, claim 6 recites the same abstract idea of data encryption of neural network. The claim recites the limitations of wherein, for the generation of the approximate polynomial, the one or more processors are configured to set a probability density function of the input data based on the mean, the standard deviation, and the correction constant, and wherein the weighted least squares is based on the probability density function, which can be done mentally with or without the use of a physical aid (e.g., pen and paper) or with a generic computer in the form of insignificant extra-solution activity which is not an inventive concept that meaningfully limits the abstract idea. Therefore, the limitation is a mental process.
Claim 7 is dependent on claim 6 and includes all the limitations of claim 6. Therefore, claim 7 recites the same abstract idea of data encryption of neural network. The claim recites the limitations of wherein, for the generation of the approximate polynomial, the one or more processors are configured to: calculate a mean square error based on the probability density function; and generate the approximate polynomial that minimizes the mean square error that is based on the degree of the approximate polynomial and the probability density function, which can be done mentally with or without the use of a physical aid (e.g., pen and paper) or with a generic computer in the form of insignificant extra-solution activity which is not an inventive concept that meaningfully limits the abstract idea. Therefore, the limitation is a mental process.
Claim 8 is dependent on claim 7 and includes all the limitations of claim 7. Therefore, claim 8 recites the same abstract idea of data encryption of neural network. The claim recites the limitations of wherein, for the generation of the approximate polynomial, the one or more processors are configured to: calculate the mean square error based on a product of the probability density function and a square of a difference between the ReLU and the updated approximate polynomial, which can be done mentally with or without the use of a physical aid (e.g., pen and paper) or with a generic computer in the form of insignificant extra-solution activity which is not an inventive concept that meaningfully limits the abstract idea. Therefore, the limitation is a mental process.
Claim 9 is dependent on claim 1 and includes all the limitations of claim 1. Therefore, claim 9 recites the same abstract idea of data encryption of neural network. The claim recites the limitations of wherein the one or more processors are configured to calculate the mean and the standard deviation based on the input data, which can be done mentally with or without the use of a physical aid (e.g., pen and paper) or with a generic computer in the form of insignificant extra-solution activity which is not an inventive concept that meaningfully limits the abstract idea. Therefore, the limitation is a mental process.
Claim 10 is dependent on claim 1 and includes all the limitations of claim 1. Therefore, claim 10 recites the same abstract idea of data encryption of neural network. The claim recites the limitations of wherein, for the generation of the approximate polynomial, the one or more processors are configured to: calculate a first coefficient and a second coefficient based on a degree of the approximate polynomial, the mean, and the standard deviation; and generate the approximate polynomial based on a product of the first coefficient and the second coefficient, which can be done mentally with or without the use of a physical aid (e.g., pen and paper) or with a generic computer in the form of insignificant extra-solution activity which is not an inventive concept that meaningfully limits the abstract idea. Therefore, the limitation is a mental process.
Claim 11 is dependent on claim 10 and includes all the limitations of claim 10. Therefore, claim 11 recites the same abstract idea of data encryption of neural network. The claim recites the limitations of wherein, for the generation of the approximate polynomial, the one or more processors are configured to: calculate the first coefficient and the second coefficient based on a value obtained by dividing the mean by the standard deviation, which can be done mentally with or without the use of a physical aid (e.g., pen and paper) or with a generic computer in the form of insignificant extra-solution activity which is not an inventive concept that meaningfully limits the abstract idea. Therefore, the limitation is a mental process.
As to claims 12-20, they have similar limitations as of claims 1-11 above. Hence, they are rejected under the same rational as of claims 1-11 above.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-20 are rejected under 35 U.S.C. 103 as being unpatentable over Lee et al. (Privacy-Preserving Machine Learning with Fully Homomorphic Encryption for Deep Neural Network) in the view of Zhang et al. (Patent No. : US 11630666 B2).
As to claim 1 Lee teaches an apparatus, comprising:
one or more processors configured to execute instructions and one or more memories storing the instructions (see abstract and introduction: the machine learning model and Deep Neural Network synonymous with processor and memory to execute instruction), wherein the execution of the instructions by the one or more processors configures the one or more processors to:
generate an approximate polynomial, approximating a neural network operation comprising a rectified linear unit ReLU), of a portion of a deep neural network model that is configured to receive input data, by using weighted least squares based on parameters corresponding to the generation of the approximate polynomial (page 11, page 7 section Exponential function, page 7 second column second paragraph from last: Algorithm 6 shows the homomorphic evaluation method for the ReLU function using the composite polynomials generated by Algorithm 5 as the input. After homomorphically evaluating the p;'s in order, we homomorphically evaluate x(l + sign(x))/2.This composition of polynomials ensures that the average approximation precision is approximately 16-bit precision and approximate the exponential function in [ -1, 1] using the least-squares method, and we find that the approximate polynomial, wherein the key-switching operation includes the decomposing, multi-sum, and mod-down operations. While the mod-down operation is linear with the level of the input ciphertext); and
generate a homomorphic encrypted data operation result based on the input data and the approximate polynomial that approximates the neural network operation (page 7 section Exponential function: Then, when we homomorphically evaluate the exponential function in [-B, B], we divide the input by B, evaluate the approximate polynomial for the exponential function).
Lee does not explicitly disclose but Zhang teaches a mean of the input data, and a standard deviation of the input data (column 29 lines 25-28: Statistical parameters of the absolute value of first input data are processed, such as a mean value a.sub.mean of the absolute value and a standard deviation a.sub.std of the absolute value). It would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to improve the processing speed and efficiency of training operations (Zhang, abstract).
As to claim 2 Lee together with Zhang teaches an apparatus according to claim 1. Lee teaches wherein the execution of the instructions by the one or more processors configures the one or more processors to: implement the deep neural network model, including a generation of the input data by implementing another portion of the deep neural network model, the generation of the approximate polynomial, and the generation of the homomorphic encrypted data operation result (page 2, section A).
As to claim 3 Lee together with Zhang teaches an apparatus according to claim 2. Lee teaches wherein the execution of the instructions by the one or more processors configures the one or more processors to: perform the generation of an approximate polynomial and the generation of respective homomorphic encrypted data operation results for plural portions of the deep neural network model that have respective neural network operations that are each configured to receive corresponding input data respectively generated by plural other portions of the deep neural network model; and generate a result of the deep neural network model dependent on the corresponding input data respectively generated by the plural other portions of the deep neural network model and the respective homomorphic encrypted data operation results (page 7 Section: Exponential function).
As to claim 4 Lee together with Zhang teaches an apparatus according to claim 1. Lee teaches wherein the parameters comprise a correction constant for correcting a degree of the approximate polynomial and the standard deviation, wherein the weighted least squares is based on a corrected standard deviation based on the correction constant, and wherein the generation of the homomorphic encrypted data operation result is based on the approximate polynomial with a corrected degree based on the correction constant (page 7 Section: Exponential function).
As to claim 5 Lee together with Zhang teaches an apparatus according to claim 4. Zhang teaches wherein, for the generation of the approximate polynomial, the one or more processors are configured to: calculate the standard deviation; and generate the corrected standard deviation by multiplying the standard deviation by the correction constant (column 29 lines 25-45).
As to claim 6 Lee together with Zhang teaches an apparatus according to claim 4. Lee teaches wherein, for the generation of the approximate polynomial, the one or more processors are configured to set a probability density function of the input data based on the mean, the standard deviation, and the correction constant, and wherein the weighted least squares is based on the probability density function (page 11 second column 2nd paragraph to last paragraph).
As to claim 7 Lee together with Zhang teaches an apparatus according to claim 6. Lee teaches wherein, for the generation of the approximate polynomial, the one or more processors are configured to: calculate a mean square error based on the probability density function; and generate the approximate polynomial that minimizes the mean square error that is based on the degree of the approximate polynomial and the probability density function (paragraph 11).
As to claim 8 Lee together with Zhang teaches an apparatus according to claim 7. Lee teaches wherein, for the generation of the approximate polynomial, the one or more processors are configured to: calculate the mean square error based on a product of the probability density function and a square of a difference between the ReLU and the updated approximate polynomial (paragraph 11 first column).
As to claim 9 Lee together with Zhang teaches an apparatus according to claim 1. Zhang teaches wherein the one or more processors are configured to calculate the mean and the standard deviation based on the input data (column 29 lines 25-45).
As to claim 10 Lee together with Zhang teaches an apparatus according to claim 1. Lee teaches wherein, for the generation of the approximate polynomial, the one or more processors are configured to: calculate a first coefficient and a second coefficient based on a degree of the approximate polynomial, the mean, and the standard deviation; and generate the approximate polynomial based on a product of the first coefficient and the second coefficient (page 11 second column last paragraph).
As to claim 11 Lee together with Zhang teaches an apparatus according to claim 10. Zhang teaches wherein, for the generation of the approximate polynomial, the one or more processors are configured to: calculate the first coefficient and the second coefficient based on a value obtained by dividing the mean by the standard deviation (column 48 lines 31-36).
As to claims 12-20, they have similar limitations as of claims 1-11 above. Hence, they are rejected under the same rational as of claims 1-11 above.
Examiner's Note: Examiner has cited particular columns and line numbers or paragraphs in the references as applied to the claims above for the convenience of the applicant. Although the specified citations are representative of the teachings of the art and are applied to the specific limitations within the individual claim, other passages and figures may apply as well. It is respectfully requested from the applicant in preparing responses, to fully consider the references in its entirety as potentially teaching of all or part of the claimed invention, as well as the context.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
The prior art made of record, listed on form PTO-892, and not relied upon, if any, is considered pertinent to applicant's disclosure.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MD I UDDIN whose telephone number is (571)270-3559. The examiner can normally be reached M-F, 8:00 am to 5:00 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Sherief Badawi can be reached at 571-272-9782. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MD I UDDIN/Primary Examiner, Art Unit 2169
Prosecution Insights