DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The filing of a terminal disclaimer by itself is not a complete reply to a nonstatutory double patenting (NSDP) rejection. A complete reply requires that the terminal disclaimer be accompanied by a reply requesting reconsideration of the prior Office action. Even where the NSDP rejection is provisional the reply must be complete. See MPEP § 804, subsection I.B.1. For a reply to a non-final Office action, see 37 CFR 1.111(a). For a reply to final Office action, see 37 CFR 1.113(c). A request for reconsideration while not provided for in 37 CFR 1.113(c) may be filed after final for consideration. See MPEP §§ 706.07(e) and 714.13.
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer.
Claims 1, 13 and 18 are provisionally rejected on the ground of nonstatutory double patenting as being unpatentable over claim 4 of copending Application No. 17715863 (reference application) in view of Yang et al., U.S. Patent No. 12153688 B2 (hereinafter Yang).
Instant Application
Copending Application 17715863
1. A method, comprising: receiving, in a computing device, a data sample as an input to an artificial neural network;
generating, by the computing device via splitting the data sample and offsetting, a plurality of first parts to represent the data sample;
shuffling, by the computing device, the first parts generated from the data sample with
second parts to mix parts as inputs to the artificial neural network;
communicating, by the computing device and to one or more entities, tasks of computing, wherein each respective task among the tasks is configured to apply a same computation of the artificial neural network to a respective part configured as one of the inputs to the artificial neural network;
receiving, by the computing device from the one or more entities, results of applying the same computation of the artificial neural network in the tasks respectively; and
generating, by the computing device based on the results received from the one or more entities, a result of applying the same computation of the artificial neural network to the data sample.
1. A method, comprising: receiving, in a computing device, a tensor having elements specifying a computation of an artificial neural network, the tensor having a first dimension and a second dimension;
1. generating, by the computing device, a plurality of computing tasks via partitioning, along the first dimension and the second dimension, the tensor into a plurality of portions respectively, each of the computing tasks configured to operate a respective portion among the portions;
4. The method of claim 3, wherein at least one of the portions is generated via offsetting, bit-wise shifting, adding a constant, multiplying by a constant, or homomorphic encryption, or any combination thereof.
1. shuffling, by the computing device, the computing tasks in distribution of the computing tasks to external entities;
1. communicating, from the computing device, the portions to the external entities according to the computing tasks being shuffled and assigned to the external entities;
1. receiving, by the computing device from the external entities, results of the computing tasks; and
1. generating, by the computing device using the results from the external entities, a result of the computation of the artificial neural network.
13. A computing device, comprising: memory; and at least one microprocessor coupled to the memory and configured via instructions to:
generate a plurality of unmodified parts from a data sample as an input to an artificial neural network, wherein a sum of the unmodified parts is equal to the data sample;
apply an offset operation to at least one of the plurality of unmodified parts to generate a plurality of first parts to represent the data sample, wherein a sum of the first parts is not equal to the data sample;
shuffle the first parts generated from the data sample with second parts to mix parts as inputs to the artificial neural network; and
communicate, to one or more entities, tasks of computing, wherein each respective task among the tasks is configured to apply a same computation of the artificial neural network to a respective part configured as one of the inputs to the artificial neural network.
12. A computing device, comprising: memory; and at least one microprocessor coupled to the memory and configured via instructions to: receive a matrix of elements specifying a computation of an artificial neural network;
generate a plurality of computing tasks via partitioning the matrix along a first dimension of the matrix, each of the computing tasks configured to operate based on a portion of the matrix;
14. sum, based on the first results from the external entities, computing results corresponding portions of the matrix partitioned according to the second dimension; and aggregate, based on the first results from the external entities, computing results corresponding portions of the matrix partitioned according to the first dimension.
16. wherein at least one of the portions is generated via offsetting, bit-wise shifting, adding a constant, multiplying by a constant, or homomorphic encryption, or any combination thereof.
shuffle the computing tasks in distribution of the computing tasks to external entities;
communicate, to the external entities, portions of the matrix shuffled according to the computing tasks being assigned to the external entities; receive, from the external entities, first results of the computing tasks; and generate, by the computing device using the first results from the external entities, a second result of the computation of the artificial neural network.
18. A non-transitory computer storage medium storing instructions which, when executed in a computing device, cause the computing device to perform a method, comprising:
receiving, from one or more entities, first results of applying a same computation of an artificial neural network in respective tasks, wherein each respective task among the tasks is configured to apply the same computation of the artificial neural network to a respective part configured as one of the inputs to the artificial neural network;
identifying, among the first results received from the one or more entities, a subset of the first results, wherein second results in the subset correspond to applying the same computation of the artificial neural network to a plurality of first parts associated with a data sample;
generate, based on the second results in the subset, a third result of applying the same computation of the artificial neural network to the data sample.
17. A non-transitory computer storage medium storing instructions which, when executed in a computing device, cause the computing device to perform a method, comprising:
17. receiving, in the computing device, a matrix of elements specifying a computation of an artificial neural network;
20. wherein the computing tasks are generated via splitting a portion of the matrix into a sum of multiple randomized portions operated upon respectively in multiple tasks among the computing tasks.
17. communicating, from the computing device, the portions to the external entities according to the computing tasks being shuffled and assigned to the external entities; receiving, by the computing device from the external entities, results of the computing tasks
17. generating, by the computing device using the results from the external entities, a result of the computation of the artificial neural network.
However, US Application No ‘863 does not explicitly disclose data sample in general the first parts generated from the data sample with second parts to mix parts as inputs to the artificial neural network. In an analogous art, Yang discloses data sample (Col. 5, lines 47-67, e.g., real data), and shuffling (Col. 5, lines 47-67, e.g., the processing device may generate random numbers R1 and R2, use them to obfuscate message M and key K. and store the results), by the computing device, the first parts generated from the data sample with second parts to mix parts as inputs to the artificial neural network (Col. 5, lines 35-46, e.g., from where the outputs may then be read as inputs into subsequent operations of S-box).
Therefore, it would have been obvious to a person having ordinary skill in the art at the time of the invention to incorporate the aforementioned elements to shuffle the data prior to using them as input for the artificial neural network in order to further defend against side-channel attacks, as suggested by Yang (Col. 1, lines 20-40).
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-12 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
The phrase “generating… a result” in claim 1 renders the claim indefinite. The claim does not provide clear boundaries of what “generating” implies. The claim contains two reasonable interpretations that make the claim indefinite, the first the claim limitation generating “based on the results received” could be interpreted that a different generated results is obtained as a “result”. However, the claim sates “generating, … based on the results received…” “a result”, which makes unclear if the results are the same results received or a different result. The later part of the claim makes it more ambiguous since it states “applying the same computation”. As the statutory language of ‘particular[ity]' and 'distinct[ness]' indicates, claims are required to be cast in clear—as opposed to ambiguous, vague, indefinite—terms. It is the claims that notify the public of what is within the protections of the patent, and what is not. (See MPEP 2173). It is unclear whether the limitation is using the term “generating” to mean to performing the computation again or merely receiving the results. The claim is indefinite because the term is not scope of the limitation is not clearly defined.
Claims 2-12 are rejected as having the same deficiencies as the claims they depend from.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status. The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-20 are rejected under 35 U.S.C. 103 as being unpatentable over Shazeer, U.S. Patent Application No. 20200042875 A1, in view of Yang et al., U.S. Patent No. 12153688 B2 (hereinafter Yang).
Regarding claim 1, Shazeer discloses a method, comprising: receiving, in a computing device, a data sample as an input to an artificial neural network ([0028], e.g., receive and assign tensor data associated with the input data);
generating, by the computing device via splitting the data sample ([0073], e.g., data is split and distributed and operated on in parallel) and offsetting ([0038], e.g., Each of the computing devices can be configured to execute an operation, e.g., matrix multiplication, by multiplying all or part of an input tensor with all or part of a weight tensor), a plurality of first parts to represent the data sample ([0102], e.g., generates as output one or more respective output tensors each having one or more respective output dimensions);
shuffling, by the computing device, as inputs to the artificial neural network ([0102], e.g., specification data specifies a respective layout for each input and output tensor that assigns each dimension of the input or output tensor to one or more of the plurality of computing devices; assigning, based on the layouts for the input and output tensors, respective device-local operations to each of the plurality of computing devices);
communicating, by the computing device and to one or more entities, tasks of computing, wherein each respective task among the tasks is configured to apply a same computation of the artificial neural network to a respective part configured as one of the inputs to the artificial neural network ([0102], e.g., wherein the specification data specifies a respective layout for each input and output tensor that assigns each dimension of the input or output tensor to one or more of the plurality of computing devices; assigning, based on the layouts for the input and output tensors, respective device-local operations to each of the plurality of computing devices; and causing the tensor computations to be executed by the plurality of computing devices by causing each of the plurality of computing devices to execute at least the respective-device local operations assigned to the computing devices);
receiving, by the computing device from the one or more entities, results of applying the same computation of the artificial neural network in the tasks respectively ([0051], e.g., The corresponding computing results received from the external entities); and
generating, by the computing device based on the results received from the one or more entities, a result of applying the same computation of the artificial neural network to the data sample ([0086], e.g., to generate a combined output that can be an output for the tensor computations or input for another computation).
Shazeer does not explicitly disclose shuffling as exactly recited, wherein “the first parts generated from the data sample with second parts to mix parts”.
However, Yang discloses shuffling (Col. 5, lines 47-67, e.g., the processing device may generate random numbers R1 and R2, use them to obfuscate message M and key K. and store the results), by the computing device, the first parts generated from the data sample with second parts to mix parts as inputs to the artificial neural network (Col. 5, lines 35-46, e.g., from where the outputs may then be read as inputs into subsequent operations of S-box).
Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the method of Shazeer to include the shuffling of Yang that the first parts generated from the data sample with second parts to mix parts as inputs to the artificial neural network to further defend against side-channel attacks, as suggested by Yang (Col. 1, lines 20-40).
Regarding claim 2, Shazeer and Yang discloses the method of claim 1. Yang further discloses generating, by the computing device, an offset key for the data sample; and applying, by the computing device according to the offset key, an offset operation to generate a modified part among the first parts representative of the data sample (Col. 3, lines 14-52, e.g., a round key may be combined (e.g., using a bitwise XOR addition) with the plaintext message).
Regarding claim 3, Shazeer and Yang discloses the method of claim 2. Shazeer and Yang further discloses wherein the generating of the result comprises: identifying, among the results received from the one or more entities, a subset of the results that are generated from applying to the same computation of the artificial neural network to the first parts (Shazeer, [0030], e.g., the layout engine 120 can generate and send layouts to the mesh execution engine 140 that in turn distributes respective tensors and operations to implement model parallelism); and performing, according to the offset key, an offset operation (Col. 3, lines 14-52, e.g., a round key may be combined (e.g., using a bitwise XOR addition) with the plaintext message) to a result of applying the same computation of the artificial neural network to the modified part to generate a corresponding result of applying the same computation of the artificial neural network to the unmodified part, wherein the corresponding result is summed with further results obtained based on the subset (Shazeer, [0086], e.g., combining individual outputs generated by executing device-local operations on the computing devices, to generate a combined output that can be an output for the tensor computations or input for another computation).
Regarding claim 4, Shazeer and Yang discloses the method of claim 2. Shazeer and Yang further discloses wherein first parts are generated by: generating a plurality of third parts that has a sum equal to the data sample (Shazeer, [0055], e.g., after the mesh execution engine 140 assigns sub-tensors of an input tensor to different computing devices across the mesh 110, the mesh execution engine 140 can execute communication operation primitives to combine output sub-tensors corresponding to assigned input sub-tensors, to form the output data 180 representing an output for the input data), wherein an unmodified part among the third parts is applied with the offset operation to generate the modified part (Yang, Col. 4, lines 8-31, e.g., Likewise, a key (initial key or a round key) may be split into its own respective shares, K=R2⊕(K⊕R2) using a different (e.g., random) number R2).
Regarding claim 5, Shazeer and Yang discloses the method of claim 4. Yang further discloses wherein the offset operation includes bit-wise shifting (Col. 3, lines 14-52, e.g., Following the S-Box, a number of other (linear) transformations may also be performed on the resulting 4×4 matrix, such as shifting cyclically various rows of the mapped matrix a certain number of row-specific steps), according to the offset key, each number in the unmodified part to generate a corresponding number in the modified part (Col. 3, lines 14-52, e.g., After repeating such operations a target number of times, a ciphertext is ultimately produced. Similar operations may then be performed to recover the plaintext message from the ciphertext).
Regarding claim 6, Shazeer and Yang discloses the method of claim 4. Yang further discloses wherein the offset operation includes adding a constant (Col. 5, lines 47-67, Copies of numbers R1 and R2 may be added, by bitwise XOR gates 126 and 128, to M and K, respectively, to obfuscate message M and key K), according to the offset key, each number in the unmodified part to generate a corresponding number in the modified part (Col. 3, lines 14-52, e.g., After repeating such operations a target number of times, a ciphertext is ultimately produced. Similar operations may then be performed to recover the plaintext message from the ciphertext).
Regarding claim 7, Shazeer and Yang discloses the method of claim 4. Yang further discloses wherein the offset operation includes multiplying by a constant (Col. 8, lines 58-67, e.g., Stage Two 320 may include GF(2.sup.4) multiplication and GF(2.sup.4) affine Square Scaling), according to the offset key, each number in the unmodified part to generate a corresponding number in the modified part (Col. 3, lines 14-52, e.g., After repeating such operations a target number of times, a ciphertext is ultimately produced. Similar operations may then be performed to recover the plaintext message from the ciphertext).
Regarding claim 8, Shazeer and Yang discloses the method of claim 4. Yang further discloses wherein the third parts are generated by: generating random numbers as data elements (Col.4, lines 8-31, e.g., any appropriate number R1 (which can be a randomly generated number)) in a part among the third parts, the random numbers generated according to the offset key to have a number of leading bits or tailing bits to be zero (Col. 3, lines 14-52, e.g., Following the S-Box, a number of other (linear) transformations may also be performed on the resulting 4×4 matrix, such as shifting cyclically various rows of the mapped matrix a certain number of row-specific steps).
Regarding claim 9, Shazeer and Yang discloses the method of claim 4. Yang further discloses wherein numbers in the modified part have a same number of bits as numbers in the unmodified part (Col. 8, lines 7-41, e.g., stages 310-360 may be processing concurrently a certain number of bytes (e.g., one, two, etc.) of share 1 and the same number of bytes of share 2 (e.g., bytes of share 1 that are in the same positions as bytes of share 2)).
Regarding claim 10, Shazeer and Yang discloses the method of claim 4. Shazeer further discloses wherein a first precision requirement to apply the same computation of the artificial neural network to the modified part is same as a second precision requirement to apply the same computation of the artificial neural network to the unmodified part ([0029], e.g., the layout engine 120 can generate and send layouts to the mesh execution engine 140 that distribute dimensions of respective tensors and operations to different computing devices, to implement data parallelism. Data parallelism refers to a class of techniques in which input data is divided and distributed across multiple computing devices, but each computing device performs the same operations on different data).
Regarding claim 11, Shazeer and Yang discloses the method of claim 10. Shazeer further discloses wherein a third precision requirement to apply the same computation of the artificial neural network ([0029], e.g., the layout engine 120 can generate and send layouts to the mesh execution engine 140 that distribute dimensions of respective tensors and operations to different computing devices, to implement data parallelism. Data parallelism refers to a class of techniques in which input data is divided and distributed across multiple computing devices, but each computing device performs the same operations on different data) to the data sample is same as the second precision requirement to apply the same computation of the artificial neural network to the unmodified part ([0055], e.g., after the mesh execution engine 140 assigns sub-tensors of an input tensor to different computing devices across the mesh 110, the mesh execution engine 140 can execute communication operation primitives to combine output sub-tensors corresponding to assigned input sub-tensors, to form the output data 180 representing an output for the input data).
Regarding claim 12, Shazeer and Yang discloses the method of claim 4. Yang further discloses wherein the same computation (Col. 3, line 54-67 through Col. 4, lines 1-7, e.g., introduce inconsequential arithmetic operations (e.g., addition, multiplication, reduction or division, and so on), or otherwise obfuscate actual computations carried out by the computing system) of the artificial neural network is configured to be performed via multiply-accumulate units (Col. 14, lines 45-67, e.g., in which weight and/or other parameter information is to be loaded to configure, logic, including integer and/or floating point units (collectively, arithmetic logic units (ALUs) or simply circuits)).
Computing device and non-transitory computer storage medium claims (13-17 and 18-20) are drawn to the device/medium corresponding to the method of using same as claimed in claims (1-4, 6 and 1, 2, 6). Therefore device/medium claims (13-17 and 18-20) correspond to method claims (1-4, 6 and 1, 2, 6), and are rejected for the same reasons of obviousness as used above (see Shazeer [0087]).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Zixuan Weng whose telephone number is (571)272-7069. The examiner can normally be reached Monday thru Friday 8:30-5:30pm, ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jorge L. Ortiz-Criado can be reached at (571) 272-7624. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ZIXUAN WENG/Examiner, Art Unit 2496
/JORGE L ORTIZ CRIADO/Supervisory Patent Examiner, Art Unit 2496
