Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
In view of the Appeal Brief filed on August 26, 2026, and responsive to an Appeal Conference, PROSECUTION IS HEREBY REOPENED. A new ground of rejection is set forth below.
To avoid abandonment of the application, appellant must exercise one of the following two options:
(1) file a reply under 37 CFR 1.111 (if this Office action is non-final) or a reply under 37 CFR 1.113 (if this Office action is final); or,
(2) initiate a new appeal by filing a notice of appeal under 37 CFR 41.31. A new notice of appeal fee and appeal brief fee will not be required for applicant to appeal from the new Office action. Any appeal brief filed on or after September 13, 2004 must comply with 37 CFR 41.37.
A Supervisory Patent Examiner (SPE) has approved of reopening prosecution by signing below.
/NICHOLAS R TAYLOR/Supervisory Patent Examiner, Art Unit 2443
3. Claims 21-40 have been examined and are pending with this action.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
4 Claim 21-40 are rejected under 35 U.S.C. 103 as being unpatentable over Bradshaw et al. (US 2021/0056405 A1) in view of Bell et al. (US 2018/0177461 A1).
INDEPENDENT:
As per claim 21, Bradshaw teaches a method comprising:
processing input data to obtain output data using a neural network (NN) comprising an NN portion (see Abstract, “A second computing device in the system includes memory that stores a local version of the ANN and user data for inputting into the local version… The first computing device includes a memory that stores the master version of the ANN, a transceiver that receives obfuscated user data transmitted from the second computing device, and a processor that trains the master version based on the received obfuscated user data using machine learning.”, NOTE: Bradshaw explicitly teaches ANN – artificial neural network that is distributed on two computing devices, hence two distinct NN portions.)
(i) receiving, from at least one upstream portion of the NN, one or more input values comprising one or more input values (see Bradshaw, Abstract: “A second computing device in the system includes memory that stores a local version of the ANN and user data for inputting into the local version… The first computing device includes a memory that stores the master version of the ANN, a transceiver that receives obfuscated user data transmitted from the second computing device, and a processor that trains the master version based on the received obfuscated user data using machine learning.”, NOTE: Bradshaw again discloses NN portion, downstream NN and upstream NN, first neural node and second neural node), and
(ii) applying neural operations to the input values to obtain one or more output values (see Bradshaw, [0003]: “Usually, an ANN is implemented by a signal at a connection (or edge) between artificial neurons being a real number, and the output of each artificial neuron being computed by a non-linear function of the sum of its inputs.”; [0009]: “Also, at least some aspects of the present disclosure are directed to a computer network that can be configured to implement obfuscating inputs for centralized training of a master version of an artificial neural network (ANN)”; [0015]: “there are many other ways of obfuscation that can be used, such as substitution, shuffling, numerical variance methods, scrambling, masking out characters, some other types of encryption not using hashing, and deletion of some values in particular fields.”; and [0038]: “The obfuscating of the extracted plurality of features can include combining different sets of inputs in the extracted plurality of features using one or more arithmetic operations to combine the different sets (e.g., see obfuscations 124a and 124b as well as step 312 illustrated in FIG. 3). The extraction of features (e.g., see extracted features 122a and 122b) can include selecting the different sets of inputs randomly for the combining of the different sets. Or, the extraction of features can include selecting the different sets of inputs deliberately for the combining of the different sets.”),
wherein the output values are used as an input into at least one downstream portion of the NN (see Bradshaw, [0011]: “he second computing device in the system can include memory that is configured to store a local version of the ANN and user data for inputting into the local version of the ANN. The second computing device can also include a processor that is configured to extract features from the user data and obfuscate the extracted features to generate obfuscated user data. The second device also can include a transceiver that is configured to transmit the obfuscated user data such as to the first computing device. The first computing device can include a memory that is configured to store the master version of the ANN, a transceiver that is configured to receive obfuscated user data transmitted from the second computing device or another device of the system hosting a local version of the ANN, and a processor that is configured to train the master version based on the received obfuscated user data using machine learning.”), and
wherein the neural operations comprise obfuscation computations (see Bradshaw, [0009]: “Also, at least some aspects of the present disclosure are directed to a computer network that can be configured to implement obfuscating inputs for centralized training of a master version of an artificial neural network (ANN)”; [0011]: “inputs for centralized training of the master version of the ANN can be obfuscated. The obfuscation can occur at any one of the multiple computing devices that host different versions of the ANN such as devices hosting local versions of the ANN.”; [0015]: “there are many other ways of obfuscation that can be used, such as substitution, shuffling, numerical variance methods, scrambling, masking out characters, some other types of encryption not using hashing, and deletion of some values in particular fields.”; [0038]: “The obfuscating of the extracted plurality of features can include combining different sets of inputs in the extracted plurality of features using one or more arithmetic operations to combine the different sets (e.g., see obfuscations 124a and 124b as well as step 312 illustrated in FIG. 3). The extraction of features (e.g., see extracted features 122a and 122b) can include selecting the different sets of inputs randomly for the combining of the different sets. Or, the extraction of features can include selecting the different sets of inputs deliberately for the combining of the different sets.”; and [0058]: “when the input is Xf=f(X1, X2) (e.g., see extracted features 122a and 122b), the expected output may not necessarily be Yf=f(Y1, Y2). Thus, Yf is an estimate. From Xf, the computer hosting the master version of the ANN 110 (e.g., see first computing device 108) cannot guess or calculate X1, and/or X2 which is hosted by one of the other computing devices hosting a local version of the ANN (e.g., see computing devices 104a and 104b).”).
Bradshaw does not explicitly teach the input value comprised on or more inconsequential input values and causing one or more output values to be independent of the inconsequential input values.
Bell teaches the input value comprised on or more inconsequential input values and causing one or more output values to be independent of the inconsequential input values (see Bell, [0068]: “For each convolutional (Cony) layer, there are 3×3 convolutions with a stride of 1, and zero padding of the input in order to ensure the sizes of the input and output match”, NOTE: Paddings are well-known, routine, and conventionally applied in NN to give the input a require length, shape, or alignment and not to supply meaningful information to the NN. Furthermore, any inconsequential input data is inherently independent of the output and vice versa.).
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify the system of Bradshaw in view of Bell so that the input value comprised on or more inconsequential input values and causing one or more output values to be independent of the inconsequential input values. One would be motivated to do so because Paddings are well-known, routine, and conventionally applied in NN to give the input a require length, shape, or alignment and not to supply meaningful information to the NN..
As per claim 28, Bradshaw and Bell teach a method to obfuscate operations of a neural network (NN), the method comprising:
modifying at least a portion of the NN (see Bradshaw, [0051]: “The method 300 begins, at step 302, with hosting, by a plurality of computers, respective different versions of an ANN and respective different user data sets.”), wherein the modified NN portion is configured to:
receive, from at least one upstream portion of the NN, one or more input values comprising one or more inconsequential input values (see Claim 21 rejection above with respect to input values being inconsequential), and
apply neural operations to the input values to obtain one or more output values (see Claim 21 rejection above),
wherein the output values are used as an input into at least one downstream portion of the NN (see Claim 21 rejection above), and
wherein the neural operations comprise obfuscation computations causing the one or more output values to be independent of the inconsequential input values (see Claim 21 rejection above).
As per claim 35, Bradshaw and Bell teach a system comprising:
a memory device (see Bradshaw, [0083]: “With respect to the method 300, method 400, or any other method, process, or operation described herein, in some embodiments, a non-transitory computer-readable storage medium stores instructions that, when executed by at least one processing device (such as processor 506 shown in FIG. 5), cause the at least one processing device to perform the method 300, method 400, or any other method, process, or operation described herein, and/or any combination thereof.”); and
a processing device communicatively coupled to the memory device (see Bradshaw, [0083]: “With respect to the method 300, method 400, or any other method, process, or operation described herein, in some embodiments, a non-transitory computer-readable storage medium stores instructions that, when executed by at least one processing device (such as processor 506 shown in FIG. 5), cause the at least one processing device to perform the method 300, method 400, or any other method, process, or operation described herein, and/or any combination thereof.”), the processing device to:
process input data to obtain output data using a neural network (NN) comprising an NN portion (see Claim 21 rejection above)
(i) receiving, from at least one upstream portion of the NN, one or more input values comprising one or more inconsequential input values (see Claim 21 rejection above), and
(ii) applying neural operations to the input values to obtain one or more output values (see Claim 21 rejection above),
wherein the output values are used as an input into at least one downstream portion of the NN (see Claim 21 rejection above), and
wherein the neural operations comprise obfuscation computations causing the one or more output values to be independent of the inconsequential input values (see Claim 21 rejection above).
DEPENDENT:
As per claims 22, 29, and 37, which respectively depend on claims 21, 28, and 35, Bradshaw and Bell further teach wherein the NN portion comprises: a neural node having an activation function that generates a null value independent of the one or more inconsequential input values into the neural node (see Bradshaw, [0015]: ‘there are many other ways of obfuscation that can be used, such as substitution, shuffling, numerical variance methods, scrambling, masking out characters, some other types of encryption not using hashing, and deletion of some values in particular fields”’ [0044]: “For example, the obfuscation can include nulling out or deleting at least part of data in the extracted plurality of features randomly. Or, the nulling out or deleting at least part of data in the extracted plurality of features can be done deliberately.”; [0047]: “Further, the second device can obfuscate the extracted plurality of features using nulling out or deleting at least part of the extracted plurality of features (e.g., either randomly or deliberately) or using a masking out or character scrambling method.”; and Claim 21 rejection above with respect to input values being inconsequential).
As per claims 23, 30, and 36, which respectively depend on claims 21, 28, and 35, Bradshaw and Bell further teach wherein the NN portion comprises:
a first neural node receiving the one or more inconsequential input values and generating an intermediate value (see Bradshaw, Abstract: “The second computing device includes a processor that extracts features from the user data and obfuscates the extracted features to generate obfuscated user data. The second device includes a transceiver that transmits the obfuscated user data.”; [0011]: “In the system, inputs for centralized training of the master version of the ANN can be obfuscated. The obfuscation can occur at any one of the multiple computing devices that host different versions of the ANN such as devices hosting local versions of the ANN. For example, the second computing device in the system can include memory that is configured to store a local version of the ANN and user data for inputting into the local version of the ANN. The second computing device can also include a processor that is configured to extract features from the user data and obfuscate the extracted features to generate obfuscated user data.”; [0029]: “The computer network 100 also includes the first computing device 108 (which can be part of a cloud or another type of distributed computing network). The first computing device 108 includes the master version of the ANN 110”; and Claim 21 rejection above with respect to input values being inconsequential); and
a second neural node performing, using the intermediate value, one or more computations that cause an output value generated by the second neural node to be independent of the intermediate value (see Bradshaw, Abstract: “The first computing device includes a memory that stores the master version of the ANN, a transceiver that receives obfuscated user data transmitted from the second computing device, and a processor that trains the master version based on the received obfuscated user data using machine learning.”; [0011]: “The first computing device can include a memory that is configured to store the master version of the ANN, a transceiver that is configured to receive obfuscated user data transmitted from the second computing device or another device of the system hosting a local version of the ANN, and a processor that is configured to train the master version based on the received obfuscated user data using machine learning.”; and [0027]: “Each computing device of the set of computing devices 102 can host and execute a local version of an ANN (e.g., see second computing device 104a and Nth computing device 104b having respective local versions of an ANN 106a and 106b).”).
As per claims 24 and 31, which respectively depend on claims 21 and 28, Bradshaw and Bell further teach wherein the NN portion:
computes a plurality of intermediate values, each of the plurality of intermediate values computed by applying a respective non-linear function of a plurality of non-linear functions to the one or more inconsequential input values (see Bradshaw, [0003]: “Usually, an ANN is implemented by a signal at a connection (or edge) between artificial neurons being a real number, and the output of each artificial neuron being computed by a non-linear function of the sum of its inputs.”; Claim 21 rejection above with respect to input values being inconsequential; and Claim 23 rejection above); and
aggregates the plurality of intermediate values to obtain the one or more output values that are independent of the inconsequential input values (see Bradshaw, [0003]: “An artificial neuron can also have a threshold in which a signal is only sent from the artificial neuron if the aggregate signal exceeds the threshold.”; and Claim 21 rejection above with respect to input values being inconsequential).
As per claims 25 and 32, which respectively depend on claims 21 and 28, Bradshaw and Bell further teach wherein the one or more input values further comprises one or more consequential input values, and wherein the NN portion:
computes a plurality of intermediate values, each of the plurality of intermediate values computed by combining the one or more consequential input values and the one or more inconsequential input values (see Bradshaw, [0003]: “An artificial neuron can also have a threshold in which a signal is only sent from the artificial neuron if the aggregate signal exceeds the threshold.”; [0016]: “… the user device can obfuscate the inputs by combining the sets and requesting a computer hosting the master version of the ANN to train the master version…”; [0038]: “The obfuscating of the extracted plurality of features can include combining different sets of inputs in the extracted plurality of features using one or more arithmetic operations to combine the different sets (e.g., see obfuscations 124a and 124b as well as step 312 illustrated in FIG. 3). The extraction of features (e.g., see extracted features 122a and 122b) can include selecting the different sets of inputs randomly for the combining of the different sets. Or, the extraction of features can include selecting the different sets of inputs deliberately for the combining of the different sets.”; and Claim 21 rejection above with respect to input values being inconsequential); and
aggregates the plurality of intermediate values to obtain the one or more output values that are independent of the inconsequential input values (see Bradshaw, [0003]: “An artificial neuron can also have a threshold in which a signal is only sent from the artificial neuron if the aggregate signal exceeds the threshold.”; and Claim 21 rejection above with respect to input values being inconsequential).
As per claims 26, 33, and 39, which respectively depend on claims 21, 28, and 35, Bradshaw and Bell further teach wherein the one or more input values further comprises one or more consequential input values, and wherein the NN portion:
applies a masking transformation to the one or more input values to obtain a plurality of masked input values, each of the plurality of masked input values being determined by the one or more consequential input values and the one or more inconsequential input values (see Bradshaw, [0015]: “Thus, hashing cannot be used for feature obfuscation in the system. But, there are many other ways of obfuscation that can be used, such as substitution, shuffling, numerical variance methods, scrambling, masking out characters, some other types of encryption not using hashing, and deletion of some values in particular fields.”; and [0045]: “The obfuscating of the extracted plurality of features (e.g., see obfuscations 124a and 124b) can include using a masking out or character scrambling method (e.g., see step 324 illustrated in FIG. 3). The masking out or character scrambling method can include masking out or character scrambling part of data in the extracted plurality of features.”; and Claim 21 rejection above with respect to input values being inconsequential);
applies an unmasking transformation to the plurality of masked input values to obtain one or more intermediate values that are independent of the inconsequential input values (see Bradshaw, [0021]: “The unmasked user data however can be used to locally train a local version of the ANN on the device of the user. For example, the user data can be only used to train a local version of the ANN on a user's mobile device (e.g., the user's smart phone, tablet, etc.). When it is shared in the system for training of other versions of the ANN it is always obfuscated by the system.”; and Claim 21 rejection above with respect to input values being inconsequential); and
computes the one or more output values using the one or more intermediate values (see Bradshaw, [0003]: “Usually, an ANN is implemented by a signal at a connection (or edge) between artificial neurons being a real number, and the output of each artificial neuron being computed by a non-linear function of the sum of its inputs”).
As per claims 27, 34, and 40, which respectively depend on claims 21, 28, and 35, Bradshaw and Bell further teach wherein the one or more inconsequential input values comprise a plurality of inconsequential input values, wherein each of the plurality of inconsequential input values is obtained by applying a respective one of a plurality of non-linear obfuscation functions, at least one non-linear obfuscation function of the plurality of non-linear obfuscation functions being linearly-dependent on one or more other non-linear obfuscation functions of the plurality of non-linear obfuscation functions (see Bradshaw, [0003]: “Usually, an ANN is implemented by a signal at a connection (or edge) between artificial neurons being a real number, and the output of each artificial neuron being computed by a non-linear function of the sum of its inputs”; and Claim 21 rejection above with respect to input values being inconsequential), and wherein the NN portion:
computes, using a combination of the inconsequential input values, one or more intermediate values that are independent of the inconsequential input values (see Bradshaw, [0003]: “An artificial neuron can also have a threshold in which a signal is only sent from the artificial neuron if the aggregate signal exceeds the threshold.”; [0016]: “… the user device can obfuscate the inputs by combining the sets and requesting a computer hosting the master version of the ANN to train the master version…”; [0038]: “The obfuscating of the extracted plurality of features can include combining different sets of inputs in the extracted plurality of features using one or more arithmetic operations to combine the different sets (e.g., see obfuscations 124a and 124b as well as step 312 illustrated in FIG. 3). The extraction of features (e.g., see extracted features 122a and 122b) can include selecting the different sets of inputs randomly for the combining of the different sets. Or, the extraction of features can include selecting the different sets of inputs deliberately for the combining of the different sets.” and Claim 21 rejection above with respect to input values being inconsequential); and
computes the one or more output values using the one or more intermediate values (see Bradshaw, [0003]: “Usually, an ANN is implemented by a signal at a connection (or edge) between artificial neurons being a real number, and the output of each artificial neuron being computed by a non-linear function of the sum of its inputs”).
As per claim 38, which depends on claims 35, Bradshaw and Bell further teach wherein the NN portion: computes a plurality of intermediate values, each of the plurality of intermediate values computed by performing at least one of:
applying a respective non-linear function of a plurality of non-linear functions to the one or more inconsequential input values (see Bradshaw, [0003]: “Usually, an ANN is implemented by a signal at a connection (or edge) between artificial neurons being a real number, and the output of each artificial neuron being computed by a non-linear function of the sum of its inputs.”; Claim 21 rejection above with respect to input values being inconsequential; and Claim 23 rejection above); or
combining one or more consequential input values and the one or more inconsequential input values (see Bradshaw, [0003]: “An artificial neuron can also have a threshold in which a signal is only sent from the artificial neuron if the aggregate signal exceeds the threshold.”; [0016]: “… the user device can obfuscate the inputs by combining the sets and requesting a computer hosting the master version of the ANN to train the master version…”; [0038]: “The obfuscating of the extracted plurality of features can include combining different sets of inputs in the extracted plurality of features using one or more arithmetic operations to combine the different sets (e.g., see obfuscations 124a and 124b as well as step 312 illustrated in FIG. 3). The extraction of features (e.g., see extracted features 122a and 122b) can include selecting the different sets of inputs randomly for the combining of the different sets. Or, the extraction of features can include selecting the different sets of inputs deliberately for the combining of the different sets.”; and Claim 21 rejection above with respect to input values being inconsequential); and
aggregates the plurality of intermediate values to obtain the one or more output values that are independent of the inconsequential input values (see Bradshaw, [0003]: “An artificial neuron can also have a threshold in which a signal is only sent from the artificial neuron if the aggregate signal exceeds the threshold.”; and Claim 21 rejection above with respect to input values being inconsequential).
Conclusion
5. For the reasons above, claims 21-40 have been rejected and remain pending.
6. Any inquiry concerning this communication or earlier communications from the examiner should be directed to MICHAEL Y WON whose telephone number is (571)272-3993. The examiner can normally be reached on Wk.1: M-F: 8-5 PST & Wk.2: M-Th: 8-7 PST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Nicholas R Taylor can be reached on 571-272-3889. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Michael Won/Primary Examiner, Art Unit 2443
