DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claims 1-2, 4-5, 7-10, 12-13, 15-18, and 20 are presented for examination.
Claim Rejections - 35 USC § 101
The text of those sections of Title 35, U.S. Code not included in this action can be found in a prior Office action.
Claims 1-2, 4-5, 7-10, 12-13, 15-18, and 20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to abstract idea without significantly more. The analysis of the claims will follow the 2019 Revised Patent Subject Matter Eligibility Guidance, 84 Fed. Reg. 50 (“2019 PEG”).
Claim 1
Step 1: The claim recites an apparatus comprising a memory and a weight gradients splitter; therefore, it is directed to the statutory category of machines.
Step 2A Prong 1: The claim recites, inter alia:
[S]plit[ting] the second weight gradients into a second set of portions of the second weight gradients including a second portion and the first set of weight gradients into a first set of portions of the first weight gradients including a first portion: This limitation could encompass the mental splitting of the weight gradients into portions.
[A]ssign[ing] the first and second sets of portions to a plurality of exchange tasks, including the first portion to a first exchange task and the second portion to a second exchange task: This limitations could encompass the mental assignment of portions of weight gradients to exchange tasks.
[B]ased on a priority of the first exchange task being higher than a priority of the second exchange task, exchange the first portion … before exchanging the second portion, wherein a priority for each of the plurality of exchange tasks is determined such that weight gradients for lower layers of the neural network are prioritized over weight gradients for higher layers of the neural network: This limitation could encompass the mental exchange of weight gradients with other weight gradients based on a priority of exchange tasks and the mental determination of which exchange tasks are to be prioritized based on the position of the weights in the network.
Step 2A Prong 2: This judicial exception is not integrated into a practical application. The claim further recites an “apparatus, comprising: a memory that stores a buffer having a plurality of entries; and a weight gradients splitter”, that “the first and second weight gradients [are] generated by a first computer system”, and that the weight gradients are exchanged “with a second computer system”. However, these are mere instructions to apply the judicial exception using a generic computer. MPEP § 2106.05(f). The claim further recites “receiv[ing] second weight gradients corresponding to a second layer of a neural network and, thereafter, receive first weight gradients corresponding to a first layer of a neural network, the second layer being higher than the first layer” and “stor[ing] the plurality of exchange tasks in the plurality of entries of the buffer, each of the plurality of exchange tasks being stored in a respective one of the plurality of entries”. However, these limitations recite the insignificant extra-solution activity of mere data gathering and output. MPEP § 2106.05(g).
Step 2B: The claim does not contain significantly more than the judicial exception. The analysis at this step mirrors that of step 2A prong 2, except insofar as the receiving limitation recites the well-understood, routine, and conventional activity of receiving or transmitting data over a network, MPEP § 2106.05(d)(II), OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015) (sending messages over a network), and the storing limitation recites the well-understood, routine, and conventional activity of storing and retrieving information in memory, Versata Dev. Group, Inc. v. SAP Am., Inc., 793 F.3d 1306, 1334, 115 USPQ2d 1681, 1701 (Fed. Cir. 2015). As an ordered whole, the claim is directed to a mentally performable process of exchanging weight gradients based on a priority of exchange tasks. Nothing in the claim provides significantly more than this. As such, the claim is not patent eligible.
Claim 2
Step 1: A machine, as above.
Step 2A Prong 1: The claim recites the same judicial exceptions as in claim 1.
Step 2A Prong 2: This judicial exception is not integrated into a practical application. The claim further recites that “each of the plurality of entries further stores an indication of whether a respective one of the plurality of exchange tasks has been completed”. This represents the insignificant extra-solution activity of mere data gathering and output. MPEP § 2106.05(g).
Step 2B: The claim does not contain significantly more than the judicial exception. The claim further recites that “each of the plurality of entries further stores an indication of whether a respective one of the plurality of exchange tasks has been completed”. This represents the well-understood, routine, and conventional activity of storing and retrieving information in memory. MPEP § 2106.05(d)(II); Versata Dev. Group, Inc. v. SAP Am., Inc., 793 F.3d 1306, 1334, 115 USPQ2d 1681, 1701 (Fed. Cir. 2015).
Claim 4
Step 1: A machine, as above.
Step 2A Prong 1: The claim recites the same judicial exceptions as in claim 1.
Step 2A Prong 2: This judicial exception is not integrated into a practical application. The claim further recites “a hardware interface, wherein the first portion and the second portion are exchanged with the second computer system via the hardware interface”. However, this is a mere instruction to apply the judicial exception using a generic computer. MPEP § 2106.05(f).
Step 2B: The claim does not contain significantly more than the judicial exception. The claim further recites “a hardware interface, wherein the first portion and the second portion are exchanged with the second computer system via the hardware interface”. However, this is a mere instruction to apply the judicial exception using a generic computer. MPEP § 2106.05(f).
Claim 5
Step 1: A machine, as above.
Step 2A Prong 1: The claim recites “detect[ing] that the hardware interface is idle at a first time, wherein the first portion are exchanged in response to determining that the hardware interface is idle at the first time; and detect[ing] that the hardware interface is idle at a second time after the first time, wherein the second portion are exchanged in response to determining that the hardware interface is idle at the second time”. This limitation could encompass mentally exchanging the weight gradients in response to mentally detecting that the hardware interface is idle at two different times.
Step 2A Prong 2: This judicial exception is not integrated into a practical application. See claim 1 analysis.
Step 2B: The claim does not contain significantly more than the judicial exception. See claim 1 analysis.
Claim 7
Step 1: A machine, as above.
Step 2A Prong 1: The claim recites, inter alia, “indicat[ing] an entry of the plurality of entries that stores an execution task being executed, … indicat[ing] an entry of the plurality of entries to store a next new exchange task”. These limitations could encompass the indication, with a pen and paper, of an entry storing a task being executed and an entry that is to store a new exchange task.
Step 2A Prong 2: This judicial exception is not integrated into a practical application. The claim further recites that these two indications are performed with “a read pointer and a write pointer”, respectively, that are associated with a buffer. However, these are mere instructions to apply the judicial exception using a generic computer. MPEP § 2106.05(f).
Step 2B: The claim does not contain significantly more than the judicial exception. The claim further recites that these two indications are performed with “a read pointer and a write pointer”, respectively, that are associated with a buffer. However, these are mere instructions to apply the judicial exception using a generic computer. MPEP § 2106.05(f).
Claim 8
Step 1: A machine, as above.
Step 2A Prong 1: The claim recites, inter alia, “after storing the plurality of exchange tasks in the plurality of entries, … indicat[ing] an entry of the plurality of entries that stores the first exchange task”. This limitation could encompass indicating an entry that stores an exchange task with a pen and paper.
Step 2A Prong 2: This judicial exception is not integrated into a practical application. The claim further recites that the “read pointer” indicates this entry. However, this is a mere instruction to apply the judicial exception using a generic computer. MPEP § 2106.05(f).
Step 2B: The claim does not contain significantly more than the judicial exception. The claim further recites that the “read pointer” indicates this entry. However, this is a mere instruction to apply the judicial exception using a generic computer. MPEP § 2106.05(f).
Claims 9-10, 12-13, 15-16
Step 1: The claims recite a method; therefore, they are directed to the statutory category of processes.
Step 2A Prong 1: The claims recite the same judicial exceptions as in claims 1-2, 4-5, and 7-8, respectively.
Step 2A Prong 2: This judicial exception is not integrated into a practical application. The analysis at this step is the same as that of claims 1-2, 4-5, and 7-8, respectively, except insofar as these claims recite a “computer-implemented method”. However, this is a mere instruction to apply the exception using a generic computer. MPEP § 2106.05(f).
Step 2B: The claim does not contain significantly more than the judicial exception. The analysis at this step is the same as that of claims 1-2, 4-5, and 7-8, respectively, except insofar as these claims recite a “computer-implemented method”. However, this is a mere instruction to apply the exception using a generic computer. MPEP § 2106.05(f).
Claims 17-18, 20
Step 1: The claims recite a system comprising one or more processors and a computer-readable medium; therefore, they are directed to the statutory category of machines.
Step 2A Prong 1: The claims recite the same judicial exceptions as in claims 1-2 and 8, respectively.
Step 2A Prong 2: This judicial exception is not integrated into a practical application. The analysis at this step is the same as in claims 1-2 and 8, respectively, except insofar as these claims recite a “system comprising: one or more processors, and a computer-readable medium comprising instructions that, when executed by the one or more processors, cause the one or more processors to perform operations”. However, this is a mere instruction to apply the judicial exception using a generic computer. MPEP § 2106.05(f).
Step 2B: The claim does not contain significantly more than the judicial exception. The analysis at this step is the same as in claims 1-2 and 8, respectively, except insofar as these claims recite a “system comprising: one or more processors, and a computer-readable medium comprising instructions that, when executed by the one or more processors, cause the one or more processors to perform operations”. However, this is a mere instruction to apply the judicial exception using a generic computer. MPEP § 2106.05(f).
Claim Rejections - 35 USC § 103
Claims 1, 4, 7-9, 12, 15-17, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Huo et al., “Decoupled Parallel Backpropagation with Convergence Guarantee,” in Int’l Conf. Machine Learning 2098-2106 (2018) (“Huo”) in view of Alvarez-Icaza Rivera et al. (US 20160086075) (“Rivera”) and further in view of Zhang et al. (US 20200175422) (“Zhang”) and Koker et al. (US 11797837) (“Koker”).
Regarding claim 1, Huo discloses “[a]n apparatus, comprising:
a memory … having a plurality of entries (Huo sec. 5, paragraph entitled “Implementation Details” discloses that the system utilizes shared memory objects in a multiprocessing package; Fig. 2 shows two portions [entries] of storage space, one for transferring error gradients from module B to module A and another for transferring error gradients from module C to module B); and
a weight gradients splitter (see below mapping and note that the software that performs the following functions is deemed a “weight gradients splitter”; note also that Huo sec. 5, paragraph labeled “Implementation Details” discloses that the method is performed on multiple processors) configured to:
receive second weight gradients corresponding to a second layer of a neural network and … receive first weight gradients corresponding to a first layer of the neural network, … the first and second weight gradients having been generated by a first computer system (Huo Fig. 2 and accompanying text show that a multilayer feedforward neural network may be split into three modules, each comprising a stack of layers; module A can perform a backward pass using a stale error gradient
δ
A
t
-
2
[weight gradient] and also receives
δ
A
t
-
1
from module B for the update of a next iteration [second weight gradients = all the
δ
B
gradients; first weight gradients = all the
δ
A
gradients; first computing system = combination of modules C and B]); and
split the second weight gradients into a second set of portions of the second weight gradients including a second portion and the first set of weight gradients into a first set of portions of the first weight gradients including a first portion (Huo Fig. 2 and accompanying text show that a multilayer feedforward neural network may be split into three modules, each comprising a stack of layers; module A can perform a backward pass using a stale error gradient
δ
A
t
-
2
and also receives
δ
A
t
-
1
from module B for the update of a next iteration; the drawing also depicts
δ
B
t
-
1
and
δ
B
t
being sent from module C to module B [first portion =
δ
A
t
-
2
; second portion =
δ
B
t
]);
wherein the apparatus is configured to:
assign the first and second sets of portions to a plurality of exchange tasks, including the first portion to a first exchange task and the second portion to a second exchange task (Huo Fig. 2 and accompanying text show that a multilayer feedforward neural network may be split into three modules, each comprising a stack of layers; module A can perform a backward pass using a stale error gradient
δ
A
t
-
2
and also receives
δ
A
t
-
1
from module B for the update of a next iteration; the drawing also depicts
δ
B
t
-
1
and
δ
B
t
being sent from module C to module B [first exchange task = exchange of
δ
A
t
-
2
from B to A; second exchange task = exchange of
δ
B
t
from C to B in the next iteration]));
store the plurality of exchange tasks in the plurality of entries …, each of the plurality of exchange tasks being stored in … the plurality of entries (Huo Fig. 2 depicts storage space for transferring error gradients from module B to module A and another storage space for transferring error gradients from module C to module B;
δ
A
t
-
2
and
δ
B
t
-
1
are stored in the storage space for the current iteration and
δ
A
t
-
1
and
δ
B
t
are stored in the storage space in the next iteration); and
based on a priority of the first exchange task being higher than a priority of the second exchange task, exchange the first portion with a second computer system before exchanging the second portion with the second computer system (Huo Fig. 2 and accompanying text show that a multilayer feedforward neural network may be split into three modules, each comprising a stack of layers; module A can perform a backward pass using a stale error gradient
δ
A
t
-
2
and also receives
δ
A
t
-
1
from module B for the update of a next iteration; the drawing also depicts
δ
B
t
-
1
and
δ
B
t
being sent from module C to module B [since the exchange of
δ
A
t
-
2
from B to A occurs in an iteration before the exchange of
δ
B
t
from C to B, the first-iteration exchanges have a higher temporal priority than the second-iteration exchanges; second computer system = combination of modules B and A]).”
Huo appears not to disclose explicitly the further limitations of the claim. However, Rivera discloses “a memory that stores a buffer (bank 106 may include a first buffer unit for spike packets scheduled for delivery when timestep t is 0, a second buffer unit for spike events scheduled for delivery when timestep t is 1, a third buffer unit for spike event packets scheduled for delivery when timestep t is 2, and a fourth buffer unit for spike event packets scheduled for delivery when timestep t is 3 – Rivera, paragraph 80; see also paragraph 198 (disclosing that the input scheduler buffer is part of memory)) …; [and] …
stor[ing] the plurality of … tasks in the plurality of entries of the buffer, each of the plurality of … tasks being stored in a respective one of the plurality of entries (bank may include a first buffer unit [entry] for spike packets scheduled for delivery [task] when timestep t is 0, a second buffer unit for spike events scheduled for delivery when timestep t is 1, a third buffer unit for spike event packets scheduled for delivery when timestep t is 2, and a fourth buffer unit for spike event packets scheduled for delivery when timestep t is 3 – Rivera, paragraph 80) ….”
Rivera and the instant application both relate to physical implementations of neural networks and are analogous. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Huo to store each task in a separate entry in the buffer, as disclosed by Rivera, and an ordinary artisan could reasonably expect to have done so successfully. Doing so would ensure that each task is executed at the correct time by separating each task in a separate memory area. See Rivera, paragraph 80.
Neither Huo nor Rivera appears to disclose explicitly the further limitations of the claim. However, Zhang discloses “receiv[ing] second weight gradients … and, thereafter, receiv[ing] first weight gradients (gradient weight compression system can encode a timestamp (e.g., a digital date and/or time) on each compressed gradient weight received from learners 114a, 114b, 114N to indicate when each compressed gradient weight was received by gradient weight compression system [i.e., in general the gradients are received at two different times] – Zhang, paragraph 75) ….”
Zhang and the instant application both relate to weight gradient transmission in machine learning systems and are analogous. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the combination of Huo and Rivera to transmit the gradients asynchronously, as disclosed by Zhang, and an ordinary artisan could reasonably expect to have done so successfully. Doing so would allow the system to be used asynchronously, thereby reducing the need for time coordination in transmission of the gradients relative to a synchronous system. See Zhang, paragraph 75.
Neither Huo, Rivera, nor Zhang appears to disclose explicitly the further limitations of the claim. However, Koker discloses that “the second layer [is] a higher layer than the first layer (Koker Fig. 12 discloses a neural network with a series of hidden layers 1204, one of which is higher than another) …; … [and]
a priority for each of the plurality of … tasks is determined such that weight gradients for lower layers of the neural network are prioritized over weight gradients for higher layers of the neural network (processor is configured to assign the first layer of a neural network to a first GPU and a second layer to a second GPU to prioritize execution [task] of the first layer of the neural network over the second [higher] layer of the neural network; gradient data for the first layer are received from the first GPU and gradient data for the second layer are received from the second GPU – Koker, claim 1 [note that since the execution of the first layer is prioritized over the execution of the second layer and the gradients for each layer are part of that layer, it follows that the gradients of the first layer are prioritized over those of the second layer]).”
Koker and the instant application both relate to distributed machine learning training and are analogous. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the combination of Huo, Rivera, and Zhang to prioritize gradient data for lower layers of the network over gradient data for higher layers, as disclosed by Koker, and an ordinary artisan could reasonably expect to have done so successfully. Doing so would allow the system to focus the network on tasks that need to be performed more quickly. See Koker, col. 23, ll. 42-49.
Claim 9 is a method claim corresponding to apparatus claim 1 and is rejected for the same reasons as given in the rejection of that claim. Similarly, claim 17 is a system claim corresponding to apparatus claim 1 and is rejected for the same reasons as given in the rejection of that claim.
Regarding claim 4, Huo, as modified by Rivera, Koker, and Zhang, discloses “a hardware interface, wherein the first portion and the second portion are exchanged with the second computer system via the hardware interface (Huo Fig. 2 and accompanying text show that a multilayer feedforward neural network may be split into three modules, each comprising a stack of layers; module A can perform a backward pass using a stale error gradient
δ
A
t
-
2
and also receives
δ
A
t
-
1
from module B for the update of a next iteration; the drawing also depicts
δ
B
t
-
1
and
δ
B
t
being sent from module C to module B [second computer system = combination of modules B and A]; sec. 5, paragraph entitled “Implementation Details”, discloses that the method is performed using multiple processors and utilizes shared memory objects [i.e., on a computer comprising a processor and memory, or a hardware interface]).”
Claim 12 is a method claim corresponding to apparatus claim 4 and is rejected for the same reasons as given in the rejection of that claim.
Regarding claim 7, the rejection of claim 1 is incorporated. Huo further discloses “an entry of the plurality of entries that stores an execution task being executed (Huo Fig. 2 shows storage space for holding error gradients
δ
A
t
-
2
and
δ
B
t
-
1
for transfer [execution task] to modules A and B, respectively, in a current iteration [being executed])” and an “exchange task (see mapping of this limitation to Huo in claim 1 supra)”.
Huo/Zhang/Koker appears not to disclose explicitly the further limitations of the claim. However, Rivera discloses that “the buffer is associated with a read pointer and a write pointer, wherein the read pointer indicates an entry of the plurality of entries that stores an execution task …, and wherein the write pointer indicates an entry of the plurality of entries to store a next new … task (input control function unit maintains a write pointer that references a memory location/buffer unit [entry] of the bank 106 to write a spike event packet to in a subsequent [next] write operation [new task], and an output control function unit maintains a read pointer that references a memory location/buffer unit of the bank 106 to read out a spike packet from in a subsequent read operation [execution task] – Rivera, paragraph 82).” It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Huo/Zhang/Koker to associate the buffer with a read pointer and a write pointer, as disclosed by Rivera, and an ordinary artisan could reasonably expect to have done so successfully. Doing so would allow the system to keep track of which tasks need to be executed and when, thereby enhancing the processing flow. See Rivera, paragraph 82.
Claim 15 is a method claim corresponding to apparatus claim 7 and is rejected for the same reasons as given in the rejection of that claim.
Regarding claim 8, the rejection of claim 7 is incorporated. Huo further discloses that “the apparatus is further configured to: …
indicate an entry of the plurality of entries that stores the first exchange task (Huo Figure 2 shows that the storage space holds weight gradients
δ
A
t
-
2
and
δ
B
t
-
1
for exchange to modules A and B, respectively [i.e., the system indicates that, at a current iteration, these two storage spaces [entries] store the gradients of the first exchange task]).”
Huo/Zhang/Koker appears not to disclose explicitly the further limitations of the claim. However, Rivera discloses “after storing the plurality of … tasks in the plurality of entries, set[ting] the read pointer to indicate an entry of the plurality of entries that stores the first … task (output control function unit maintains a read pointer that references a memory location/buffer unit [entry] of the bank 106 to read out a spike event packet from in a subsequent read operation [first task] – Rivera, paragraph 82 [note that the event packet must have already been stored in the buffer unit prior to the read pointer being able to reference it]).” It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Huo/Zhang/Koker to associate the buffer with a read pointer that indicates an entry that stores a particular task, as disclosed by Rivera, and an ordinary artisan could reasonably expect to have done so successfully. Doing so would allow the system to keep track of which tasks need to be executed and when, thereby enhancing the processing flow. See Rivera, paragraph 82.
Claim 16 is a method claim corresponding to apparatus claim 8 and is rejected for the same reasons as given in the rejection of that claim. Similarly, claim 20 is a system claim corresponding to apparatus claim 8 and is rejected for the same reasons as given in the rejection of that claim.
Claims 2, 10, and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Huo in view of Rivera, Koker, and Zhang and further in view of McBride et al. (US 20200409757) (“McBride”).
Regarding claim 2, the rejection of claim 1 is incorporated. Huo further discloses that “each of the plurality of entries further stores …exchange tasks (Huo Fig. 2 depicts storage space [entries] for transferring error gradients from module B to module A and another storage space for transferring [exchanging] error gradients from module C to module B;
δ
A
t
-
2
and
δ
B
t
-
1
are stored in the storage space for the current iteration and
δ
A
t
-
1
and
δ
B
t
are stored in the storage space in the next iteration) ….”
Neither Huo, Zhang, Koker, nor Rivera appears to disclose explicitly the further limitations of the claim. However, McBride discloses “stor[ing] an indication of whether a respective one of the plurality of … tasks has been completed (workload fragment may, for example, store data indicating that it has completed 25% of its execution [task] – McBride, paragraph 11).”
McBride and the instant application both relate to neural networks and are analogous. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the combination of Huo, Zhang, Koker, and Rivera to store data indicating whether a task has been completed, as disclosed by McBride, and an ordinary artisan could reasonably expect to have done so successfully. Doing so would apprise a user of the progress of the tasks, thereby allowing him to make more informed decisions about the assignment of tasks. See McBride, paragraph 11.
Claim 10 is a method claim corresponding to apparatus claim 2 and is rejected for the same reasons as given in the rejection of that claim. Similarly, claim 18 is a system claim corresponding to apparatus claim 2 and is rejected for the same reasons as given in the rejection of that claim.
Claims 5 and 13 are rejected under 35 U.S.C. 103 as being unpatentable over Huo in view of Rivera, Koker, and Zhang and further in view of Liu et al. (US 20190370642) (“Liu”).
Regarding claim 5, the rejection of claim 4 is incorporated. Huo further discloses that “the apparatus is further configured to:
detect [a condition] at a first time, wherein the first portion are exchanged in response to [the condition] (Huo Figure 2 and accompanying text disclose that, at a first iteration, a module A performs a backward pass with error gradient
δ
A
t
-
2
and a module B performs a backward pass with error gradient
δ
B
t
-
1
[condition = weight gradients for the iteration are in the storage space]); and
detect [the condition] at a second time after the first time, wherein the second portion are exchanged in response to [the condition] (Huo Figure 2 and accompanying text disclose that, at a next iteration [second time after the first time], a module A performs a backward pass with error gradient
δ
A
t
-
1
and a module B performs a backward pass with error gradient
δ
B
t
[note the computation of the backward pass with the error gradient implies transferal of the error gradient]).”
Neither Huo, Zhang, Koker, nor Rivera appears to disclose explicitly the further limitations of the claim. However, Liu discloses that “the apparatus is further configured to:
detect that the hardware interface is idle …, wherein the [data] are exchanged in response to determining that the hardware interface is idle (chip [hardware interface] may include at least N units; the working state information of the chip may include working state information of at least S units in the at least N units; a voltage frequency scaling unit may be configured to: in response to determining that a unit A is in an idle state, send [exchange] voltage frequency scaling information [data] to the unit A, where unit A is any one of the at least S units – Liu, paragraphs 199-200) ….”
Liu and the instant application both relate to data transfer in neural networks and are analogous. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the combination of Huo, Zhang, Koker, and Rivera to send the data in response to determining that the hardware is idle, as disclosed by Liu, and an ordinary artisan could reasonably expect to have done so successfully. Doing so would ensure that the data may be immediately processed by the hardware by ensuring that it is not executing some other process, thereby enhancing the efficiency of the system. See Liu, paragraphs 199-200.
Claim 13 is a method claim corresponding to apparatus claim 5 and is rejected for the same reasons as given in the rejection of that claim.
Response to Arguments
Applicant's arguments filed April 3, 2026 (“Remarks”) have been fully considered but they are not persuasive.
Applicant first argues that the claims as amended are eligible under 35 USC § 101 because (a) they are allegedly directed to a technological improvement in a distributed computer system by reducing backward locking and network latency; and (b) processing weight gradients for neural networks allegedly cannot be performed mentally because they involve “massive matrices of floating-point values” that cannot be practically manipulated in the mind. Remarks at 7-8. However, taking the last argument first, the claims do not limit the size of the matrices involved, nor, for that matter, are matrices ever recited in the claims. Regarding (a), it is unclear what, if any, additional elements of the claims, beyond those constituting the judicial exception itself, are related to solving the problem of “backward locking” and “network latency”. Indeed, all of the additional elements of the claims are directed either to generic computer implementation of the judicial exception or data gathering and output, neither of which provides an inventive concept. Note that the judicial exception itself cannot provide the inventive concept. MPEP § 2106.05(I).
Regarding the art rejection, Applicant argues that the cited references allegedly fail to teach the claim as amended because (a) the claim now allegedly recites that the portions are fragments of the weight gradients of a single layer, whereas the portions of Huo that Examiner mapped to the claimed “portions” are whole gradients for different modules/layers; and (b) the combination of Huo and Koker allegedly fails to teach prioritizing the exchange of a portion of split layer gradients because Koker relates to the computation priority of layers on GPUs rather than the exchange priority of gradient portions. Remarks at 8-12. However, regarding (a), the claims do not require that the portions correspond to a single layer. At most, they require that the portions come from first and second weight gradients that “correspond[] to” a first and second layer of the neural network, respectively. Nothing in the claim prevents these first and second weight gradients from also “corresponding to” other layers, so long as they at least partially “correspond[] to” the first and second layers, respectively. Nor do the claims specify how the splitting takes place. Regarding (b), the independent claims require merely that “weight gradients for lower layers of the neural network [be] prioritized over weight gradients for higher layers of the neural network.” While the claim does require that the priority be for “exchange tasks,” Examiner never represented that Koker prioritizes certain exchange tasks over others, but rather only that it prioritizes certain tasks over others (namely, the computation of the first layer over the computation of the second). Huo teaches the performance of exchange tasks, and it would have been obvious to an ordinary artisan to have combined Huo and Koker such that the tasks being prioritized are the exchange tasks for the reasoning given in the rejection.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to RYAN C VAUGHN whose telephone number is (571)272-4849. The examiner can normally be reached M-R 7:00a-5:00p ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kamran Afshar, can be reached at 571-272-7796. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/RYAN C VAUGHN/ Primary Examiner, Art Unit 2125
Prosecution Insights