DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This action is in response to communications filed on 03/05/2026.
Claims 4 and 14 have been canceled.
Claims 1-3, 5-13, and 15-20 are pending and have been examined.
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 03/05/2026 has been entered.
Priority
Applicant’s claim for the benefit of a prior-filed application under 35 U.S.C. 119(e) or under 35 U.S.C. 120, 121, 365(c), or 386(c) is acknowledged.
Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55.
Specification
The disclosure is objected to because of the following informalities:
The use of a trade name or a mark used in commerce (e.g. WIMAX, WIGIG, etc.) has been noted in this application. It should be capitalized (each letter) wherever it appears and be accompanied by the generic terminology or, where appropriate, include a proper symbol indicating use in commerce, such as ™, SM, or ® following the word. Although the use of trade names and marks used in commerce (i.e., trademarks, service marks, certification marks, and collective marks) are permissible in patent applications, the proprietary nature of the marks should be respected and every effort made to prevent their use in any manner which might adversely affect their validity as commercial marks.
Appropriate correction is required.
Response to Arguments
With respect to objections to the specification, some trademarks have not been addressed. See above.
Previous objections to the claims have been withdrawn in view of amendments.
Previous rejections under 35 USC 112 have been withdrawn in view of amendments.
Applicant's arguments filed have been fully considered but they are not persuasive.
Applicant argues in substance that the references allegedly do not teach maintaining the batch size and adjusting the number of batches, and that Jain is allegedly related to a different technical problem. However, examiner respectfully disagrees.
A “data batch” merely refers to a “number of data samples” (such as noted in applicant’s remarks). Since a number can include 1, a batch can refer to 1 sample. Cui teaches “for…128 samples, and N=4 accelerators, the initial mini-batch dataset would be partioned into four (4) sub-batch datasets, each with [data items] of 32 (i.e., 128/4), i.e., 32:32:32:32” (e.g. in paragraph 48 and 54, i.e. 32 data items with item size of 1) and “epoch iteration is then repeated… determine an optimal job partition ratio for partitioning a mini-batch dataset into sub-batch datasets for processing by the accelerator resources 320. The self-adaptive batch dataset partitioning is performed to rebalance the processing loads among the accelerator resources… determine (through an iterative load balancing process) that an optimal job partition ratio of 48:48:16:16 among the accelerators”, i.e. adjusted to 48 data items of a “maintained”/“constant” size 1 for accelerator 320-1 (e.g. in paragraphs 5, 49, 50, and 54). With the interpretation that a data “batch” refers to the above “data item”, the teachings of Cui read on (or at least are functionally equivalent to) the claimed features. In the interest of advancing prosecution, Jain is relied upon to explicitly teach a data item including a “batch” with a “batch” size (e.g. in paragraph 58, “each batch may represent equal distribution of data… equal size batches may be formed from large data pool”). In response to applicant's argument that Jain is nonanalogous art, it has been held that a prior art reference must either be in the field of the inventor’s endeavor or, if not, then be reasonably pertinent to the particular problem with which the inventor was concerned, in order to be relied upon as a basis for rejection of the claimed invention. See In re Oetiker, 977 F.2d 1443, 24 USPQ2d 1443 (Fed. Cir. 1992). In this case, Jain is generally related to processing batches, i.e. related field, and specifically describes “identify optimum number of batches that need to be selected, based on…processing time [i.e. operation time] per batch” (e.g. in paragraph 58), i.e. pertinent problem. In response to applicant's argument that the references fail to show certain features of the invention, it is noted that the features upon which applicant relies (i.e., shared model weights, latency reduction) are not recited in the rejected claim(s). Although the claims are interpreted in light of the specification, limitations from the specification are not read into the claims. See In re Van Geuns, 988 F.2d 1181, 26 USPQ2d 1057 (Fed. Cir. 1993). Such features are not required by the claimed limitations. Furthermore, in response to applicant's arguments against the references individually (e.g. Jain allegedly not teaching adjusting a number of batches per epoch for reallocation), one cannot show nonobviousness by attacking references individually where the rejections are based on combinations of references. See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981); In re Merck & Co., 800 F.2d 1091, 231 USPQ 375 (Fed. Cir. 1986). In this case, Cui teaches adjusting a number of data items per epoch for reallocation and Jain is relied upon to teach a data item being a “batch” (see above cited portions). In response to applicant's argument that the examiner's conclusion of obviousness is based upon improper hindsight reasoning, it must be recognized that any judgment on obviousness is in a sense necessarily a reconstruction based upon hindsight reasoning. But so long as it takes into account only knowledge which was within the level of ordinary skill at the time the claimed invention was made, and does not include knowledge gleaned only from the applicant's disclosure, such a reconstruction is proper. See In re McLaughlin, 443 F.2d 1392, 170 USPQ 209 (CCPA 1971). In this case, Cui teaches adjusting a number of data items per epoch for reallocation and Jain is relied upon to teach a data item being a “batch” with a “batch” size (see above cited portions). This can be for determining an “optimum number of batches that need to be selected, based on…processing time per batch” (e.g. Jain, in paragraph 58). The combination also amounts to a simple substitution that yields predictable results [e.g. see KSR Int'l Co v. Teleflex Inc., 550 US 398,82 USPQ2d 1385,1396 (U.S. 2007) and MPEP 2143(B)]; substituting a data item with a data batch. As such, the combination teaches the claimed features.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1-3, 5-9, 11-13, 15-18, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Cui et al. (US 20200042362 A1) in view of Jain et al. (US 20220172258 A1).
As per independent claim 1, Cui teaches a method, performed by a server, of performing distributed learning, the method comprising:
building a computer cluster by selecting worker nodes configured to perform distributed learning, from among a plurality of nodes, wherein nodes in the computer cluster comprise the server that is a master node and the worker nodes (e.g. in paragraphs 19 and 75, “distributed computing system 100 comprises... The parameter server nodes 110-1, 110-2,…, 110-S manage a respective set of globally shared model parameters 112-1, 112-2,…, 112-S [i.e. master node]. The worker nodes 130-1, 130-2,…, 130-N comprise respective accelerator devices 132-1, 132-2,…, 132-N (collectively, accelerator devices 132)… computing resource scheduling and provisioning module 642 can implement any suitable method or protocol for selecting, allocating, scheduling and provisioning one or more GPU server nodes and associated accelerator resources (e.g., GPU devices) for executing HPC workloads associated with client service requests”);
determining data subsets by splitting a training dataset wherein each of the data subsets corresponds to each of the nodes in the computer cluster (e.g. in paragraph 47, “partitions the current mini-batch dataset 390 into a plurality of sub-batch datasets 390-1, 390-2, 390-3, and 390-4. The sub-batch datasets 390-1, 390-2, 390-3, and 390-4 are copied to the respective accelerators 320-1, 320-2, 320-3, and 320-4”) and includes a number N of data items, wherein N is an integer of 1 or more (e.g. in paragraph 54, “for…128 samples, and N=4 accelerators, the initial mini-batch dataset would be partioned into four (4) sub-batch datasets, each with [data items] of 32 (i.e., 128/4), i.e., 32:32:32:32”, i.e. “N”), and each of the data items has an equal number of data samples corresponding to a item size (e.g. in paragraph 54, each data item has 1 sample);
obtaining training results from the nodes in the computer cluster by training each artificial intelligence (Al) model stored in each of the nodes in the computer cluster based on each of the data subsets (e.g. in paragraph 47, “accelerators 320-1, 320-2, 320-3, and 320-4 execute a model training task by processing the respective sub-batch datasets 390-1, 390-2, 390-3, and 390-4… compute gradients (G) using the respective sub-batch datasets”);
updating weights of an Al model stored in the server based on the training results (e.g. in paragraph 47, “parameter server 330 updates the model weights using the gradients received from the accelerators”);
identifying, with respect to each of the nodes in the computer cluster, an operation time taken for each of the nodes in the computer cluster to perform training (e.g. in paragraph 5, “determining a job completion time for each of the accelerator resources to complete processing of the corresponding one of the sub-batch datasets of the initial mini-batch dataset”); and
adjusting, for each epoch, for each of the nodes in the computer cluster, a number N of data items included in each of the data subsets while maintaining the item size of the data items constant, based on the operation time of each of the nodes in the computer cluster (e.g. in paragraphs 5, 48-49, 50, and 54, “epoch iteration is then repeated… determine an optimal job partition ratio for partitioning a mini-batch dataset into sub-batch datasets for processing by the accelerator resources 320. The self-adaptive batch dataset partitioning is performed to rebalance the processing loads among the accelerator resources… determine (through an iterative load balancing process) that an optimal job partition ratio of 48:48:16:16 among the accelerators”, e.g. adjusted to 48 data items of size 1 for accelerator 320-1),
but does not specifically teach wherein data items include data batches and item size includes batch size.
However, Jain teaches data items including data batches having an item size including a batch size (e.g. in paragraph 58, “each batch may represent equal distribution of data… equal size batches may be formed from large data pool… identify optimum number of batches that need to be selected, based on…processing time per batch”). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Cui to include the teachings of Jain because one of ordinary skill in the art would have recognized the benefit of optimizing a number of batches (also amounts to a simple substitution that yields predictable results [e.g. see KSR Int'l Co v. Teleflex Inc., 550 US 398,82 USPQ2d 1385,1396 (U.S. 2007) and MPEP 2143(B)]). See also response to arguments above.
As per claim 2, the rejection of claim 1 is incorporated and the combination further teaches wherein the obtaining of the training results from the nodes in the computer cluster comprises: obtaining, from the first worker node, a first training result generated based on a first worker node among the worker nodes training an Al model stored in the first worker node using a first data subset corresponding to the first worker node (e.g. Cui, in paragraph 47, “accelerators 320-1 [i.e. first worker node], 320-2, 320-3, and 320-4 execute a model training task by processing the respective sub-batch datasets 390-1, 390-2, 390-3, and 390-4... compute gradients (G) using the respective sub-batch datasets 390-1, 390-2, 390-3, and 390-4 and send the processing results (e.g., gradients G) to the parameter server… their local model (e.g., copy of model computation graph 375)”); and obtaining, from the second worker node, a second training result generated based on a second worker node from among the worker nodes training an Al model stored in the second worker node using a second data subset corresponding to the second worker node (e.g. Cui, in paragraph 47, “accelerators 320-1, 320-2 [i.e. second worker node], 320-3, and 320-4 execute a model training task by processing the respective sub-batch datasets 390-1, 390-2, 390-3, and 390-4... compute gradients (G) using the respective sub-batch datasets 390-1, 390-2, 390-3, and 390-4 and send the processing results (e.g., gradients G) to the parameter server… their local model (e.g., copy of model computation graph 375)”).
As per claim 3, the rejection of claim 1 is incorporated and the combination further teaches performing next training for each Al model stored in each of the nodes in the computer cluster using each of the nodes in the computer cluster, wherein the nodes in the computer cluster perform the next training using the data subsets in which the number of data batches has been adjusted (e.g. Cui, in paragraphs 5, 47-49, and 59, “update their local model (e.g., copy of model computation graph 375) with the updated weights… determine an optimal job partition ratio for partitioning a mini-batch dataset into sub-batch datasets for processing by the accelerator resources 320. The self-adaptive batch dataset partitioning is performed to rebalance the processing loads among the accelerator resources… For the next mini-batch iteration, the self-adaptive batch dataset partitioning control module will partition the mini-batch dataset into sub-batch datasets with adjusted [number of data items] based on the adjusted job partition ratio”; Jain, in paragraph 58, “each batch may represent equal distribution of data… equal size batches may be formed from large data pool… identify optimum number of batches that need to be selected, based on…processing time per batch”).
As per claim 5, the rejection of claim 1 is incorporated and the combination further teaches determining an average operation time of the nodes in the computer cluster (e.g. Cui, in paragraph 56, “SD (σ) for the job completion times T.sub.i can be determined [which is a comparison to] a mean (or average) of all the job completion times”), wherein the adjusting of the number of data batches included in each of the data subsets comprises: comparing an average operation time of the nodes in the computer cluster with the operation time of each of the nodes in the computer cluster and adjusting the number of data batches included in each of the data subsets based on a result of the comparing (e.g. Cui, in paragraphs 63 and 65, “job completion times T.sub.i of the accelerators A1, A2, and A3 in the second mini-batch iteration 502 to be 3.15 sec, 4.61 sec, and 3.39 sec, which results in a SD=0.63924 for the job completion times in the second mini-batch iteration… Since SD>L.sub.0 in the second mini-batch iteration 502,…utilizes the partition ratio adjustment value K.sub.0=4 to adjust the job partition load between the second and first accelerators A2 and A1 which are found to have the slowest and fastest job completion times… job partition (size of sub-batch dataset) for the second accelerator A2 is further reduced by K.sub.0=4 samples, while the job partition ([number of data items] of sub-batch dataset) for the first accelerator A1 is increased by K.sub.0=4 samples” and figure 5; Jain, in paragraph 58, “each batch may represent equal distribution of data… equal size batches may be formed from large data pool… identify optimum number of batches that need to be selected, based on…processing time per batch”).
As per claim 6, the rejection of claim 5 is incorporated and the combination further teaches wherein the adjusting of the number of data batches included in each of the data subsets based on the comparing comprises adjusting the number of data batches included in each of the data subsets based on the result of the comparing being greater than or equal to or a threshold value (e.g. Cui, in paragraphs 57 and 65, “compared with the pre-specified SD threshold value L.sub.0… Since SD>L.sub.0 in the second mini-batch iteration 502, the control process utilizes the partition ratio adjustment value K.sub.0=4 to adjust the job partition load [number of data items] between the second and first accelerators”; Jain, in paragraph 58, “each batch may represent equal distribution of data… equal size batches may be formed from large data pool… identify optimum number of batches that need to be selected, based on…processing time per batch”).
As per claim 7, the rejection of claim 4 is incorporated and the combination further teaches wherein the adjusting of the number of data included in each of the data subsets comprises adjusting some data of a data subset corresponding to a node with an operation time longer than the average operation time to be included in a data subset corresponding to a node with an operation time shorter than the average operation time (e.g. Cui, in paragraphs 56 and 63, “a mean (or average) of all the job completion times… adjust the job partition load [number of data items] between the second and third accelerators A2 and A3 which are found to have the slowest and fastest job completion times, respectively. In particular, in an exemplary embodiment, the job partition ([number of data items] of sub-batch dataset) for the second accelerator A2 is reduced by K.sub.0=4 samples, while the job partition ([number of data items] of sub-batch dataset) for the third accelerator A3 is increased by K.sub.0=4 samples” and figure 5 items 501-502 showing data taken from node A2 with longer than average time (about 4s) to be included in A3 with less that average time; Jain, in paragraph 58, “each batch may represent equal distribution of data… equal size batches may be formed from large data pool… identify optimum number of batches that need to be selected, based on…processing time per batch”).
As per claim 8, the rejection of claim 1 is incorporated and the combination further teaches wherein the determining of the data subsets comprises splitting the training dataset wherein a number of data batches in a data subset corresponding to the master node is less than a number of data batches in each of the data subsets corresponding to the worker nodes (e.g. Cui, in paragraph 47, “partitions the current mini-batch dataset 390 into a plurality of sub-batch datasets 390-1, 390-2, 390-3, and 390-4. The sub-batch datasets 390-1, 390-2, 390-3, and 390-4 are copied to the respective accelerators 320-1, 320-2, 320-3, and 320-4”, i.e. master node has a subset of 0 as the data items are split to worker nodes; Jain, in paragraph 58, “each batch may represent equal distribution of data… equal size batches may be formed from large data pool… identify optimum number of batches that need to be selected, based on…processing time per batch”).
As per claim 9, the rejection of claim 1 is incorporated and the combination further teaches wherein the adjusting of the number of data batches included in each of the data subsets comprises adjusting some data batches of a data subset corresponding to a node having a longest operation time among the nodes in the computer cluster to be included in the data subsets corresponding to the other nodes in the computer cluster (e.g. Cui, in paragraphs 63 and 65, “adjust the job partition load [number of data items] between the second and first accelerators A2 and A1 which are found to have the slowest and fastest job completion times, respectively… the job partition ([number of data items] of sub-batch dataset) for the second accelerator A2 is further reduced by K.sub.0=4 samples, while the job partition ([number of data items] of sub-batch dataset) for the first accelerator A1 is increased by K.sub.0=4 samples” and figure 5 items 501-503; Jain, in paragraph 58, “each batch may represent equal distribution of data… equal size batches may be formed from large data pool… identify optimum number of batches that need to be selected, based on…processing time per batch”).
Claims 11-13 and 15-18 are the server claims corresponding to method claims 1-3, 5, and 7-9, and are rejected under the same reasons set forth and the combination further teaches a communication interface comprising communication circuitry (e.g. Cui, in paragraph 33, “network interface circuitry”); a memory storing one or more instructions (e.g. Cui, in paragraphs 33-35 and 39, “the system memory 210, the storage resources 260, and other local storage and off-infrastructure storage media… software modules that are persistently stored in the local storage resources and loaded into the system memory 210 resources”); at least one processor, comprising processing circuitry, configured to execute the one or more instructions stored in the memory and the at least one processor is individually and/or collectively configured based on executing the one or more instructions to perform the method (e.g. Cui, in paragraphs 33-35, “one or more types of hardware processors that are configured to process program instructions ”)
Claims 20 is the medium claim corresponding to method claim 1, and is rejected under the same reasons set forth and the combination further teaches a non-transitory computer-readable recording medium having recorded thereon an executable program for instructing a computer to perform (e.g. Cui, in paragraphs 33-35 and 39, “the system memory 210, the storage resources 260, and other local storage and off-infrastructure storage media… software modules that are persistently stored in the local storage resources and loaded into the system memory 210 resources”).
Claims 10 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Cui et al. (US 20200042362 A1) in view of Jain et al. (US 20220172258 A1) and further in view of Hasha et al. (US 20080288646 A1).
As per claim 10, the rejection of claim 1 is incorporated and the combination further teaches selecting one or more worker nodes from among the nodes in the computer cluster, based on the operation time of each of the nodes in the computer cluster (e.g. Cui, in paragraphs 63 and 65, “adjust the job partition load between the second and third accelerators A2 and A3 which are found to have the slowest and fastest job completion times, respectively… computing resource scheduling and provisioning module 642 can implement any suitable method or protocol for selecting, allocating, scheduling and provisioning one or more GPU server nodes and associated accelerator resources (e.g., GPU devices) for executing HPC workloads associated with client service requests…depending on various factors”), but does not specifically teach removing the selected one or more worker nodes from the computer cluster and incorporating one or more other nodes into the computer cluster as worker nodes. However, Hasha teaches selecting one or more nodes from a computer cluster based on a time of each node, removing the selected one or more nodes from a computer cluster and incorporating one or more other nodes into the computer cluster as worker nodes (e.g. in paragraphs 329, 365, 405, and 487, “an act of the monitor node establishing a monitor side time-to-die time based on the monitor side time-to-live duration value… remove failed nodes from failed node list 1947 after appropriate periods of time… Rings can be reconfigured from time to time, such as, for example, when a new node joins a ring or when an existing node departs a ring…as a result of node monitoring… secondary nodes”). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of the combination to include the teachings of Hasha because one of ordinary skill in the art would have recognized the benefit of improving efficiency based on relevant monitoring.
Claim 19 is the server claim corresponding to method claim 10, and is rejected under the same reasons set forth.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
For example,
Prakash et al. (US 20190220703 A1) teaches “the load balancing policy may define criteria to be used by the MEC system 200 for determining threshold criteria or a desired level of reliability for selecting a particular edge compute node 101, 201 to perform computational tasks β. In one example, the threshold criteria may aim to “evenly” distributing training data to available edge compute nodes 101, 201 so that individual desired epoch times for computing each partial at each edge compute node 101, 201 are the same or as close as possible” (e.g. in paragraph 43).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to WILLIAM WONG whose telephone number is (571)270-1399. The examiner can normally be reached Monday-Friday 9am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, TAMARA KYLE can be reached at (571)272-4241. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/W.W/Examiner, Art Unit 2144 04/02/2026
/TAMARA T KYLE/Supervisory Patent Examiner, Art Unit 2144
Prosecution Insights