Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The filing of a terminal disclaimer by itself is not a complete reply to a nonstatutory double patenting (NSDP) rejection. A complete reply requires that the terminal disclaimer be accompanied by a reply requesting reconsideration of the prior Office action. Even where the NSDP rejection is provisional the reply must be complete. See MPEP § 804, subsection I.B.1. For a reply to a non-final Office action, see 37 CFR 1.111(a). For a reply to final Office action, see 37 CFR 1.113(c). A request for reconsideration while not provided for in 37 CFR 1.113(c) may be filed after final for consideration. See MPEP §§ 706.07(e) and 714.13.
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer.
Claims 1 – 3, 5, and 6 are rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1, 3, 4, and 6 of U.S. Patent No. US-11425195-B1. Although the claims at issue are not identical, they are not patentably distinct from each other because each and every element of the above pending claims is anticipated by a corresponding element of the above patented claims, as is illustrated below.
Pending claim 1
A method comprising:
at each compute node of a plurality of compute nodes:
generating a local vector comprising values for a vector element subset in a common set of vector elements; and
sending a vector chunk comprising the values for the vector element subset out of a port of the compute node that is associated with the vector element
subset; and
at a compute-enabled switch:
receiving vector chunks over a plurality of switch ports, each compute node of the plurality of compute nodes connected to the compute-enabled switch via a
different port of the plurality of switch ports;
reducing the vector chunks into a single result chunk;
and sending the result chunk to each compute node of the plurality of compute nodes.
Patented claim 1
A method comprising:
at each compute node of a plurality of compute nodes:
generating . . . a local vector comprising values for a common set of vector elements; and
for each vector element subset of a plurality of subsets in the common set of vector elements, based on each generated local vector, sending a vector chunk comprising values for the vector element subset
. . .
at . . . compute-enabled switches:
receiving vector chunks over a plurality of switch ports, each compute node of the plurality of compute nodes connected to the compute-enabled switch via a different port of the plurality of switch ports;
reducing the vector chunks into a single result chunk, . . .
and sending the result chunk to each compute node of the plurality of compute nodes.
Pending claim 2
The method of Claim 1, wherein reducing the vector chunks comprises performing
reduction operations including one or more of: summation, averaging, multiplying,
selecting a minimum value, or selecting a maximum value.
Patented claim 3
The method of claim 1, wherein reducing the vector chunks comprises . . . performing one or more reduction operations . . . including one or more of summation, averaging, multiplying,
selecting a minimum value, or selecting a maximum value.
Pending claim 3
The method of Claim 1, wherein a plurality of computing processes at the plurality
of compute nodes belongs to a worker set executing a common distributed application.
Patented claim 4
The method of claim 1, wherein each compute process of the plurality of computing processes at each of the plurality of compute nodes belongs to a worker set executing a common distributed application . . .
Pending claim 5
The method of Claim 1, wherein
the local vector represents a local gradient computed from test results of a machine learning model.
Patented claim 6
The method of claim 1 . . . generating
the local vector comprises . . . a local gradient based on results of the testing, the local vector being the local gradient . . .
Pending claim 6
The method of Claim 1, wherein a result gradient is formed based at least in part on the single result chunk; wherein the result gradient is used to adjust parameters of a machine learning model.
Patented claim 6
The method of claim 1. . .
adjusting the parameters based on a result gradient formed by combining each result chunk generated by the plurality of compute-enabled switches, the result gradient having a single value for each vector element of the common set of vector elements.
Claims 4 and 11 – 16 are rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1, 3, 4, and 6 of U.S. Patent No. US-11425195-B1 (‘195 patent) in view of claims 7 and 11 of US-12353921-B2 (‘921 patent). Although the claims at issue are not identical, they are not patentably distinct from each other because each and every element of the above pending claims is anticipated by a corresponding element of a combination of the above patented claims.
Pending claim 4
The method of Claim 3, wherein the common distributed application represents one or more of: an artificial intelligence application, a machine learning application, or a computing application implementing one or more artificial neural networks.
Patented claim 7 (of ‘921)
The method of claim 5, wherein the common distributed application represents one or more of: deep learning applications, machine learning applications or artificial intelligence applications.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify claim 1 of the ‘195 patent with claim 7 of the ‘921 patent in order to better leverage the AI/ML/neural network training process in different but related computer intelligence applications.
Regarding pending claim 11, patented claim 1 anticipates said claim as illustrated below:
Pending claim 11
. . .
at each compute node of a plurality of compute nodes:
generating a local vector comprising values for a vector element subset in a common set of vector elements; and
sending a vector chunk comprising the values for the vector element subset out of a port of the compute node that is associated with the vector element
subset; and
at a compute-enabled switch:
receiving vector chunks over a plurality of switch ports, each compute node of the plurality of compute nodes connected to the compute-enabled switch via a
different port of the plurality of switch ports;
reducing the vector chunks into a single result chunk;
and sending the result chunk to each compute node of the plurality of compute nodes.
Patented claim 1
at each compute node of a plurality of compute nodes:
generating . . . a local vector comprising values for a common set of vector elements; and
for each vector element subset of a plurality of subsets in the common set of vector elements, based on each generated local vector, sending a vector chunk comprising values for the vector element subset
. . .
at . . . compute-enabled switches:
receiving vector chunks over a plurality of switch ports, each compute node of the plurality of compute nodes connected to the compute-enabled switch via a different port of the plurality of switch ports;
reducing the vector chunks into a single result chunk, . . .
and sending the result chunk to each compute node of the plurality of compute nodes.
The claim of the ‘195 patent fails to anticipate the system-based implementation of pending claim 11, and thus fails to show: a system comprising:
a plurality of compute nodes implemented by one or more hardware processors; and
a compute-enabled switch.
Claim 11 of the ‘921 patent shows a system comprising: a plurality of switch ports; and a plurality of compute nodes implemented by one or more hardware processors.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify claim 1 of the ‘195 patent with claim 11 of the ‘921 patent in order to apply the method in a hardware system, thus providing clear implementation details and hardware requirements for a full actualization and application of said method.
Regarding pending claim 12, claim 3 of the ‘195 patent anticipates said claim in the same manner claim 3 of the ‘195 patent anticipates pending claim 2.
Regarding pending claim 13, claim 4 of the ‘195 patent anticipates said claim in the same manner claim 3 of the ‘195 patent anticipates pending claim 3.
Regarding pending claim 15, claim 6 of the ‘195 patent anticipates said claim in the same manner claim 6 of the ‘195 patent anticipates pending claim 5.
Regarding pending claim 16, claim 6 of the ‘195 patent anticipates said claim in the same manner claim 6 of the ‘195 patent anticipates pending claim 6.
Claims 7, 8, and 9 are rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1 and 6 of U.S. Patent No. US-11425195-B1 (‘195 patent) in view of Luo (Luo, Liang, et al. “Parameter hub: a rack-scale parameter server for distributed deep neural network training.” Proceedings of the ACM Symposium on Cloud Computing. (Year: 2018)). Although the claims at issue are not identical, they are not patentably distinct from each other because each and every element of the above pending claims is anticipated by a corresponding element of the above combination.
Regarding pending claim 7, patented claims 1 and 6 of the ‘195 patent anticipates pending claim 6. The patented claims do not show wherein a result gradient is formed based at least in part on the single result chunk; wherein the result gradient is used to adjust parameters of a machine learning model.
Luo shows wherein a result gradient is formed based at least in part on the single result chunk; wherein the result gradient is used to adjust parameters of a machine learning model (pg. 42 section 2).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify the patented claims with the distributed neural network training and bottleneck prevention techniques of Luo in order to ensure the resultant model improves throughout the training process while also lowering the resources consumed by said training process.
Regarding pending claim 8, patented claim 1 and 6 anticipates pending claim 6. The patented claims do not show: wherein an error occurs in processing the vector chunks received from the plurality of compute nodes; wherein a message is sent to each compute nodes in the plurality of compute nodes to inform the error.
Luo shows: wherein an error occurs in processing the vector chunks received from the plurality of compute nodes; wherein a message is sent to each compute nodes in the plurality of compute nodes to inform the error (pg. 42 Section 2).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify the patented claims with the distributed neural network training and bottleneck prevention techniques of Luo in order to ensure the resultant model improves throughout the training process while also lowering the resources consumed by said training process.
Regarding pending claim 9, patented claim 1 and 6 of the ‘195 patent anticipates pending claim 6. The patented claims do not show: wherein each compute node in the plurality of compute nodes is implemented at least in part with one or more of: central processing units, graphics processing units, tensor processing units, floating point units, hardware accelerators, or other computing processor.
Luo shows: wherein each compute node in the plurality of compute nodes is implemented at least in part with one or more of: central processing units, graphics processing units, tensor processing units, floating point units, hardware accelerators, or other computing processor (pg. 46 L38-L58).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify the patented claims with the distributed neural network training and bottleneck prevention techniques of Luo in order to ensure the resultant model improves throughout the training process while also lowering the resources consumed by said training process.
Claims 17, 18, and 19 are rejected on the ground of nonstatutory double patenting as being unpatentable over claim 1 U.S. Patent No. US-11425195-B1 (‘195 patent) in view of claim 11 of US-12353921-B2 (‘921 patent) and Luo (Luo, Liang, et al. "Parameter hub: a rack-scale parameter server for distributed deep neural network training." Proceedings of the ACM Symposium on Cloud Computing. (Year: 2018)). Although the claims at issue are not identical, they are not patentably distinct from each other because each and every element of the above pending claims is anticipated by a corresponding element of a combination of the above combination.
Regarding pending claim 17, patented claim 1 and 6 of the ‘195 patent in view of claim 11 of the ‘921 patent anticipates pending claim 16. The patented claims do not show wherein a result gradient is formed based at least in part on the single result chunk; wherein the result gradient is used to adjust parameters of a machine learning model.
Luo shows wherein a result gradient is formed based at least in part on the single result chunk; wherein the result gradient is used to adjust parameters of a machine learning model (pg. 42 section 2).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify the patented claims with the distributed neural network training and bottleneck prevention techniques of Luo in order to ensure the resultant model improves throughout the training process while also lowering the resources consumed by said training process.
Regarding pending claim 18, patented claim 1 and 6 of the ‘195 patent in view of claim 11 of the ‘921 patent anticipates pending claim 16. The patented claims do not show: wherein an error occurs in processing the vector chunks received from the plurality of compute nodes; wherein a message is sent to each compute nodes in the plurality of compute nodes to inform the error.
Luo shows: wherein an error occurs in processing the vector chunks received from the plurality of compute nodes; wherein a message is sent to each compute nodes in the plurality of compute nodes to inform the error (pg. 42 Section 2).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify the patented claims with the distributed neural network training and bottleneck prevention techniques of Luo in order to ensure the resultant model improves throughout the training process while also lowering the resources consumed by said training process.
Regarding pending claim 19, patented claim 1 and 6 of the ‘195 patent in view of claim 11 of the ‘921 patent anticipates pending claim 16. The patented claims do not show: wherein each compute node in the plurality of compute nodes is implemented at least in part with one or more of: central processing units, graphics processing units, tensor processing units, floating point units, hardware accelerators, or other computing processor.
Luo shows: wherein each compute node in the plurality of compute nodes is implemented at least in part with one or more of: central processing units, graphics processing units, tensor processing units, floating point units, hardware accelerators, or other computing processor (pg. 46 L38-L58).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify the patented claims with the distributed neural network training and bottleneck prevention techniques of Luo in order to ensure the resultant model improves throughout the training process while also lowering the resources consumed by said training process.
Subject Matter Allowable over Prior Art
Claims 1 - 20 would be allowable, absent the pending Double Patenting rejections. In addition to the above cited Luo non-patent literature reference, the closet prior art of record includes: Matveev (US-20190156215-A1), illustrating the transition from a input vector (analogous to the claimed “local vector”) to an output vector (analogous to the claimed “result chunk”) ins Fig. 1,3, and 5. [3-4] and [9-10] discuss how forward and backward passes are used when training neural network models (similar to the subject matter of pending claims 7 and 8). How weights/biases are used in neural network training (similar to the subject matter of claim 9) is discussed in [12,25-28] and [31].
Graham (Graham, Richard L., et al. "Scalable hierarchical aggregation and reduction protocol (sharp) tm streaming-aggregation hardware design and evaluation." High Performance Computing: 35th International Conference, ISC High Performance 2020. (Year: 2020)), discussing techniques for distributing vector chunks to particular switch ports (Figs. 1A-1B, pg. 5 lines 20-24, Section 3.6, pgs. 8 - 9) when executing distributed ML-applications or training ML / DNN models (pg. 10 lines 15-21 and pg. 15 lines 13-29, discussing execution of DL-POLY, a distributed neural network application).
Sapio (Sapio, Amedeo, et al. "Scaling distributed machine learning with {In-Network} aggregation." 18th USENIX Symposium on Networked Systems Design and Implementation. (Year: 2019); Sapio was made of record via the 8/27/2025 IDS), who provides further discussion in the applicability of programmable switches and their use with ML frameworks (pg. 1, L19 – L38 and L51 – L55, R17 – R19). Both arithmetic and averaging operations are supported (pg. 1, R34-R36; pg. 4, L50 – L53).
Chen (US-20110173413-A1) who further discusses techniques for dividing input data for processing and combining the results for distribution in an switch-based, compute-node environment ([14,36-38]). Chung (US-10167800-B1) discusses, in a multi-CPU / GPU ML environment (col. 3 lines 10-15 and 60-65), use of distributed vectors to perform operations such as multiplication and addition (col. 4 lines 1 – 7 and lines 30-56). Node-to-node data exchange via couplings performed with switches is discussed (col. 6 lines 1 – 53).
Thakur (Thakur, Rajeev, Rolf Rabenseifner, and William Gropp. "Optimization of collective communication operations in MPICH." The International Journal of High Performance Computing Applications 19.1. (Year: 2005); Thakur was made of record via the 8/27/2025 IDS) provides an overview of multiple types of implementations of distributed data processing techniques, and their work, particularly the algorithm introduced by Rabenseifner, is noted by the above noted NPL reference Graham as being influential in the common distributed processing architectures utilized by the prior art of record.
Additionally, Sinz (Sinz, Fabian H., et al. "Engineering a less artificial intelligence." Neuron 103.6: 967-979. (Year: 2019)) provides relevant background teachings regarding AI/ML training highly relevant to the claimed subject matter.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JOHN M MACILWINEN whose telephone number is (571)272-9686. The examiner can normally be reached Monday - Friday, 9:00 - 5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Glenton B Burgess can be reached at (571) 272 - 3949. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
JOHN MACILWINEN
Primary Examiner
Art Unit 2442
/JOHN M MACILWINEN/ Primary Examiner, Art Unit 2454