Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Patel et al. (U.S. 2024/0273346) in view of Bernat (U.S. 2019/0068693).
Re claim 1, Patel et al. disclose in Figures 1-10 a system (e.g. abstract and Figure 2) comprising: a plurality of hardware accelerators including at least one mixture-of-experts (MoE) layer having multiple experts (e.g. Figures 2-4 and paragraphs [0001-0002, 0058-0059 and 0101] wherein multiple experts X are within the MoE layers); and wherein at least one expert of the multiple experts is offloaded from the plurality of hardware accelerators to another accelerators (e.g. abstract, Figures 4-6 and paragraphs [0021, 0079, and 0083-0084] wherein the experts are redistributed to different accelerators).
Patel et al. fail to disclose a plurality of network interface cards (NICs) coupled to the plurality of hardware accelerators and the another accelerators are the plurality of NICs. However, Bernat discloses in Figures 1-21 a plurality of network interface cards (NICs) coupled to the plurality of hardware accelerators and the another accelerators are the plurality of NICs (e.g. abstract, Figures 16 and 18-19, and paragraphs [0103, 0105 and 0123-0124] wherein NIC can be considered as another accelerator for receiving the offload from another processing devices).
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of claimed invention to add a plurality of network interface cards (NICs) coupled to the plurality of hardware accelerators and the another accelerators are the plurality of NICs as seen in Bernat’s invention into Patel et al.’s invention because it would enable to increase the efficiency and durability of the system.
Re claim 2, Patel et al. in view of Bernat disclose the plurality of hardware accelerators are graphics processing units (GPUs) (e.g. Patel et al. – paragraphs [0002, 0023, and 0025] and Bernat – paragraphs [0057 and 0063] wherein GPUs are highly considered as accelerators for performing specialized functions).
Re claim 3, Patel et al. in view of Bernat disclose all of the multiple experts are offloaded from the plurality of hardware accelerators to the plurality of NICs (e.g. Patel et al. – paragraphs [0021, 0063, 0079, and 0083-0084] wherein the experts can be redistributed to different accelerators and Bernat - abstract, Figures 16 and 18-19, and paragraphs [0103, 0105 and 0123-0124] wherein NIC can be considered as another accelerator for receiving the offload from another processing devices).
Re claim 4, Patel et al. in view of Bernat disclose a subset of the multiple experts are selectively offloaded from the plurality of hardware accelerators to the plurality of NICs (e.g. Patel et al. – paragraphs [0021, 0063, 0079, and 0083-0084] wherein the experts can be redistributed to different accelerators and Bernat - abstract, Figures 16 and 18-19, and paragraphs [0103, 0105 and 0123-0124] wherein NIC can be considered as another accelerator for receiving the offload from another processing devices).
Re claim 5, Patel et al. in view of Bernat disclose the subset of the multiple experts are selected based on memory and computational capacity available on the plurality of NICs (e.g. Patel et al. – paragraphs [0036, 0063, and 0072] and Bernat - paragraphs [0062, 0081-0083, and 0103] as offloading/redistributing the experts to balance resources).
Re claim 6, Patel et al. in view of Bernat disclose the multiple experts are designated as either hot experts or cold experts (e.g. Patel et al. – Figures 5-6 and paragraphs [0033 and 0044-0046] with different level of experts).
Re claim 7, Patel et al. in view of Bernat disclose the cold experts are offloaded from the plurality of hardware accelerators to the plurality of NICs (e.g. Patel et al. – Figures 5-6 and paragraphs [0021, 0064, and 0072-0074] where loads/experts are balance/redistribute and Bernat - abstract, Figures 16 and 18-19, and paragraphs [0103, 0105 and 0123-0124] wherein NIC can be considered as another accelerator for receiving the offload from another processing devices).
Re claim 8, Patel et al. in view of Bernat disclose the hot experts are duplicated for each of the plurality of hardware accelerators (e.g. Patel et al. – paragraphs [0076-0077 and 0095-0096] for improve processing).
Re claim 9, Patel et al. in view of Bernat disclose a portion of the multiple experts are designated as the hot experts by gathering expert temperature statistics to create a global view of expert temperatures (e.g. Patel et al. – paragraphs [0025-0026 and 0056-0058]).
Re claim 10, Patel et al. in view of Bernat disclose at least one expert of the multiple experts of the MoE layer is sharded across the plurality of hardware accelerators (e.g. Patel et al. – Figures 7 and paragraphs [0057 and 0085-0088] where experts are sharded among accelerators).
Re claim 11, Patel et al. in view of Bernat disclose a subset of the multiple experts designated as sharded experts are offloaded to the plurality of NICs (e.g. Patel et al. – Figures 7 and paragraphs [0057 and 0085-0088] where experts are sharded among accelerators and Bernat - abstract, Figures 16 and 18-19, and paragraphs [0103, 0105 and 0123-0124] wherein NIC can be considered as another accelerator for receiving the offload from another processing devices).
Re claim 12, Patel et al. disclose in Figures 1-10 a method (e.g. abstract and Figure 2) comprising: providing at least one mixture-of-experts (MoE) layer having multiple experts to a plurality of hardware accelerators coupled to another accelerators (e.g. Figures 2-4 and paragraphs [0001-0002, 0058-0059 and 0101] wherein multiple experts X are within the MoE layers); and offloading at least one expert of the multiple experts from the plurality of hardware accelerators to the another accelerators (e.g. abstract, Figures 4-6 and paragraphs [0021, 0079, and 0083-0084] wherein the experts are redistributed to different accelerators).
Patel et al. fail to disclose coupling another accelerators as to a plurality of network interface cards (NICs). However, Bernat discloses in Figures 1-21 coupling another accelerators as to a plurality of network interface cards (NICs) (e.g. abstract, Figures 16 and 18-19, and paragraphs [0103, 0105 and 0123-0124] wherein NIC can be considered as another accelerator for receiving the offload from another processing devices).
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of claimed invention to add a coupling another accelerators as to a plurality of network interface cards (NICs) as seen in Bernat’s invention into Patel et al.’s invention because it would enable to increase the efficiency and durability of the system.
Re claim 13, it is a method claim having similar limitations as cited in claim 2. Thus, claim 13 is also rejected under the same rationale as cited in the rejection of claim 2 above.
Re claim 14, it is a method claim having similar limitations as cited in claim 4. Thus, claim 14 is also rejected under the same rationale as cited in the rejection of claim 4 above.
Re claim 15, it is a method claim having similar limitations as cited in claim 5. Thus, claim 15 is also rejected under the same rationale as cited in the rejection of claim 5 above.
Re claim 16, it is a method claim having similar limitations as cited in claim 6. Thus, claim 16 is also rejected under the same rationale as cited in the rejection of claim 6 above.
Re claim 17, it is a method claim having similar limitations as cited in claim 7. Thus, claim 17 is also rejected under the same rationale as cited in the rejection of claim 7 above.
Re claim 18, it is a method claim having similar limitations as cited in claim 8. Thus, claim 18 is also rejected under the same rationale as cited in the rejection of claim 8 above.
Re claim 19, Patel et al. disclose in Figures 1-10 a system (e.g. abstract and Figure 2) comprising: a plurality of hardware accelerators (e.g. Figures 4 and paragraphs [0027-0028] with hardware accelerators as GPU); and a neural network architecture (e.g. paragraphs [0026-027]) including multiple experts distributed across the plurality of hardware accelerators (e.g. Figures 2-4 and paragraphs [0001-0002, 0058-0059 and 0101] wherein multiple experts X are within the MoE layers), wherein at least one expert of the multiple experts is offloaded from the plurality of hardware accelerators to another accelerators (e.g. abstract, Figures 4-6 and paragraphs [0021, 0079, and 0083-0084] wherein the experts are redistributed to different accelerators).
Patel et al. fail to disclose the another accelerators as a plurality of network interface cards (NICs). However, Bernat discloses in Figures 1-21 the another accelerators as a plurality of network interface cards (NICs) (e.g. abstract, Figures 16 and 18-19, and paragraphs [0103, 0105 and 0123-0124] wherein NIC can be considered as another accelerator for receiving the offload from another processing devices).
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of claimed invention to add the another accelerators as a plurality of network interface cards (NICs) as seen in Bernat’s invention into Patel et al.’s invention because it would enable to increase the efficiency and durability of the system.
Re claim 20, it is a system claim having similar limitations as cited in claims 6, 7, and 8. Thus, claim 20 is also rejected under the same rationale as cited in the rejections of claims 6, 7, and 8 above.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
U.S. Patent Application Publication No. 2026/0156176 discloses an accelerator offload device includes: a traffic collection part that collects traffic information; a device emulation part that conceals connection destination switching so as to pretend that an application is communicating with an accelerator.
U.S. Patent Application Publication No. 2023/0385118 discloses an elective execution of workloads using hardware accelerators includeing a client application submitting a command for execution of a workload directly to a hardware accelerator, where the command includes an indication of a performance expectation from the hardware accelerator.
U.S. Patent Application Publication No. 2023/0281052 discloses strategically harvesting untapped compute capacity of hardware accelerators to manage transient workload spikes at computing systems.
U.S. Patent No. 12,289,239 discloses a dynamic load balancing for multi-core computing environments includes a first and a plurality of second cores of a processor, and circuitry in a die of the processor separate from the first and the second cores, the circuitry to enqueue identifiers in one or more queues in the circuitry associated with respective ones of data packets of a packet flow.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Chat C Do whose telephone number is (571)272-3721. The examiner can normally be reached {M - Th} 4:30am - 2:30pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Dede Zecher can be reached at 571-272-0800. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Chat C Do/Supervisory Patent Examiner, Art Unit 2193
Prosecution Insights