Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
Claims 1-20 are presented for examination
As to claims 9-16, since applicant’s specification specifically equating server to computing device(s) [paragraph 40] and referred to interchangeably as server computer [paragraphs 43-44 and 47], the examiner is interpreting the claimed “deployment server” as computing hardware, hence, not 35 USC 101 rejection is warranted.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1, 5-6, 8-9, 13-14, 16-17 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by US PG Pub. 2018/0341525 to Gupta et al. (hereafter Gupta).
Gupta was cited in applicant’s IDS filed on 12/11/24.
As to claim 1, Gupta teaches the invention as claimed including a computer-implemented method for a dynamic routing system, comprising:
deploying a central processing unit (CPU)-bound instance of a machine learning model configured to run on a CPU;
deploying a graphics processing unit (GPU)-bound instance of the machine learning model configured to run on a GPU [loading a deep learning model onto worker node’s CPU or GPU, paragraphs 71-72];
receiving an inference request to invoke the machine learning model [assign tasks in a receive job to a CPU or GPU having a model or set of models loaded, paragraphs 70-72]; and
dynamically routing, by an inference gateway, the inference request to either the CPU-bound instance or the GPU-bound instance of the machine learning model based on status feedback and profile information associated with each processing unit, wherein the profile information includes at least one of the following: speed, throughput, latency, accuracy, rate of learning, energy efficiency, or any combination thereof [assignment/distribution of tasks via a job manager based in part on balancing load and minimal completion time, paragraphs 60, 75 and 96].
As to claim 5, Gupta teaches the invention as claimed including further comprising:
routing the inference request to the GPU-bound instance based on determining that the inference request fits into an existing batch for processing by the GPU based on a memory and core usage footprint of the machine learning model [GPU-bound task types being fed to GPUs for execution in batch mode in fully utilizing the GPU, paragraphs 26-27, 53, 76-77 and 97-98; paragraph 38, lines 9-12; paragraph 73, lines 1-11];
routing the inference request to an available CPU-bound instance based on determining that the GPU-bound instance is unavailable; or
deploying a new GPU-bound instance or a new CPU-bound instance based on a predetermined time threshold, based on determining that available hardware resources exceed a capacity reserved for inference requests, or based on determining that the GPU-bound instance and the CPU-bound instance are not available to run the machine learning model.
As to claim 6, Gupta teaches the invention as claimed including wherein the inference gateway further includes a gateway component and a router component for performing steps comprising: receiving the inference request from at least one inference source; and routing the inference request to one or more instances of the machine learning model based on determining a most suitable processing unit [job manager receives a job, divide job into tasks and assign tasks with deep learning model, paragraphs 11, 23, 70 and 94; Fig. 1A and corresponding text].
As to claim 8, Gupta teaches the invention as claimed including wherein the inference gateway includes at least one of a user interface for facilitating entry of the inference request or at least one interface (API) for direct and uniform input of inference requests from disparate systems [wrapper/interface layer allowing IMC client to communicate with the system, paragraphs 69 and 73-74].
As to claims 9, 13-14 and 16, Gupta teaches the computer-implemented method for a dynamic routing system as recited in claims 1, 5-6 and 8, therefore Gupta teaches the system for implementing the method. Furthermore, Gupta teaches at least one deployment server [system architecture including various nodes having CPUs and memory, paragraphs 41-45].
As to claim 17, Gupta teaches the computer-implemented method for a dynamic routing system as recited in claim 1, therefore Gupta teaches the computer program product comprising at least one non-transitory computer-readable medium including one or more instructions that, when executed by at least one processor, cause the at least one processor to implement the method.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 7 and 15 is/are rejected under 35 U.S.C. 103 as being unpatentable over Gupta as applied to claims 1 and 9 above.
As to claim 7, Gupta does not specifically teach wherein the inference gateway is implemented in hardware as an application-specific integrated circuit (ASIC) or field-programmable gate array (FPGA) to process inference requests. However, Gupta teaches a job manager implemented in hardware as CPU in communication with GPU(s) [paragraphs 45, 48, 67-68]. Furthermore, integrated circuits such as FPGA and ASIC are well known in the art. It would have been obvious before the effective filing date of the claimed invention to have implemented an inference gateway in known integrated circuits as a matter of design choice.
As to claim 15, this claim is rejected for the same reason as claim 7 above.
Allowable Subject Matter
Claims 2-4, 10-12 and 18-20 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
The following is a statement of reasons for the indication of allowable subject matter:
The prior arts of record when taken individually or in combination do not expressly teach or render obvious, in the context of the claims taken as a whole, the invention as recited in claims 2-4, 10-12 and 18-20.
Gupta disclosed a deep learning model configured to assign tasks to a CPU or a GPU such that the assignment can be a predetermined setting such as detecting and classifying request/task type [paragraphs 39-40, 50-52 and 70]. US PG Pub. 2019/0156247 to Faulhaber teaches a model selector that dynamically route inference requests to particular ML models based at least in part on characteristics of the inference request, ML models performance such as accuracy [paragraphs 19, 32, 34, 52, 72]; machine to train a machine learning model being GPU or CPU instance type [paragraph 77]. Faulhaber was cited in applicant’s IDS filed on 12/11/24. The prior arts of record when taken individually or in combination do not expressly teach or render obvious the limitations “dynamically modifying the static assignment based on feedback from the CPU or GPU” as a whole as recited in claims 2, 10 and 18.
Neither a reference uncovered that would have provided a basis of evidence for asserting a motivation, nor one of ordinary skilled in the art before the effective filing date of the claimed invention, knowing the teaching of the prior arts of record would have combined them to arrive at the present invention as recited in the context of claims 2-4, 10-12 and 18-20 as a whole.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to QING YUAN WU whose telephone number is (571)272-3776. The examiner can normally be reached M-F 9AM-6PM EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Lewis Bullock can be reached on 571-272-3759. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/QING YUAN WU/Primary Examiner, Art Unit 2199
Prosecution Insights