DETAILED ACTION
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This office action is in response to application filed on 7/7/2023, claiming priority based on KR 10-2022-0084124 dated 7/2/2022.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1-5, 13-17, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Choi et al. ("Multi-model machine learning inference serving with gpu spatial partitioning." arXiv preprint arXiv:2109.01611 (2021)), in view of Shen et al. (“Nexus: A GPU Cluster Engine for Accelerating DNN-Based video Analysis”, SOSP ’19, Oct 27, 2019).
As for claim 1, Choi teaches an electronic device comprising one or more processors and a memory storing instructions configured to cause the one or more processors (Pg. 1, Abstract, “…the scheduler …”, Pg. 7, Section 5 – Implementation, “…multi-GPU inference serving system …front end inference scheduler and backend inference executors…these software modules are instantiated…” and Pg. 8, Section 6.1 – Methodology, “we use a multi-GPU inference server, which is equipped with a Xeon E5-2630 v4 CPU …and 192 GB of host DRAM memory…”) to:
based on resource utilization of partitions of different sizes, for batches of different sizes input to the partitions of different sizes, determine correspondences between the sizes of the batches and the sizes of the partitions (Pg. 3, Section 3.1, “…for each batch size, we sweep through the increasing fractions of GPU compute resource (i.e., gpu-let size), which range from 20% to 100% to observe how efficiently the inference execution utilizes the additional compute resource…”);
determine numbers of partitions for the respective determined sizes of the partitions based on number of the batches and the correspondences between the sizes of the batches and the sizes of the partitions (Pg. 6, Table 2, Section 4.3 – “Scheduling Algorithm”, and “Algorithm 1”. “p Partition size … b Batch size … gpulet_size Actual partition size of gpu_let…”, “…peff…preq…pideal…gpulet…FindBestFit (p_ideal…sort all gpulet by gpulet_size in ascending order…”, and “for each model…profiled execution latency of batch b on partition size p…elastic partitioning for models that are running …the incoming request rates …are tracked …the scheduler allocates one or more gpu-lets so that they can handle the entirety of request rate…” based on correspondence (relationship) between batch with partition size, represented by execution latency, and the size of the batches represented by incoming rate, determining the peff, preq and pideal to find the best fit one or more gpu-lets. The number of the one or more gpu-lets determined is understood as the number of partitions); and
partition the accelerator into partitions based on the determined numbers of the respective sizes of the partitions (Pg. 7, “…the scheduler allocates one or more gpu-lets so that they can handle the entirety of request…” in view of Pg. 7, “…determining ideal gpu-let size…” and Pg. 6, “algorithm 1” teaching finding most cost effective gpulet size, required size to avoid SLO violation and iteratively find ideal partition size matching gpu-lets that are partitioned accelerators) .
While the claims nor the specification limits the interpretation of “number of the batches” and the incoming rate for a model can reasonably be understood as number of batches for the model. Nevertheless, in the interest of compact prosecution, examiner note Choi does not explicitly teach where the limitation requires frequency information by batch size rather than the rate of requests coming in.
Shen teaches a method of GPU accelerated neural network learning execution including determining GPU resource allocation based on number of the batch sizes (Pg. 328, “…for session Si….with a batch size b, ….latency for any given request is 2Lki(b)…Denote batch size Bi as the maximum value for b that meats the constraint 2Lki(b) <= Li, therefore…maximal throughput, denoted by Ti, for session Si on a single GPU is ….the number of GPU nodes we allocate to execute just Si request s is n=[Ri/Ti]…” teaching for a given batch size and rate of requests, it determines number of batches needed to process the requests for a session at the given request rate that equals to n in the exemplary calculation. Alternatively, system determines, based on how many batches of each batch size fits into a partition, to determine how many partitions are needed per duty cycle (Pg. 326, Fig. 2), which can also be understood as utilizing information on number of the batch sizes as claimed). This known technique is applicable to the system of Choi as they both share characteristics and capabilities, namely, they are directed to batch based execution of neural network learning workloads on GPU accelerators.
One of ordinary skill in the art before the effective filing date of the application would have recognized that applying the known technique of Shen would have yielded predictable results and resulted in an improved system. It would have been recognized that applying the technique of Shen to the teachings of Choi would have yielded predictable results because the level of ordinary skill in the art demonstrated by the references applied shows the ability to incorporate such task scheduling features into similar systems. Further, applying determining GPU resource allocation based on number of the batch sizes to Choi with determine numbers of partitions for the respective determined sizes of the partitions based on number of the batches and the correspondences between the sizes of the batches and the sizes of the partitions accordingly, would have been recognized by those of ordinary skill in the art as resulting in an improved system that would allow improved execution efficiency for deep neural networks workloads executions on GPUs (Shen, Abstract)
As for claim 2, Choi also teaches the resource utilization is determined based on a neural network (NN) model executed by the partitions (Pg. 2, “…each ML model, its computational characteristics are measured and registered…based on …information…the scheduler routes requests where throughput would be maximized while satisfying the SLO constraints.” And Pg. 3, “…sweep through the increasing fractions of GPU compute resource….to observe how efficiently the inference execution utilizes the additional compute resource…”).
As for claim 3, Choi also teaches determine the correspondences between the sizes of the batches and the sizes of the partitions based on a size of a batch corresponding to when the resource utilization of a partition corresponds to a preset threshold resource utilization (Pg. 7, “…Determining ideal gpu-let size…the knee, where the curvature has the local maximum, implies the most cost-effective sweet spot…uses the gpulet size at the knee as peff…” ).
In addition, Shen also teaches the same limitation (Pg. 329, “greedy scheduling …largest batch size and the corresponding duty cycle in order to meet the throughput and SLO needs…”). Rationale to combine same as claim 1 above.
As for claim 4, Choi also teaches determine the numbers of partitions of respective sizes according to throughput based on the sizes of the partitions and the sizes of the batches (Pg. 6, “Algorithm 1” and Section 4.3 – scheduling aogirthm, teaching L(b,p) execution latency of batch b on partition size p, to determine the gpulets of gpulet.size to allocate. In addition, Pg. 7 teaches “…allocates the one or more gpu-lets so that they can handle the entirety of request rate…” here, throughput can be understood to correlate with the request rate that is processible by the gpulets allocated.).
In addition, Shen explicitly teaches partition of respective sizes according to throughput based on the sizes of the batches (Section 4.1, “batch execution latency and throughput at different batch sizes …batching profile…”). Rationale to combine same as claim 1 above.
As for claim 5, Choi also teaches determine the numbers of partitions of respective sizes such that the number of batches by size corresponds to the number of batches by size which are processible based on the numbers of partitions of respective sizes (Pg. 6-7, “…scheduler allocates one or more gpu-lets so that they can handle the entirety of request rate…” and “Algorithm 1” teaching “… incoming_rate <- rate m; assigned_rate <-0, …while incoming_rate > assigned_rate do….” Teaching continue to add number of partitions of gpulets such that the assigned capacity can process the workload).
As for claim 13, Choi also teaches a method of managing an accelerator device that can be reconfigured to have different partitions that are capable of executing batches (Pg. 5, Section 4.1, “…scheduler makes …decisions…the size of gpulets…”), the method comprising:
providing associations between batch sizes and respective partition sizes ((Pg. 3, Section 3.1, “…for each batch size, we sweep through the increasing fractions of GPU compute resource (i.e., gpu-let size), which range from 20% to 100% to observe how efficiently the inference execution utilizes the additional compute resource…” Pg. 6, Table 2, “…p Partition size b Batch size…L(b,p) Latency function of b and p…” and Algorithm 1), the batch sizes comprising amounts of data in the corresponding batches, the partition sizes comprising amounts of processing resources of the corresponding partitions (Pg. 5, section 4.1, “…batch size and gpulet size…”, Pg. 6, Table 2, “…p Partition size b Batch size…L(b,p) Latency function of b and p…” );
instantiating a number of instances of partitions for each of the respective partition sizes based on information about frequencies of batches for the respective batch sizes (Pg. 6, “Algorithm 1” teaches allocating one or more gpu-lets based on rate of incoming requests, adding gpu-lets to alloc-gpulets. And Pg. 7, “…the scheduler allocates one or more gpu-lets so that they can handle the entirety of request rate…”)and
assigning batches to the instantiated instances of partitions, wherein the batches are assigned to instances of partitions that have partition sizes associated with, according to the associations, the sizes of the batches (Pg. 8, “…the desired batch size per model is determined….when the desired size of request is formed or a duty-cycle is passed, the scheduler dispatches the batch to one of the backend inference executor processes…which is in charge of a gpu-let…initiates the inference execution for the give model and batched input…”, and Pg 6, “Algorithm 1” teaching the gpulets sorted by size and finding gpu-let whose sizes are greater than or equal to ideal for allocation.).
While the claims nor the specification limits the interpretation of “frequencies of batches for the respective batch sizes” and the incoming rate for a model can reasonably be understood as number of batches for the model. Nevertheless, in the interest of compact prosecution, examiner note Choi does not explicitly teach where the limitation requires frequency information by batch size rather than the rate of requests coming in.
Shen teaches a method of GPU accelerated neural network learning execution including determining GPU resource allocation based on frequencies of batches for the respective batch sizes (Pg. 328, “…for session Si….with a batch size b, ….latency for any given request is 2Lki(b)…Denote batch size Bi as the maximum value for b that meats the constraint 2Lki(b) <= Li, therefore…maximal throughput, denoted by Ti, for session Si on a single GPU is ….the number of GPU nodes we allocate to execute just Si request s is n=[Ri/Ti]…” teaching for a given batch size and rate of requests, it determines number of batches needed to process the requests for a session at the given request rate that equals to n in the exemplary calculation. Alternatively, system determines, based on how many batches of each batch size fits into a partition, to determine how many partitions are needed per duty cycle (Pg. 326, Fig. 2), which can also be understood as utilizing information on number of the batch sizes as claimed). This known technique is applicable to the system of Choi as they both share characteristics and capabilities, namely, they are directed to batch based execution of neural network learning workloads on GPU accelerators.
One of ordinary skill in the art before the effective filing date of the application would have recognized that applying the known technique of Shen would have yielded predictable results and resulted in an improved system. It would have been recognized that applying the technique of Shen to the teachings of Choi would have yielded predictable results because the level of ordinary skill in the art demonstrated by the references applied shows the ability to incorporate such task scheduling features into similar systems. Further, applying determining GPU resource allocation based on frequencies of batches for the respective batch sizes to Choi with determine numbers of partitions for the respective determined sizes of the partitions based on number of the batches and the correspondences between the sizes of the batches and the sizes of the partitions accordingly, would have been recognized by those of ordinary skill in the art as resulting in an improved system that would allow improved execution efficiency for deep neural networks workloads executions on GPUs (Shen, Abstract)
As for claim 14, Choi also teaches wherein the associations are determined based on a resource utilization threshold (Pg. 7, “…knee…the most cost-effective sweet spot…calculates the curvature at the profiled gpulet size and uses the gpulet size at the knee…”).
In addition, Shen also teaches the same limitation (Pg. 329, “greedy scheduling …largest batch size and the corresponding duty cycle in order to meet the throughput and SLO needs…”). Rationale to combine same as claim 1 above.
As for claim 15, Choi also teaches statistics of the sizes of partitions and the sizes of batches . The method of claim 13, wherein the information about frequencies of batches for the respective batch sizes is determined based on statistics of the sizes of partitions and the sizes of batches (Pg. 6, “…profiled execution latency of batch b on partition size p…”).
Shen also teaches frequency of batches for the respective batch sizes determined (Pg. 328, “…for session Si….with a batch size b, ….latency for any given request is 2Lki(b)…Denote batch size Bi as the maximum value for b that meats the constraint 2Lki(b) <= Li, therefore…maximal throughput, denoted by Ti, for session Si on a single GPU is ….the number of GPU nodes we allocate to execute just Si request s is n=[Ri/Ti]…” teaching for a given batch size and rate of requests, it determines number of batches needed to process the requests for a session at the given request rate that equals to n in the exemplary calculation. Alternatively, system determines, based on how many batches of each batch size fits into a partition, to determine how many partitions are needed per duty cycle (Pg. 326, Fig. 2), which can also be understood as utilizing information on number of the batch sizes as claimed). Rationale to combine is same as claim 13 above.
As for claim 16, Choi teaches numbers of instances of partitions for the respective partition sizes are determined of particular partition sizes (Pg. 7, “…scheduler allocates one or more gpu-lets so that they can handle the entirety of request rate…” and Pg. 6, “Algorithm 1”).
In addition, Shen also teaches determining the number of instances of resources a given size is proportional to a frequency of processing batches of a corresponding size (Pg. 328, “…for session Si….with a batch size b, ….latency for any given request is 2Lki(b)…Denote batch size Bi as the maximum value for b that meats the constraint 2Lki(b) <= Li, therefore…maximal throughput, denoted by Ti, for session Si on a single GPU is ….the number of GPU nodes we allocate to execute just Si request s is n=[Ri/Ti]…” teaching for a given batch size and rate of requests, it determines number of batches needed to process the requests for a session at the given request rate that equals to n in the exemplary calculation. Alternatively, system determines, based on how many batches of each batch size fits into a partition, to determine how many partitions are needed per duty cycle (Pg. 326, Fig. 2), which can also be understood as frequency of processing batches of the respective sizes as claimed). Rationale to combine is same as claim 13 above.
As for claim 17, Choi teaches the accelerator device comprises a multi-instance graphics processing unit (GPU) (Abstract, “…abstraction layer of GPU resources is created with configurable GPU resources…….concurrently in a GPU…” and Pg. 1, Section 1, “…efficient spatial partitioning of GPUs resources (called MPS mechanism…supports that the computational resources of a GPU can be partitioned to run different contexts simultaneously…to serve multiple ML tasks concurrently….partitionable GPUs…gpu-let….multiple virtual gpus out of a single physical GPU with spatial partitioning…”).
As for claim 20, Choi also teaches the associations between the batch sizes and the respective partition sizes are determined based on historical statistics of executions of previous batches by previous partitions of the accelerator having the partition sizes (Pg. 6, “…profiled execution latency of batch b on partition size p…” profiled means previously profiled statistics, thus, historical statistics.).
Claim(s) 6-8 are rejected under 35 U.S.C. 103 as being unpatentable over Choi et al. ("Multi-model machine learning inference serving with gpu spatial partitioning." arXiv preprint arXiv:2109.01611 (2021)), in view of Shen et al. (“Nexus: A GPU Cluster Engine for Accelerating DNN-Based video Analysis”, SOSP ’19, Oct 27, 2019), in view of Gujarati et al. (“Serving DNNs like Clockwork: Performance Predictability from the Bottom Up”, OSDI 20, Nov 4, 2020).
As for claim 6. Choi also teaches when a batch is scheduled to the accelerator, predicted execution times processing the batch by each of the respective partition sizes, based on a processing time determined based on the size of one or more partitions obtained by partitioning the accelerator and the size of the batch input to the partitions, and assign the batch to one of the partitions by comparing the predicted execution times with an execution time requirement that is associated with the batch (Pg. 6, “L(b,p):…execution latency of batch b on partition size p…int f: interference function, and …SLO: per-model SLO….for each scheduling period…” Table 2, “L(b,P) latency function of b and p…SLO (in latency) of model i…” are used to calculate allocation at Algorithm 1, “…L(b, gpulet.size)+int f <= SLO…”).
While the L(b, p) is clearly derived in Choi, thus, a form of calculation, regardless of method of profiling. Nevertheless, in the interest of compact prosecution, Examiner note Choi and Shen do not explicitly teach calculating said predicted execution time.
However, Gujarati teaches a known method of managing DNN workloads on GPU accelerators including calculate predicted execution times for completing processing of DNN workloads (Pg. 450, “action profiles, which are measurements of past 10 actions durations, stratified by model, worker, and batch size, to predict the duration of future action…” teaching using of plurality of measurements to derive a predicted execution time for the batch in future.). This known technique is applicable to the system of Choi and Shen as they both share characteristics and capabilities, namely, they are directed to batch based execution of neural network learning workloads on GPU accelerators.
One of ordinary skill in the art before the effective filing date of the application would have recognized that applying the known technique of Gujarati would have yielded predictable results and resulted in an improved system. It would have been recognized that applying the technique of Gujarati to the teachings of Choi and Shen would have yielded predictable results because the level of ordinary skill in the art demonstrated by the references applied shows the ability to incorporate such task scheduling features into similar systems. Further, applying calculate predicted execution times for completing processing of DNN workloads to Choi and Shen with calculate predicted execution times for completing processing of the batch for each of the respective sizes of the partitions accordingly, would have been recognized by those of ordinary skill in the art as resulting in an improved system that would allow improved conformity with tight, request-level service – level objectives (Gujarati, Abstract)
As for claim 7, Choi also teaches the processing times are determined based on a neural network (NN) model executed by the partitions (Pg. 6, “…L(b, p)…”).
In addition, Gujarati also teaches the processing times are determined based on a neural network (NN) model executed by the partitions (Pg. 450, “…measurements….stratified by model, worker, and batch size…”). Rational to combine same as claim 6 above.
As for claim 8, Choi also teaches schedule the batch to a smallest partition among partitions for which the respectively corresponding predicted execution times meet the execution time requirement (Pg. 2, “…creating and assigning the most efficient GPU share for a given ML model…” and Pg. 7, “choosing the “ideal” gpulet size between the most cost-effective gpulet size….and the minimum required gpulet size to serve rate m…always picks the minimum of peff and preq…”).
Claim(s) 9-12 are rejected under 35 U.S.C. 103 as being unpatentable over Choi et al. ("Multi-model machine learning inference serving with gpu spatial partitioning." arXiv preprint arXiv:2109.01611 (2021)), in view of Gujarati et al. (“Serving DNNs like Clockwork: Performance Predictability from the Bottom Up”, OSDI 20, Nov 4, 2020).
As for claim 9, Choi also teaches an electronic device comprising:
one or more processors and a memory storing instructions executable by the processor, wherein, in response to the instructions being executed by the one or more processors (Table 3 – “cpu” “memory capacity”), the one or more processors:
based on a processing time determined based on a size of one or more partitions obtained by partitioning an accelerator and a size of a batch input to each of the partitions, when the batch is scheduled to the accelerator, calculate predicted execution times for completing processing of the batch for each of the respective sizes of the partitions (Table 2 – “L(b, p) Latency function of b and p…SLOi SLO (in latency) of model i…”, Algorithm 1: “…if L(b, gpulet_size) + inf f <= SLO…” Pg. 6, “…profiled execution latency of batch b on partition size p…” teaching latency (processing time) based on size of partition and size of batch input, the calculated predicted execution times for completing the processing of the batch can be understood as the latency plus inference overhead summed to compare against SLO.), and
schedule the batch to one of the partitions by comparing the predicted execution times with an execution time requirement associated with the batch (Pg. 7, “…the scheduler allocates one or more gpu-lets so that they can handle the entirety of request…” in view of Algorithm 1 teaching finding GPU-lets that does not violate SLO for execution of the workload).
Choi clearly teaches calculating a execution time for completing the execution of the batch of respective sizes of the partitions with the calculation of the duration of processing batch with particular partition size and overhead time, because it is based on profiled historical data, it is clearly predicting execution time for the batch for scheduling purposes. Nevertheless, in the interest of compact prosecution, Examiner note in the event calculate predicted execution times means using profiled historical data to then derive a secondary predicted execution time, Choi does not explicitly teach such an interpretation of the claim language.
However, Gujarati teaches a known method of managing DNN workloads on GPU accelerators including calculate predicted execution times for completing processing of batch sizes (Pg. 450, “action profiles, which are measurements of past 10 actions durations, stratified by model, worker, and batch size, to predict the duration of future action…” teaching using of plurality of measurements to derive a predicted execution time for the batch in future.). This known technique is applicable to the system of Choi as they both share characteristics and capabilities, namely, they are directed to batch based execution of neural network learning workloads on GPU accelerators.
One of ordinary skill in the art before the effective filing date of the application would have recognized that applying the known technique of Gujarati would have yielded predictable results and resulted in an improved system. It would have been recognized that applying the technique of Gujarati to the teachings of Choi would have yielded predictable results because the level of ordinary skill in the art demonstrated by the references applied shows the ability to incorporate such task scheduling features into similar systems. Further, applying calculate predicted execution times for completing processing of DNN workloads to Choi with calculate predicted execution times for completing processing of the batch for each of the respective sizes of the partitions accordingly, would have been recognized by those of ordinary skill in the art as resulting in an improved system that would allow improved conformity with tight, request-level service – level objectives (Gujarati, Abstract)
As for claim 10, Gujarati also teaches the processing times are determined based on a neural network (NN) model executed by the partitions (Pg. 450, “….measurements of …actions duration, stratified by model, worker, and batch size…”). Rationale to combine same as claim 9 above.
As for claim 11, Gujarati also teaches calculate the predicted durations based on a remaining processing time of a batch that is currently processed by one of the partitions, a processing time of a batch that is already scheduled to one of the partitions, and a processing time of the batch (pg. 450, “…predict the duration of future action…pending actions, which tracks submitted actions and estimates when each executor will next be available…predict when candidate actions will complete…”).
As for claim 12, Choi teaches schedule the batch to a smallest partition size among partition sizes for which the corresponding predicted execution times are earlier than the execution time requirements (Pg. 6, “Algorithm 1…sort all gpulet by gpulet.size in ascending order….if L (b, gpulet.size) + int f <= SLO…” Thus, it would be obvious to pick the smallest partition size satisfying the SLO as it is tested based on ascending order, thus, the gpulet.size that satisfies the latency requirement with the smallest size would be found first.).
Claim(s) 18 are rejected under 35 U.S.C. 103 as being unpatentable over Choi and Shen, further in view of Kerr et al. (US PGPUB 2021/0124582).
As for claim 18, Choi explicitly teach the use of spatial partitioning in Nvidia GPU processors, which is well known to be done by clustering. Nevertheless, in the interest of compact prosecution, Choi does not explicitly teach partition size correspond to a number of graphics processing clusters.
However, Kerr teaches the NVidia GPUs are spatially partitioned where a partition size correspond to a number of graphics processing clusters (paragraph 132, 137, “various GPCs to process tasks defined by the one or more streams. Thus, arbitrary number of GPCs can be used for a single task).
It would be obvious to a person of ordinary skill in the art before the effective filing date of the application to incorporate Kerr’s teaching of partition size corresponds to a number of graphics processing clusters into Choi and Shen using spatial partitioning in NVidia GPU processors because they are directed towards the same Nvidia GPU processor spatial partitioned multiprocessing and because doing so allows for the efficient distribution of work to portions of GPU processors that manages memory bandwidth demands and increase throughput. (Kerr, paragraph 13 and 137).
Claim(s) 19 are rejected under 35 U.S.C. 103 as being unpatentable over Choi and Shen, in view of Gujarati et al. (“Serving DNNs like Clockwork: Performance Predictability from the Bottom Up”, OSDI 20, Nov 4, 2020).
As for claim 19, Choi also teaches assigning the batch to an instance of a partition having the lowest predicted execution time (Pg. 6, “profiled execution latency of batch b on partition size p…” and Pg. 2, “…lower the item weight by choosing a larger batch size…”, and Pg. 4, “…selecting the gpu-let placed at the curve knee is considered to be the most cost-effective choice, when batch size is large, the latency significantly drops as more resource is added…” teaching attempt to allocate gpu-let sized at the knee, which is the amount of resource that effectively reduce the latency. Additional resource no longer reduce latency, thus, the knee can constructively be understood as the lowest predicted execution time/latency for the workload using gpu-let of different sizes).
While Choi clearly teaches calculating an execution time for completing the execution of the batch of respective sizes of the partitions with the calculation of the duration of processing batch with particular partition size and overhead time, because it is based on profiled historical data, it is clearly predicting execution time for the batch for scheduling purposes. Nevertheless, in the interest of compact prosecution, Examiner note in the event calculate predicted execution times means using profiled historical data to then derive a secondary predicted execution time, Choi and Shen do not explicitly teach such an interpretation of the claim language.
However, Gujarati teaches a known method of managing DNN workloads on GPU accelerators including calculate predicted execution times for completing processing of batches of particular size using particular partition sizes (Pg. 450, “action profiles, which are measurements of past 10 actions durations, stratified by model, worker, and batch size, to predict the duration of future action…” teaching using of plurality of measurements to derive a predicted execution time for the batch in future.). This known technique is applicable to the system of Choi as they both share characteristics and capabilities, namely, they are directed to batch based execution of neural network learning workloads on GPU accelerators.
One of ordinary skill in the art before the effective filing date of the application would have recognized that applying the known technique of Gujarati would have yielded predictable results and resulted in an improved system. It would have been recognized that applying the technique of Gujarati to the teachings of Choi would have yielded predictable results because the level of ordinary skill in the art demonstrated by the references applied shows the ability to incorporate such task scheduling features into similar systems. Further, applying calculate predicted execution times for completing processing of batch for respective partition sizes to Choi with calculate predicted execution times for completing processing of the batch for each of the respective sizes of the partitions accordingly, would have been recognized by those of ordinary skill in the art as resulting in an improved system that would allow improved conformity with tight, request-level service – level objectives (Gujarati, Abstract)
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to KEVIN X LU whose telephone number is (571)270-1233. The examiner can normally be reached M-F 10am-6pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Lewis Bullock can be reached on 5712723759. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/KEVIN X LU/Examiner, Art Unit 2199
/LEWIS A BULLOCK JR/Supervisory Patent Examiner, Art Unit 2199
Prosecution Insights