DETAILED ACTION
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claims 1-20 are pending in this application.
Specification
The title of the invention is not descriptive. A new title is required that is clearly indicative of the invention to which the claims are directed.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) are: “scheduler" in claims 1, 3-6, 15-16 and 18-19, “compiler” in claims 2-3 and 17, and “capacity planner” in claim 20.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
Claim limitations “scheduler" in claims 1, 3-6, 15-16 and 18-19, “compiler” in claims 2-3 and17, and “capacity planner” in claim 20 invokes 35 U.S.C. 112(f). The specification paragraph [0153] that discloses “Any of the steps, operations, or processes described herein can be performed or implemented with one or more hardware or software modules, alone or in combination with other devices. In one embodiment, a software module is implemented with a computer program product comprising a computer-readable medium containing computer program” and [0155] of the specification discloses “some embodiments of the present disclosure can further relate to a system comprising a processor (e.g., a tensor streaming processor or an artificial intelligence processor), at least one computer processor (e.g., a host server), and a non-transitory computer-readable storage medium. The storage medium can store computer executable instructions, which when executed by the compiler operating on the at least one computer processor, cause the at least one computer processor to be operable for performing the operations and techniques described herein.” as performing corresponding structure. However, said scheduler" in claims 1, 3-6, 15-16 and 18-19, “compiler” in claims 2-3 and17, and “capacity planner” in claim 20 without the detail about the means to accomplish the functions are not an adequate disclosure of corresponding structure (i.e., they are general purpose computer and they are not sufficient structure to be corresponding structure under 112(f). That is, the general purpose computer must be transformed into a specially programmed computer by way of an algorithm). MPEP § 2181(II)(B) specifically indicated that “For a computer-implemented 35 U.S.C. 112(f) claim limitation, the specification must disclose an algorithm for performing the claimed specific computer function, or else the claim is indefinite under 35 U.S.C. 112(b). See Net MoneyIN, Inc. v. Verisign. Inc., 545 F.3d 1359, 1367, 88 USPQ2d 1751, 1757 (Fed. Cir. 2008). See also In re Aoyama, 656 F.3d 1293, 1297, 99 USPQ2d 1936, 1939 (Fed. Cir. 2011) ("[W]hen the disclosed structure is a computer programmed to carry out an algorithm, ‘the disclosed structure is not the general purpose computer, but rather that special purpose computer programmed to perform the disclosed algorithm.’") (quoting WMS Gaming, Inc. v. Int’l Game Tech., 184 F.3d 1339, 1349, 51 USPQ2d 1385, 1391 (Fed. Cir. 1999))” and “The corresponding structure is not simply a general purpose computer by itself but the special purpose computer as programmed to perform the disclosed algorithm. Aristocrat, 521 F.3d at 1333, 86 USPQ2d at 1239. Thus, the specification must sufficiently disclose an algorithm to transform a general purpose microprocessor to the special purpose computer” Therefore, the claims (i.e., 1-9 and 15-20) are indefinite and is rejected under 35 U.S.C. 112(b).
Applicant may:
(a) Amend the claim so that the claim limitation will no longer be interpreted as a limitation under 35 U.S.C. 112(f);
(b) Amend the written description of the specification such that it expressly recites what structure, material, or acts perform the entire claimed function, without introducing any new matter (35 U.S.C. 132(a)); or
(c) Amend the written description of the specification such that it clearly links the structure, material, or acts disclosed therein to the function recited in the claim, without introducing any new matter (35 U.S.C. 132(a)).
If applicant is of the opinion that the written description of the specification already implicitly or inherently discloses the corresponding structure, material, or acts and clearly links them to the function so that one of ordinary skill in the art would recognize what structure, material, or acts perform the claimed function, applicant should clarify the record by either:
(a) Amending the written description of the specification such that it expressly recites the corresponding structure, material, or acts for performing the claimed function and clearly links or associates the structure, material, or acts to the claimed function, without introducing any new matter (35 U.S.C. 132(a)); or
(b) Stating on the record what the corresponding structure, material, or acts, which are implicitly or inherently set forth in the written description of the specification, perform the claimed function. For more information, see 37 CFR 1.75(d) and MPEP §§ 608.01(o) and 2181.
Claim Rejections - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claims 1-9 and 15-20 are rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement.
The claims 1-6 and 15-20 contain subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for pre-AIA the inventor(s), at the time the application was filed, had possession of the claimed invention. As described above in 112(f) (i.e., “scheduler" in claims 1, 3-6, 15-16 and 18-19, “compiler” in claims 2-3 and17, and “capacity planner” in claim 20), the disclosure does not provide adequate structure to perform the claimed functions. The specification does not demonstrate that applicant has made an invention that achieves the claimed function because the invention is not described with sufficient detail such that one of ordinary skill in the art can reasonably conclude that the inventor had possession of the claimed invention. See MPEP § 2181(II)(B) “When a claim containing a computer-implemented 35 U.S.C. 112(f) claim limitation is found to be indefinite under 35 U.S.C. 112(b) for failure to disclose sufficient corresponding structure (e.g., the computer and the algorithm) in the specification that performs the entire claimed function, it will also lack written description under 35 U.S.C. 112(a)”.
Claims 2-9 and 16-20, they are depend on claims 1 and 15 and do not overcome the deficiencies thereof, therefore they are rejected for the same reason as claims 1 and 15 above.
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
Claims 1-9 and 15-20 are rejected under 35 U.S.C. 112(b), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor, or for pre-AIA the applicant regards as the invention.
As per claims 1-9 and 15-20:
As described above in 112(f) (i.e., “scheduler" in claims 1, 3-6, 15-16 and 18-19, “compiler” in claims 2-3 and17, and “capacity planner” in claim 20) without the detail about the means to accomplish the functions are not an adequate disclosure of corresponding structure. The MPEP § 2181(II)(B) specifically indicated that “For a computer-implemented 35 U.S.C. 112(f) claim limitation, the specification must disclose an algorithm for performing the claimed specific computer function, or else the claim is indefinite under 35 U.S.C. 112(b). See Net MoneyIN, Inc. v. Verisign. Inc., 545 F.3d 1359, 1367, 88 USPQ2d 1751, 1757 (Fed. Cir. 2008). See also In re Aoyama, 656 F.3d 1293, 1297, 99 USPQ2d 1936, 1939 (Fed. Cir. 2011) ("[W]hen the disclosed structure is a computer programmed to carry out an algorithm, ‘the disclosed structure is not the general purpose computer, but rather that special purpose computer programmed to perform the disclosed algorithm.’") (quoting WMS Gaming, Inc. v. Int’l Game Tech., 184 F.3d 1339, 1349, 51 USPQ2d 1385, 1391 (Fed. Cir. 1999))”. Therefore, the claims (i.e., 1-9 and 15-20) are indefinite and is rejected under 35 U.S.C. 112(b).
As per claims 2-9 and 16-20:
They are depend on claims 1 and 15 and do not overcome the deficiencies thereof, therefore they are rejected for the same reason as claims 1 and 15 above.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to a judicial exception (i.e., a law of nature, a natural phenomenon, or an abstract idea) without significantly more.
Claim 1 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1, Statutory Category: Yes, the claim 1 is a deterministic streaming system that recites a series of steps and therefore falls in the statutory category of a machine.
Step 2A- Prong 1: Judicial Exception Recited: Yes, the claim recites: “evaluate a latency for each task of a plurality of tasks to be run at the deterministic streaming system, and adjust at least one of an accuracy metric and a quality metric for an output of each of the plurality of tasks based on the evaluated latency until the plurality of tasks can be completed before expiration of one or more contractual deadlines” As drafted, the claim as a whole recites a method including steps that could be performed in the human mind, but for the recitation of generic computing components. The human mind can easily judging/evaluating a latency for each task of a plurality of tasks to be run at the deterministic streaming system, adjusting/changing/modifying at least one of an accuracy metric and a quality metric for an output of each of the plurality of tasks based on the evaluated latency until the plurality of tasks can be completed before expiration of one or more contractual deadlines. Therefore, but for the recitation of generic computing components, these steps may be a Mental Processes that can be performed in the human mind (including an observation, evaluation, judgment, opinion).
Therefore, yes, the claims do recite judicial exceptions. No, this judicial exception is not integrated into a practical application. In particular, the claim recites an additional limitations that “a plurality of deterministic streaming processors, each deterministic streaming processor including an array of processing elements; and a scheduler configured to” and “wherein at least a subset of the plurality of deterministic streaming processors is configured to run the plurality of tasks each having the output with at least one of the adjusted accuracy metric and the adjusted quality metric” which is directed to Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a generic computer as a tool to perform an abstract idea (see MPEP 2106.05(f)). In addition, the limitation of “wherein at least a subset of the plurality of deterministic streaming processors is configured to run” which is merely applying the judicial exception or abstract idea (See MPEP 2106.05(f)). (i.e., “configured to run” does not necessary mean that is actually running). Accordingly, even in combination, these additional elements do not integrate the abstract idea into a practical application because they not impose any meaningful limits on practicing the abstract idea. Therefore, the claim is directed to the abstract idea.
Step 2B: Claim provides an Inventive Concept: No. The additional element “a plurality of deterministic streaming processors, each deterministic streaming processor including an array of processing elements; and a scheduler configured to” and “wherein at least a subset of the plurality of deterministic streaming processors is configured to run the plurality of tasks each having the output with at least one of the adjusted accuracy metric and the adjusted quality metric” which is directed to Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a generic computer as a tool to perform an abstract idea (see MPEP 2106.05(f)). In addition, the limitation of “wherein at least a subset of the plurality of deterministic streaming processors is configured to run” which is merely applying the judicial exception or abstract idea (See MPEP 2106.05(f)). (i.e., “configured to run” does not necessary mean that is actually running). These additional elements and combination of the elements does not amount to significant more than the exception itself or provide an inventive concept in Step 2B.
For these reasons, there is no inventive concept in the claim, and thus the claim is ineligible.
Independent claims 10 and 15 are rejected for the same reason as claim 1 above. Claim 15 further recites “adjust a level of accuracy of one or more other tasks of the plurality of tasks in the queue to increase a quality metric of the one or more other tasks, based on deterministic information about an amount of computation that can be performed at the processor farm within a defined time period, and adjust, based on the deterministic information, at least one of an accuracy metric and a quality metric of results generated by the plurality of tasks until the plurality of tasks can be completed by defined contractual deadlines” these steps may be a Mental Processes that can be performed in the human mind (including an observation, evaluation, judgment, opinion). In addition, “a processor farm” and “achieve a level of confidence for a first task of the plurality of tasks in a queue to generate a result having an accuracy metric above a threshold accuracy” which is directed to Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a generic computer as a tool to perform an abstract idea (see MPEP 2106.05(f)).
With respect to the dependent claim 2, the claim elaborates that a compiler configured to: calculate an amount of computation that can be performed within a period of time for each of the plurality of tasks; and provide information about the calculated amount of computation to the scheduler for the evaluation of latency for each of the plurality of tasks (“calculate an amount of computation” which is a Mental Processes that can be performed in the human mind (including an observation, evaluation, judgment, opinion). In addition, “compiler” is directed to generic computing components/functions merely applying the abstract idea (MPEP § 2106.05(f)). Further, “provide information” which is insignificant extra solution activity (i.e., transmitting data) See MPEP 2106.05(g)) and Courts have identified “receiving and transmitting data, storing and retrieving information”, et cetera as well understood, routine, conventional and mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (see MPEP 2106.05(f))).
With respect to the dependent claim 3, the claim elaborates that a compiler configured to: compile a source code of each model of a plurality of models associated with the plurality of tasks into an intermediate representation, wherein the scheduler is further configured to generate quality information associated with a plurality of binary executables, based on the intermediate representation, and the compiler is further configured to compile the intermediate representation into the plurality of binary executables using the generated quality information (these limitation is directed to Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a generic computer as a tool to perform an abstract idea (see MPEP 2106.05(f)).
With respect to the dependent claim 4, the claim elaborates that wherein the scheduler is further configured to generate the quality information while performing one or more static capacity planning jobs when one or more new models of the plurality of models are being registered (these limitation is directed to Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a generic computer as a tool to perform an abstract idea (see MPEP 2106.05(f)).
With respect to the dependent claim 5, the claim elaborates that wherein the scheduler is further configured to: select a binary executable of the plurality of binary executables for execution at one or more of the deterministic streaming processors, based on a number of computational cycles required for each of the plurality of binary executables to be executed (“select” are being treated as part of abstract idea and is analogous to Mental processes, such that concept can be performed in the human mind. Further, the claim as a whole is a Mental Processes that can be performed in the human mind (including an observation, evaluation, judgment, opinion)).
With respect to the dependent claim 6, the claim elaborates that select at least the subset of the plurality of deterministic streaming processors to run the plurality of tasks based on a resource availability map identifying each deterministic streaming processor of the plurality of deterministic streaming processors (“select… based on a resource availability map…” are being treated as part of abstract idea and is analogous to Mental processes, such that concept can be performed in the human mind. Further, the claim as a whole is a Mental Processes that can be performed in the human mind (including an observation, evaluation, judgment, opinion)).
With respect to the dependent claim 7, the claim elaborates that wherein the resource availability map comprises a list of each deployed deterministic streaming processor of the plurality of deterministic streaming processors and information about a configuration of each deployed deterministic streaming processor (these limitation is directed to Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a generic computer as a tool to perform an abstract idea (see MPEP 2106.05(f)).
With respect to the dependent claim 8, the claim elaborates that wherein the resource availability map comprises information about a defect classification identifying a defect associated with each deterministic streaming processor of the plurality of deterministic streaming processors (these limitation is directed to Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a generic computer as a tool to perform an abstract idea (see MPEP 2106.05(f)).
With respect to the dependent claim 9, the claim elaborates that wherein the deterministic streaming system meets at least one of a defined quality of experience (QoE) metric and a defined quality of service (QoS) metric, based on at least the subset of the plurality of deterministic streaming processors running the plurality of tasks each having the output with at least one of the adjusted accuracy metric and the adjusted quality metric (these limitation is directed to Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a generic computer as a tool to perform an abstract idea (see MPEP 2106.05(f)).
Dependent claims 11, 12, 13 and 14 recite the same features as applied to claims (2), (3 and 5), (6), (7 and 8) respectively above, therefore they are also rejected under the same rationale.
With respect to the dependent claim 16, the claim elaborates that wherein the scheduler is further configured to: assign the plurality of tasks to one or more processors in the processor farm in accordance with the deterministic information provided by a compiler of the system; and dynamically change the quality metric of the results in response to changes in a workload associated with the plurality of tasks (“assign” and “dynamically change the quality metric of the results” are being treated as part of abstract idea and is analogous to Mental processes, such that concept can be performed in the human mind. Further, the claim as a whole is a Mental Processes that can be performed in the human mind (including an observation, evaluation, judgment, opinion)).
With respect to the dependent claim 17, the claim elaborates that a compiler configured to: produce a plurality of binary executables from a source code of a model; and characterize the processor farm in advance of an arrival of each task of the plurality of tasks to account for availability of resources within the processor farm (“produce a plurality of binary executables from a source code” and “characterize the processor farm” are being treated as part of abstract idea and is analogous to Mental processes, such that concept can be performed in the human mind).
With respect to the dependent claim 18, the claim elaborates that wherein the scheduler is further configured to: produce quality information for the plurality of binary executables, the quality information including information about at least one of an accuracy metric and a latency for each of the plurality of binary executables when executed at specific resources of the processor farm (“produce quality information for the plurality of binary executables” are being treated as part of abstract idea and is analogous to Mental processes, such that concept can be performed in the human mind).
With respect to the dependent claim 19, the claim elaborates that wherein the scheduler is further configured to: provide the quality information to the compiler for compiling an intermediate representation of the model to generate the plurality of binary executables; and in response to a plurality of requests for the plurality of tasks, serve the plurality of requests with a binary executable of the plurality of binary executables, the binary executable yields a better performance at lower quality results to meet the defined contractual deadlines.(“provide the quality information to the compiler” which is insignificant extra solution activity (i.e., transmitting data) See MPEP 2106.05(g)) and Courts have identified “receiving and transmitting data, storing and retrieving information”, et cetera as well understood, routine, conventional and mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (see MPEP 2106.05(f))). “serve the plurality of requests with a binary executable” is directed to Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a generic computer as a tool to perform an abstract idea (see MPEP 2106.05(f)).
With respect to the dependent claim 20, the claim elaborates that wherein the scheduler comprises a capacity planner configured to: simulate the processor farm consisting of simulated leaky buckets for all existing and newly registered models that are filled with tasks representing a maximum load that any of the leaky buckets is configured to allow, a simulation cluster of deterministic streaming processors, and a simulation scheduler that mimics scheduling decisions of the scheduler, wherein the capacity planner uses worst case load conditions and information about a number of the existing registered models to statically accept or reject the newly registered models (these limitations are directed to Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a generic computer as a tool to perform an abstract idea (see MPEP 2106.05(f)).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1 and 9-10 are rejected under 35 U.S.C. 103 as being unpatentable over Verhaegh et al. (US Pub. 2006/0192850 A1) in view of Puppala et al. (US Pub. 2019/0188038 A1).
As per claim 1, Verhaegh teaches the invention substantially as claimed including A deterministic streaming system comprising (Verhaegh, Fig. 2; Fig. 9):
a scheduler configured to (Verhaegh, Fig. 2, 202, controller (as scheduler); :
evaluate a latency for each task of a plurality of tasks to be run at the deterministic streaming system (Verhaegh, Fig. 2, 208 queue include frames need to be processed individually (as task of plurality of tasks to be run); [0051] trying to optimize the local QoS within the allocated budget, in the context of high-quality video processing. It is assume that the video processing task is scalable, i.e. that it can trade picture quality for resource usage at the level of individual frames, and that the task works ahead, i.e. that it can start processing the next frame immediately after completing the previous one, provided that the data are available. These scalable video algorithms provide a limited number of QoS levels that can be chosen for each frame. The extent to which working ahead can be applied is determined by latency and buffer constraints. The QoS specification for high-quality video combines three elements, which have to be balanced: processing quality, deadline misses, and quality changes; [0065] the processing time (as latency for each task) for each frame is known in advance, finding a control strategy that maximizes the average revenue can be computed. In that case, the optimal quality levels can be computed off-line using dynamic programming), and
adjust at least one of an accuracy metric and a quality metric for an output of each of the plurality of tasks based on the evaluated latency until the plurality of tasks can be completed before expiration of one or more contractual deadlines (Verhaegh, Fig. 3, Fig. 4, Fig. 5; [0060] As mentioned before, at each start point the controller has to select the quality level at which the upcoming frame is processed. Preferably, a control strategy is chosen that finds an optimal balance to meet the following three objectives: [0061] because deadline misses and the accompanying frame skips result in artifacts in the output, deadline misses should be as sparse as possible. To prevent deadline misses, it may be necessary to process frames at lower quality levels. [0062] to obtain a high output quality, frames should be processed at an as high as possible quality level. [0063] the number and size of quality-level changes should be as low as possible, because (bigger) changes in the quality level may result in (better) perceivable artifacts; [0082] select the quality level (as a quality metric) for the next frame to be processed, i.e. the frame that corresponds to the start point); [0082] the just-completed frame was processed at quality level q and that the frame before that was processed at quality level pq. The revenue is composed of a (high) negatively-valued penalty on the number of deadlines that were missed since the previous start point, a positively-valued reward for the quality level q at which the frame was processed, and a negatively-valued quality-change penalty qcp(pq, q) for changing the quality level from pq to q; [0085] To estimate the processing time for the frame at a different quality level, the off-line determined ept-values (expected processing time) are used that were also used for budget scaling. For example, if a frame, processed at quality level q.sub.2_l , yields a processing time of 20 ms, and if ept (q.sub.0)=15 ms and ept (q.sub.2)=22 ms, then the estimated processing time for the frame at quality level q.sub.0 is 20 msept(q.sub.0)/ept(q.sub.2)=13.6 ms. The estimated processing times are used to simulate processing the frame. Starting at a grid point state s.sub.t, and taking a particular quality-level action q.sub.i, using the estimated processing time for quality level q.sub.i the resulting (non-grid point) state s.sub.i+1 after processing the frame, the corresponding greedy quality-level action q.sub.t+1, and the resulting revenue r.sub.i+1. can be computed. In this computation, first the processing time for budget scaling (normalization step) is corrected. Using this information, the Sarsa update rule is applied. At each start point this is done preferably for all grid point states and all quality-level actions);
wherein at least a processor is configured to run the plurality of tasks each having the output with at least one of the adjusted accuracy metric and the adjusted quality metric (Verhaegh, Fig. 9, 914 microprocessor; [0058] the task has a private processor; [0051] Here local QoS control is concerned, i.e. trying to optimize the local QoS within the allocated budget, in the context of high-quality video processing. It is assume that the video processing task is scalable, i.e. that it can trade picture quality for resource usage at the level of individual frames, and that the task works ahead, i.e. that it can start processing the next frame immediately after completing the previous one, provided that the data are available. These scalable video algorithms provide a limited number of QoS levels that can be chosen for each frame. The extent to which working ahead can be applied is determined by latency and buffer constraints. The QoS specification for high-quality video combines three elements, which have to be balanced: processing quality, deadline misses, and quality changes; [0061] because deadline misses and the accompanying frame skips result in artifacts in the output, deadline misses should be as sparse as possible. To prevent deadline misses, it may be necessary to process frames at lower quality levels. [0062] to obtain a high output quality, frames should be processed at an as high as possible quality level. [0063] the number and size of quality-level changes should be as low as possible, because (bigger) changes in the quality level may result in (better) perceivable artifacts).
Verhaegh fails to specifically teach a plurality of deterministic streaming processors, each deterministic streaming processor including an array of processing elements; and wherein at least a subset of the plurality of deterministic streaming processors is configured to run the plurality of tasks.
However, Puppala teaches a plurality of deterministic streaming processors, each deterministic streaming processor including an array of processing elements; and wherein at least a subset of the plurality of deterministic streaming processors is configured to run the plurality of tasks (Puppala, Abstract, Methods, systems and apparatuses for graph stream processing are disclosed. One apparatus includes a cascade of graph streaming processors, wherein each of the graph streaming processor includes a processor array, and a graph streaming processor scheduler; Fig. 4, 410 and 420 (as at least a subset of the plurality of deterministic streaming processors) that is processing the plurality of tasks (i.e., Fig. 4, 411, 412, 421, 423); Fig. 5).
It would have been obvious to one having ordinary skill in the art before the effective filling date of the claimed invention to have combined the teaching of Verhaegh with Puppala because Puppala’s teaching of utilizing the graph streaming processors for processing the different stages of the tasks would have provided Verhaegh’s system with the advantage and capability to allow the system to execute the different tasks simultaneously in order to improve the throughput and system performance (see Puppala, [0004] “executed on several processors simultaneously to improve the throughput”).
As per claim 9, Verhaegh and Puppala teach the invention according to claim 1 above. Verhaegh further teaches wherein the deterministic streaming system meets at least one of a defined quality of experience (QoE) metric and a defined quality of service (QoS) metric, based on processor running the plurality of tasks each having the output with at least one of the adjusted accuracy metric and the adjusted quality metric (Verhaegh, [0111] FIG. 9 illustrates the main parts of the system according to the invention in a schematic way. The system 900 comprises a microprocessor 914; [0051] Here local QoS control is concerned, i.e. trying to optimize the local QoS within the allocated budget, in the context of high-quality video processing. It is assume that the video processing task is scalable, i.e. that it can trade picture quality for resource usage at the level of individual frames, and that the task works ahead, i.e. that it can start processing the next frame immediately after completing the previous one, provided that the data are available. These scalable video algorithms provide a limited number of QoS levels that can be chosen for each frame. The extent to which working ahead can be applied is determined by latency and buffer constraints. The QoS specification for high-quality video combines three elements, which have to be balanced: processing quality, deadline misses, and quality changes. [0060] As mentioned before, at each start point the controller has to select the quality level at which the upcoming frame is processed. Preferably, a control strategy is chosen that finds an optimal balance to meet the following three objectives: [0061] because deadline misses and the accompanying frame skips result in artifacts in the output, deadline misses should be as sparse as possible. To prevent deadline misses, it may be necessary to process frames at lower quality levels. [0062] to obtain a high output quality, frames should be processed at an as high as possible quality level. [0063] the number and size of quality-level changes should be as low as possible, because (bigger) changes in the quality level may result in (better) perceivable artifacts).
In addition, Puppala teaches at least the subset of the plurality of deterministic streaming processors (Puppala, Abstract, Methods, systems and apparatuses for graph stream processing are disclosed. One apparatus includes a cascade of graph streaming processors, wherein each of the graph streaming processor includes a processor array, and a graph streaming processor scheduler; Fig. 4, 410 and 420 (as at least a subset of the plurality of deterministic streaming processors) that is processing the plurality of tasks (i.e., Fig. 4, 411, 412, 421, 423); Fig. 5).
As per claim 10, it is a method claim of claim 1 above. Therefore, it is rejected for the same reason as claim 1 above.
Claims 2 and 11 are rejected under 35 U.S.C. 103 as being unpatentable over Verhaegh and Puppala, as applied to claims 1 and 10 respectively above, and further in view of Che et al. (US Pub. 2016/0335064 A1) and Heaton et al. (US Patent. 11,561,833 B1).
As per claim 2, Verhaegh and Puppala teach the invention according to claim 1 above. Verhaegh further teaches calculate an amount of computation that can be performed within a period of time for each of the plurality of tasks; and calculated amount of computation for the evaluation of latency for each of the plurality of tasks (Verhaegh, [0006] Since resources are finite, deadline misses are likely to occur. In order to alleviate this, the media algorithms can run in lower than default quality levels, leading to correspondingly lower resource demands; [0057] Starting at d.sub.0, in the period between each pair of successive deadlines the task is assigned a guaranteed processing time budget b (0.ltoreq.b.ltoreq.P). Based on this guaranteed budget, a measure called progress is introduced. Progress .rho..sub.1, calculated at a start point s.sub.1, is the total amount of guaranteed budget left until d.sub.i-1, divided by b. This progress indicates how much budget is left after completing the previous frame i-1; [0059] FIGS. 5 and 6 illustrate two example timelines for b=P/2. The task has to process 5 frames. The frames actually processed are denoted in FIG. 5 by reference numerals 501, 502, 504, and 505 and in FIG. 6 by reference numerals 601, 602, 603, 604, and 605. Again, it is assumed that P=1, .rho.=2, and d.sub.0=0. It is further assumed that s.sub.1 is the moment at which the task is assigned budget for the first time. In FIG. 5, the progress at the successive start points is given by .rho..sub.1=0, .rho..sub.2=0.5, .rho..sub.4=0.75, and .rho..sub.5=0.5, respectively, and in FIG. 6 by .rho..sub.1=0, .rho..sub.2=0.5, .rho..sub.3=0.75, .rho..sub.4=1, and .rho..sub.5=0.5, respectively. Note that in each period the budget is distributed differently, as determined by an underlying scheduler. In FIG. 6, at m.sub.3 the task has consumed half of its budget for that period. The other half of the budget is lost due to blocking; also see [0051] The QoS specification for high-quality video combines three elements, which have to be balanced: processing quality, deadline misses, and quality changes).
Verhaegh and Puppala fail to specifically teach a compiler configured to: calculate an amount of computation and provide information about the calculated amount of computation to the scheduler.
However, Che teaches a compiler configured to: calculate an amount of computation (Che, Claim 4, a compiler evaluating the application determines that an amount of data needed to perform computations in the part of the application exceeds a first predetermined threshold).
It would have been obvious to one having ordinary skill in the art before the effective filling date of the claimed invention to have combined the teaching of Verhaegh and Puppala with Che because Che’s teaching of using the compiler to determining an amount of data needed to perform computations would have provided Verhaegh and Puppala’s system with the advantage and capability to allow the system to easily determining the data resource needed with regarding the computation which improving the system performance and efficiency.
Verhaegh, Puppala and Che fail to specifically teach the compiler provide information about the calculated amount of computation to the scheduler.
However, Heaton teaches the compiler provide information about the calculated amount of computation to the scheduler (Heaton, Col 1, line 64-Col 2, line 5, The neural network processor may include internal memory and an array of processing elements to perform neural network computations. The compiler engine and the runtime engine may operate in different computing systems of the computing environment. The compiler engine can allocate the memory and computation resources for the neural network processing operations, and provide information about the allocated memory and computation resources to the runtime engine).
It would have been obvious to one having ordinary skill in the art before the effective filling date of the claimed invention to have combined the teaching of Verhaegh, Puppala and Che with Heaton because Heaton’s teaching of compiler providing the computation information (i.e., allocated resources) to the runtime engine would have provided Verhaegh, Puppala and Che’s system with the advantage and capability to allow the system to easily performing the resource allocation based on the provided information in order to improving the system efficiency and resource utilization.
As per claim 11, it is a method claim of claim 2 above. Therefore, it is rejected for the same reason as claim 2 above.
Claim 3 is rejected under 35 U.S.C. 103 as being unpatentable over Verhaegh and Puppala, as applied to claim 1 above, and further in view of Tristan et al. (US Pub. 2015/0095277 A1) and Brady et al. (US Pub. 2019/0392296 A1).
As per claim 3, Verhaegh and Puppala teach the invention according to claim 1 above. Verhaegh further teaches wherein the scheduler is further configured to generate quality information (Verhaegh, Abstract, provide the output quality of a plurality of output qualities of the next media frame; and control means (904) conceived to set the output quality of the next media frame based upon a self-learning control strategy that uses a processing time and an output quality of a previous media frame to determine the output quality of the next media frame).
Verhaegh and Puppala fail to specifically teach a compiler configured to: compile a source code of each model of a plurality of models associated with the plurality of tasks into an intermediate representation, the quality information associated with a plurality of binary executables, based on the intermediate representation, and the compiler is further configured to compile the intermediate representation into the plurality of binary executables using the generated quality information.
However, Tristan teaches a compiler configured to: compile a source code of each model of a plurality of models associated with the plurality of tasks into an intermediate representation (Tristan, [0072] Compiler 122 compiles source code (such as specification 200) representing an LDA model, of a body of data, into an intermediate representation of the model, such as intermediate representation expression 700 of FIG. 7; Abstract, generates data-parallel inference code to sample from probability distributions in models provided to the compiler).
It would have been obvious to one having ordinary skill in the art before the effective filling date of the claimed invention to have combined the teaching of Verhaegh and Puppala with Tristan because Tristan’s teaching of compiler that compiles source code representing an model into an intermediate representation would have provided Verhaegh and Puppala’s system with the advantage and capability to allow the system to easily utilizes an intermediate representation that symbolically represents features of probability distributions which improving the system performance and efficiency.
Verhaegh, Puppala and Tristan fail to specifically teach the quality information associated with a plurality of binary executables, based on the intermediate representation, and the compiler is further configured to compile the intermediate representation into the plurality of binary executables using the generated quality information.
However, Brady teaches the quality information associated with a plurality of binary executables, based on the intermediate representation (Brady, Fig. 1, 140 intermediate representation; 115, 120 (as quality information); 150, binary; [0046] an improved compiler may be configured to consume a machine learning framework's (e.g., TensorFlow, Caffe™, etc.) representation (e.g., 110) of a Deep Neural Network (DNN), adapt and optimize it for a selected target (e.g., 125) and produce a binary executable (e.g., 150) corresponding to the selected target hardware 125 in a way that allows for compile time target specific optimizations. FIG. 10 is a simplified block diagram 1000 illustrating the generation of an example serialized binary 150 from a graph data structure 110 defining a trained neural network model for use in deep learning applications. The binary 150 may be generated to optimize the resources available at a particular target machine learning hardware device (e.g., 125). To produce such a binary 150, an improved compiler 105 may be provided that is implemented to optimize performance of deep learning applications. In some implementations, the compiler 105 may access the neural network model 110, together with information (e.g., target descriptor file 120) concerning the application and the target hardware 125 and generate an improved intermediate representation (IR) 140…a data model 1010, and a control model 1015. The intermediate representation 140 may also be provided with data (e.g., structural data 1020) describing attributes of the target hardware device (e.g., as extracted from an example target descriptor file 120), among other example sub-models and information; [0059] a target descriptor file 120 may be provided as input to specify attributes of a particular neural network computing device that is to implement the neural network and for which the executable code 150 is to be tuned or optimized);
the compiler is further configured to compile the intermediate representation into the plurality of binary executables using the generated quality information (Brady, Fig. 1, 120, 105 complier, 140 to 150 binary; [0046] an improved compiler may be configured to consume a machine learning framework's (e.g., TensorFlow, Caffe™, etc.) representation (e.g., 110) of a Deep Neural Network (DNN), adapt and optimize it for a selected target (e.g., 125) and produce a binary executable (e.g., 150) corresponding to the selected target hardware 125 in a way that allows for compile time target specific optimizations. FIG. 10 is a simplified block diagram 1000 illustrating the generation of an example serialized binary 150 from a graph data structure 110 defining a trained neural network model for use in deep learning applications. The binary 150 may be generated to optimize the resources available at a particular target machine learning hardware device (e.g., 125). To produce such a binary 150, an improved compiler 105 may be provided that is implemented to optimize performance of deep learning applications. In some implementations, the compiler 105 may access the neural network model 110, together with information (e.g., target descriptor file 120) concerning the application and the target hardware 125 and generate an improved intermediate representation (IR) 140…a data model 1010, and a control model 1015. The intermediate representation 140 may also be provided with data (e.g., structural data 1020) describing attributes of the target hardware device (e.g., as extracted from an example target descriptor file 120), among other example sub-models and information).
It would have been obvious to one having ordinary skill in the art before the effective filling date of the claimed invention to have combined the teaching of Verhaegh, Puppala and Tristan with Brady because Brady’s teaching of using the compiler to compile the intermediate representation in to the binary executable which is specific for a target hardware would have provided Verhaegh, Puppala and Tristan’s system with the advantage and capability to allow the system to utilizing the different types of the hardware for processing the neural network model by using the compiler in order to improving the system performance and efficiency.
Claims 6-7 and 13 are rejected under 35 U.S.C. 103 as being unpatentable over Verhaegh and Puppala, as applied to claims 1 and 10 respectively above, and further in view of Balle et al. (US Pub. 2018/0026905 A1)
As per claim 6, Verhaegh and Puppala teach the invention according to claim 1 above. Puppala further teaches select at least the subset of the plurality of deterministic streaming processors to run the plurality of tasks based on identifying each deterministic streaming processor of the plurality of deterministic streaming processors (Puppala, Fig. 4; [0007] The method includes processing scheduling, by a thread manager of each of a plurality of graph streaming processors, a plurality of threads operating on an array of processors of the graph streaming processor; [0024] Fig. 1 shows an acyclic graph mapped to stages of a graph streaming processor, according to an embodiment. For the example shown in FIG. 1, stage-0 is responsible for dispatching threads of the nodes n-00 (111) and n-01 (112). Further, stage-1 schedules the threads for the node n-10 (114), and the stage-2 schedules the threads of for the node n-20 (115), so on and so forth. The last stage of the pipeline is stage-N which schedules the threads for the node n-N0 (116), where is N is fixed for a specific GSP; [0048] the last stage 413 of the first GSP-0 (410) and the first stage 421 of the second GSP-1 (420) share the shared command buffer 430. For embodiments, the shared command buffer 430 can be physically located in memory of either of the GSPs 410, 420, but is located between the delineated stages 413, 421).
Verhaegh and Puppala fail to specifically teach when select at least the subset of the plurality of deterministic streaming processors to run the plurality of tasks, it is based on a resource availability map.
However, Balle teaches when select at least the subset of the plurality of deterministic streaming processors to run the plurality of tasks, it is based on a resource availability map (Balle, [0047] the orchestrator server 1240 is configured determine, from the telemetry data, patterns of resource utilization of the workloads, and adjust the allocations of resources across the managed nodes 1260 to provide additional resources to workloads that are presently or are predicted to have inadequate resources available to them to improve the performance of workloads (e.g., increase the speed of execution of the workloads), and deallocate resources from workloads that are not making sufficient use (e.g., in satisfaction of a threshold amount of use) of those resources. As such, the orchestrator server 1240 may reduce the amount of idle resources in the data center and increase the achievement of one or more of the resource allocation objectives; [0083] generate a resource allocation map indicative of allocations of resources among the managed nodes; determine, as a function of the telemetry data and the resource allocation map, a dynamic adjustment to allocation of resources to at least one of the managed nodes to improve performance of at least one of the workloads executed on the at least one of the managed nodes; and apply the adjustment to the allocation of resources among the managed nodes as the workloads are executed).
It would have been obvious to one having ordinary skill in the art before the effective filling date of the claimed invention to have combined the teaching of Verhaegh and Puppala with Balle because Balle’s teaching of using the resource allocation map for scheduling/allocating/selecting the resources for processing the tasks would have provided Verhaegh and Puppala’s system with the advantage and capability to allow the system to improving the resource utilization in order to improving the performance of workload execution (see Balle, [0083] “improve performance of at least one of the workloads executed on the at least one of the managed nodes”).
As per claim 7, Verhaegh, Puppala and Balle teach the invention according to claim 6 above. Puppala teaches deterministic streaming processor of the plurality of deterministic streaming processors (Puppala, [0007] The method includes processing scheduling, by a thread manager of each of a plurality of graph streaming processors, a plurality of threads operating on an array of processors of the graph streaming processor). In addition, Balle further teaches wherein the resource availability map comprises a list of each deployed deterministic streaming processor of the plurality of deterministic streaming processors and information about a configuration of each deployed deterministic streaming processor (Balle, [0059] the resource allocation map 1410 may include, for any given resource, an identification of the resource type (e.g., memory, data storage, compute, accelerator, etc.), an address of the resource (e.g., a unique identifier, such as a media access control (MAC) address, of the managed node 1260 where the resource is physically located and, in some embodiments, an internal address of the resource within the managed node 1260, such as a logical address of a block of data storage), and an identification of the managed node 1260 (e.g., a MAC address) that has received the allocation of the resource… such as target power usage of the components, processor capacity (e.g., a number of cores to be used, a clock speed, a percentage of available processor cycles, etc.) available to one or more workloads, memory resource capacity (e.g., amount of memory to be used and/or frequency of memory accesses to volatile memory and/or non-volatile memory) available to one or more workloads, communication circuitry capacity (e.g., network bandwidth) available to one or more workloads, and/or target operating temperatures and fan speeds.).
As per claim 13, it is a method claim of claim 6 above. Therefore, it is rejected for the same reason as claim 6 above.
Claims 8 and 14 are rejected under 35 U.S.C. 103 as being unpatentable over Verhaegh, Puppala and Balle, as applied to claims 6 and 13 respectively above, and further in view of Cepulis et al. (US Patent. 6,463,550 B1).
As per claim 8, Verhaegh, Puppala and Balle teach the invention according to claim 6 above. Puppala teaches deterministic streaming processor of the plurality of deterministic streaming processors (Puppala, Fig. 4; [0007] The method includes processing scheduling, by a thread manager of each of a plurality of graph streaming processors, a plurality of threads operating on an array of processors of the graph streaming processor).
Verhaegh, Puppala and Balle fail to specifically teach wherein the resource availability map comprises information about a defect classification identifying a defect associated with each deterministic streaming processor of the plurality of deterministic streaming processors.
However, Cepulis teaches wherein the resource availability map comprises information about a defect classification identifying a defect associated with each deterministic streaming processor of the plurality of deterministic streaming processors (Cepulis, Col 1, line 51, devices (such as the CPU); Col 12, lines 4-8, memory storage unit 104 of that device contains an error flag during block 404, indicating that the component has failed, then the boot strap processor configures the logical resource map in main memory 112 to indicate that the device has failed).
It would have been obvious to one having ordinary skill in the art before the effective filling date of the claimed invention to have combined the teaching of Verhaegh, Puppala and Balle with Cepulis because Cepulis’s teaching of the resource map indicating that processors are failed would have provided Verhaegh, Puppala and Balle’s system with the advantage and capability to allow the system to easily determining the status of the processors in order to improving the resource utilization and system performance.
As per claim 14, Verhaegh, Puppala and Balle teach the invention according to claim 13 above. Puppala teaches deterministic streaming processor of the plurality of deterministic streaming processors (Puppala, [0007] The method includes processing scheduling, by a thread manager of each of a plurality of graph streaming processors, a plurality of threads operating on an array of processors of the graph streaming processor). In addition, Balle further teaches wherein the resource availability map comprises a list of each deployed deterministic streaming processor of the plurality of deterministic streaming processors and information about a configuration of each deployed deterministic streaming processor (Balle, [0059] the resource allocation map 1410 may include, for any given resource, an identification of the resource type (e.g., memory, data storage, compute, accelerator, etc.), an address of the resource (e.g., a unique identifier, such as a media access control (MAC) address, of the managed node 1260 where the resource is physically located and, in some embodiments, an internal address of the resource within the managed node 1260, such as a logical address of a block of data storage), and an identification of the managed node 1260 (e.g., a MAC address) that has received the allocation of the resource… such as target power usage of the components, processor capacity (e.g., a number of cores to be used, a clock speed, a percentage of available processor cycles, etc.) available to one or more workloads, memory resource capacity (e.g., amount of memory to be used and/or frequency of memory accesses to volatile memory and/or non-volatile memory) available to one or more workloads, communication circuitry capacity (e.g., network bandwidth) available to one or more workloads, and/or target operating temperatures and fan speeds.).
Verhaegh, Puppala and Balle fail to specifically teach wherein the resource availability map comprises information about a defect classification identifying a defect associated with each deterministic streaming processor of the plurality of deterministic streaming processors.
However, Cepulis teaches wherein the resource availability map comprises information about a defect classification identifying a defect associated with each deterministic streaming processor of the plurality of deterministic streaming processors (Cepulis, Col 1, line 51, devices (such as the CPU); Col 12, lines 4-8, memory storage unit 104 of that device contains an error flag during block 404, indicating that the component has failed, then the boot strap processor configures the logical resource map in main memory 112 to indicate that the device has failed).
It would have been obvious to one having ordinary skill in the art before the effective filling date of the claimed invention to have combined the teaching of Verhaegh, Puppala and Balle with Cepulis because Cepulis’s teaching of the resource map indicating that processors are failed would have provided Verhaegh, Puppala and Balle’s system with the advantage and capability to allow the system to easily determining the status of the processors in order to improving the resource utilization and system performance.
Claim 15 is rejected under 35 U.S.C. 103 as being unpatentable over Verhaegh et al. (US Pub. 2006/0192850 A1) in view of Lipton et al. (US Pub. 2014/0355905 A1), and further in view of GUO et al. (US Pub. 2012/0294545 A1) and Puppala et al. (US Pub. 2019/0188038 A1).
As per claim 15, Verhaegh teaches the invention substantially as claimed including A system for executing a plurality of tasks, the system comprising: (Verhaegh, Fig. 2; Fig. 9; Abstract, The invention relates to a method and system (900) to set an output quality of a next media frame):
a scheduler configured to: (Verhaegh, Fig. 2, 202, controller (as scheduler):
adjust a level of quality of one or more other tasks of the plurality of tasks in the queue to increase a quality metric of the one or more other tasks, based on deterministic information about an amount of computation that can be performed within a defined time period, (Verhaegh, Fig. 2, 208 queue include frames need to be processed individually (as task of plurality of tasks to be run); [0006] Since resources are finite, deadline misses are likely to occur. In order to alleviate this, the media algorithms can run in lower than default quality levels, leading to correspondingly lower resource demands; [0057] Starting at d.sub.0, in the period between each pair of successive deadlines the task is assigned a guaranteed processing time budget b (0.ltoreq.b.ltoreq.P). Based on this guaranteed budget, a measure called progress is introduced. Progress .rho..sub.1, calculated at a start point s.sub.1, is the total amount of guaranteed budget left until d.sub.i-1, divided by b. This progress indicates how much budget is left after completing the previous frame i-1; [0051] trying to optimize the local QoS (as quality metric) within the allocated budget, in the context of high-quality video processing. It is assume that the video processing task is scalable, i.e. that it can trade picture quality for resource usage at the level of individual frames, and that the task works ahead, i.e. that it can start processing the next frame immediately after completing the previous one, provided that the data are available. These scalable video algorithms provide a limited number of QoS levels that can be chosen for each frame. The extent to which working ahead can be applied is determined by latency and buffer constraints. The QoS specification for high-quality video combines three elements, which have to be balanced: processing quality, deadline misses, and quality changes; [0060] As mentioned before, at each start point the controller has to select the quality level at which the upcoming frame is processed. Preferably, a control strategy is chosen that finds an optimal balance to meet the following three objectives: [0061] because deadline misses and the accompanying frame skips result in artifacts in the output, deadline misses should be as sparse as possible. To prevent deadline misses, it may be necessary to process frames at lower quality levels. [0062] to obtain a high output quality, frames should be processed at an as high as possible quality level. [0063] the number and size of quality-level changes should be as low as possible, because (bigger) changes in the quality level may result in (better) perceivable artifacts; [0065] the processing time for each frame is known in advance, finding a control strategy that maximizes the average revenue can be computed. In that case, the optimal quality levels can be computed off-line using dynamic programming), and
adjust, based on the deterministic information, at least one of an accuracy metric and a quality metric of results generated by the plurality of tasks until the plurality of tasks can be completed by defined contractual deadlines.
(Verhaegh, Fig. 3, Fig. 4, Fig. 5; [0060] As mentioned before, at each start point the controller has to select the quality level at which the upcoming frame is processed. Preferably, a control strategy is chosen that finds an optimal balance to meet the following three objectives: [0061] because deadline misses and the accompanying frame skips result in artifacts in the output, deadline misses should be as sparse as possible. To prevent deadline misses, it may be necessary to process frames at lower quality levels. [0062] to obtain a high output quality, frames should be processed at an as high as possible quality level. [0063] the number and size of quality-level changes should be as low as possible, because (bigger) changes in the quality level may result in (better) perceivable artifacts; [0082] select the quality level (as a quality metric) for the next frame to be processed, i.e. the frame that corresponds to the start point); [0082] the just-completed frame was processed at quality level q and that the frame before that was processed at quality level pq. The revenue is composed of a (high) negatively-valued penalty on the number of deadlines that were missed since the previous start point, a positively-valued reward for the quality level q at which the frame was processed, and a negatively-valued quality-change penalty qcp(pq, q) for changing the quality level from pq to q; [0085] To estimate the processing time for the frame at a different quality level, the off-line determined ept-values (expected processing time) are used that were also used for budget scaling. For example, if a frame, processed at quality level q.sub.2_l , yields a processing time of 20 ms, and if ept (q.sub.0)=15 ms and ept (q.sub.2)=22 ms, then the estimated processing time for the frame at quality level q.sub.0 is 20 msept(q.sub.0)/ept(q.sub.2)=13.6 ms. The estimated processing times are used to simulate processing the frame. Starting at a grid point state s.sub.t, and taking a particular quality-level action q.sub.i, using the estimated processing time for quality level q.sub.i the resulting (non-grid point) state s.sub.i+1 after processing the frame, the corresponding greedy quality-level action q.sub.t+1, and the resulting revenue r.sub.i+1. can be computed. In this computation, first the processing time for budget scaling (normalization step) is corrected. Using this information, the Sarsa update rule is applied. At each start point this is done preferably for all grid point states and all quality-level actions);
Verhaegh fails to specifically teach achieve a level of confidence for a first task of the plurality of tasks in a queue to generate a result having an accuracy metric above a threshold accuracy.
However, Lipton teaches achieve a level of confidence for a first task of the plurality of tasks in a queue to generate a result having an accuracy metric above a threshold accuracy (Lipton, [0019] In order to ensure a high quality end result, the system will set a minimum "accuracy score" (as level of confidence) that should be met by an image before it is placed in the mosaic. The system will only place (or assign) images into the mosaic if their accuracy score is higher than the pre-determined threshold. If a given cell is currently occupied, and an incoming photo has an accuracy score that is both above the quality threshold and also above the accuracy score of the current image, the new image may replace the current image in order to improve the overall quality of the mosaic; please note, queued tasks was taught by Verhaegh).
It would have been obvious to one having ordinary skill in the art before the effective filling date of the claimed invention to have combined the teaching of Verhaegh with Lipton because Lipton’s teaching of generated result that having accuracy is higher than the threshold would have provided Verhaegh’s system with the advantage and capability to allow the system to ensuring that the result is above the quality and accuracy threshold in order to improve the overall quality of the mosaic.
Verhaegh and Lipton fail to specifically teach when adjust a level of quality to increase a quality metric, the adjusted level of quality is accuracy…to increase a quality metric.
However, GUO teaches when adjust a level of quality to increase a quality metric, the adjusted level of quality is accuracy…to increase a quality metric (GUO, [0035] the predictive coding model for coding each macroblock of the image frame 10 is set by the encoder so as to control the predictive coding model that codes the macroblocks and improve the accuracy (as to adjust accuracy)…For example, in HD images, Full HD images, or D1 resolution images, the ratio is adjusted to 2:3 so as to increase image quality and accuracy (as adjusted accuracy which will lead to increase a quality metric)).
It would have been obvious to one having ordinary skill in the art before the effective filling date of the claimed invention to have combined the teaching of Verhaegh and Lipton with GUO because GUO’s teaching of increasing the accuracy which will also increasing the quality would have provided Verhaegh and Lipton’s system with the advantage and capability to allow the system to further ensuring the system to providing a quality result regarding to adjusted/improved accuracy which improving the system performance and efficiency.
Verhaegh, Lipton and GUO fail to specifically teach executing a plurality of tasks at a processor farm.
However, Puppala teaches executing a plurality of tasks at a processor farm.
(Puppala, Abstract, Methods, systems and apparatuses for graph stream processing are disclosed. One apparatus includes a cascade of graph streaming processors (as processor farm), wherein each of the graph streaming processor includes a processor array, and a graph streaming processor scheduler; Fig. 4, 410 and 420 that is processing the plurality of tasks (i.e., Fig. 4, 411, 412, 421, 423); Fig. 5).
It would have been obvious to one having ordinary skill in the art before the effective filling date of the claimed invention to have combined the teaching of Verhaegh, Lipton and GUO with Puppala because Puppala’s teaching of utilizing the graph streaming processors for processing the different stages of the tasks would have provided Verhaegh, Lipton and GUO’s system with the advantage and capability to allow the system to execute the different tasks simultaneously in order to improve the throughput and system performance (see Puppala, [0004] “executed on several processors simultaneously to improve the throughput”).
Claim 16 is rejected under 35 U.S.C. 103 as being unpatentable over Verhaegh, Lipton, GUO and Puppala, as applied to claim 15 above, and further in view of Heaton et al. (US Patent. 11,561,833 B1).
As per claim 16, Verhaegh, Lipton, GUO and Puppala teach the invention according to claim 15 above. Verhaegh teaches wherein the scheduler is further configured to: dynamically change the quality metric of the results in response to changes in a workload associated with the plurality of tasks (Verhaegh, [0008] processing the previous media-frame; determine a state comprising of a relative progress value of the processed previous media-frame; a scaled budget value of the processed previous media-frame; and the output quality of the processed previous media-frame; determine a revenue based upon the state and a possible output quality of the next media-frame; [0049] Soft timing requirements for tasks with fluctuating load such as acceptance of occasional deadline misses or an average-case response time requirement can be viewed as a special case of Quality of Service (QoS); [0052] The balancing control strategies are concerned with two types of load fluctuations: short-term (or stochastic), and structural. To control the short-term load fluctuations; [0082] After processing a frame, at the start point of the subsequent frame to be processed, the agent first updates the scaled budget using the processing time of the just-completed frame. This updated scaled budget is part of the state at the start point. Next, the agent computes the revenue for the just-completed frame. For notational convenience, it is assumed that the just-completed frame was processed at quality level q and that the frame before that was processed at quality level pq. The revenue is composed of a (high) negatively-valued penalty on the number of deadlines that were missed since the previous start point, a positively-valued reward for the quality level q at which the frame was processed, and a negatively-valued quality-change penalty qcp(pq, q) for changing the quality level from pq to q. Note that the agent computes the revenue based on information provided by the environment (number of deadline misses, quality levels), instead of receiving the revenue directly from the environment. Using the revenue, the agent updates (learns) its action-values. After that, the updated action-values are used to select the quality level for the next frame to be processed, i.e. the frame that corresponds to the start point).
In addition, Puppala teaches assign the plurality of tasks to one or more processors in the processor farm in accordance with the deterministic information (Puppala, Fig. 4; [0007] The method includes processing scheduling, by a thread manager of each of a plurality of graph streaming processors, a plurality of threads operating on an array of processors of the graph streaming processor; [0024] Fig. 1 shows an acyclic graph mapped to stages of a graph streaming processor, according to an embodiment. For the example shown in FIG. 1, stage-0 is responsible for dispatching threads of the nodes n-00 (111) and n-01 (112). Further, stage-1 schedules the threads for the node n-10 (114), and the stage-2 schedules the threads of for the node n-20 (115), so on and so forth. The last stage of the pipeline is stage-N which schedules the threads for the node n-N0 (116), where is N is fixed for a specific GSP; [0048] the last stage 413 of the first GSP-0 (410) and the first stage 421 of the second GSP-1 (420) share the shared command buffer 430. For embodiments, the shared command buffer 430 can be physically located in memory of either of the GSPs 410, 420, but is located between the delineated stages 413, 421).
Verhaegh, Lipton, GUO and Puppala fail to specifically teach the deterministic information provided by a compiler of the system.
However, Heaton teaches the deterministic information provided by a compiler of the system. (Heaton, Col 1, line 64-Col 2, line 5, The neural network processor may include internal memory and an array of processing elements to perform neural network computations. The compiler engine and the runtime engine may operate in different computing systems of the computing environment. The compiler engine can allocate the memory and computation resources for the neural network processing operations, and provide information about the allocated memory and computation resources to the runtime engine).
It would have been obvious to one having ordinary skill in the art before the effective filling date of the claimed invention to have combined the teaching of Verhaegh, Lipton, GUO and Puppala with Heaton because Heaton’s teaching of compiler providing the computation information (i.e., allocated resources) to the runtime engine would have provided Verhaegh, Lipton, GUO and Puppala’s system with the advantage and capability to allow the system to easily performing the resource allocation based on the provided information in order to improving the system efficiency and resource utilization.
Claim 17 is rejected under 35 U.S.C. 103 as being unpatentable over Verhaegh, Lipton, GUO and Puppala, as applied to claim 15 above, and further in view of WANNER et al. (US Pub. 2020/0379911 A1).
As per claim 17, Verhaegh, Lipton, GUO and Puppala teach the invention according to claim 15 above. Verhaegh, Lipton, GUO and Puppala fail to specifically teach a compiler configured to: produce a plurality of binary executables from a source code of a model; and characterize the processor farm in advance of an arrival of each task of the plurality of tasks to account for availability of resources within the processor farm.
However, WANNER teaches a compiler configured to: produce a plurality of binary executables from a source code of a model (WANNER, [0023] The electronic device 110 includes a compiler 215, and a memory 240. The memory 240 includes neural network (NN) model source code 244, which after being compiled by the compiler 215, generates neural network (NN) binary executables 242 that can be deployed to different target platforms for execution); and
characterize the processor farm in advance of an arrival of each task of the plurality of tasks to account for availability of resources within the processor farm (WANNER, [0026] In an implementation, the compiler 215 analyzes the NN model source code 244 and determines which data of a given neural network (NN) model that would benefit from being placed in faster memory (e.g., the memcache 257) instead of slower memory (e.g., the DRAM 258). Such data may include, for example, data corresponding to the aforementioned input and output feature(s), and/or data structures of the NN model. By way of example, respective outputs of operations by the NN model can be in the form of data structures such as a container (e.g., tensor) that can store data in N dimensions (e.g., a matrix, a vector, array, array of arrays, etc.). [0027]; [0028] As referred to herein, a cache indicator may include information indicating whether to request an allocation of memory in the shared cache, or to perform another operation such as evicting or invalidating data already stored in the shared cache. Such information, in an example, may be included in an instruction (e.g., as part of a memory transaction) sent to a processor (e.g., CPU, GPU, NP) which is then processed by the processor for determine whether to request an allocation of memory within cache or slower memory or to evict a portion of memory. For allocating the memcache 257, the compiler 215 may use knowledge regarding a size of the memcache 257 available on a target device to determine whether an allocation of the memcache 257 is feasible; [0033] As further illustrated, the electronic device 115, in one or more implementations, includes a system-on-chip (SOC) 250. The SOC 250 may include an L2 cache 252 (e.g., on-chip memory), a CPU 254, a GPU 255, and a neural processor 256. (as processor farm) The electronic device 115 further includes a memcache 257 and a DRAM 258 (e.g., off-chip memory)).
It would have been obvious to one having ordinary skill in the art before the effective filling date of the claimed invention to have combined the teaching of Verhaegh, Lipton, GUO and Puppala with WANNER because WANNER’s teaching of allocation of the available faster memory resource within the processor farm in advance for processing the tasks would have provided Verhaegh, Lipton, GUO and Puppala’s system with the advantage and capability to allow the system to increasing the task processing speed by utilizing the fast memory resource which improving the system performance and efficiency.
Allowable Subject Matter
Claims 4-5, 12 and 18-20 are objected to as being dependent upon a rejected base claim, but would be allowable if overcomes the rejections under 35 U.S.C. 112 (a), 112(b) and 101, and rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Reasons of Allowable Subject Matter:
The closest prior arts of record Verhaegh et al. (US Pub. 2006/0192850 A1) teaches a computing system that set an output quality of a next media frame to provide the output quality of a plurality of output qualities of the next media frame based upon a self-learning control strategy that uses a processing time and an output quality of a previous media frame to determine the output quality of the next media frame (see Abstract, [0060-0063], [0051], [0068]; [0082-0085]).
Puppala et al. (US Pub. 2019/0188038 A1) teaches a computing system includes a cascade of graph streaming processors, wherein each of the graph streaming processor includes a processor array, and a graph streaming processor scheduler which is utilized for scheduling the tasks to be processed across the different graph streaming processors (see Fig. 4, Fig. 5 and [0064]).
Tristan et al. (US Pub. 2015/0095277 A1) teaches a system that having a probabilistic programming compiler that (a) generates data-parallel inference code to sample from probability distributions in models provided to the compiler; and (b) utilizes a modular framework to allow addition and removal of inference algorithm information based on which the compiler generates the inference code (see Tristan, [0072] Compiler 122 compiles source code (such as specification 200) representing an LDA model, of a body of data, into an intermediate representation of the model, such as intermediate representation expression 700 of FIG. 7; Abstract, generates data-parallel inference code to sample from probability distributions in models provided to the compiler).
Brady et al. (US Pub. 2019/0392296 A1) teaches a computing system having a compiler receives a graph describing a neural network and accesses data to describe a target computing device to implement the neural network. The compiler generates an intermediate representation from the graph and the data, where the intermediate representation includes an operator model, a data model, and a control model. The compiler generates a binary executable using each of the operator model, data model, and control model of the intermediate representation (see Abstract, Fig. 1, 140 intermediate representation; 115, 120 (as quality information); 150, binary; [0046]).
The features “wherein the scheduler is further configured to generate the quality information while performing one or more static capacity planning jobs when one or more new models of the plurality of models are being registered”, “wherein the scheduler is further configured to: select a binary executable of the plurality of binary executables for execution at one or more of the deterministic streaming processors, based on a number of computational cycles required for each of the plurality of binary executables to be executed”, “compiling, by a compiler of the deterministic streaming system, a source code of each model of a plurality of models associated with the plurality of tasks into an intermediate representation; generating, by the scheduler, quality information associated with a plurality of binary executables, based on the intermediate representation; compiling, by the compiler, the intermediate representation into the plurality of binary executables using the generated quality information; and selecting, by the scheduler, a binary executable of the plurality of binary executables for execution at one or more of the deterministic streaming processors, based on a number of computational cycles required for each of the plurality of binary executables to be executed”, “wherein the scheduler is further configured to: produce quality information for the plurality of binary executables, the quality information including information about at least one of an accuracy metric and a latency for each of the plurality of binary executables when executed at specific resources of the processor farm”, “wherein the scheduler is further configured to: provide the quality information to the compiler for compiling an intermediate representation of the model to generate the plurality of binary executables; and in response to a plurality of requests for the plurality of tasks, serve the plurality of requests with a binary executable of the plurality of binary executables, the binary executable yields a better performance at lower quality results to meet the defined contractual deadlines” and “wherein the scheduler comprises a capacity planner configured to: simulate the processor farm consisting of simulated leaky buckets for all existing and newly registered models that are filled with tasks representing a maximum load that any of the leaky buckets is configured to allow, a simulation cluster of deterministic streaming processors, and a simulation scheduler that mimics scheduling decisions of the scheduler, wherein the capacity planner uses worst case load conditions and information about a number of the existing registered models to statically accept or reject the newly registered models” when taken in the context of the claims as a whole, were not found in the prior art teachings.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ZUJIA XU whose telephone number is (571)272-0954. The examiner can normally be reached M-F 9:30-5:30 EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Aimee J Li can be reached at (571) 272-4169. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ZUJIA XU/Examiner, Art Unit 2195
Prosecution Insights