DETAILED ACTION
This action is in response to the application field on 09/29/2023. Claims 1-20 are pending and have been examined.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
The information disclosure statement (IDS) submitted on 04/01/2024. The submission is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are:
“a training operation acquiring module” as claimed in claim 8 stating “a training operation acquiring module, a priority order placement module, and an operation placement optimization module”.
“a priority order placement module” as claimed in claim 8 stating “a training operation acquiring module, a priority order placement module, and an operation placement optimization module”.
“an operation placement optimization module” as claimed in claim 8 stating “a training operation acquiring module, a priority order placement module, and an operation placement optimization module”.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 8 rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Regarding claim 8 , “A system for operation resource placement of deep learning, comprising a training operation acquiring module, a priority order placement module, and an operation placement optimization module; wherein the training operation acquiring module is configured for acquiring training operations to be placed and corresponding priorities; the priority order placement module is configured for selecting a network structure for operation placement according to required resource amount of the training operations in sequence based on an order of the priorities; wherein the network structure comprises a server, a top of rack, a container group set denoted as Podset, and a trunk layer switch; and the operation placement optimization module is configured for taking a transmission amount of network data in a training process as an optimization target to perform minimization optimization based on the selected network structure, and obtaining a corresponding operation placement scheme” invokes 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. In this instance, the corresponding structure refers to computer implemented means-plus function. The written description fails to disclose the corresponding structure, material, or acts for performing the entire claimed function and to clearly link the structure, material, or acts to the function. The specification does not provide sufficient details of any structure that is used to implement the training operation acquiring module, and a priority order placement module. Therefore, the claim is indefinite and is rejected under 35 U.S.C. 112(b) or pre-AIA 35 U.S.C. 112, second paragraph.
Applicant may:
(a) Amend the claim so that the claim limitation will no longer be interpreted as a limitation under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph;
(b) Amend the written description of the specification such that it expressly recites what structure, material, or acts perform the entire claimed function, without introducing any new matter (35 U.S.C. 132(a)); or
(c) Amend the written description of the specification such that it clearly links the structure, material, or acts disclosed therein to the function recited in the claim, without introducing any new matter (35 U.S.C. 132(a)).
If applicant is of the opinion that the written description of the specification already implicitly or inherently discloses the corresponding structure, material, or acts and clearly links them to the function so that one of ordinary skill in the art would recognize what structure, material, or acts perform the claimed function, applicant should clarify the record by either:
(a) Amending the written description of the specification such that it expressly recites the corresponding structure, material, or acts for performing the claimed function and clearly links or associates the structure, material, or acts to the claimed function, without introducing any new matter (35 U.S.C. 132(a)); or
(b) Stating on the record what the corresponding structure, material, or acts, which are implicitly or inherently set forth in the written description of the specification, perform the claimed function. For more information, see 37 CFR 1.75(d) and MPEP §§ 608.01(o) and 2181.
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claim 8 rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention.
Regarding claim 8, “A system for operation resource placement of deep learning, comprising a training operation acquiring module, a priority order placement module, and an operation placement optimization module; wherein the training operation acquiring module is configured for acquiring training operations to be placed and corresponding priorities; the priority order placement module is configured for selecting a network structure for operation placement according to required resource amount of the training operations in sequence based on an order of the priorities; wherein the network structure comprises a server, a top of rack, a container group set denoted as Podset, and a trunk layer switch; and the operation placement optimization module is configured for taking a transmission amount of network data in a training process as an optimization target to perform minimization optimization based on the selected network structure, and obtaining a corresponding operation placement scheme” as described above, does not provide adequate structure to perform the claimed function (See 112(b) rejection above). Therefore, the specification does not appear to provide sufficient detail such that on of ordinary skill can reasonable conclude that the inventor has possession of the claimed invention.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claim 10, and 17-20 rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter. The claim(s) does/do not fall within at least one of the four categories of patent eligible subject matter
Claims 10 and 17-20 recite a computer-readable storage medium, which may be interpreted as a signal or carrier wave, and thus is not one of the four statutory categories of patentable subject matter. The examiner suggests amending the claims to recite a non-transitory computer readable storage medium.
Claim 1-20 rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Regarding claim 1:
Subject Matter of Eligibility Analysis Step 1:
Claim 1 recites a method, which is directed to a process, and thus is one of the four statutory categories of patentable subject matter.
Subject Matter of Eligibility Analysis Step 2A Prong 1:
Claim 1 recites
based on an order of the priorities, selecting a network structure for operation placement according to required resource amount of the training operations in sequence (this limitation is a mental process as it encompasses a human mentally choosing a network structure to use).
based on the selected network structure, taking a transmission amount of network data in a training process as an optimization target to perform minimization optimization, and obtaining a corresponding operation placement scheme (this limitation is mental process as it encompasses a human mentally obtaining an operation placement scheme by using the minimization optimization model given in paragraph 0071).
Therefore, claim 1 recites an abstract idea.
Subject Matter of Eligibility Analysis Step 2A Prong 2:
Claim 1 further recites additional elements of
acquiring training operations to be placed and corresponding priorities (this element does not integrate the abstract idea into a practical application because it is merely data gathering, which is an insignificant extra-solution activity (see MPEP 2106.05(g))).
wherein the network structure comprises a server, a top of rack, a container group set denoted as Podset, and a trunk layer switch (this element does not integrate an abstract idea because it recites a field of use limitation to apply a judicial exception (see MPEP 2106.05(h))).
Therefore, claim 1 is not integrated into a practical application.
Subject Matter of Eligibility Analysis Step 2B:
The additional elements of claim 1 do not provide significantly more than the abstract idea itself, taken alone and in combination because
acquiring training operations to be placed and corresponding priorities is well understood, routine, and conventional. The court has ruled that “Receiving or transmitting data over a network, e.g., using the Internet to gather data” is recognized as a computer function that is well‐understood, routine, and conventional (buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014)).
wherein the network structure comprises a server, a top of rack, a container group set denoted as Podset, and a trunk layer switch recites a field of use limitation to apply a judicial exception (see MPEP 2106.05(h).
Therefore, claim 1 is subject-matter ineligible.
Regarding claim 2:
Subject Matter of Eligibility Analysis Step 1:
Claim 2 recites a method, which is directed to a process, and thus is one of the four statutory categories of patentable subject matter.
Subject Matter of Eligibility Analysis Step 2A Prong 1:
Claim 2 recites
classifying the training operations entering a cluster and adjusting resources of the training operations entering a cluster and adjusting resources of the training operations (this limitation is a mental process as it encompasses a human mentally classifying training operations and adjusting resources).
determining the priorities of the training operations according to a classification of the training operations and placing the training operations into a queue of training operations (this limitation is a mental process as it encompasses a human mentally determining the priority of an operation and placing it into a queue).
Therefore, claim 2 recites an abstract idea.
Subject Matter of Eligibility Analysis Step 2A Prong 2:
Claim 2 does not further recite any additional elements. Therefore, claim 2 is not integrated into a practical application.
Subject Matter of Eligibility Analysis Step 2B:
Since there are not additional elements, claim 2 does not provide significantly more than the abstract idea itself, taken alone or in combination. Therefore, claim 2 is subject matter ineligible.
Regarding claim 3:
Subject Matter of Eligibility Analysis Step 1:
Claim 3 recites a method, which is directed to a process, and thus is one of the four statutory categories of patentable subject matter.
Subject Matter of Eligibility Analysis Step 2A Prong 1:
Claim 3 recites
dividing cluster resources according to the number of network hops to obtain a multi- layer network structure (this limitation is a mental process as it encompasses a human mentally dividing resources based on the number of network hops).
selecting, layer by layer, the network structure adapted to the required resource amount of the training operations, based on a resource amount in each layer of the network structure (this limitation is a mental process as it encompasses a human mentally selecting a network structure to use based on the resource amount of each layer).
Therefore, claim 3 recites an abstract idea.
Subject Matter of Eligibility Analysis Step 2A Prong 2:
Claim 3 further recites additional elements of
extracting, from the queue of training operations the training operations to be placed according to the priorities (this element does not integrate the abstract idea into a practical application because it amounts to mere instructions to apply (see MPEP 2106.05(f))).
Therefore, claim 3 is not integrated into a practical application.
Subject Matter of Eligibility Analysis Step 2B:
The additional elements of claim 3 do not provide significantly more than the abstract idea itself, taken alone and in combination because
extracting, from the queue of training operations the training operations to be placed according to the priorities is an instruction to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(f).
Therefore, claim 3 is subject-matter ineligible.
Regarding claim 4:
Subject Matter of Eligibility Analysis Step 1:
Claim 4 recites a method, which is directed to a process, and thus is one of the four statutory categories of patentable subject matter.
Subject Matter of Eligibility Analysis Step 2A Prong 1:
Because claim 4 is dependent on claim 1, the Subject Matter of Eligibility Analysis Step 2A Prong 1 from claim 1 is applied here. Therefore claim 4 recites an abstract idea.
Subject Matter of Eligibility Analysis Step 2A Prong 2:
Claim 4 further recites additional elements of
indicating the transmission amount of network data in the training process based on parameter servers, workers, and the number of parameters of each training operation jointly, and obtaining the optimization target (this element does not integrate the abstract idea into a practical application because it amounts to mere instructions to apply (see MPEP 2106.05(f))).
establishing an optimization model of the transmission amount of network data based on the optimization target and a capacity of processing resources in a cluster as an optimization constraint (this element does not integrate the abstract idea into a practical application because it amounts to mere instructions to apply (see MPEP 2106.05(f))).
based on an optimization result of the optimization model of the transmission amount of network data, assigning the number of the parameter servers and the number of the workers, and processing resources of the parameter servers and processing resources of the workers, to each training operation in the network structure to obtain the operation placement scheme (this element does not integrate the abstract idea into a practical application because it amounts to mere instructions to apply (see MPEP 2106.05(f))).
Therefore, claim 4 is not integrated into a practical application.
Subject Matter of Eligibility Analysis Step 2B:
The additional elements of claim 4 do not provide significantly more than the abstract idea itself, taken alone and in combination because
indicating the transmission amount of network data in the training process based on parameter servers, workers, and the number of parameters of each training operation jointly, and obtaining the optimization target is an instruction to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(f).
establishing an optimization model of the transmission amount of network data based on the optimization target and a capacity of processing resources in a cluster as an optimization constraint is an instruction to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(f).
based on an optimization result of the optimization model of the transmission amount of network data, assigning the number of the parameter servers and the number of the workers, and processing resources of the parameter servers and processing resources of the workers, to each training operation in the network structure to obtain the operation placement scheme is an instruction to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(f).
Therefore, claim 4 is subject-matter ineligible.
Regarding claim 5:
Subject Matter of Eligibility Analysis Step 1:
Claim 5 recites a method, which is directed to a process, and thus is one of the four statutory categories of patentable subject matter.
Subject Matter of Eligibility Analysis Step 2A Prong 1:
Because claim 5 is dependent on claim 1, the Subject Matter of Eligibility Analysis Step 2A Prong 1 from claim 1 is applied here. Therefore claim 5 recites an abstract idea.
Subject Matter of Eligibility Analysis Step 2A Prong 2:
Claim 5 further recites additional elements of
when a plurality of training operations share the same processing resource, obtaining a raw time of the training operations by fitting, and obtaining a training time for an entire processing resource by normalization (this element does not integrate the abstract idea into a practical application because it amounts to mere instructions to apply (see MPEP 2106.05(f))).
Therefore, claim 5 is not integrated into a practical application.
Subject Matter of Eligibility Analysis Step 2B:
The additional elements of claim 5 do not provide significantly more than the abstract idea itself, taken alone and in combination because
when a plurality of training operations share the same processing resource, obtaining a raw time of the training operations by fitting, and obtaining a training time for an entire processing resource by normalization is an instruction to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(f).
Therefore, claim 5 is subject-matter ineligible.
Regarding claim 6:
Subject Matter of Eligibility Analysis Step 1:
Claim 6 recites a method, which is directed to a process, and thus is one of the four statutory categories of patentable subject matter.
Subject Matter of Eligibility Analysis Step 2A Prong 1:
Claim 6 recites
obtaining the raw time by measuring a forward propagation time and a backpropagation time of the training operations and fitting the forward propagation time and the backpropagation time of the training operations in conjunction with a gradient aggregation time (this limitation is a mental process as it encompasses a human mentally obtaining a raw time for training operations, if a forward propagation and back propagation algorithm was given).
Therefore, claim 6 recites an abstract idea.
Subject Matter of Eligibility Analysis Step 2A Prong 2:
Claim 6 further recites additional elements of
obtaining the raw time by measuring a forward propagation time and a backpropagation time of the training operations and fitting the forward propagation time and the backpropagation time of the training operations in conjunction with a gradient aggregation time (this element does not integrate the abstract idea into a practical application because it amounts to mere instructions to apply (see MPEP 2106.05(f))).
Therefore, claim 6 is not integrated into a practical application.
Subject Matter of Eligibility Analysis Step 2B:
The additional elements of claim 6 do not provide significantly more than the abstract idea itself, taken alone and in combination because
obtaining the raw time by measuring a forward propagation time and a backpropagation time of the training operations and fitting the forward propagation time and the backpropagation time of the training operations in conjunction with a gradient aggregation time is an instruction to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(f)).
Therefore, claim 6 is subject-matter ineligible.
Regarding claim 7:
Subject Matter of Eligibility Analysis Step 1:
Claim 7 recites a method, which is directed to a process, and thus is one of the four statutory categories of patentable subject matter.
Subject Matter of Eligibility Analysis Step 2A Prong 1:
Because claim 7 is dependent on claim 1, the Subject Matter of Eligibility Analysis Step 2A Prong 1 from claim 1 is applied here. Therefore claim 7 recites an abstract idea.
Subject Matter of Eligibility Analysis Step 2A Prong 2:
Claim 7 further recites additional elements of
establishing an overall scheduling algorithm of the training operations based on the number of remaining services required for the training operations and a capacity of processing resources in a cluster as an optimization constraint (this element does not integrate the abstract idea into a practical application because it amounts to mere instructions to apply (see MPEP 2106.05(f))).
periodically traversing processing resources of the training operations based on the overall scheduling algorithm of the training operations and obtaining an optimization result with a minimum number of remaining services (this element does not integrate the abstract idea into a practical application because it amounts to mere instructions to apply (see MPEP 2106.05(f))).
Therefore, claim 7 is not integrated into a practical application.
Subject Matter of Eligibility Analysis Step 2B:
The additional elements of claim 7 do not provide significantly more than the abstract idea itself, taken alone and in combination because
establishing an overall scheduling algorithm of the training operations based on the number of remaining services required for the training operations and a capacity of processing resources in a cluster as an optimization constraint is an instruction to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(f)).
periodically traversing processing resources of the training operations based on the overall scheduling algorithm of the training operations and obtaining an optimization result with a minimum number of remaining services is an instruction to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(f)).
Therefore, claim 7 is subject-matter ineligible.
Regarding claim 8:
Subject Matter of Eligibility Analysis Step 1:
Claim 8 recites a system, which is directed to a machine, and thus is one of the four statutory categories of patentable subject matter.
Subject Matter of Eligibility Analysis Step 2A Prong 1:
Claim 8 recites
the priority order placement module is configured for selecting a network structure for operation placement according to required resource amount of the training operations in sequence based on an order of the priorities (this limitation is a mental process as it encompasses a human mentally choosing a network structure to use).
the operation placement optimization module is configured for taking a transmission amount of network data in a training process as an optimization target to perform minimization optimization based on the selected network structure, and obtaining a corresponding operation placement scheme (this limitation is mental process as it encompasses a human mentally obtaining an operation placement scheme by using the minimization optimization model given in paragraph 0071).
Therefore, claim 8 recites an abstract idea.
Subject Matter of Eligibility Analysis Step 2A Prong 2:
Claim 8 further recites additional elements of
a training operation acquiring module, a priority order placement module, and an operation placement optimization module (this element does not integrate the abstract idea into a practical application because it recites generic computing components on which to perform the abstract idea (see MPEP 2106.05(b))).
the training operation acquiring module is configured for acquiring training operations to be placed and corresponding priorities (this element does not integrate the abstract idea into a practical application because it is merely data gathering, which is an insignificant extra-solution activity (see MPEP 2106.05(g))).
wherein the network structure comprises a server, a top of rack, a container group set denoted as Podset, and a trunk layer switch (this element does not integrate an abstract idea because it recites a field of use limitation to apply a judicial exception (see MPEP 2106.05(h))).
Therefore, claim 8 is not integrated into a practical application.
Subject Matter of Eligibility Analysis Step 2B:
The additional elements of claim 8 do not provide significantly more than the abstract idea itself, taken alone and in combination because
a training operation acquiring module, a priority order placement module, and an operation placement optimization module are generic components used to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(b)).
acquiring training operations to be placed and corresponding priorities is well understood, routine, and conventional. The court has ruled that “Receiving or transmitting data over a network, e.g., using the Internet to gather data” is recognized as a computer function that is well‐understood, routine, and conventional (buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014)).
wherein the network structure comprises a server, a top of rack, a container group set denoted as Podset, and a trunk layer switch recites a field of use limitation to apply a judicial exception (see MPEP 2106.05(h).
Therefore, claim 8 is subject-matter ineligible.
Regarding claim 9:
Subject Matter of Eligibility Analysis Step 1:
Claim 9 recites a computer device, which is directed to a machine, and thus is one of the four statutory categories of patentable subject matter.
Subject Matter of Eligibility Analysis Step 2A Prong 1:
Because claim 9 is dependent on claim 1, the Subject Matter of Eligibility Analysis Step 2A Prong 1 from claim 1 is applied here. Therefore claim 9 recites an abstract idea.
Subject Matter of Eligibility Analysis Step 2A Prong 2:
Claim 9 further recites additional elements of
A computer device, comprising a processor and a memory, wherein a computer program is stored by the memory and executable by the processor to implement the steps of the method for operation resource placement of deep learning of claim 1(this element does not integrate the abstract idea into a practical application because it recites generic computing components on which to perform the abstract idea (see MPEP 2106.05(b))).
Therefore, claim 9 is not integrated into a practical application.
Subject Matter of Eligibility Analysis Step 2B:
The additional elements of claim 9 do not provide significantly more than the abstract idea itself, taken alone and in combination because
A computer device, comprising a processor and a memory, wherein a computer program is stored by the memory and executable by the processor to implement the steps of the method for operation resource placement of deep learning of claim 1 are generic components used to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(b)).
Therefore, claim 9 is subject-matter ineligible.
Regarding claim 10:
Subject Matter of Eligibility Analysis Step 1:
Claim 10 recites a computer-readable storage medium, which may be interpreted as a signal or carrier wave, and thus is not one of the four statutory categories of patentable subject matter.
Subject Matter of Eligibility Analysis Step 2A Prong 1:
Because claim 10 is dependent on claim 1, the Subject Matter of Eligibility Analysis Step 2A Prong 1 from claim 1 is applied here. Therefore claim 10 recites an abstract idea.
Subject Matter of Eligibility Analysis Step 2A Prong 2:
Claim 10 further recites addition elements of
A computer-readable storage medium having stored a computer program, wherein the computer program is executed by a processor to implement the steps of the method for operation resource placement of deep learning of claim 1 (this element does not integrate the abstract idea into a practical application because it recites a generic computing component on which to perform the abstract idea (see MPEP 2106.05(b))).
Therefore, claim 10 is not integrated into a practical application.
Subject Matter of Eligibility Analysis Step 2B:
The additional elements of claim 10 do not provide significantly more than the abstract idea itself, taken alone and in combination because
A computer-readable storage medium having stored a computer program, wherein the computer program is executed by a processor to implement the steps of the method for operation resource placement of deep learning of claim 1 uses a computer as a tool to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(b)).
Therefore, claim 10 is subject-matter ineligible.
Regarding claim 11:
Subject Matter of Eligibility Analysis Step 1:
Claim 11 recites a computer device, which is directed to a machine, and thus is one of the four statutory categories of patentable subject matter.
Subject Matter of Eligibility Analysis Step 2A Prong 1:
Claim 11 recites
classifying the training operations entering a cluster and adjusting resources of the training operations entering a cluster and adjusting resources of the training operations (this limitation is a mental process as it encompasses a human mentally classifying training operations and adjusting resources).
determining the priorities of the training operations according to a classification of the training operations and placing the training operations into a queue of training operations (this limitation is a mental process as it encompasses a human mentally determining the priority of an operation and placing it into a queue).
Therefore, claim 11 recites an abstract idea.
Subject Matter of Eligibility Analysis Step 2A Prong 2:
Claim 11 does not further recite any additional elements. Therefore, claim 11 is not integrated into a practical application.
Subject Matter of Eligibility Analysis Step 2B:
Since there are not additional elements, claim 11 does not provide significantly more than the abstract idea itself, taken alone or in combination. Therefore, claim 11 is subject matter ineligible.
Regarding claim 12:
Subject Matter of Eligibility Analysis Step 1:
Claim 12 recites a computer device, which is directed to a machine, and thus is one of the four statutory categories of patentable subject matter.
Subject Matter of Eligibility Analysis Step 2A Prong 1:
Claim 12 recites
dividing cluster resources according to the number of network hops to obtain a multi- layer network structure (this limitation is a mental process as it encompasses a human mentally dividing resources based on the number of network hops).
selecting, layer by layer, the network structure adapted to the required resource amount of the training operations, based on a resource amount in each layer of the network structure (this limitation is a mental process as it encompasses a human mentally selecting a network structure to use based on the resource amount of each layer).
Therefore, claim 12 recites an abstract idea.
Subject Matter of Eligibility Analysis Step 2A Prong 2:
Claim 12 further recites additional elements of
extracting, from the queue of training operations the training operations to be placed according to the priorities (this element does not integrate the abstract idea into a practical application because it amounts to mere instructions to apply (see MPEP 2106.05(f))).
Therefore, claim 12 is not integrated into a practical application.
Subject Matter of Eligibility Analysis Step 2B:
The additional elements of claim 12 do not provide significantly more than the abstract idea itself, taken alone and in combination because
extracting, from the queue of training operations the training operations to be placed according to the priorities is an instruction to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(f).
Therefore, claim 12 is subject-matter ineligible.
Regarding claim 13:
Subject Matter of Eligibility Analysis Step 1:
Claim 13 recites a computer device, which is directed to a machine, and thus is one of the four statutory categories of patentable subject matter.
Subject Matter of Eligibility Analysis Step 2A Prong 1:
Because claim 13 is dependent on claim 9, the Subject Matter of Eligibility Analysis Step 2A Prong 1 from claim 9 is applied here. Therefore claim 13 recites an abstract idea.
Subject Matter of Eligibility Analysis Step 2A Prong 2:
Claim 13 further recites additional elements of
indicating the transmission amount of network data in the training process based on parameter servers, workers, and the number of parameters of each training operation jointly, and obtaining the optimization target (this element does not integrate the abstract idea into a practical application because it amounts to mere instructions to apply (see MPEP 2106.05(f))).
establishing an optimization model of the transmission amount of network data based on the optimization target and a capacity of processing resources in a cluster as an optimization constraint (this element does not integrate the abstract idea into a practical application because it amounts to mere instructions to apply (see MPEP 2106.05(f))).
based on an optimization result of the optimization model of the transmission amount of network data, assigning the number of the parameter servers and the number of the workers, and processing resources of the parameter servers and processing resources of the workers, to each training operation in the network structure to obtain the operation placement scheme (this element does not integrate the abstract idea into a practical application because it amounts to mere instructions to apply (see MPEP 2106.05(f))).
Therefore, claim 13 is not integrated into a practical application.
Subject Matter of Eligibility Analysis Step 2B:
The additional elements of claim 13 do not provide significantly more than the abstract idea itself, taken alone and in combination because
indicating the transmission amount of network data in the training process based on parameter servers, workers, and the number of parameters of each training operation jointly, and obtaining the optimization target is an instruction to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(f).
establishing an optimization model of the transmission amount of network data based on the optimization target and a capacity of processing resources in a cluster as an optimization constraint is an instruction to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(f).
based on an optimization result of the optimization model of the transmission amount of network data, assigning the number of the parameter servers and the number of the workers, and processing resources of the parameter servers and processing resources of the workers, to each training operation in the network structure to obtain the operation placement scheme is an instruction to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(f).
Therefore, claim 13 is subject-matter ineligible.
Regarding claim 14:
Subject Matter of Eligibility Analysis Step 1:
Claim 14 recites a computer device, which is directed to a machine, and thus is one of the four statutory categories of patentable subject matter.
Subject Matter of Eligibility Analysis Step 2A Prong 1:
Because claim 14 is dependent on claim 9, the Subject Matter of Eligibility Analysis Step 2A Prong 1 from claim 9 is applied here. Therefore claim 14 recites an abstract idea.
Subject Matter of Eligibility Analysis Step 2A Prong 2:
Claim 14 further recites additional elements of
when a plurality of training operations share the same processing resource, obtaining a raw time of the training operations by fitting, and obtaining a training time for an entire processing resource by normalization (this element does not integrate the abstract idea into a practical application because it amounts to mere instructions to apply (see MPEP 2106.05(f))).
Therefore, claim 14 is not integrated into a practical application.
Subject Matter of Eligibility Analysis Step 2B:
The additional elements of claim 14 do not provide significantly more than the abstract idea itself, taken alone and in combination because
when a plurality of training operations share the same processing resource, obtaining a raw time of the training operations by fitting, and obtaining a training time for an entire processing resource by normalization is an instruction to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(f).
Therefore, claim 14 is subject-matter ineligible.
Regarding claim 15:
Subject Matter of Eligibility Analysis Step 1:
Claim 15 recites a computer device, which is directed to a machine, and thus is one of the four statutory categories of patentable subject matter.
Subject Matter of Eligibility Analysis Step 2A Prong 1:
Claim 15 recites
obtaining the raw time by measuring a forward propagation time and a backpropagation time of the training operations and fitting the forward propagation time and the backpropagation time of the training operations in conjunction with a gradient aggregation time (this limitation is a mental process as it encompasses a human mentally obtaining a raw time for training operations, if a forward propagation and back propagation algorithm was given).
Therefore, claim 15 recites an abstract idea.
Subject Matter of Eligibility Analysis Step 2A Prong 2:
Claim 15 further recites additional elements of
obtaining the raw time by measuring a forward propagation time and a backpropagation time of the training operations and fitting the forward propagation time and the backpropagation time of the training operations in conjunction with a gradient aggregation time (this element does not integrate the abstract idea into a practical application because it amounts to mere instructions to apply (see MPEP 2106.05(f))).
Therefore, claim 15 is not integrated into a practical application.
Subject Matter of Eligibility Analysis Step 2B:
The additional elements of claim 15 do not provide significantly more than the abstract idea itself, taken alone and in combination because
obtaining the raw time by measuring a forward propagation time and a backpropagation time of the training operations and fitting the forward propagation time and the backpropagation time of the training operations in conjunction with a gradient aggregation time is an instruction to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(f)).
Therefore, claim 15 is subject-matter ineligible.
Regarding claim 16:
Subject Matter of Eligibility Analysis Step 1:
Claim 16 recites a computer device, which is directed to a machine, and thus is one of the four statutory categories of patentable subject matter.
Subject Matter of Eligibility Analysis Step 2A Prong 1:
Because claim 16 is dependent on claim 9, the Subject Matter of Eligibility Analysis Step 2A Prong 1 from claim 9 is applied here. Therefore claim 16 recites an abstract idea.
Subject Matter of Eligibility Analysis Step 2A Prong 2:
Claim 16 further recites additional elements of
establishing an overall scheduling algorithm of the training operations based on the number of remaining services required for the training operations and a capacity of processing resources in a cluster as an optimization constraint (this element does not integrate the abstract idea into a practical application because it amounts to mere instructions to apply (see MPEP 2106.05(f))).
periodically traversing processing resources of the training operations based on the overall scheduling algorithm of the training operations and obtaining an optimization result with a minimum number of remaining services (this element does not integrate the abstract idea into a practical application because it amounts to mere instructions to apply (see MPEP 2106.05(f))).
Therefore, claim 16 is not integrated into a practical application.
Subject Matter of Eligibility Analysis Step 2B:
The additional elements of claim 16 do not provide significantly more than the abstract idea itself, taken alone and in combination because
establishing an overall scheduling algorithm of the training operations based on the number of remaining services required for the training operations and a capacity of processing resources in a cluster as an optimization constraint is an instruction to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(f)).
periodically traversing processing resources of the training operations based on the overall scheduling algorithm of the training operations and obtaining an optimization result with a minimum number of remaining services is an instruction to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(f)).
Therefore, claim 16 is subject-matter ineligible.
Regarding claim 17:
Subject Matter of Eligibility Analysis Step 1:
Claim 17 recites a computer-readable storage medium, which may be interpreted as a signal or carrier wave, and thus is not one of the four statutory categories of patentable subject matter.
Subject Matter of Eligibility Analysis Step 2A Prong 1:
Claim 17 recites
classifying the training operations entering a cluster and adjusting resources of the training operations entering a cluster and adjusting resources of the training operations (this limitation is a mental process as it encompasses a human mentally classifying training operations and adjusting resources).
determining the priorities of the training operations according to a classification of the training operations and placing the training operations into a queue of training operations (this limitation is a mental process as it encompasses a human mentally determining the priority of an operation and placing it into a queue).
Therefore, claim 17 recites an abstract idea.
Subject Matter of Eligibility Analysis Step 2A Prong 2:
Claim 17 does not further recite any additional elements. Therefore, claim 11 is not integrated into a practical application.
Subject Matter of Eligibility Analysis Step 2B:
Since there are not additional elements, claim 17 does not provide significantly more than the abstract idea itself, taken alone or in combination. Therefore, claim 17 is subject matter ineligible.
Regarding claim 18:
Subject Matter of Eligibility Analysis Step 1:
Claim 18 recites a computer-readable storage medium, which may be interpreted as a signal or carrier wave, and thus is not one of the four statutory categories of patentable subject matter.
Subject Matter of Eligibility Analysis Step 2A Prong 1:
Claim 18 recites
dividing cluster resources according to the number of network hops to obtain a multi- layer network structure (this limitation is a mental process as it encompasses a human mentally dividing resources based on the number of network hops).
selecting, layer by layer, the network structure adapted to the required resource amount of the training operations, based on a resource amount in each layer of the network structure (this limitation is a mental process as it encompasses a human mentally selecting a network structure to use based on the resource amount of each layer).
Therefore, claim 18 recites an abstract idea.
Subject Matter of Eligibility Analysis Step 2A Prong 2:
Claim 18 further recites additional elements of
extracting, from the queue of training operations the training operations to be placed according to the priorities (this element does not integrate the abstract idea into a practical application because it amounts to mere instructions to apply (see MPEP 2106.05(f))).
Therefore, claim 18 is not integrated into a practical application.
Subject Matter of Eligibility Analysis Step 2B:
The additional elements of claim 18 do not provide significantly more than the abstract idea itself, taken alone and in combination because
extracting, from the queue of training operations the training operations to be placed according to the priorities is an instruction to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(f).
Therefore, claim 18 is subject-matter ineligible.
Regarding claim 19:
Subject Matter of Eligibility Analysis Step 1:
Claim 19 recites a computer-readable storage medium, which may be interpreted as a signal or carrier wave, and thus is not one of the four statutory categories of patentable subject matter.
Subject Matter of Eligibility Analysis Step 2A Prong 1:
Because claim 19 is dependent on claim 10, the Subject Matter of Eligibility Analysis Step 2A Prong 1 from claim 10 is applied here. Therefore claim 19 recites an abstract idea.
Subject Matter of Eligibility Analysis Step 2A Prong 2:
Claim 19 further recites additional elements of
indicating the transmission amount of network data in the training process based on parameter servers, workers, and the number of parameters of each training operation jointly, and obtaining the optimization target (this element does not integrate the abstract idea into a practical application because it amounts to mere instructions to apply (see MPEP 2106.05(f))).
establishing an optimization model of the transmission amount of network data based on the optimization target and a capacity of processing resources in a cluster as an optimization constraint (this element does not integrate the abstract idea into a practical application because it amounts to mere instructions to apply (see MPEP 2106.05(f))).
based on an optimization result of the optimization model of the transmission amount of network data, assigning the number of the parameter servers and the number of the workers, and processing resources of the parameter servers and processing resources of the workers, to each training operation in the network structure to obtain the operation placement scheme (this element does not integrate the abstract idea into a practical application because it amounts to mere instructions to apply (see MPEP 2106.05(f))).
Therefore, claim 19 is not integrated into a practical application.
Subject Matter of Eligibility Analysis Step 2B:
The additional elements of claim 19 do not provide significantly more than the abstract idea itself, taken alone and in combination because
indicating the transmission amount of network data in the training process based on parameter servers, workers, and the number of parameters of each training operation jointly, and obtaining the optimization target is an instruction to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(f).
establishing an optimization model of the transmission amount of network data based on the optimization target and a capacity of processing resources in a cluster as an optimization constraint is an instruction to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(f).
based on an optimization result of the optimization model of the transmission amount of network data, assigning the number of the parameter servers and the number of the workers, and processing resources of the parameter servers and processing resources of the workers, to each training operation in the network structure to obtain the operation placement scheme is an instruction to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(f).
Therefore, claim 19 is subject-matter ineligible.
Regarding claim 20:
Subject Matter of Eligibility Analysis Step 1:
Claim 20 recites a computer-readable storage medium, which may be interpreted as a signal or carrier wave, and thus is not one of the four statutory categories of patentable subject matter.
Subject Matter of Eligibility Analysis Step 2A Prong 1:
Because claim 20 is dependent on claim 10, the Subject Matter of Eligibility Analysis Step 2A Prong 1 from claim 10 is applied here. Therefore claim 20 recites an abstract idea.
Subject Matter of Eligibility Analysis Step 2A Prong 2:
Claim 20 further recites additional elements of
when a plurality of training operations share the same processing resource, obtaining a raw time of the training operations by fitting, and obtaining a training time for an entire processing resource by normalization (this element does not integrate the abstract idea into a practical application because it amounts to mere instructions to apply (see MPEP 2106.05(f))).
Therefore, claim 20 is not integrated into a practical application.
Subject Matter of Eligibility Analysis Step 2B:
The additional elements of claim 20 do not provide significantly more than the abstract idea itself, taken alone and in combination because
when a plurality of training operations share the same processing resource, obtaining a raw time of the training operations by fitting, and obtaining a training time for an entire processing resource by normalization is an instruction to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(f).
Therefore, claim 20 is subject-matter ineligible.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claim(s) 1-3 is/are rejected under 35 U.S.C. 103 as being unpatentable over Gu et al. (Tiresias: A GPU Cluster Manager for Distributed Deep Learning) (hereafter referred to as Gu) in view of Al-Fares et al. (A Scalable, Commodity Data Center Network Architecture) (hereafter referred to as Al-Fares).
Regarding claim 1, Gu teaches
acquiring training operations to be placed and corresponding priorities (Gu, Abstract, “We present Tiresias, a GPU cluster manager tailored for distributed DL training jobs, which efficiently schedules and places DL jobs to reduce their job completion times (JCTs)” and “At a high-level, 2DAS assigns each job a priority based on its attained service. The attained service of a job is calculated based on the number of GPUs it uses (WJ) and the amount of time it has been running so far (tJ). The former becomes known upon the job arrival, while the latter continuously increases” (Gu, Section 3.2.2).
based on an order of the priorities, selecting a network structure for operation placement according to required resource amount of the training operations in sequence (Gu, Section 1, “We implement an RDMA network profiling library in Tiresias that can determine the model structure of DDL jobs through network-level activities. By leveraging the profiling library and the iterative nature of DDL training, Tiresias can transparently and intelligently place jobs. Tiresias first runs the job in a trial environment for a few iterations, and then determines the best placement strategy according to the criteria summarized from previous measurements”).
based on the selected network structure, taking a transmission amount of network data in a training process as an optimization target to perform minimization optimization, and obtaining a corresponding operation placement scheme (Gu, Section 2.2, “Trying to minimize network communication during model aggregation is a common optimization in distributed training because the network can be a performance bottleneck and waste GPU cycles [31]. Hence, many existing GPU cluster managers blindly follow a consolidation constraint when placing DDL jobs– specifically, they assign all components (parameter servers and workers) of the job to the same or the minimum number of servers” and “Taking this viewpoint to its logical extreme, we created an ILP formulation to optimally allocate resources in the cluster to minimize and balance network transfers among machines” (Gu, Section 3.3)).
Gu does not teach, but Al-Fares does teach
the network structure comprises a server, a top of rack, a container group set denoted as Podset, and a trunk layer switch (Al-Fares, Section 2.2, “We organize a k-ary fat-tree as shown in Figure 3. There are k pods, each containing two layers of k/2 switches. Each k-port switch in the lower layer is directly connected to k/2 hosts. Each of the remaining k/2 ports is connected to k/2 of the k ports in the aggregation layer of the hierarchy”, “A three-tiered design has a core tier in the root of the tree, an aggregation tier in the middle and an edge tier at the leaves of the tree… Switches at the leaves of the tree have some number of GigE ports (48–288) as well as some number of10GigEuplinks to one or more layers of network elements that aggregate and transfer packets between the leaf switches” (Al-Fares, Section 2.1.1), and “There are k pods, each containing two layers of k/2 switches. Each k-port switch in the lower layer is directly connected to k/2 hosts… There are (k/2)2 k-port core switches. Each core switch has one port connected to each of k pods. The ith port of any core switch is connected to pod i such that consecutive ports in the aggregation layer of each pod switch are connected to core switches on (k/2) strides… As an example instance of this topology, a fat-tree built from 48 port GigE switches would consist of 48 pods, each containing an edge layer and an aggregation layer with 24 switches each. The edge switches in every pod are assigned 24 hosts each” (Al-Farse, Section 2.2).
PNG
media_image1.png
330
683
media_image1.png
Greyscale
Examiner notes that the host is mapped to a server, the edge tier switch is mapped to a top of rack, the pod is mapped to the podset, and the core switch is mapped to a trunk layer switch).
Gu and Al-Fares are considered analogous to the claimed invention because they both deal with network communications. It would have been obvious to one having ordinary skill in the art prior to the effective filing date to have modified Gu to have the network structure from Al-Fares. Al-Fares teaches “We show that by interconnecting commodity switches in a fat tree architecture, we can achieve the full bisection bandwidth of clusters consisting of tens of thousands of nodes... By leveraging strictly commodity switches, we achieve lower cost than existing solutions while simultaneously delivering more bandwidth” (Al-Fares, Section 1) (See MPEP 2141 (III)(G) Some teaching, suggestion, or motivation in the prior art that would have led one of ordinary skill to modify the prior art reference or to combine prior art reference teachings to arrive at the claimed invention.
Regarding claim 2, Gu and Al-Fares teach the method of claim 1, Gu further teaches
classifying the training operations entering a cluster and adjusting resources of the training operations (Gu, Section 1, “Our second idea is to use model structure to loosen the consolidated placement constraint whenever possible. We observe that only certain types of DL models are sensitive to whether they are consolidated or not, and their sensitivity is due to skew in tensor size distributions in their models. We use this insight to separate jobs into two categories: jobs that are sensitive to consolidation (high skew) and the rest” and “Given a job J that needs PSJ parameter servers and WJ workers, if there are enough resources in the cluster, Tiresias must determine how to allocate them. More specifically, it must determine whether to consolidate the job’s GPUs in as few machines as possible or to distribute them. The former is currently enforced in Microsoft production clusters; as a result, a job may be placed in the WAITQUEUE even if there are GPUs available across the cluster” (Gu, Section 3.3). Examiner notes that the insight to separate jobs into two categories maps to classifying the training operations).
determining the priorities of the training operations according to a classification of the training operations and placing the training operations into a queue of training operations (Gu, Section 3.2.2, “The priority function in 2DAS can be changed based on different prior knowledge. When no job duration information is provided, the priority function applies the LAS algorithm where a job’s priority is inverse to its attained service. If the cluster operator provides the distribution of job duration from previous experience, then a job’s priority equals its Gittins index value” and “Instead of using a continuous priority spectrum, we maintain K logical queues (Q1,Q2,...,QK), with queue priorities decreasing from Q1 to QK. The i-th queue contains jobs of attained service (WJtJ) values within [Qlo i ,Qhi i ).” (Gu, Section 3.2.3)).
Regarding claim 3, Gu and Al-Fares teach the method of claim 2, Gu further teaches
dividing cluster resources according to the number of network hops to obtain a multi- layer network structure (Gu, Section 3.3, “Given a job J that needs PSJ parameter servers and WJ workers, if there are enough resources in the cluster, Tiresias must determine how to allocate them. More specifically, it must determine whether to consolidate the job’s GPUs in as few machines as possible or to distribute them. The former is currently enforced in Microsoft production clusters; as a result, a job may be placed in the WAITQUEUE even if there are GPUs available across the cluster. Taking this viewpoint to its logical extreme, we created an ILP formulation to optimally allocate resources in the cluster to minimize and balance network transfers among machines”).
extracting, from the queue of training operations the training operations to be placed according to the priorities (Gu, Section 3.2.3, “Instead of using a continuous priority spectrum, we maintain K logical queues (Q1,Q2,...,QK), with queue priorities decreasing from Q1 to QK. The i-th queue contains jobs of attained service (WJtJ) values within [Qlo i ,Qhi i ).”).
selecting, layer by layer, the network structure adapted to the required resource amount of the training operations, based on a resource amount in each layer of the network structure (Gu, Section 3.3, “Given a job J that needs PS_J parameter servers and W_J workers, if there are enough resources in the cluster, Tiresias must determine how to allocate them. More specifically, it must determine whether to consolidate the job’s GPUs in as few machines as possible or to distribute them”).
Claim(s) 4 and 7 is/are rejected under 35 U.S.C. 103 as being unpatentable over Gu in view of Al-Fares and Peng et al. (Optimus: An Efficient Dynamic Resource Scheduler for Deep Learning Clusters) (hereafter referred to as Peng).
Regarding claim 4, Gu and Al-Fares teach the method of claim 1, Gu and Al-Fares does not teach, but Peng does teach
indicating the transmission amount of network data in the training process based on parameter servers, workers, and the number of parameters of each training operation jointly, and obtaining the optimization target (Peng, Section 3.2, “In a typical DL job, the time taken to complete one training step on a worker includes the time for doing forward propagation (i.e., loss computation) and backward propagation (i.e., gradients computation) at the worker, the worker pushing gradients to parameter servers, parameter servers updating parameters, and the worker pulling updated parameters from parameter servers, plus extra communication overhead. Suppose there are p parameter servers and w workers in the job. The bandwidth capacity of each parameter server is B, and the model size (i.e., total bytes of parameters) is S”).
establishing an optimization model of the transmission amount of network data based on the optimization target and a capacity of processing resources in a cluster as an optimization constraint (Peng, Section 4.1, “Our scheduler aims to minimize the average completion time of these jobs. We can solve the following optimization problem to decide the numbers of workers/parameter servers for each job j ∈ J, where (7) is the capacity constraint:
PNG
media_image2.png
116
332
media_image2.png
Greyscale
).
based on an optimization result of the optimization model of the transmission amount of network data, assigning the number of the parameter servers and the number of the workers, and processing resources of the parameter servers and processing resources of the workers, to each training operation in the network structure to obtain the operation placement scheme (Peng, Section 4.1, “Our resource allocation algorithm in each scheduling interval works as follows. We first allocate one worker and one parameter server to each active job to avoid starvation, and then sort all jobs in order of their marginal gains computed using (9). Then we iteratively select the job with the largest marginal gain and add one worker or parameter server to the job, according to which of the two terms in (9) is larger (i.e., whether adding a worker or adding a parameter server brings larger marginal gain). Marginal gains of the jobs are updated when their resource allocation changes. The procedure repeats until some resource in the cluster is used up, or marginal gains of all jobs become non-positive”).
Gu, Al-Fares, and Peng are considered analogous to the claimed invention because they deal with deep learning clusters. It would have been obvious to one having ordinary skill in the art prior to the effective filing date to have modified Gu and Al-Fares use the resource allocation algorithm from Peng. One of the ordinary skill in the art would have known to apply the known technique of optimizing resource allocation for deep learning tasks. Therefore, apply Peng’s technique would yield the predicable result of increasing speed and efficiency of completing deep learning tasks (See MPEP 2141 (III)(D) Applying a known technique to a known device ready for improvement to yield predicable results).
Regarding claim 7, Gu and Al-Fare teach the method of claim 1, Gu further teaches
establishing an overall scheduling algorithm of the training operations based on the number of remaining services required for the training operations and a capacity of processing resources in a cluster as an optimization constraint (Gu, Section 3.2.1, “To quantify the approaches, we ran trace-driven simulations on three different schedulers using the Microsoft production trace: (1) smallest-first (SF); (2) SRTF; and (3) shortest-remaining-service-first (SRSF). Of them, the first two are single-dimensional schedulers; the last one considers both spatial and temporal aspects. The remaining service in SRSF is the multiplication of a job’s remaining time and the number of GPUs. For this simulation, we assume that job durations are given when needed. Table 1 shows that SRSF outperforms the rest in minimizing the average JCT. SRSF has a much smaller tail JCT than the single-dimensional counterparts as well. Altogether, we move forward in building a DDL scheduler that considers both spatial and temporal aspects of resource usage”).
Gu does not teach, but Peng does teach
periodically traversing processing resources of the training operations based on the overall scheduling algorithm of the training operations and obtaining an optimization result with a minimum number of remaining services (Peng, Section 4.1, “Our resource allocation algorithm in each scheduling interval works as follows. We first allocate one worker and one parameter server to each active job to avoid starvation, and then sort all jobs in order of their marginal gains computed using (9). Then we iteratively select the job with the largest marginal gain and add one worker or parameter server to the job ... The procedure repeats until some resource in the cluster is used up, or marginal gains of all jobs become non-positive”).
Gu, Al-Fares, and Peng are considered analogous to the claimed invention because they deal with deep learning clusters. It would have been obvious to one having ordinary skill in the art prior to the effective filing date to have modified Gu and Al-Fares use the resource allocation algorithm from Peng. One of the ordinary skill in the art would have known to apply the known technique of optimizing resource allocation for deep learning tasks. Therefore, apply Peng’s technique would yield the predicable result of increasing speed and efficiency of completing deep learning tasks (See MPEP 2141 (III)(D) Applying a known technique to a known device ready for improvement to yield predicable results).
Claim(s) 5 and 6 is/are rejected under 35 U.S.C. 103 as being unpatentable over Gu in view of Al-Fares, Peng, and Xiao et al. (Gandiva: Introspective Cluster Scheduling for Deep Learning) (hereafter referred to as Xiao).
Regarding claim 5, Gu, Al-Fares, and Peng teach the method of claim 4, Gu and Al-Fares does not teach, but Xiao does teach
when a plurality of training operations share the same processing resource, obtaining a raw time of the training operations by fitting, and obtaining a training time for an entire processing resource by normalization (Xiao, Section 4.1, “An alternative to suspend-resume for time-slicing is to run multiple DLT jobs on a GPU simultaneously and let the GPU time-share the jobs. We call this packing. Packing in GPU is efficient only when the packed jobs do not exceed the GPU resources (cores, memory) and do not adversely impact each other … Gandiva estimates a DLT job’s mini batch time, the time to do one forward/backward pass over a batch of input data, as the time taken between two minimums of the GPU memory usage cycles (Figure 5(a)). Because DLT jobs typically perform millions of such mini batch operations in their lifetime, the scheduler compares the mini batch time of a DLT prior to and post a scheduling decision to determine its effectiveness … By comparing the mini batch time of each of the two DLT jobs before and after packing, Gandiva can decide whether packing is effective.”. Examiner notes that the mini batch time is mapped to the raw time).
Gu, Al-Fares, and Xiao are considered analogous to the claimed invention because they deal with deep learning clusters. It would have been obvious to one having ordinary skill in the art prior to the effective filing date to have modified Gu and Al-Fares use Gandiva’s scheduler from Xiao. Xiao teaches “in a real workload of jobs running in a 180-GPU cluster, Gandiva improves aggregate cluster utilization by 26%, pointing to a new way of managing large GPU clusters for deep learning” (Xiao, Abstract) (See MPEP 2141 (III)(G) Some teaching, suggestion, or motivation in the prior art that would have led one of ordinary skill to modify the prior art reference or to combine prior art reference teachings to arrive at the claimed invention.
Regarding claim 6, Gu, Al-Fares, Peng, and Xiao teach the method of claim 5, Xiao further teaches
obtaining the raw time by measuring a forward propagation time and a backpropagation time of the training operations and fitting the forward propagation time and the backpropagation time of the training operations in conjunction with a gradient aggregation time (Xiao, Section 2, “It computes a set of scores for each mini-batch by performing numerical computations using the model weights, called the forward pass. Based on the desired task, an objective function is defined that measures an error between the computed scores and desired scores. The error is populated via a backward pass over the model, where it first computes a gradient for each weight (i.e., the impact of each weight on the error) and then applies a negative of the gradient, scaled by a parameter called the learning rate, to each weight to decrease the error. Both the forward and backward passes typically involve billions of floating point operations and thus leverage GPUs. Each forward backward pass is called a mini-batch iteration”).
Gu, Al-Fares, and Xiao are considered analogous to the claimed invention because they deal with deep learning clusters. It would have been obvious to one having ordinary skill in the art prior to the effective filing date to have modified Gu and Al-Fares use Gandiva’s scheduler from Xiao. Xiao teaches “in a real workload of jobs running in a 180-GPU cluster, Gandiva improves aggregate cluster utilization by 26%, pointing to a new way of managing large GPU clusters for deep learning” (Xiao, Abstract) (See MPEP 2141 (III)(G) Some teaching, suggestion, or motivation in the prior art that would have led one of ordinary skill to modify the prior art reference or to combine prior art reference teachings to arrive at the claimed invention.
Claim(s) 8-12, 17, and 18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Gu in view of Al-Fares, and Phanishayee et al. (US 20240160471 A1) (hereafter referred to as Phanishayee).
Regarding claim 8, Gu teaches
a training operation acquiring module, a priority order placement module, and an operation placement optimization module (Gu, Section 2.2, “Trying to minimize network communication during model aggregation is a common optimization in distributed training because the network can be a performance bottleneck and waste GPU cycles [31]. Hence, many existing GPU cluster managers blindly follow a consolidation constraint when placing DDL jobs– specifically, they assign all components (parameter servers and workers) of the job to the same or the minimum number of servers”. Examiner notes that the GPU clusters manager is mapped to the modules).
the training operation acquiring module is configured for acquiring training operations to be placed and corresponding priorities (Hu, Abstract, “We present Tiresias, a GPU cluster manager tailored for distributed DL training jobs, which efficiently schedules and places DL jobs to reduce their job completion times (JCTs)” and “At a high-level, 2DAS assigns each job a priority based on its attained service. The attained service of a job is calculated based on the number of GPUs it uses (WJ) and the amount of time it has been running so far (tJ). The former becomes known upon the job arrival, while the latter continuously increases” (Gu, Section 3.2.2).
the priority order placement module is configured for selecting a network structure for operation placement according to required resource amount of the training operations in sequence based on an order of the priorities (Gu, Section 1, “We implement an RDMA network profiling library in Tiresias that can determine the model structure of DDL jobs through network-level activities. By leveraging the profiling library and the iterative nature of DDL training, Tiresias can transparently and intelligently place jobs. Tiresias first runs the job in a trial environment for a few iterations, and then determines the best placement strategy according to the criteria summarized from previous measurements”).
the operation placement optimization module is configured for taking a transmission amount of network data in a training process as an optimization target to perform minimization optimization based on the selected network structure, and obtaining a corresponding operation placement scheme (Gu, Section 2.2, “Trying to minimize network communication during model aggregation is a common optimization in distributed training because the network can be a performance bottleneck and waste GPU cycles [31]. Hence, many existing GPU cluster managers blindly follow a consolidation constraint when placing DDL jobs– specifically, they assign all components (parameter servers and workers) of the job to the same or the minimum number of servers” and “Taking this viewpoint to its logical extreme, we created an ILP formulation to optimally allocate resources in the cluster to minimize and balance network transfers among machines” (Gu, Section 3.3)).
Gu does not teach, but Al-Fares does teach
the network structure comprises a server, a top of rack, a container group set denoted as Podset, and a trunk layer switch (Al-Fares, Section 2.2, “We organize a k-ary fat-tree as shown in Figure 3. There are k pods, each containing two layers of k/2 switches. Each k-port switch in the lower layer is directly connected to k/2 hosts. Each of the remaining k/2 ports is connected to k/2 of the k ports in the aggregation layer of the hierarchy”, “A three-tiered design has a core tier in the root of the tree, an aggregation tier in the middle and an edge tier at the leaves of the tree… Switches at the leaves of the tree have some number of GigE ports (48–288) as well as some number of10GigEuplinks to one or more layers of network elements that aggregate and transfer packets between the leaf switches” (Al-Fares, Section 2.1.1), and “There are k pods, each containing two layers of k/2 switches. Each k-port switch in the lower layer is directly connected to k/2 hosts… There are (k/2)2 k-port core switches. Each core switch has one port connected to each of k pods. The ith port of any core switch is connected to pod i such that consecutive ports in the aggregation layer of each pod switch are connected to core switches on (k/2) strides… As an example instance of this topology, a fat-tree built from 48 port GigE switches would consist of 48 pods, each containing an edge layer and an aggregation layer with 24 switches each. The edge switches in every pod are assigned 24 hosts each” (Al-Farse, Section 2.2).
PNG
media_image1.png
330
683
media_image1.png
Greyscale
Examiner notes that the host is mapped to a server, the edge tier switch is mapped to a top of rack, the pod is mapped to the podset, and the core switch is mapped to a trunk layer switch).
Gu and Al-Fares are considered analogous to the claimed invention because they both deal with network communications. It would have been obvious to one having ordinary skill in the art prior to the effective filing date to have modified Gu to have the network structure from Al-Fares. Al-Fares teaches “We show that by interconnecting commodity switches in a fat tree architecture, we can achieve the full bisection bandwidth of clusters consisting of tens of thousands of nodes... By leveraging strictly commodity switches, we achieve lower cost than existing solutions while simultaneously delivering more bandwidth” (Al-Fares, Section 1) (See MPEP 2141 (III)(G) Some teaching, suggestion, or motivation in the prior art that would have led one of ordinary skill to modify the prior art reference or to combine prior art reference teachings to arrive at the claimed invention.
Gu and Al-Fares does not teach, but Phanishayee does teach
a training operation acquiring module, a priority order placement module, and an operation placement optimization module (Phanishayee, paragraphs 0104-0107, “FIG. 7 shows another example deep learning cluster scheduler modular toolkit method or technique 700. Block 702 can provide a deep learning cluster scheduler modular toolkit that includes multiple modular deep learning scheduler abstractions and interaction paths between the multiple modular deep learning scheduler abstractions. Block 704 can receive user input for individual modular deep learning scheduler abstractions. Block 706 can compose multiple modular deep learning scheduler abstraction modules to realize a DL scheduler from the multiple modular deep learning scheduler abstractions and the user input that follows the interaction paths“).
Gu, Al-Fares, and Phanishayee are considered analogous to the claimed invention because they deep learning scheduling. It would have been obvious to one having ordinary skill in the art prior to the effective filing date to have modified Gu and Al-Farse to use the deep learning cluster scheduler modular toolkit from Phanishayee. One of the ordinary skill in the art would have known to apply the known technique of using a computer machine/module to perform instructions. Therefore, applying Phanishayee’s technique would yield the predictable result of running instructions on computer modules (See MPEP 2141 (III)(D) Applying a known technique to a known device ready for improvement to yield predicable results.
Regarding claim 9, Gu and Al-Fares teach the method of claim 1, Phanishayee teaches
A computer device, comprising a processor and a memory, wherein a computer program is stored by the memory and executable by the processor (Phanishayee, paragraph 0101, “The term “device,” “computer,” or “computing device” as used herein can mean any type of device that has some amount of processing capability and/or storage capability. Processing capability can be provided by one or more processors that can execute data in the form of computer-readable instructions to provide a functionality”).
Gu, Al-Fares, and Phanishayee are considered analogous to the claimed invention because they deep learning scheduling. It would have been obvious to one having ordinary skill in the art prior to the effective filing date to have modified Gu and Al-Farse to include one or more processors from Phanishayee. One of the ordinary skill in the art would have known to apply the known technique of using a processor to perform instructions. Therefore, applying Phanishayee’s technique would yield the predictable result of running instructions on a computer (See MPEP 2141 (III)(D) Applying a known technique to a known device ready for improvement to yield predicable results.
Regarding claim 10, Gu and Al-Fares teach the method of claim 1, Gu and Al-Fares does not teach, but Phanishayee does teach
A computer-readable storage medium having stored a computer program, wherein the computer program is executed by a processor (Phanishayee, paragraph 0101, “The storage can include any one or more of volatile or non-volatile memory, hard drives, flash storage devices, and/or optical storage devices (e.g., CDs, DVDs etc.), remote storage (e.g., cloud-based storage), among others. As used herein, the term “computer-readable media” can include signals. In contrast, the term ‘computer-readable storage media’ excludes signals. Computer-readable storage media includes ‘computer-readable storage devices’”) .
Gu, Al-Fares, and Phanishayee are considered analogous to the claimed invention because they deep learning scheduling. It would have been obvious to one having ordinary skill in the art prior to the effective filing date to have modified Gu and Al-Farse to include a computer-readable storage media from Phanishayee. One of the ordinary skill in the art would have known to apply the known technique of using a computer-readable storage media to store data. Therefore, applying Phanishayee’s technique would yield the predictable result of holding data on a computer-readable storage media (See MPEP 2141 (III)(D) Applying a known technique to a known device ready for improvement to yield predicable results.
Regarding claim 11, Gu, Al-Fares, and Phanishayee teach the system of claim 9, Gu further teaches
classifying the training operations entering a cluster and adjusting resources of the training operations (Gu, Section 1, “Our second idea is to use model structure to loosen the consolidated placement constraint whenever possible. We observe that only certain types of DL models are sensitive to whether they are consolidated or not, and their sensitivity is due to skew in tensor size distributions in their models. We use this insight to separate jobs into two categories: jobs that are sensitive to consolidation (high skew) and the rest” and “Given a job J that needs PSJ parameter servers and WJ workers, if there are enough resources in the cluster, Tiresias must determine how to allocate them. More specifically, it must determine whether to consolidate the job’s GPUs in as few machines as possible or to distribute them. The former is currently enforced in Microsoft production clusters; as a result, a job may be placed in the WAITQUEUE even if there are GPUs available across the cluster” (Gu, Section 3.3). Examiner notes that the insight to separate jobs into two categories maps to classifying the training operations).
determining the priorities of the training operations according to a classification of the training operations and placing the training operations into a queue of training operations (Gu, Section 3.2.2, “The priority function in 2DAS can be changed based on different prior knowledge. When no job duration information is provided, the priority function applies the LAS algorithm where a job’s priority is inverse to its attained service. If the cluster operator provides the distribution of job duration from previous experience, then a job’s priority equals its Gittins index value” and “Instead of using a continuous priority spectrum, we maintain K logical queues (Q1,Q2,...,QK), with queue priorities decreasing from Q1 to QK. The i-th queue contains jobs of attained service (WJtJ) values within [Qlo i ,Qhi i ).” (Gu, Section 3.2.3)).
Regarding claim 12, Gu, Al-Fares, and Phanishayee teach the system of claim 11, Gu further teaches
dividing cluster resources according to the number of network hops to obtain a multi- layer network structure (Gu, Section 3.3, “Given a job J that needs PSJ parameter servers and WJ workers, if there are enough resources in the cluster, Tiresias must determine how to allocate them. More specifically, it must determine whether to consolidate the job’s GPUs in as few machines as possible or to distribute them. The former is currently enforced in Microsoft production clusters; as a result, a job may be placed in the WAITQUEUE even if there are GPUs available across the cluster. Taking this viewpoint to its logical extreme, we created an ILP formulation to optimally allocate resources in the cluster to minimize and balance network transfers among machines”).
extracting, from the queue of training operations the training operations to be placed according to the priorities (Gu, Section 3.2.3, “Instead of using a continuous priority spectrum, we maintain K logical queues (Q1,Q2,...,QK), with queue priorities decreasing from Q1 to QK. The i-th queue contains jobs of attained service (WJtJ) values within [Qlo i ,Qhi i ).”).
selecting, layer by layer, the network structure adapted to the required resource amount of the training operations, based on a resource amount in each layer of the network structure (Gu, Section 3.3, “Given a job J that needs PS_J parameter servers and W_J workers, if there are enough resources in the cluster, Tiresias must determine how to allocate them. More specifically, it must determine whether to consolidate the job’s GPUs in as few machines as possible or to distribute them”).
Regarding claim 17, Gu, Al-Fares, and Phanishayee teach the computer-readable storage medium of claim 10, Gu further teaches
classifying the training operations entering a cluster and adjusting resources of the training operations (Gu, Section 1, “Our second idea is to use model structure to loosen the consolidated placement constraint whenever possible. We observe that only certain types of DL models are sensitive to whether they are consolidated or not, and their sensitivity is due to skew in tensor size distributions in their models. We use this insight to separate jobs into two categories: jobs that are sensitive to consolidation (high skew) and the rest” and “Given a job J that needs PSJ parameter servers and WJ workers, if there are enough resources in the cluster, Tiresias must determine how to allocate them. More specifically, it must determine whether to consolidate the job’s GPUs in as few machines as possible or to distribute them. The former is currently enforced in Microsoft production clusters; as a result, a job may be placed in the WAITQUEUE even if there are GPUs available across the cluster” (Gu, Section 3.3). Examiner notes that the insight to separate jobs into two categories maps to classifying the training operations).
determining the priorities of the training operations according to a classification of the training operations and placing the training operations into a queue of training operations (Gu, Section 3.2.2, “The priority function in 2DAS can be changed based on different prior knowledge. When no job duration information is provided, the priority function applies the LAS algorithm where a job’s priority is inverse to its attained service. If the cluster operator provides the distribution of job duration from previous experience, then a job’s priority equals its Gittins index value” and “Instead of using a continuous priority spectrum, we maintain K logical queues (Q1,Q2,...,QK), with queue priorities decreasing from Q1 to QK. The i-th queue contains jobs of attained service (WJtJ) values within [Qlo i ,Qhi i ).” (Gu, Section 3.2.3)).
Regarding claim 18, Gu, Al-Fares, and Phanishayee teach the computer-readable storage medium of claim 10, Gu further teaches
dividing cluster resources according to the number of network hops to obtain a multi- layer network structure (Gu, Section 3.3, “Given a job J that needs PSJ parameter servers and WJ workers, if there are enough resources in the cluster, Tiresias must determine how to allocate them. More specifically, it must determine whether to consolidate the job’s GPUs in as few machines as possible or to distribute them. The former is currently enforced in Microsoft production clusters; as a result, a job may be placed in the WAITQUEUE even if there are GPUs available across the cluster. Taking this viewpoint to its logical extreme, we created an ILP formulation to optimally allocate resources in the cluster to minimize and balance network transfers among machines”).
extracting, from the queue of training operations the training operations to be placed according to the priorities (Gu, Section 3.2.3, “Instead of using a continuous priority spectrum, we maintain K logical queues (Q1,Q2,...,QK), with queue priorities decreasing from Q1 to QK. The i-th queue contains jobs of attained service (WJtJ) values within [Qlo i ,Qhi i ).”).
selecting, layer by layer, the network structure adapted to the required resource amount of the training operations, based on a resource amount in each layer of the network structure (Gu, Section 3.3, “Given a job J that needs PS_J parameter servers and W_J workers, if there are enough resources in the cluster, Tiresias must determine how to allocate them. More specifically, it must determine whether to consolidate the job’s GPUs in as few machines as possible or to distribute them”).
Claim(s) 13, 16, and 19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Gu in view of Al-Fares, Phanishayee, and Peng
Regarding claim 13, Gu, Al-Fares, and Phanishayee teach the system of claim 9, Gu and Al-Fares does not teach, but Peng does teach
indicating the transmission amount of network data in the training process based on parameter servers, workers, and the number of parameters of each training operation jointly, and obtaining the optimization target (Peng, Section 3.2, “In a typical DL job, the time taken to complete one training step on a worker includes the time for doing forward propagation (i.e., loss computation) and backward propagation (i.e., gradients computation) at the worker, the worker pushing gradients to parameter servers, parameter servers updating parameters, and the worker pulling updated parameters from parameter servers, plus extra communication overhead. Suppose there are p parameter servers and w workers in the job. The bandwidth capacity of each parameter server is B, and the model size (i.e., total bytes of parameters) is S”).
establishing an optimization model of the transmission amount of network data based on the optimization target and a capacity of processing resources in a cluster as an optimization constraint (Peng, Section 4.1, “Our scheduler aims to minimize the average completion time of these jobs. We can solve the following optimization problem to decide the numbers of workers/parameter servers for each job j ∈ J, where (7) is the capacity constraint:
PNG
media_image2.png
116
332
media_image2.png
Greyscale
).
based on an optimization result of the optimization model of the transmission amount of network data, assigning the number of the parameter servers and the number of the workers, and processing resources of the parameter servers and processing resources of the workers, to each training operation in the network structure to obtain the operation placement scheme (Peng, Section 4.1, “Our resource allocation algorithm in each scheduling interval works as follows. We first allocate one worker and one parameter server to each active job to avoid starvation, and then sort all jobs in order of their marginal gains computed using (9). Then we iteratively select the job with the largest marginal gain and add one worker or parameter server to the job, according to which of the two terms in (9) is larger (i.e., whether adding a worker or adding a parameter server brings larger marginal gain). Marginal gains of the jobs are updated when their resource allocation changes. The procedure repeats until some resource in the cluster is used up, or marginal gains of all jobs become non-positive”).
Gu, Al-Fares, Phanishayee, and Peng are considered analogous to the claimed invention because they deal with deep learning clusters. It would have been obvious to one having ordinary skill in the art prior to the effective filing date to have modified Gu and Al-Fares use the resource allocation algorithm from Peng. One of the ordinary skill in the art would have known to apply the known technique of optimizing resource allocation for deep learning tasks. Therefore, apply Peng’s technique would yield the predicable result of increasing speed and efficiency of completing deep learning tasks (See MPEP 2141 (III)(D) Applying a known technique to a known device ready for improvement to yield predicable results).
Regarding claim 16, Gu, Al-Fares, and Phanishayee teach the system of claim 9, Gu further teaches
establishing an overall scheduling algorithm of the training operations based on the number of remaining services required for the training operations and a capacity of processing resources in a cluster as an optimization constraint (Gu, Section 3.2.1, “To quantify the approaches, we ran trace-driven simulations on three different schedulers using the Microsoft production trace: (1) smallest-first (SF); (2) SRTF; and (3) shortest-remaining-service-first (SRSF). Of them, the first two are single-dimensional schedulers; the last one considers both spatial and temporal aspects. The remaining service in SRSF is the multiplication of a job’s remaining time and the number of GPUs. For this simulation, we assume that job durations are given when needed. Table 1 shows that SRSF outperforms the rest in minimizing the average JCT. SRSF has a much smaller tail JCT than the single-dimensional counterparts as well. Altogether, we move forward in building a DDL scheduler that considers both spatial and temporal aspects of resource usage”).
Gu does not teach, but Peng does teach
periodically traversing processing resources of the training operations based on the overall scheduling algorithm of the training operations and obtaining an optimization result with a minimum number of remaining services (Peng, Section 4.1, “Our resource allocation algorithm in each scheduling interval works as follows. We first allocate one worker and one parameter server to each active job to avoid starvation, and then sort all jobs in order of their marginal gains computed using (9). Then we iteratively select the job with the largest marginal gain and add one worker or parameter server to the job ... The procedure repeats until some resource in the cluster is used up, or marginal gains of all jobs become non-positive”).
Gu, Al-Fares, Phanishayee, and Peng are considered analogous to the claimed invention because they deal with deep learning clusters. It would have been obvious to one having ordinary skill in the art prior to the effective filing date to have modified Gu and Al-Fares use the resource allocation algorithm from Peng. One of the ordinary skill in the art would have known to apply the known technique of optimizing resource allocation for deep learning tasks. Therefore, apply Peng’s technique would yield the predicable result of increasing speed and efficiency of completing deep learning tasks (See MPEP 2141 (III)(D) Applying a known technique to a known device ready for improvement to yield predicable results).
Regarding claim 19, Gu, Al-Fares, and Phanishayee teach the computer-readable storage medium of claim 10, Gu and Al-Fares does not teach, but Peng does teach
indicating the transmission amount of network data in the training process based on parameter servers, workers, and the number of parameters of each training operation jointly, and obtaining the optimization target (Peng, Section 3.2, “In a typical DL job, the time taken to complete one training step on a worker includes the time for doing forward propagation (i.e., loss computation) and backward propagation (i.e., gradients computation) at the worker, the worker pushing gradients to parameter servers, parameter servers updating parameters, and the worker pulling updated parameters from parameter servers, plus extra communication overhead. Suppose there are p parameter servers and w workers in the job. The bandwidth capacity of each parameter server is B, and the model size (i.e., total bytes of parameters) is S”).
establishing an optimization model of the transmission amount of network data based on the optimization target and a capacity of processing resources in a cluster as an optimization constraint (Peng, Section 4.1, “Our scheduler aims to minimize the average completion time of these jobs. We can solve the following optimization problem to decide the numbers of workers/parameter servers for each job j ∈ J, where (7) is the capacity constraint:
PNG
media_image2.png
116
332
media_image2.png
Greyscale
).
based on an optimization result of the optimization model of the transmission amount of network data, assigning the number of the parameter servers and the number of the workers, and processing resources of the parameter servers and processing resources of the workers, to each training operation in the network structure to obtain the operation placement scheme (Peng, Section 4.1, “Our resource allocation algorithm in each scheduling interval works as follows. We first allocate one worker and one parameter server to each active job to avoid starvation, and then sort all jobs in order of their marginal gains computed using (9). Then we iteratively select the job with the largest marginal gain and add one worker or parameter server to the job, according to which of the two terms in (9) is larger (i.e., whether adding a worker or adding a parameter server brings larger marginal gain). Marginal gains of the jobs are updated when their resource allocation changes. The procedure repeats until some resource in the cluster is used up, or marginal gains of all jobs become non-positive”).
Gu, Al-Fares, Phanishayee, and Peng are considered analogous to the claimed invention because they deal with deep learning clusters. It would have been obvious to one having ordinary skill in the art prior to the effective filing date to have modified Gu and Al-Fares use the resource allocation algorithm from Peng. One of the ordinary skill in the art would have known to apply the known technique of optimizing resource allocation for deep learning tasks. Therefore, apply Peng’s technique would yield the predicable result of increasing speed and efficiency of completing deep learning tasks (See MPEP 2141 (III)(D) Applying a known technique to a known device ready for improvement to yield predicable results).
Claim(s) 14, 15, and 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Gu in view of Al-Fares, Phanishayee, Peng, and Xiao.
Regarding claim 14, Gu, Al-Fares, Phanishayee, and Peng teach the system of claim 9, Gu and Al-Fares does not teach, but Xiao does teach
when a plurality of training operations share the same processing resource, obtaining a raw time of the training operations by fitting, and obtaining a training time for an entire processing resource by normalization (Xiao, Section 4.1, “An alternative to suspend-resume for time-slicing is to run multiple DLT jobs on a GPU simultaneously and let the GPU time-share the jobs. We call this packing. Packing in GPU is efficient only when the packed jobs do not exceed the GPU resources (cores, memory) and do not adversely impact each other … Gandiva estimates a DLT job’s mini batch time, the time to do one forward/backward pass over a batch of input data, as the time taken between two minimums of the GPU memory usage cycles (Figure 5(a)). Because DLT jobs typically perform millions of such mini batch operations in their lifetime, the scheduler compares the mini batch time of a DLT prior to and post a scheduling decision to determine its effectiveness … By comparing the mini batch time of each of the two DLT jobs before and after packing, Gandiva can decide whether packing is effective.”. Examiner notes that the mini batch time is mapped to the raw time).
Gu, Al-Fares, Phanishayee, Peng, and Xiao are considered analogous to the claimed invention because they deal with deep learning clusters. It would have been obvious to one having ordinary skill in the art prior to the effective filing date to have modified Gu and Al-Fares use Gandiva’s scheduler from Xiao. Xiao teaches “in a real workload of jobs running in a 180-GPU cluster, Gandiva improves aggregate cluster utilization by 26%, pointing to a new way of managing large GPU clusters for deep learning” (Xiao, Abstract) (See MPEP 2141 (III)(G) Some teaching, suggestion, or motivation in the prior art that would have led one of ordinary skill to modify the prior art reference or to combine prior art reference teachings to arrive at the claimed invention.
Regarding claim 15, Gu, Al-Fares, Phanishayee, and Peng teach the system of claim 14, Xiao further teaches
obtaining the raw time by measuring a forward propagation time and a backpropagation time of the training operations and fitting the forward propagation time and the backpropagation time of the training operations in conjunction with a gradient aggregation time (Xiao, Section 2, “It computes a set of scores for each mini-batch by performing numerical computations using the model weights, called the forward pass. Based on the desired task, an objective function is defined that measures an error between the computed scores and desired scores. The error is populated via a backward pass over the model, where it first computes a gradient for each weight (i.e., the impact of each weight on the error) and then applies a negative of the gradient, scaled by a parameter called the learning rate, to each weight to decrease the error. Both the forward and backward passes typically involve billions of floating point operations and thus leverage GPUs. Each forward backward pass is called a mini-batch iteration”).
Gu, Al-Fares, Phanishayee, Peng, and Xiao are considered analogous to the claimed invention because they deal with deep learning clusters. It would have been obvious to one having ordinary skill in the art prior to the effective filing date to have modified Gu and Al-Fares use Gandiva’s scheduler from Xiao. Xiao teaches “in a real workload of jobs running in a 180-GPU cluster, Gandiva improves aggregate cluster utilization by 26%, pointing to a new way of managing large GPU clusters for deep learning” (Xiao, Abstract) (See MPEP 2141 (III)(G) Some teaching, suggestion, or motivation in the prior art that would have led one of ordinary skill to modify the prior art reference or to combine prior art reference teachings to arrive at the claimed invention.
Regarding claim 20, Gu, Al-Fares, Phanishayee, and Peng teach the computer-readable storage medium of claim 10, Gu and Al-Fares does not teach, but Xiao does teach
when a plurality of training operations share the same processing resource, obtaining a raw time of the training operations by fitting, and obtaining a training time for an entire processing resource by normalization (Xiao, Section 4.1, “An alternative to suspend-resume for time-slicing is to run multiple DLT jobs on a GPU simultaneously and let the GPU time-share the jobs. We call this packing. Packing in GPU is efficient only when the packed jobs do not exceed the GPU resources (cores, memory) and do not adversely impact each other … Gandiva estimates a DLT job’s mini batch time, the time to do one forward/backward pass over a batch of input data, as the time taken between two minimums of the GPU memory usage cycles (Figure 5(a)). Because DLT jobs typically perform millions of such mini batch operations in their lifetime, the scheduler compares the mini batch time of a DLT prior to and post a scheduling decision to determine its effectiveness … By comparing the mini batch time of each of the two DLT jobs before and after packing, Gandiva can decide whether packing is effective.”. Examiner notes that the mini batch time is mapped to the raw time).
Gu, Al-Fares, Phanishayee, Peng, and Xiao are considered analogous to the claimed invention because they deal with deep learning clusters. It would have been obvious to one having ordinary skill in the art prior to the effective filing date to have modified Gu and Al-Fares use Gandiva’s scheduler from Xiao. Xiao teaches “in a real workload of jobs running in a 180-GPU cluster, Gandiva improves aggregate cluster utilization by 26%, pointing to a new way of managing large GPU clusters for deep learning” (Xiao, Abstract) (See MPEP 2141 (III)(G) Some teaching, suggestion, or motivation in the prior art that would have led one of ordinary skill to modify the prior art reference or to combine prior art reference teachings to arrive at the claimed invention.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Kang et al. (Cost Efficient GPU Cluster Management for Training and Inference of Deep Learning) discloses a cost efficient deep learning job allocation (CE-DLA) approach minimizing the energy consumption cost for the DL cluster operation while guaranteeing the performance requirements of user requests.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to STEVEN VO whose telephone number is (571)272-9622. The examiner can normally be reached Monday - Friday from 7-3 pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Michelle Bechtold can be reached at (571) 431-0762. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/S.V./Examiner, Art Unit 2148 /MICHELLE T BECHTOLD/Supervisory Patent Examiner, Art Unit 2148
Prosecution Insights