DETAILED ACTION
This action is responsive to claims filed on 8 March 2024.
Claims 1-15 are pending for examination.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Objections
Claim 2 is objected to because of the following informalities: “the last node” in line 13 should be “a last node”. Appropriate correction is required.
Claim 13 is objected to because of the following informalities: “the last node” in line 18 should be “a last node”. Appropriate correction is required.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-12, 15 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 1 recites the limitation "the identification" in line 16. There is insufficient antecedent basis for this limitation in the claim. For examination purposes, the term "the identification" has been construed to be “an identification”. Claims 2-6, which are dependent on claim 1, are similarly rejected.
Claim 4 recites the limitation "the added one more nodes" in lines 4-5. There is insufficient antecedent basis for this limitation in the claim. For examination purposes, the term "the added one more nodes" has been construed to be “the added one more node” in lines 13-14 of claim 1. Claim 5, which is dependent on claim 4, is similarly rejected.
Claim 5 recites the limitation "the added one more nodes" in line 5. There is insufficient antecedent basis for this limitation in the claim. For examination purposes, the term "the added one more nodes" has been construed to be “the added one more node” in lines 13-14 of claim 1.
Claim 7 recites the limitation "the identification" in line 14. There is insufficient antecedent basis for this limitation in the claim. For examination purposes, the term "the identification" has been construed to be “identify whether a first node for executing an inference on the artificial intelligence models operate without errors based on supporting the operation function on the target processor”. Claims 8-12, which are dependent on claim 7, are similarly rejected.
Claim 7 recites the limitation "the identification" in line 17. There is insufficient antecedent basis for this limitation in the claim. For examination purposes, the term "the identification" has been construed to be “identify whether a first node for executing an inference on the artificial intelligence models operate without errors based on supporting the operation function on the target processor”. Claims 8-12, which are dependent on claim 7, are similarly rejected.
Claim 7 recites the limitation "the identification" in line 20. There is insufficient antecedent basis for this limitation in the claim. For examination purposes, the term "the identification" has been construed to be “identify whether a first node for executing an inference on the artificial intelligence models operate without errors based on supporting the operation function on the target processor”. Claims 8-12, which are dependent on claim 7, are similarly rejected.
Claim 7 recites the limitation "the identification" in line 24. There is insufficient antecedent basis for this limitation in the claim. For examination purposes, the term "the identification" has been construed to be “identify whether a first node for executing an inference on the artificial intelligence models operate without errors based on supporting the operation function on the target processor”. Claims 8-12, which are dependent on claim 7, are similarly rejected.
Claim 7 recites the limitation "one more nodes" in line 14. As “one” is singular, but “nodes” is plural, this renders the claim indefinite. For examination purposes, the term "one more nodes" has been construed to be “one more node”. Claims 8-12, which are dependent on claim 7, are similarly rejected.
Claim 7 recites the limitation "the one more nodes" in line 20. As “one” is singular, but “nodes” is plural, this renders the claim indefinite. For examination purposes, the term "the one more nodes" has been construed to be “the one more node” of line 14 in claim 7. Claims 8-12, which are dependent on claim 7, are similarly rejected.
Claim 7 recites the limitation "the error" in line 23. There is insufficient antecedent basis for this limitation in the claim. For examination purposes, the term "the error" has been construed to be “an error”. Claims 8-12, which are dependent on claim 7, are similarly rejected.
Claim 10 recites the limitation "the added one or more nodes" in lines 4-5. There is insufficient antecedent basis for this limitation in the claim. For examination purposes, the term "the added one or more nodes" has been construed to be “the added one more node” in line 14 of claim 7. Claim 11, which is dependent on claim 10, is similarly rejected.
Claim 11 recites the limitation "the added one or more nodes" in lines 5, 6. There is insufficient antecedent basis for this limitation in the claim. For examination purposes, the term "the added one or more nodes" has been construed to be “the added one more node” in line 14 of claim 7.
Claim 15 recites the limitation "the added one or more nodes" in line 4. There is insufficient antecedent basis for this limitation in the claim. For examination purposes, the term "the added one or more nodes" has been construed to be “the added one more node” in line 11 of claim 13.
In Claim 5 and analogous claim 11, “difficult” in line 6 is a relative term which renders the claim indefinite. The term “difficult” is not defined by the claim, the specification does not provide a standard for ascertaining the requisite degree, and one of ordinary skill in the art would not be reasonably apprised of the scope of the invention. In such a product, any number of factors can impact machine learning inferencing to be considered "difficult". For examination purposes, the term "difficult" has been construed to be determining the difficulty to execute the inference without errors by using any feature of calculations in the graph of nodes which optimize operations.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-15 are rejected under 35 U.S.C. 101 because the claimed invention is directed to a judicial exception, abstract idea, without significantly more.
Step 1: This part of the eligibility analysis evaluates whether the claim(s) falls within any statutory
category. MPEP 2106.03:
According to the first part of the Alice analysis, in the instant case, the claims were determined
to be directed to one of the four statutory categories: an article of manufacture, a method/process, a machine/system/product (Claims 1-15), and a composition of matter. Based on the claims being determined to be within of the four categories (i.e., process, machine, manufacture, or composition of matter), (Step 1), it must be determined if the claims are directed to a judicial exception (i.e., law of nature, natural phenomenon, and abstract idea).
Step 2A Prong One: This part of the eligibility analysis evaluates whether the claim(s) recites a
judicial exception.
Regarding independent claims 1, 7, 13, the claims recite a judicial exception (i.e., an abstract idea enumerated in the 2019 PEG) without significantly more (Step-2A: Prong One). The applicant's claim limitations under broadest reasonable interpretation covers activities classified under mental processes - concepts performed in the human mind (including an observation, evaluation, judgment, opinion) (see MPEP § 2106.04(a)(2), subsection Ill) and the 2019 PEG. As evaluated below:
Claim 1:
“identify whether an operation function is supported on a target processor” (mental process of judgement)
“identify whether a first node for executing an inference on the artificial intelligence models operate without errors based on supporting the operation function on the target processor” (mental process of judgement)
“identify whether a next node operates without error by adding one more node in case that the first node operates without errors” (mental process of judgement)
If the identified limitation(s) falls within at least one of the groupings of abstract ideas, it is
reasonable to conclude that the claim(s) recites an abstract idea in Step 2A Prong One.
Step 2A Prong Two: This part of the eligibility analysis evaluates whether the claim(s) as a whole integrates the recited judicial exception into a practical application of the exception. As evaluated below:
“form a first group by creating a partition from the first node to an identified N-1st node based on the identification that an error occurred on an Nth node”
“form a second group by creating a partition for the Nth node on which the error occurred”
These recitations are deemed insufficient to transform the judicial exception to a patentable invention because the recitation is directed to instructions for mere data gathering or data output, see MPEP 2106.05(g).
“load the artificial intelligence models stored in the memory and execute a runtime engine of a framework”
The recitation is directed to mere instructions to implement an abstract idea on a computer, or
merely uses a computer as a tool to perform an abstract idea and are considered to adding the words "apply it" (or an equivalent) with the judicial exception, See MPEP 2106.05(f).
Accordingly, these additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea when considered as an ordered combination and as a whole.
Step 2B: This part of the eligibility analysis evaluates whether the claim, as a whole, amounts to
significantly more than the recited exception, i.e., whether any additional element, or combination of
additional elements, adds an inventive concept to the claim. MPEP 2106.05.
First, the additional elements considered as part of the preamble and the additional elements
directed to the use of computer technology are deemed insufficient to transform the judicial exception
to a patentable invention to a patentable invention because they generally link the judicial exception to
the technology environment, see MPEP 2106.05(h).
Second, the additional elements directed to mere application of the abstract idea or mere instructions to implement an abstract idea on a computer are deemed insufficient to transform the judicial exception to a patentable invention to a patentable invention because the limitations generally apply the use of a generic computer and/or process with the judicial exception, see MPEP 2106.05(f).
Third, the claims are directed to instructions merely indicating a field of use or technological environment in which to apply a judicial exception. The courts have found these types of limitations insufficient to transform the judicial exception to a patentable invention, see MPEP 2106.05(g).
Lastly, the claims directed to data gathering activity as noted above, are deemed directed to an insignificant extra-solution activity. The courts have found these types of limitations insufficient to
qualify as "significantly more", see MPEP 2106.05(g).
Furthermore, when considering evidence in view of Berkheimer v. HP, Inc., 881 F.3d 1360, 1368, 125 USPQ2d 1649, 1654 (Fed. Cir. 2018), see USPTO Berkheimer Memorandum (April 2018). Examiner notes Berkheimer: Option 2 - A citation to one or more of the court decisions discussed in MPEP § 2106.05(d}(II} as noting the well understood, routine, conventional nature of the additional element (s) (e.g., limitations directed to mere data gathering):
The courts have recognized the following computer functions as well understood, routine, and conventional functions when they are claimed in a merely generic manner (e.g., at a high level of generality) or as insignificant extra-solution activity, see MPEP 2106.05(d).
The additional limitations, as analyzed, failed to integrate a judicial exception into a practical application at Step 2A and provide an inventive concept in Step 2B, per the analysis above. Thus, considering the additional elements individually and in combination and the claims as a whole, the additional elements do not provide significantly more than the abstract idea. This claim is not patent eligible. Therefore, in examining elements as recited by the limitations individually and as an ordered combination, as a whole, claim 1 does not recite what the courts have identified as "significantly more".
Claim 7:
“identify whether an operation function is supported on a target processor” (mental process of judgement)
“identify whether a first node for executing an inference on the artificial intelligence models operate without errors based on supporting the operation function on the target processor” (mental process of judgement)
“repeat the identification until a last node by adding one more nodes based on the identification being successful” (mental process of judgement)
“identify whether the Nth node operates without errors” (mental process of judgement)
“repeat the identification until the last node by adding the one more nodes based on the Nth node operating without errors” (mental process of judgement)
“repeat the identification until the last node whether the error occurs” (mental process of judgement)
If the identified limitation(s) falls within at least one of the groupings of abstract ideas, it is
reasonable to conclude that the claim(s) recites an abstract idea in Step 2A Prong One.
Step 2A Prong Two: This part of the eligibility analysis evaluates whether the claim(s) as a whole integrates the recited judicial exception into a practical application of the exception. As evaluated below:
“obtain information about hardware of the electronic device”
“form a first group by creating a partition from the first node to an identified N-1st node based on the identification not being successful in a Nth node”
“form a second group by creating a partition from the Nth node to an identified K-1st node based on a Kth node on which the error occurred”
These recitations are deemed insufficient to transform the judicial exception to a patentable invention because the recitation is directed to instructions for mere data gathering or data output, see MPEP 2106.05(g).
“load the artificial intelligence models stored in the memory and execute a runtime engine of a framework”
The recitation is directed to mere instructions to implement an abstract idea on a computer, or
merely uses a computer as a tool to perform an abstract idea and are considered to adding the words "apply it" (or an equivalent) with the judicial exception, See MPEP 2106.05(f).
“wherein the N and the K are natural numbers”
“wherein the K is greater than the N”
These recitations are deemed insufficient to transform the judicial exception to a patentable invention because the recitation is directed to instructions merely indicating a field of use or technological environment in which to apply a judicial exception, see MPEP 2106.05(h).
Accordingly, these additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea when considered as an ordered combination and as a whole.
Step 2B: This part of the eligibility analysis evaluates whether the claim, as a whole, amounts to
significantly more than the recited exception, i.e., whether any additional element, or combination of
additional elements, adds an inventive concept to the claim. MPEP 2106.05.
First, the additional elements considered as part of the preamble and the additional elements
directed to the use of computer technology are deemed insufficient to transform the judicial exception
to a patentable invention to a patentable invention because they generally link the judicial exception to
the technology environment, see MPEP 2106.05(h).
Second, the additional elements directed to mere application of the abstract idea or mere instructions to implement an abstract idea on a computer are deemed insufficient to transform the judicial exception to a patentable invention to a patentable invention because the limitations generally apply the use of a generic computer and/or process with the judicial exception, see MPEP 2106.05(f).
Third, the claims are directed to instructions merely indicating a field of use or technological environment in which to apply a judicial exception. The courts have found these types of limitations insufficient to transform the judicial exception to a patentable invention, see MPEP 2106.05(g).
Lastly, the claims directed to data gathering activity as noted above, are deemed directed to an insignificant extra-solution activity. The courts have found these types of limitations insufficient to
qualify as "significantly more", see MPEP 2106.05(g).
Furthermore, when considering evidence in view of Berkheimer v. HP, Inc., 881 F.3d 1360, 1368, 125 USPQ2d 1649, 1654 (Fed. Cir. 2018), see USPTO Berkheimer Memorandum (April 2018). Examiner notes Berkheimer: Option 2 - A citation to one or more of the court decisions discussed in MPEP § 2106.05(d}(II} as noting the well understood, routine, conventional nature of the additional element (s) (e.g., limitations directed to mere data gathering):
The courts have recognized the following computer functions as well understood, routine, and conventional functions when they are claimed in a merely generic manner (e.g., at a high level of generality) or as insignificant extra-solution activity, see MPEP 2106.05(d).
The additional limitations, as analyzed, failed to integrate a judicial exception into a practical application at Step 2A and provide an inventive concept in Step 2B, per the analysis above. Thus, considering the additional elements individually and in combination and the claims as a whole, the additional elements do not provide significantly more than the abstract idea. This claim is not patent eligible. Therefore, in examining elements as recited by the limitations individually and as an ordered combination, as a whole, claim 7 does not recite what the courts have identified as "significantly more".
Claim 13:
“identifying whether an operation is supported on a target processor” (mental process of judgement)
“identifying whether a first node for executing an inference on the artificial intelligence model operates without errors based on supporting the operation on the target processor” (mental process of judgement)
“identifying whether a next node operates without error by adding one more node in case that the first node operates without errors” (mental process of judgement)
If the identified limitation(s) falls within at least one of the groupings of abstract ideas, it is
reasonable to conclude that the claim(s) recites an abstract idea in Step 2A Prong One.
Step 2A Prong Two: This part of the eligibility analysis evaluates whether the claim(s) as a whole integrates the recited judicial exception into a practical application of the exception. As evaluated below:
“forming a first group by creating a partition from the first node to an identified N-1st node”
“forming a second group by creating a partition for an Nth node on which an error occurred”
These recitations are deemed insufficient to transform the judicial exception to a patentable invention because the recitation is directed to instructions for mere data gathering or data output, see MPEP 2106.05(g).
“loading an artificial intelligence model stored in memory and executing a runtime engine of a framework”
“storing instructions that repeatedly identify whether the inference is executed without errors on the artificial intelligence model from an N+1th node to the last node”
The recitation is directed to mere instructions to implement an abstract idea on a computer, or
merely uses a computer as a tool to perform an abstract idea and are considered to adding the words "apply it" (or an equivalent) with the judicial exception, See MPEP 2106.05(f).
Accordingly, these additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea when considered as an ordered combination and as a whole.
Step 2B: This part of the eligibility analysis evaluates whether the claim, as a whole, amounts to
significantly more than the recited exception, i.e., whether any additional element, or combination of
additional elements, adds an inventive concept to the claim. MPEP 2106.05.
First, the additional elements considered as part of the preamble and the additional elements
directed to the use of computer technology are deemed insufficient to transform the judicial exception
to a patentable invention to a patentable invention because they generally link the judicial exception to
the technology environment, see MPEP 2106.05(h).
Second, the additional elements directed to mere application of the abstract idea or mere instructions to implement an abstract idea on a computer are deemed insufficient to transform the judicial exception to a patentable invention to a patentable invention because the limitations generally apply the use of a generic computer and/or process with the judicial exception, see MPEP 2106.05(f).
Third, the claims are directed to instructions merely indicating a field of use or technological environment in which to apply a judicial exception. The courts have found these types of limitations insufficient to transform the judicial exception to a patentable invention, see MPEP 2106.05(g).
Lastly, the claims directed to data gathering activity as noted above, are deemed directed to an insignificant extra-solution activity. The courts have found these types of limitations insufficient to
qualify as "significantly more", see MPEP 2106.05(g).
Furthermore, when considering evidence in view of Berkheimer v. HP, Inc., 881 F.3d 1360, 1368, 125 USPQ2d 1649, 1654 (Fed. Cir. 2018), see USPTO Berkheimer Memorandum (April 2018). Examiner notes Berkheimer: Option 2 - A citation to one or more of the court decisions discussed in MPEP § 2106.05(d}(II} as noting the well understood, routine, conventional nature of the additional element (s) (e.g., limitations directed to mere data gathering):
The courts have recognized the following computer functions as well understood, routine, and conventional functions when they are claimed in a merely generic manner (e.g., at a high level of generality) or as insignificant extra-solution activity, see MPEP 2106.05(d).
The additional limitations, as analyzed, failed to integrate a judicial exception into a practical application at Step 2A and provide an inventive concept in Step 2B, per the analysis above. Thus, considering the additional elements individually and in combination and the claims as a whole, the additional elements do not provide significantly more than the abstract idea. This claim is not patent eligible. Therefore, in examining elements as recited by the limitations individually and as an ordered combination, as a whole, claim 13 does not recite what the courts have identified as "significantly more".
Furthermore, regarding dependent claims 2-6, which depend from claim 1, claims 8-12, which depend from claim 7, claims 14-15, which depend from claim 13, the claims are directed to a judicial exception (i.e., an abstract idea enumerated in the 2019 PEG, a law of nature, or a natural phenomenon) without significantly more as highlighted below in the claim limitations by evaluating the claim limitations under the Step2A and 2B:
Claim 2:
Incorporates the rejection of claim 1.
“wherein the target processor corresponds to any one of devices (XPUs) performing compiling and/or interpreting”
“wherein the nodes included in the first group are executed by the devices (XPUs) performing compiling and/or interpreting”
“wherein the nodes included in the second group are executed by devices other than the first group among the devices (XPUs) performing compiling and/or interpreting”
“wherein the one or more programs further include computer-executable instructions that, when executed by the one or more processors, cause the electronic device to”
These recitations are deemed insufficient to transform the judicial exception to a patentable invention because the recitation is directed to instructions merely indicating a field of use or technological environment in which to apply a judicial exception, see MPEP 2106.05(h).
“repeat identifying whether the inference on the artificial intelligence models is executed without errors from an N+1 node to the last node” (mental process of judgement)
The recitation is directed to mere instructions to implement an abstract idea on a computer, or
merely uses a computer as a tool to perform an abstract idea and are considered to adding the words "apply it" (or an equivalent) with the judicial exception, See MPEP 2106.05(f).
Limitations directed to mere instructions indicating a field of use or technological environment in which to apply a judicial exception or directed to mere instructions to implement an abstract idea on a computer cannot integrate a judicial exception into a practical application at Step 2A or provide an inventive concept in Step 2B.
Claims 3, 9:
Incorporates the rejections of claims 1, 8, respectively.
“control the operation function using any one of devices (XPUs) for compiling and/or interpreting in case that the target processor does not support the operation function”
These recitations are deemed insufficient to transform the judicial exception to a patentable invention because the recitation is directed to instructions merely indicating a field of use or technological environment in which to apply a judicial exception, see MPEP 2106.05(h).
Limitations directed to mere instructions indicating a field of use or technological environment in which to apply a judicial exception cannot integrate a judicial exception into a practical application at Step 2A or provide an inventive concept in Step 2B.
Claims 4, 10:
Incorporates the rejections of claims 1, 8, respectively.
“identify whether an inference can be executed without errors even if the added one more nodes are included using the target processor in a situation where a new node is added” (mental process of judgement)
The recitation is directed to mere instructions to implement an abstract idea on a computer, or
merely uses a computer as a tool to perform an abstract idea and are considered to adding the words "apply it" (or an equivalent) with the judicial exception, See MPEP 2106.05(f).
Limitations directed to mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea and are considered to adding the words "apply it" (or an equivalent) cannot integrate a judicial exception into a practical application at Step 2A or provide an inventive concept in Step 2B.
Claims 5, 11:
Incorporates the rejections of claims 4, 10, respectively.
“perform the identification using the target processor” (mental process of judgement)
The recitation is directed to mere instructions to implement an abstract idea on a computer, or
merely uses a computer as a tool to perform an abstract idea and are considered to adding the words "apply it" (or an equivalent) with the judicial exception, See MPEP 2106.05(f).
“create a partition between old nodes and the added one more nodes based on a determination that it is difficult to execute the inference without errors if the added one more nodes are included”
These recitations are deemed insufficient to transform the judicial exception to a patentable invention because the recitation is directed to instructions for mere data gathering or data output, see MPEP 2106.05(g).
Limitations directed to instructions for mere data gathering or data output or directed to mere instructions to implement an abstract idea on a computer/using computer as a tool cannot integrate a judicial exception into a practical application at Step 2A or provide an inventive concept in Step 2B.
Claim 6:
Incorporates the rejection of claim 1.
“store a kernel compiled or interpreted for use in the target processor based on a completion of a compilation of the artificial intelligence models in the memory”
These recitations are deemed insufficient to transform the judicial exception to a patentable invention because the recitation is directed to instructions for mere data gathering or data output, see MPEP 2106.05(g).
Limitations directed to instructions for mere data gathering or data output cannot integrate a judicial exception into a practical application at Step 2A or provide an inventive concept in Step 2B.
Claim 8:
Incorporates the rejection of claim 7.
“wherein the information about the hardware on the electronic device includes a size of the memory and a speed of an operating frequency”
These recitations are deemed insufficient to transform the judicial exception to a patentable invention because the recitation is directed to instructions merely indicating a field of use or technological environment in which to apply a judicial exception, see MPEP 2106.05(h).
Limitations directed to mere instructions indicating a field of use or technological environment in which to apply a judicial exception cannot integrate a judicial exception into a practical application at Step 2A or provide an inventive concept in Step 2B.
Claim 12:
Incorporates the rejection of claim 8.
“store a compiled result in the memory so that it can be used in the target processor based on a completion of a compilation of the artificial intelligence models”
These recitations are deemed insufficient to transform the judicial exception to a patentable invention because the recitation is directed to instructions for mere data gathering or data output, see MPEP 2106.05(g).
“compiled result contains a kernel or an image”
These recitations are deemed insufficient to transform the judicial exception to a patentable invention because the recitation is directed to instructions merely indicating a field of use or technological environment in which to apply a judicial exception, see MPEP 2106.05(h).
Limitations directed to instructions for mere data gathering or data output or directed to mere instructions indicating a field of use or technological environment in which to apply a judicial exception cannot integrate a judicial exception into a practical application at Step 2A or provide an inventive concept in Step 2B.
Claim 14:
Incorporates the rejection of claim 13.
“wherein the target processor corresponds to any one of devices (XPUs) performing compiling and/or interpreting”
“wherein the nodes included in the first group are executed by the devices (XPUs) performing compiling and/or interpreting”
“wherein the nodes included in the second group is executed by a device different from the first group among the devices (XPUs) performing compile and/or interpreting”
“wherein the devices (XPUs) include at least one of a central processing unit (CPU) or a graphical processing unit (GPU)”
These recitations are deemed insufficient to transform the judicial exception to a patentable invention because the recitation is directed to instructions merely indicating a field of use or technological environment in which to apply a judicial exception, see MPEP 2106.05(h).
Limitations directed to mere instructions indicating a field of use or technological environment in which to apply a judicial exception cannot integrate a judicial exception into a practical application at Step 2A or provide an inventive concept in Step 2B.
Claim 15:
Incorporates the rejection of claim 13.
“identifying whether the inference is executed using the target processor without errors even if the added one or more nodes are included” (mental process of judgement)
The recitation is directed to mere instructions to implement an abstract idea on a computer, or
merely uses a computer as a tool to perform an abstract idea and are considered to adding the words "apply it" (or an equivalent) with the judicial exception, See MPEP 2106.05(f).
Limitations directed to mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea and are considered to adding the words "apply it" (or an equivalent) cannot integrate a judicial exception into a practical application at Step 2A or provide an inventive concept in Step 2B.
The dependent claims as analyzed above, do not recite limitations that integrated the judicial exception into a practical application. In addition, the claim limitations do not include additional elements that are sufficient to amount to significantly more than the judicial exception (Step-2B). Therefore, the claims do not recite any limitations, when considered individually or as a whole, that recite what have the courts have identified as "significantly more", see MPEP 2106.05; and therefore, as a whole the claims are not patent eligible. As shown above, the dependent claims do not provide any additional elements that when considered individually or as an ordered combination, amount to significantly more than the abstract idea identified. Therefore, as a whole, the dependent claims do not recite what have the courts have identified as "significantly more" than the recited judicial exception. Therefore, claims 2-6, 8-12, 14-15 are rejected under 35 U.S.C. 101 because the claimed invention is directed to a judicial exception and does not recite, when claim elements are examined individually and as a whole, elements that the courts have identified as "significantly more" than the recited judicial exception.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1-2, 4-5, 13, 15 are rejected under 35 U.S.C. 103 as being unpatentable over Yang et al. (U.S. Pre-Grant Publication No. 20200151579, hereinafter ‘Yang'), in view of Fontaine et al. (U.S. Pre-Grant Publication No. 20240168799, hereinafter 'Fontaine').
Regarding claim 1, Yang teaches An electronic device comprising: memory storing artificial intelligence models and one or more programs including computer-executable instructions; and one or more processors, wherein the one or more programs include computer-executable instructions that, when executed by the one or more processors, cause the electronic device to ([0107] The electronic system 1000 may include a processor 1010, RAM 1020, a model processor 1030, memory 1040, a sensor module 1050, and a resource 1060. The components of the electronic system 1000 may be connected to each other in a communicable manner via a bus. The processor 1010 may control an overall operation of the electronic system 1000. For example, the processor 1010 may control a function of the model processor 1030 by executing programs stored in the RAM 1020. In addition, the processor 1010 may output an input model, on which deep learning is to be performed based on a neural network, to the model processor 1030 by controlling the RAM 1020, the memory 1040, and the like. The RAM 1020 may temporarily store programs, data, applications, or instructions.):
load the artificial intelligence models stored in the memory and execute a runtime engine of a framework ([0030] The model analyzer 110 may receive the input model IM including data, which uses an ANN, from the plurality of applications APP and may generate a calculation processing graph including a plurality of subgraphs by analyzing the input model IM. The term “calculation processing graph” used herein may refer to a data structure represented by a graph including a plurality of calculation nodes. The model analyzer 110 may generate the plurality of subgraphs by dividing the calculation processing graph into structures suitable to be processed respectively by the plurality of resources including a first resource 210, a second resource 220, and Nth resource 230. The model analyzer 110 may output the calculation processing graph and the plurality of subgraphs to the task manager 120.; ([0036] The plurality of resources including a first resource 210, a second resource 220, and Nth resource 230 may include heterogeneous resources. In one example, the first resource 210 may include an NPU and the second resource 220 may include a GPU. load the artificial intelligence models stored in the memory and execute a runtime engine of a framework Each of the plurality of resources including a first resource 210, a second resource 220, and Nth resource 230 may execute a calculation for a subgraph by executing the execution data ED received from a corresponding compiler.; [0037] According to the inventive concept, the plurality of resources including a first resource 210, a second resource 220, and Nth resource 230 may load the execution data ED independently of each other.),
identify whether an operation function is supported on a target processor ([0047] In the example of FIG. 2, the first compiler 131 may correspond to a first resource (not shown), and the second compiler 132 may correspond to a second resource (not shown). That is, the first compiler 131 may compile a received input into a program language used in the first resource, and the second compiler 132 may compile a received input into a program language used in the second resource.; [0048] The task manager 120 may allocate the received first subgraph SG1 of the calculation processing graph CPG to the first resource and allocate the received second subgraph SG2 of the calculation processing graph CPG to the second resource, identify whether an operation function is supported on a target processor based on the information on the hardware capability of each resource.),
form a first group by creating a partition from the first node to an identified N-1st node based on the identification that an error occurred on an Nth node, and form a second group by creating a partition for the Nth node on which the error occurred ([0060] Referring to FIGS. 2 and 5, the model analyzer 110 may extract the model information Info_M from the input model IM (S210). The model analyzer 110 may generate the calculation processing graph CPG′ by analyzing the input model IM based on the model information Info_M (S220). The model analyzer 110 may optimize the calculation processing graph CPG′ (S230). In one example, the model analyzer 110 may generate the optimized calculation processing graph CPG by performing at least one of a graph substitution operation for substituting at least a portion of the calculation processing graph CPG′ with another graph, a graph merging operation for merging at least portions of the calculation processing graph CPG′, and a graph division operation for dividing at least a portion of the calculation processing graph CPG′. Hereinafter, the expression, “at least one of a, b, and c,” should be understood as including only a, only b, only c, both a and b, both a and c, both b and c, all of a, b, and c, or all variations thereof. The model analyzer 110 may sequentially form a first group by creating a partition from the first node to an identified N-1st node generate the plurality of subgraphs SG1 and form a second group by creating a partition for the Nth node on which the error occurred SG2 by dividing the based on the identification that an error occurred on an Nth node optimized calculation processing graph CPG according to graph features (S240).).
Yang fails to teach identify whether a first node for executing an inference on the artificial intelligence models operate without errors based on supporting the operation function on the target processor, identify whether a next node operates without error by adding one more node in case that the first node operates without errors,
Fontaine teaches identify whether a first node for executing an inference on the artificial intelligence models operate without errors based on supporting the operation function on the target processor, identify whether a next node operates without error by adding one more node in case that the first node operates without errors ([0100] In at least one embodiment, identify whether a first node for executing an inference on the artificial intelligence models operate without errors based on supporting the operation function on the target processor at step 404, it is determined whether a graph is a valid graph. In at least one embodiment, at step 404, it is identify whether a next node operates without error by adding one more node in case that the first node operates without errors determined whether a graph is a valid graph by traversing a graph to determine whether nodes of a graph are valid nodes. In at least one embodiment, it is determined whether a graph is a valid graph by traversing a graph to determine whether dependencies between nodes of a graph are valid dependencies. In at least one embodiment, it is determined whether a graph is a valid graph by evaluating a topology of a graph. In at least one embodiment, it is determined whether a graph is a valid graph by evaluating shape information associated with a graph.; [0101] In at least one embodiment, if at step 404, it is determined that a graph is not a valid graph (“NO” branch), execution advances to step 406. In at least one embodiment, if at step 404, it is determined that a graph is a valid graph (“YES” branch), execution advances to step 408.; [0102] In at least one embodiment, at step 406, an error is returned.),
Yang and Fontaine are considered to be analogous to the claimed invention because they are in the same field of machine learning. In view of the teachings of Yang, it would have been obvious for a person of ordinary skill in the art to apply the teachings of Fontaine to Yang before the effective filing date of the claimed invention in order to improve amount of memory, processor time, or other computing resources used to execute graphs (cf. Fontaine, [0002] Parallel computing programs may use significant amounts of memory, processor time, or other computing resources to execute graphs used to represent computing operations. Amount of memory, processor time, or other computing resources used to execute graphs may be improved.).
Regarding claim 2, Yang, as modified by Fontaine, teaches The electronic device of claim 1.
Yang teaches wherein the target processor corresponds to any one of devices (XPUs) performing compiling and/or interpreting ([0036] The plurality of resources including a first resource 210, a second resource 220, and Nth resource 230 may include heterogeneous resources. In one example, the first resource 210 may include an NPU and the second resource 220 may include a GPU. Each of the plurality of resources including a first resource 210, a second resource 220, and Nth resource 230 may execute a calculation for a subgraph by executing the execution data ED received from a corresponding compiler.),
wherein the nodes included in the first group are executed by the devices (XPUs) performing compiling and/or interpreting, wherein the nodes included in the second group are executed by devices other than the first group among the devices (XPUs) performing compiling and/or interpreting ([0031] The task manager 120 may allocate the plurality of subgraphs including calculation nodes to the plurality of resources, wherein the nodes included in the first group are executed by the devices (XPUs) performing compiling and/or interpreting including a first resource 210, wherein the nodes included in the second group are executed by devices other than the first group among the devices (XPUs) performing compiling and/or interpreting a second resource 220, and Nth resource 230, whereby there may be additional resources between the second resource 220 and the Nth resource 230. According to the inventive concept, the task manager 120 may allocate a suitable resource (for example, a first resource 210, a second resource 220, or Nth resource 230) to each of the plurality of subgraphs based on structural features of each subgraph and kinds of calculations included in each subgraph.), and
Fontaine teaches wherein the one or more programs further include computer-executable instructions that, when executed by the one or more processors, cause the electronic device to ([0047] In at least one embodiment, processor 102 has included thereon, instructions that, when executed, define a graph 106. In at least one embodiment, a graph 106 is a graph that includes one or more tasks. In at least one embodiment, instructions that, when executed, define graph 106 are loaded from processor memory 104. In at least one embodiment, instructions that, when executed, define graph 106 are loaded from computer system 100. In at least one embodiment, instructions for processor 102 that, when executed, define graph 106, are stored in processor memory 104. In at least one embodiment, instructions that, when executed, define graph 106, are executed by a process, processor, thread, thread group, or some other such entity where that process, processor, thread, thread group, or some other such entity has access to processor memory 104. In at least one embodiment, instructions for a process, processor, thread, thread group, or some other such entity that, when executed, define graph 106, are stored in processor memory 104. In at least one embodiment, when instructions are executed that define graph 106, a graph template 108 is created.):
repeat identifying whether the inference on the artificial intelligence models is executed without errors from an N+1 node to the last node ([0100] In at least one embodiment, at step 404, it is determined whether a graph is a valid graph. In at least one embodiment, at step 404, it is determined whether a graph is a valid graph by repeat identifying whether the inference on the artificial intelligence models is executed without errors from an N+1 node to the last node traversing a graph to determine whether nodes of a graph are valid nodes. In at least one embodiment, it is determined whether a graph is a valid graph by traversing a graph to determine whether dependencies between nodes of a graph are valid dependencies. In at least one embodiment, it is determined whether a graph is a valid graph by evaluating a topology of a graph.).
Yang and Fontaine are combinable for the same rationale as set forth above with respect to claim 1.
Regarding claim 4, Yang, as modified by Fontaine, teaches The electronic device of claim 1.
Fontaine teaches wherein the one or more programs further include computer-executable instructions that, when executed by the one or more processors, cause the electronic device to: identify whether an inference can be executed without errors even if the added one more nodes are included using the target processor in a situation where a new node is added ([0071] In at least one embodiment, an API that adds a child graph node to a graph returns an error code to a calling process that identify whether an inference can be executed without errors indicates a success or failure of an operation even if the added one more nodes are included using the target processor in a situation where a new node is added to add a child graph node to a graph.; [0048] In at least one embodiment, instructions that, when executed, instantiate graph instance 116, cause creation of an executable instance of graph from a graph instantiation API. In at least one embodiment, instructions that, when executed, instantiate graph instance 116, cause creation of an executable instance of graph from a stream. In at least one embodiment, stream is an execution stream.; [0049] In at least one embodiment, within each stream, inference a sequence of work components executes in issue-order on computer system 100.).
Yang and Fontaine are combinable for the same rationale as set forth above with respect to claim 1.
Regarding claim 5, Yang, as modified by Fontaine, teaches The electronic device of claim 4.
Fontaine teaches wherein the one or more programs further include computer-executable instructions that, when executed by the one or more processors, cause the electronic device to: perform the identification using the target processor ([0048] In at least one embodiment, processor 102 has included thereon, instructions that, when executed, instantiate graph 110. In at least one embodiment, instructions that, when executed, instantiate graph instance 116, cause creation of an executable instance of graph from a graph instantiation API. In at least one embodiment, instructions that, when executed, perform the identification using the target processor instantiate graph instance 116, cause creation of an executable instance of graph from a stream. In at least one embodiment, stream is an execution stream.); and
Yang teaches create a partition between old nodes and the added one more nodes based on a determination that it is difficult to execute the inference without errors if the added one more nodes are included ([0064] The model analyzer 110 may output, to the task manager 120, the second subgraph SG1 generated as a result of analyzing the input model (T160). The model analyzer 110 create a partition between old nodes and the added one more nodes may generate the second subgraph SG2 by partitioning the calculation processing graph based on a determination that it is difficult to execute the inference without errors if the added one more nodes are included based on features of calculations. The task manager 120 may allocate the second subgraph SG2 to a second resource RS2 based on features of calculations included in the second subgraph SG2 and the hardware capabilities of the plurality of resources (T170). The task manager 120 may output the second subgraph SG2 to the second compiler 132 corresponding to the second resource RS2 (T180), and the second compiler 132 may compile the second subgraph SG2 (T190).; [0043] The model optimizer 113 may perform an optimization operation on the calculation processing graph CPG′ and may output, to the task manager 120, a calculation processing graph CPG generated as a result of the optimization operation. The optimization operation may refer to an operation of minimizing and optimizing an amount of calculation processing for the calculation processing graph CPG′. In one embodiment, the optimization operation may include a graph substitution operation for substituting at least a portion of the calculation processing graph CPG′ with another graph, a graph merging operation for merging at least portions of the calculation processing graph CPG′, and a graph division operation for dividing at least a portion of the calculation processing graph CPG′.).
Yang and Fontaine are combinable for the same rationale as set forth above with respect to claim 1.
Regarding claim 13, Yang teaches loading an artificial intelligence model stored in memory and executing a runtime engine of a framework ([0030] The model analyzer 110 may receive the input model IM including data, which uses an ANN, from the plurality of applications APP and may generate a calculation processing graph including a plurality of subgraphs by analyzing the input model IM. The term “calculation processing graph” used herein may refer to a data structure represented by a graph including a plurality of calculation nodes. The model analyzer 110 may generate the plurality of subgraphs by dividing the calculation processing graph into structures suitable to be processed respectively by the plurality of resources including a first resource 210, a second resource 220, and Nth resource 230. The model analyzer 110 may output the calculation processing graph and the plurality of subgraphs to the task manager 120.; ([0036] The plurality of resources including a first resource 210, a second resource 220, and Nth resource 230 may include heterogeneous resources. In one example, the first resource 210 may include an NPU and the second resource 220 may include a GPU. loading an artificial intelligence model stored in memory and executing a runtime engine of a framework Each of the plurality of resources including a first resource 210, a second resource 220, and Nth resource 230 may execute a calculation for a subgraph by executing the execution data ED received from a corresponding compiler.; [0037] According to the inventive concept, the plurality of resources including a first resource 210, a second resource 220, and Nth resource 230 may load the execution data ED independently of each other.);
identifying whether an operation is supported on a target processor ([0047] In the example of FIG. 2, the first compiler 131 may correspond to a first resource (not shown), and the second compiler 132 may correspond to a second resource (not shown). That is, the first compiler 131 may compile a received input into a program language used in the first resource, and the second compiler 132 may compile a received input into a program language used in the second resource.; [0048] The task manager 120 may allocate the received first subgraph SG1 of the calculation processing graph CPG to the first resource and allocate the received second subgraph SG2 of the calculation processing graph CPG to the second resource, identifying whether an operation is supported on a target processor based on the information on the hardware capability of each resource.);
forming a first group by creating a partition from the first node to an identified N-1st node; forming a second group by creating a partition for an Nth node on which an error occurred ([0060] Referring to FIGS. 2 and 5, the model analyzer 110 may extract the model information Info_M from the input model IM (S210). The model analyzer 110 may generate the calculation processing graph CPG′ by analyzing the input model IM based on the model information Info_M (S220). The model analyzer 110 may optimize the calculation processing graph CPG′ (S230). In one example, the model analyzer 110 may generate the optimized calculation processing graph CPG by performing at least one of a graph substitution operation for substituting at least a portion of the calculation processing graph CPG′ with another graph, a graph merging operation for merging at least portions of the calculation processing graph CPG′, and a graph division operation for dividing at least a portion of the calculation processing graph CPG′. Hereinafter, the expression, “at least one of a, b, and c,” should be understood as including only a, only b, only c, both a and b, both a and c, both b and c, all of a, b, and c, or all variations thereof. The model analyzer 110 may sequentially forming a first group by creating a partition from the first node to an identified N-1st node generate the plurality of subgraphs SG1 and forming a second group by creating a partition for an Nth node on which an error occurred SG2 by dividing the optimized calculation processing graph CPG according to graph features (S240).); and
Yang fails to teach One or more non-transitory computer-readable storage media storing one or more programs including computer-executable instructions that, when executed by one or more processors of an electronic device, cause the electronic device to perform operations, the operations comprising: identifying whether a first node for executing an inference on the artificial intelligence model operates without errors based on supporting the operation on the target processor; identifying whether a next node operates without error by adding one more node in case that the first node operates without errors; storing instructions that repeatedly identify whether the inference is executed without errors on the artificial intelligence model from an N+1th node to the last node.
Fontaine teaches One or more non-transitory computer-readable storage media storing one or more programs including computer-executable instructions that, when executed by one or more processors of an electronic device, cause the electronic device to perform operations, the operations comprising ([0421] In at least one embodiment, a computer-readable storage medium is a non-transitory computer-readable storage medium that excludes transitory signals (e.g., a propagating transient electric or electromagnetic transmission) but includes non-transitory data storage circuitry (e.g., buffers, cache, and queues) within transceivers of transitory signals. In at least one embodiment, code (e.g., executable code or source code) is stored on a set of one or more non-transitory computer-readable storage media having stored thereon executable instructions (or other memory to store executable instructions) that, when executed (e.g., as a result of being executed) by one or more processors of a computer system, cause computer system to perform operations described herein.):
identifying whether a first node for executing an inference on the artificial intelligence model operates without errors based on supporting the operation on the target processor; identifying whether a next node operates without error by adding one more node in case that the first node operates without errors ([0100] In at least one embodiment, identifying whether a first node for executing an inference on the artificial intelligence model operates without errors based on supporting the operation on the target processor at step 404, it is determined whether a graph is a valid graph. In at least one embodiment, at step 404, it is identifying whether a next node operates without error by adding one more node in case that the first node operates without errors determined whether a graph is a valid graph by traversing a graph to determine whether nodes of a graph are valid nodes. In at least one embodiment, it is determined whether a graph is a valid graph by traversing a graph to determine whether dependencies between nodes of a graph are valid dependencies. In at least one embodiment, it is determined whether a graph is a valid graph by evaluating a topology of a graph. In at least one embodiment, it is determined whether a graph is a valid graph by evaluating shape information associated with a graph.; [0101] In at least one embodiment, if at step 404, it is determined that a graph is not a valid graph (“NO” branch), execution advances to step 406. In at least one embodiment, if at step 404, it is determined that a graph is a valid graph (“YES” branch), execution advances to step 408.; [0102] In at least one embodiment, at step 406, an error is returned.);
storing instructions that repeatedly identify whether the inference is executed without errors on the artificial intelligence model from an N+1th node to the last node ([0099] In at least one embodiment, a processor such as processor 102 storing instructions executes instructions to perform at least a portion of example process 400. In at least one embodiment, a graphics processor such as graphics processor 112 executes instructions to perform at least a portion of example process 400. In at least one embodiment, at step 402, a graph is built using systems and methods such as those described herein. In at least one embodiment, a graph is built from a graph template. In at least one embodiment, a graph is built using a graph construction API. In at least one embodiment, a graph is built from a stream. In at least one embodiment, after step 402, execution advances to step 604.; [0100] In at least one embodiment, at step 404, it is determined whether a graph is a valid graph. In at least one embodiment, at step 404, it is determined whether a graph is a valid graph by that repeatedly identify whether the inference is executed without errors on the artificial intelligence model from an N+1th node to the last node traversing a graph to determine whether nodes of a graph are valid nodes. In at least one embodiment, it is determined whether a graph is a valid graph by traversing a graph to determine whether dependencies between nodes of a graph are valid dependencies. In at least one embodiment, it is determined whether a graph is a valid graph by evaluating a topology of a graph.).
Yang and Fontaine are combinable for the same rationale as set forth above with respect to claim 1.
Regarding claim 15, Yang, as modified by Fontaine, teaches The one or more non-transitory computer-readable storage media of claim 13.
Fontaine teaches identifying whether the inference is executed using the target processor without errors even if the added one or more nodes are included ([0071] In at least one embodiment, an API that adds a child graph node to a graph returns an error code to a calling process that identifying whether the inference is executed using the target processor without errors indicates a success or failure of an operation even if the added one or more nodes are included to add a child graph node to a graph.; [0048] In at least one embodiment, instructions that, when executed, instantiate graph instance 116, cause creation of an executable instance of graph from a graph instantiation API. In at least one embodiment, instructions that, when executed, instantiate graph instance 116, cause creation of an executable instance of graph from a stream. In at least one embodiment, stream is an execution stream.; [0049] In at least one embodiment, within each stream, inference a sequence of work components executes in issue-order on computer system 100.).
Yang and Fontaine are combinable for the same rationale as set forth above with respect to claim 1.
Claims 3, 7-11, 14 are rejected under 35 U.S.C. 103 as being unpatentable over Yang, in view of Fontaine, and further in view of Kovvuri et al. (U.S. Pre-Grant Publication No. 20190286973, hereinafter 'Kovvuri').
Regarding claim 3, Yang, as modified by Fontaine, teaches The electronic device of claim 1.
Yang, as modified by Fontaine, fails to teach control the operation function using any one of devices (XPUs) for compiling and/or interpreting in case that the target processor does not support the operation function.
Kovvuri teaches control the operation function using any one of devices (XPUs) for compiling and/or interpreting in case that the target processor does not support the operation function ([0052] The control the operation function using any one of devices (XPUs) for compiling and/or interpreting control unit 160 can be used to execute a tool flow for compiling, training, installing, and executing a deep neural network graph. As one example, different portions of the tool flow can use different components of the multiprocessor 100. The compilation and training steps can be performed by the CPU 165. After the neural network is trained, the neural network can be used in an inference mode where new data is presented to the neural network for classification. The neural network can be divided into different subgraphs, where a portion of the subgraphs are executed by the CPU 165 and a portion of the subgraphs are executed by the plurality 110 of neural processing cores. The control unit 160 can schedule the data transfer between the CPU 165 and the plurality 110 of neural processing cores so that a latency between the CPU 165 and the plurality 110 of neural processing cores is in case that the target processor does not support the operation function optimized for the particular division of the subgraphs on the different hardware components.).
Yang, Fontaine, and Kovvuri are considered to be analogous to the claimed invention because they are in the same field of machine learning. In view of the teachings of Yang and Fontaine, it would have been obvious for a person of ordinary skill in the art to apply the teachings of Kovvuri to Yang before the effective filing date of the claimed invention in order to improve processing speed and reduce latency (cf. Kovvuri, [0021] Methods and apparatus are disclosed for partitioning artificial neural network (NN) models, including deep neural network (DNN), into subgraphs that can be provided to a neural network accelerator, therefore providing improved processing speed and reduced latency.).
Regarding claim 7, Yang teaches An electronic device comprising: memory storing artificial intelligence models and one or more programs including computer-executable instructions; and one or more processors, wherein the one or more programs includes computer-executable instructions that, when executed by the one or more processors, cause the electronic device to ([0107] The electronic system 1000 may include a processor 1010, RAM 1020, a model processor 1030, memory 1040, a sensor module 1050, and a resource 1060. The components of the electronic system 1000 may be connected to each other in a communicable manner via a bus. The processor 1010 may control an overall operation of the electronic system 1000. For example, the processor 1010 may control a function of the model processor 1030 by executing programs stored in the RAM 1020. In addition, the processor 1010 may output an input model, on which deep learning is to be performed based on a neural network, to the model processor 1030 by controlling the RAM 1020, the memory 1040, and the like. The RAM 1020 may temporarily store programs, data, applications, or instructions.):
load the artificial intelligence models stored in the memory and execute a runtime engine of a framework ([0030] The model analyzer 110 may receive the input model IM including data, which uses an ANN, from the plurality of applications APP and may generate a calculation processing graph including a plurality of subgraphs by analyzing the input model IM. The term “calculation processing graph” used herein may refer to a data structure represented by a graph including a plurality of calculation nodes. The model analyzer 110 may generate the plurality of subgraphs by dividing the calculation processing graph into structures suitable to be processed respectively by the plurality of resources including a first resource 210, a second resource 220, and Nth resource 230. The model analyzer 110 may output the calculation processing graph and the plurality of subgraphs to the task manager 120.; ([0036] The plurality of resources including a first resource 210, a second resource 220, and Nth resource 230 may include heterogeneous resources. In one example, the first resource 210 may include an NPU and the second resource 220 may include a GPU. load the artificial intelligence models stored in the memory and execute a runtime engine of a framework Each of the plurality of resources including a first resource 210, a second resource 220, and Nth resource 230 may execute a calculation for a subgraph by executing the execution data ED received from a corresponding compiler.; [0037] According to the inventive concept, the plurality of resources including a first resource 210, a second resource 220, and Nth resource 230 may load the execution data ED independently of each other.),
identify whether an operation function is supported on a target processor ([0047] In the example of FIG. 2, the first compiler 131 may correspond to a first resource (not shown), and the second compiler 132 may correspond to a second resource (not shown). That is, the first compiler 131 may compile a received input into a program language used in the first resource, and the second compiler 132 may compile a received input into a program language used in the second resource.; [0048] The task manager 120 may allocate the received first subgraph SG1 of the calculation processing graph CPG to the first resource and allocate the received second subgraph SG2 of the calculation processing graph CPG to the second resource, identify whether an operation function is supported on a target processor based on the information on the hardware capability of each resource.),
form a first group by creating a partition from the first node to an identified N-1st node based on the identification not being successful in a Nth node, form a second group by creating a partition from the Nth node to an identified K-1st node based on a Kth node on which the error occurred ([0060] Referring to FIGS. 2 and 5, the model analyzer 110 may extract the model information Info_M from the input model IM (S210). The model analyzer 110 may generate the calculation processing graph CPG′ by analyzing the input model IM based on the model information Info_M (S220). The model analyzer 110 may optimize the calculation processing graph CPG′ (S230). In one example, the model analyzer 110 may generate the optimized calculation processing graph CPG by performing at least one of a graph substitution operation for substituting at least a portion of the calculation processing graph CPG′ with another graph, a graph merging operation for merging at least portions of the calculation processing graph CPG′, and a graph division operation for dividing at least a portion of the calculation processing graph CPG′. Hereinafter, the expression, “at least one of a, b, and c,” should be understood as including only a, only b, only c, both a and b, both a and c, both b and c, all of a, b, and c, or all variations thereof. The model analyzer 110 may sequentially form a first group by creating a partition from the first node to an identified N-1st node generate the plurality of subgraphs SG1 and form a second group by creating a partition from the Nth node to an identified K-1st node based on a Kth node on which the error occurred SG2 by dividing the based on the identification not being successful in a Nth node optimized calculation processing graph CPG according to graph features (S240).), and
wherein the N and the K are natural numbers, and wherein the K is greater than the N ([0030] The model analyzer 110 may receive the input model IM including data, which uses an ANN, from the plurality of applications APP and may generate a calculation processing graph including a plurality of subgraphs by analyzing the input model IM. The term “calculation processing graph” used herein may refer to a data structure represented by a graph including a plurality of calculation nodes. The model analyzer 110 may generate the plurality of subgraphs by dividing the calculation processing graph into structures suitable to be processed respectively by the plurality of resources including a first resource 210, a second resource 220, and Nth resource 230.; [0055] The model analyzer 110 may partition the generated calculation processing graph CPG into a plurality of subgraphs SG1, SG2, and SG3 based on kinds of calculations, calculation preferences, shapes of graphs, and the like. In the example of FIG. 4, the model analyzer 110 may classify first to fourth calculations OP00 to OP03 as the first subgraph SG1, classify fifth to eleventh calculations OP04 to OP10 as the second subgraph SG2, and classify twelfth to fourteenth calculations OP11 to OP13 as a third subgraph SG3.; As taught by Yang, calculation nodes correspond to subgraphs of calculations processed by plurality of resources. The calculations in the subgraphs are described and partitioned sequentially in order, with the larger integers in the latter subgraphs, which respectively correspond to the resources and calculation nodes, in a similar manner to the Nth and Kth nodes of the claimed invention, wherein K is greater than N.).
Yang fails to teach obtain information about hardware of the electronic device,
identify whether a first node for executing an inference on the artificial intelligence models operate without errors based on supporting the operation function on the target processor, repeat the identification until a last node by adding one more nodes based on the identification being successful, identify whether the Nth node operates without errors, repeat the identification until the last node by adding the one more nodes based on the Nth node operating without errors,
repeat the identification until the last node whether the error occurs,
Fontaine teaches identify whether a first node for executing an inference on the artificial intelligence models operate without errors based on supporting the operation function on the target processor, repeat the identification until a last node by adding one more nodes based on the identification being successful ([0100] In at least one embodiment, identify whether a first node for executing an inference on the artificial intelligence models operate without errors based on supporting the operation function on the target processor at step 404, it is determined whether a graph is a valid graph. In at least one embodiment, at step 404, it is repeat the identification until a last node by adding one more nodes based on the identification being successful determined whether a graph is a valid graph by traversing a graph to determine whether nodes of a graph are valid nodes. In at least one embodiment, it is determined whether a graph is a valid graph by traversing a graph to determine whether dependencies between nodes of a graph are valid dependencies. In at least one embodiment, it is determined whether a graph is a valid graph by evaluating a topology of a graph. In at least one embodiment, it is determined whether a graph is a valid graph by evaluating shape information associated with a graph.; [0101] In at least one embodiment, if at step 404, it is determined that a graph is not a valid graph (“NO” branch), execution advances to step 406. In at least one embodiment, if at step 404, it is determined that a graph is a valid graph (“YES” branch), execution advances to step 408.; [0102] In at least one embodiment, at step 406, an error is returned.),
identify whether the Nth node operates without errors, repeat the identification until the last node by adding the one more nodes based on the Nth node operating without errors,
repeat the identification until the last node whether the error occurs ([0100] In at least one embodiment, identify whether the Nth node operates without errors at step 404, it is determined whether a graph is a valid graph. In at least one embodiment, at step 404, it is repeat the identification until the last node by adding the one more nodes based on the Nth node operating without errors determined whether a graph is a valid graph by traversing a graph to determine whether nodes of a graph are valid nodes. In at least one embodiment, it is determined whether a graph is a valid graph by traversing a graph to determine whether dependencies between nodes of a graph are valid dependencies. In at least one embodiment, it is determined whether a graph is a valid graph by evaluating a topology of a graph. In at least one embodiment, it is determined whether a graph is a valid graph by evaluating shape information associated with a graph.; [0101] In at least one embodiment, if at step 404, repeat the identification until the last node whether the error occurs it is determined that a graph is not a valid graph (“NO” branch), execution advances to step 406. In at least one embodiment, if at step 404, it is determined that a graph is a valid graph (“YES” branch), execution advances to step 408.; [0102] In at least one embodiment, at step 406, an error is returned.),
Yang and Fontaine are combinable for the same rationale as set forth above with respect to claim 1.
Kovvuri teaches obtain information about hardware of the electronic device ([0020] Each of the subgraphs can be executed by either a CPU, a GPU, or programmable hardware. Specifically, the obtain information about hardware of the electronic device target hardware for executing each of the subgraphs can be identified using a programming interface, an application programming interface, and/or marker nodes. The hardware used to execute the subgraphs can be selected based on the suitability of the subgraph for the particular hardware.; [0043] As discussed herein, techniques can be applied to such operations to reduce the demands for computation as well as memory bandwidth in a given system, whether it is an FPGA, CPU or another hardware platform. As used herein, the use of the term “element” herein refers to a member of such a matrix or vector.),
Yang, Fontaine, and Kovvuri are combinable for the same rationale as set forth above with respect to claim 3.
Regarding claim 8, Yang, as modified by Fontaine and Kovvuri, teaches The electronic device of claim 7.
Kovvuri teaches wherein the information about the hardware on the electronic device includes a size of the memory and a speed of an operating frequency ([0020] Each of the subgraphs can be executed by either a CPU, a GPU, or programmable hardware. Specifically, the wherein the information about the hardware on the electronic device target hardware for executing each of the subgraphs can be identified using a programming interface, an application programming interface, and/or marker nodes. The hardware used to execute the subgraphs can be selected based on the suitability of the subgraph for the particular hardware.; [0043] As discussed herein, techniques can be applied to such operations to reduce the and a speed of an operating frequency demands for computation as well as includes a size of the memory memory bandwidth in a given system, whether it is an FPGA, CPU or another hardware platform. As used herein, the use of the term “element” herein refers to a member of such a matrix or vector.).
Yang, Fontaine, and Kovvuri are combinable for the same rationale as set forth above with respect to claim 3.
Regarding claim 9, Yang, as modified by Fontaine and Kovvuri, teaches The electronic device of claim 8.
Kovvuri teaches control the operation function using any one of devices (XPUs) for compiling and/or interpreting in case that the target processor does not support the operation function ([0052] The control the operation function using any one of devices (XPUs) for compiling and/or interpreting control unit 160 can be used to execute a tool flow for compiling, training, installing, and executing a deep neural network graph. As one example, different portions of the tool flow can use different components of the multiprocessor 100. The compilation and training steps can be performed by the CPU 165. After the neural network is trained, the neural network can be used in an inference mode where new data is presented to the neural network for classification. The neural network can be divided into different subgraphs, where a portion of the subgraphs are executed by the CPU 165 and a portion of the subgraphs are executed by the plurality 110 of neural processing cores. The control unit 160 can schedule the data transfer between the CPU 165 and the plurality 110 of neural processing cores so that a latency between the CPU 165 and the plurality 110 of neural processing cores is in case that the target processor does not support the operation function optimized for the particular division of the subgraphs on the different hardware components.).
Yang, Fontaine, and Kovvuri are combinable for the same rationale as set forth above with respect to claim 3.
Regarding claim 10, Yang, as modified by Fontaine and Kovvuri, teaches The electronic device of claim 8.
Fontaine teaches identify whether an inference can be executed without errors even if the added one more nodes are included using the target processor in a situation where a new node is added ([0071] In at least one embodiment, an API that adds a child graph node to a graph returns an error code to a calling process that identify whether an inference can be executed without errors indicates a success or failure of an operation even if the added one more nodes are included using the target processor in a situation where a new node is added to add a child graph node to a graph.; [0048] In at least one embodiment, instructions that, when executed, instantiate graph instance 116, cause creation of an executable instance of graph from a graph instantiation API. In at least one embodiment, instructions that, when executed, instantiate graph instance 116, cause creation of an executable instance of graph from a stream. In at least one embodiment, stream is an execution stream.; [0049] In at least one embodiment, within each stream, inference a sequence of work components executes in issue-order on computer system 100.).
Yang, Fontaine, and Kovvuri are combinable for the same rationale as set forth above with respect to claim 3.
Regarding claim 11, Yang, as modified by Fontaine and Kovvuri, teaches The electronic device of claim 10.
Fontaine teaches perform the identification using the target processor ([0048] In at least one embodiment, processor 102 has included thereon, instructions that, when executed, instantiate graph 110. In at least one embodiment, instructions that, when executed, instantiate graph instance 116, cause creation of an executable instance of graph from a graph instantiation API. In at least one embodiment, instructions that, when executed, perform the identification using the target processor instantiate graph instance 116, cause creation of an executable instance of graph from a stream. In at least one embodiment, stream is an execution stream.); and
Yang teaches create a partition between old nodes and the added one more nodes based on a determination that it is difficult to execute the inference without errors if the added one more nodes are included ([0064] The model analyzer 110 may output, to the task manager 120, the second subgraph SG1 generated as a result of analyzing the input model (T160). The model analyzer 110 create a partition between old nodes and the added one more nodes may generate the second subgraph SG2 by partitioning the calculation processing graph based on a determination that it is difficult to execute the inference without errors if the added one more nodes are included based on features of calculations. The task manager 120 may allocate the second subgraph SG2 to a second resource RS2 based on features of calculations included in the second subgraph SG2 and the hardware capabilities of the plurality of resources (T170). The task manager 120 may output the second subgraph SG2 to the second compiler 132 corresponding to the second resource RS2 (T180), and the second compiler 132 may compile the second subgraph SG2 (T190).; [0043] The model optimizer 113 may perform an optimization operation on the calculation processing graph CPG′ and may output, to the task manager 120, a calculation processing graph CPG generated as a result of the optimization operation. The optimization operation may refer to an operation of minimizing and optimizing an amount of calculation processing for the calculation processing graph CPG′. In one embodiment, the optimization operation may include a graph substitution operation for substituting at least a portion of the calculation processing graph CPG′ with another graph, a graph merging operation for merging at least portions of the calculation processing graph CPG′, and a graph division operation for dividing at least a portion of the calculation processing graph CPG′.).
Yang, Fontaine, and Kovvuri are combinable for the same rationale as set forth above with respect to claim 3.
Regarding claim 14, Yang, as modified by Fontaine, teaches The one or more non-transitory computer-readable storage media of claim 13.
Yang teaches wherein the target processor corresponds to any one of devices (XPUs) performing compiling and/or interpreting ([0036] The plurality of resources including a first resource 210, a second resource 220, and Nth resource 230 may include heterogeneous resources. In one example, the first resource 210 may include an NPU and the second resource 220 may include a GPU. Each of the plurality of resources including a first resource 210, a second resource 220, and Nth resource 230 may execute a calculation for a subgraph by executing the execution data ED received from a corresponding compiler.),
wherein the nodes included in the first group are executed by the devices (XPUs) performing compiling and/or interpreting, and wherein the nodes included in the second group is executed by a device different from the first group among the devices (XPUs) performing compile and/or interpreting ([0031] The task manager 120 may allocate the plurality of subgraphs including calculation nodes to the plurality of resources, wherein the nodes included in the first group are executed by the devices (XPUs) performing compiling and/or interpreting including a first resource 210, wherein the nodes included in the second group is executed by a device different from the first group among the devices (XPUs) performing compile and/or interpreting a second resource 220, and Nth resource 230, whereby there may be additional resources between the second resource 220 and the Nth resource 230. According to the inventive concept, the task manager 120 may allocate a suitable resource (for example, a first resource 210, a second resource 220, or Nth resource 230) to each of the plurality of subgraphs based on structural features of each subgraph and kinds of calculations included in each subgraph.),
Yang fails to teach wherein the devices (XPUs) include at least one of a central processing unit (CPU) or a graphical processing unit (GPU).
Kovvuri teaches wherein the devices (XPUs) include at least one of a central processing unit (CPU) or a graphical processing unit (GPU) ([0020] As described herein, a machine learning model can include a graph of computational nodes. The machine learning model can be partitioned into different subgraphs, where each of the subgraphs comprises a subset of the computational nodes of the machine learning model. Each of the subgraphs can be executed by either a CPU, a GPU, or programmable hardware.).
Yang, Fontaine, and Kovvuri are combinable for the same rationale as set forth above with respect to claim 3.
Claim 6 is rejected under 35 U.S.C. 103 as being unpatentable over Yang, in view of Fontaine, and further in view of Liu et al. (U.S. Pre-Grant Publication No. 20200334083, hereinafter 'Liu').
Regarding claim 6, Yang, as modified by Fontaine, teaches The electronic device of claim 1.
Yang, as modified by Fontaine, fails to teach store a kernel compiled or interpreted for use in the target processor based on a completion of a compilation of the artificial intelligence models in the memory.
Liu teaches store a kernel compiled or interpreted for use in the target processor based on a completion of a compilation of the artificial intelligence models in the memory ([0035] The typical workflow of the host code may comprise: initializing the target dedicated processing unit device; reading in the IR computation graph; for each computation node in the computation path in the graph, choosing the corresponding CUDA kernel function implementation from the device code library, store allocating the in the memory memory for the input and output for the a kernel compiled or interpreted kernel function, allocating the workspace memory for the kernel function execution, copying the input parameters of the kernel function from the central processing unit to the dedicated processing unit, for use in the target processor based on a completion of a compilation of the artificial intelligence models invoking the kernel function, executing the kernel function in the dedicated processing unit, copying the output of the kernel function from the dedicated processing unit back to the central processing unit; releasing the target dedicated processing unit device; and outputting the computation result.).
Yang, Fontaine, and Liu are considered to be analogous to the claimed invention because they are in the same field of machine learning. In view of the teachings of Yang and Fontaine, it would have been obvious for a person of ordinary skill in the art to apply the teachings of Liu to Yang before the effective filing date of the claimed invention in order to effectively determine a resource required for performing a task (cf. Liu, [0037] In order to at least partly solve one or more of the above and other potential problems, embodiments of the present disclosure propose a solution for determining a resource required for executing a code segment. The solution presents a novel method for judging a resource required for performing a task by creating a dummy DL compiler to collect the memory and threads/blocks requirement to run a DL task in the dedicated processing unit. With the technical solution of the present disclosure, a resource required for performing a task may be determined effectively and efficiently.).
Claim 12 is rejected under 35 U.S.C. 103 as being unpatentable over Yang, in view of Fontaine, Kovvuri, and further in view of Liu et al. (U.S. Pre-Grant Publication No. 20200334083, hereinafter 'Liu').
Regarding claim 12, Yang, as modified by Fontaine and Kovvuri, teaches The electronic device of claim 8.
Yang, as modified by Fontaine and Kovvuri, fails to teach store a compiled result in the memory so that it can be used in the target processor based on a completion of a compilation of the artificial intelligence models, and the compiled result contains a kernel or an image.
Liu teaches store a compiled result in the memory so that it can be used in the target processor based on a completion of a compilation of the artificial intelligence models, and the compiled result contains a kernel or an image ([0035] The typical workflow of the host code may comprise: initializing the target dedicated processing unit device; reading in the IR computation graph; for each computation node in the computation path in the graph, choosing the corresponding CUDA kernel function implementation from the device code library, store a compiled result in the memory allocating the memory for the input and output for the compiled result contains a kernel or an image kernel function, allocating the workspace memory for the kernel function execution, copying the input parameters of the kernel function from the central processing unit to the dedicated processing unit, so that it can be used in the target processor based on a completion of a compilation of the artificial intelligence models invoking the kernel function, executing the kernel function in the dedicated processing unit, copying the output of the kernel function from the dedicated processing unit back to the central processing unit; releasing the target dedicated processing unit device; and outputting the computation result.).
Yang, Fontaine, Kovvuri, and Liu are considered to be analogous to the claimed invention because they are in the same field of machine learning. In view of the teachings of Yang, Fontaine, and Kovvuri, it would have been obvious for a person of ordinary skill in the art to apply the teachings of Liu to Yang before the effective filing date of the claimed invention in order to effectively determine a resource required for performing a task (cf. Liu, [0037] In order to at least partly solve one or more of the above and other potential problems, embodiments of the present disclosure propose a solution for determining a resource required for executing a code segment. The solution presents a novel method for judging a resource required for performing a task by creating a dummy DL compiler to collect the memory and threads/blocks requirement to run a DL task in the dedicated processing unit. With the technical solution of the present disclosure, a resource required for performing a task may be determined effectively and efficiently.).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MAGGIE MAIDO whose telephone number is (703) 756-1953. The examiner can normally be reached M-Th: 6am - 4pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Michael Huntley can be reached on (303) 297-4307. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MM/Examiner, Art Unit 2129
/MICHAEL J HUNTLEY/Supervisory Patent Examiner, Art Unit 2129
Prosecution Insights