DETAILED ACTION
Remarks
The present application was filed 19 December 2023 and claims priority to KR10-2022-0178720 filed on 19 December 2022 and KR10-2023-0042202 filed on 30 March 2023.
Claims 1-20 are pending.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Examiner Notes
Examiner cites particular columns, paragraphs, figures and line numbers in the references as applied to the claims below for the convenience of the applicant. Although the specified citations are representative of the teachings in the art and are applied to the specific limitations within the individual claim, other passages and figures may apply as well. It is respectfully requested that, in preparing responses, the applicant fully consider the references in their entirety as potentially teaching all or part of the claimed invention, as well as the context of the passage as taught by the prior art or disclosed by the examiner.
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Specification
The disclosure is objected to because of the following informalities:
It uses the trademarks PYTORCH, APACHE, INTEL, GOOGLE, META, TENSORFLOW, at pp. 2, 8, and 15 without capitalizing every letter of the mark or, or, where appropriate, include a proper symbol indicating use in commerce such as ™, SM, or ® following the term. See M.P.E.P. § 608.01(v).
Drawings
The drawings filed 19 December 2023 are acceptable for examination purposes.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 5-6 and 15-16 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
As to claim 5, the claim refers to “the” nodes stored in the memory. There is insufficient antecedent basis for this limitation in the claim and it is unclear to which previously recited element, if any, the claim is referring. Note that claim 13 only refers to storing “node information” in the memory. For the purposes of examination, storing node information in claim 3 will be construed as comprising storing all the extracted nodes of that claim.
As to claim 6, the claim is dependent on claim 5 and asl refers to “the” nodes stored in the memory. Accordingly, it is rejected for the same reasons.
As to claims 14 and 16, the language of this claim is indefinite and rejected for substantially the same reasons noted above with respect to claims 5-6.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to a judicial exception without significantly more.
As to claim 1, the claim recites:
a method of generating a deep learning model graph and an abstract syntax tree (AST) for an integrated compiler, the method comprising:
generating, by a processor, a forward graph for inference of a deep learning model and a backward graph for training the deep learning model through a graph generator using the same library according to the same grammatical structure; and
generating, by the processor, an AST for the forward graph and the backward graph using one parser.
Although a process is claimed (Step 1), under the broadest reasonable interpretation in light of the specification the above underlined elements recite a mental process because they describe a process performable by the human mind with aid of pen and paper. The claim therefore recites an abstract idea. (Step 2A Prong 1).
None of the additional elements integrate the judicial exception into a practical application. (Step 2A Prong 2).
Reference to the method steps as being performed “by a processor”, “through a graph generator using the same library” or “using one parser” are mere instructions to apply the mental process because they only amount to instructions to implement the abstract idea on a computer. See M.P.E.P. § 2106.05(f).
Looking at the claim limitations as an ordered combination yields the same conclusion as that reached when looking at the elements individually. Their collective function is merely to implement the abstract idea using a generic computer.
The claim does not include additional elements that amount to significantly more than the judicial exception for substantially the same reasons discussed above with respect to a practical application. (Step 2B).
As to claims 2 and 7-10 the features of these claims do not integrate the abstract idea into a practical application or amount to significantly more at least because, apart from the same additional elements recited by claim 1, they only further describe steps performable by the human mind with aid of pen and paper and therefore only further describe the abstract idea itself.
As to claim 3-4, the features of these claims do not integrate the abstract idea into a practical application or amount to significantly more at least because they only recite mental steps performed by a processor as with claim 1. Further, storing certain information in memory only amounts to extra solution activity because it is only a nominal or tangential addition to the claims which courts have recognized as well-understood, routine and conventional. See M.P.E.P. § 2106.05(d)(II).
As to claims 5-6¸ the features of this claim do not integrate the abstract idea into a practical application or amount to significantly more at least because outputting is merely an additional mental step performable with aid of pen and paper and referring to that step as being performed by a processor only amounts to implement the abstract idea on a computer as with claim 1. To the extent outputting is considered an additional element, it only amounts to insignificant extra-solution activity (see M.P.E.P. § 2106.05(g) item (3)) and would include well-understood, routine and conventional activities such as transmission over the internet. See M.P.E.P. § 2106.05(d)(II).
As to claim 11, the claim recites the same abstract idea as claim 1 and does not include additional elements that integrate abstract idea into a practical application or amount to significantly more than the abstract idea for substantially the same reasons.
As to claims 12-20, the features of this claim do not integrate the abstract idea into a practical application or amount to significantly more for reasons substantially the same as those set forth above with respect to claims 2-10.
Claims 11-20 are rejected under 35 U.S.C. 101 because the claim the claimed invention is directed to non-statutory subject matter.
As to claim 11, the claim is directed to a device comprising a graph generator and a parser, which, in light of the last paragraph of page 15 of the specification, are construed as software. The claim is thus directed only to software, i.e., software per se. Software per se is non-statutory subject matter. See M.P.E.P. § 2106.03(I).
Note that while the claim recites that the generator and parser are “of” a processor this does not mean that the device itself comprises that processor. As claimed, the device only comprises the generator and the parser.
As to claims 12-20, the claims are dependent on claim 11 but do not cure the deficiencies of that claim. Note that while some of these claims refer to a processor performing certain activities, the device itself does not necessarily comprise that processor as with claim 1.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1 and 11 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Kwon et al., “Backward Graph Construction and Lowering in DL Compiler for Model Training on AI Accelerators” (art made of record – hereinafter Kwon).
As to claim 1, Kwon discloses a method of generating a deep learning model graph and an abstract syntax tree (AST) for an integrated compiler, (e.g., Kwon, p. 91 abstract: a deep learning compiler; p. 91 Fig. 3, right col. 2nd bullet point: we adopt a multi-level IR compiler to implement the proposed compiler [and see figure, a forward graph, backward graph and AST are generated from a Pytorch model (deep learning model). Note too the line from II to III in the figure labeled “AST”]) the method comprising:
generating, by a processor, a forward graph for inference of a deep learning model and a backward graph for training the deep learning model using the same library according to the same grammatical structure; (e.g., Kwon, p. 91 right col. Figure 3 [see figure] a forward graph and backward graph [forward graphs are necessarily used for inference and backward graphs are necessarily used for training ]1 are generated from a Pytorch [a library] model [deep learning model and same grammatical structure]; and
generating, by the processor an AST for the forward graph and the backward graph using one parser (e.g., Kwon, p. 91 right col. Figure 3 [see figure], the forward and backward graph are input into a single parser at II and the output is an AST, as indicated by the line labeled “AST” from II to III).
As to claim 11, it is a device claim having limitations substantially the same as claim 1. Accordingly, it is rejected for substantially the same reasons.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-2 and 11-12 are rejected under 35 U.S.C. 103 as being unpatentable over “AOT Autograd – How to Use and Optimize?” (art made of record – hereinafter AOTAutograd) in view of Zhu et al. “Matrix-DSP back-end support based on TVM compilation structure” (art made of record – hereinafter Zhu).
As to claim 1, AOTAutograd discloses a method of generating a deep learning model graph for an integrated compiler (e.g., AOTAutograd, p. 1 “Background”: AOT Autograd provides simple mechanisms to compile the extracted forward and backward graphs through deep learning compilers such as TVM and others) the method comprising:
generating, by a processor, a forward graph for inference of a deep learning model and a backward graph for training the deep learning model through a graph generator using the same library according to the same grammatical structure; (e.g., AOTAutograd, p. 1 “Background”: we will learn how to use AOT Autograd [a software program, so its functions are necessarily performed by a process] to speedup training of deep learning models; p. 1 “Use AOT Autograd” AOT uses __torch_dispatch__ based tracing mechanism to extract forward and backward graphs and wraps them in torch.fx GraphModule containers [again, forward graphs are necessarily used for inference and backward graphs are necessarily used for training]; pp. 1-2 “Use AOT Autograd”: lets write a compiler that just prints the graph. The above code prints the FX graph for the forward and backward graph [and see the output of the code in the second code block of p. 2]) and
the forward graph and the backward graph (see immediately above).
AOTAutograd does not explicitly disclose generating an abstract syntax tree (AST) for an integrated compiler or generating, by the processor, an AST for the forward graph and the backward graph using one parser.
However, in an analogous art, Zhu discloses:
generating an abstract syntax tree (AST) for an integrated compiler; (e.g., Zhu, [/ 2 Sec. 2.1: the deep learning compiler TVM; p. 3 Sec. 2.3: TVM back-end: completing the basic tensor ir syntax tree2 construction) and
generating, by the processor, an AST for the graph using one parser (e.g., Zhu, p. 2 Sec. 2.1: after performing a graph optimization on the relay IR, it is divided [parsed] into a tensor expression representing the function of the operator. The back-end optimizes the operator resolved into tensor IR form; p. 3 Sec. 2.3 pars. 1-2: the operators in the calculation graph will be parsed into operator expressions described in tensor language. The optimization process of the operator is actually the processing process of the tensor IR syntax tree [AST]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the forward and backward graph of AOTAutograd such that an AST is generated for those graphs using one parser, as taught by Zhu, as Zhu would provide the advantage of a means of compiling the graphs using TVM, as suggested by AOTAutograd. (See Zhu, p. 2 Sec. 2.1, AOTAutograd, p. 1 “Background”).
As to claim 2, AOTAutograd/Zhu discloses the method of claim 1 (see rejection of claim 1 above), AOTAutograd further discloses:
wherein the generating of the backward graph using the same library according to the same grammatical structure comprises extracting, by the processor, all trainable nodes among nodes in the deep learning model and edge information for connecting the nodes and then outputting the nodes and the edge information as input data for graph generation (e.g., AOTAutograd, p. 3 “Recomputation”: at the end of __torch_dispatch__ tracing, AOT Autograd has [i.e., as output from __torch_dispatch__] a forward graph and a joint forward-backward graph AOT Autograd then uses a partitioner to isolate the forward and backward graph [i.e., the graphs from __torch_dispatch__ (nodes and edge information for connecting the nodes) are input to the partitioner]).
As to claim 11, it is a device claim having limitations substantially the same as claim 1. Accordingly, it is rejected for substantially the same reasons.
As to claim 12, it is a device claim having limitations substantially the same as claim 2. Accordingly, it is rejected for substantially the same reasons.
Claims 3-6 and 13-16 are rejected under 35 U.S.C. 103 as being unpatentable over AOTAutograd (“AOT Autograd – How to Use and Optimize?”) in view of Zhu (“Matrix-DSP back-end support based on TVM compilation structure”) in further view of Russell (US 10,949,178) (art made of record – hereinafter Russell).
As to claim 3, AOTAutograd/Zhu discloses the method of claim 2 (see rejection of claim 2), and further discloses trainable nodes (see rejection of claim 2 above) but does not explicitly disclose further comprising, after the extracting of the all the trainable nodes and the edge information for connecting the nodes: determining, by the processor, types of all the nodes from a first node; and storing only node information or the node information and attribute information in a memory depending on a determined type of a node.
However, in an analogous art, Russell discloses further comprising, after the extracting of the all the nodes and the edge information for connecting the nodes (e.g., Russell, Fig. 2 and associated text, col. 9 ll. 23-25: a subgraph projection generated from the full graph projection):
determining, by the processor, types of all the nodes from a first node; (e.g., Rusell, Fig. 2 and associated text, col. 9 l. 66 – col. 10 l. 2: nodes may be added as leaf nodes) and
storing only node information or the node information and attribute information in a memory depending on a determined type of a node (e.g., Rusell, Fig. 2 and associated text, col. 9 l. 66 – col. 10 l. 2: nodes may be added as leaf nodes [storing node information] that do not include references [attribute information] to any other objects).
It would have been obvious to modify the graph of trainable nodes taught by AOTAutograd/Zhu such that after the extracting of the all the nodes and the edge information for connecting the nodes: determining types of all the nodes from a first node; and storing only node information or the node information and attribute information in a memory depending on a determined type of a node, as taught by Russell, as Russell would provide the advantage of a means of generating a subgraph of the graph, including leaf nodes, that excludes functions irrelevant to a particular workflow. (See Russell, col. 5 ll. 42-25, col. 9 l. 66 – col. 10 l. 2).
As to claim 4, AOTAutograd/Zhu/Russell discloses the method of claim 3 (see rejection of claim 4 above), but AOTAutograd/Zhu does not explicitly disclose wherein the storing of only the node information or the node information and the attribute information in the memory comprises: determining, by the processor, a type of current node; storing only node information in the memory when the current node is a leaf node; and storing not only the node information but also attribute information in the memory when the current node is not a leaf node.
However, in an analogous art, Russell discloses:
wherein the storing of only the node information or the node information and the attribute information in the memory (see below) comprises:
determining, by the processor, a type of current node; (see below)
storing only node information in the memory when the current node is a leaf node; (e.g., Rusell, Fig. 2 and associated text, col. 9 l. 66 – col. 10 l. 2: nodes may be added as leaf nodes [storing node information] that do not include references [attribute information] to any other objects) and
storing not only the node information but also attribute information in the memory when the current node is not a leaf node (see immediately above, note the non leaf nodes in the figure that do include references to other objects [attribute information])
It would have been obvious to modify the graph of trainable nodes taught by AOTAutograd/Zhu such that after the extracting of the all the nodes and the edge information for connecting the nodes: determining types of all the nodes from a first node; and storing only node information or the node information and attribute information in a memory depending on a determined type of a node, as taught by Russell, as Russell would provide the advantage of a means of generating a subgraph of the graph, including leaf nodes, that excludes functions irrelevant to a particular workflow. (See Russell, col. 5 ll. 42-25, col. 9 l. 66 – col. 10 l. 2).
As to claim 5, AOTAutograd/Zhu/Russell discloses the method of claim 3 (see rejection of claim 3 above), but AOTAutograd/Zhu does not explicitly disclose further comprising, after the storing of only the node information or the node information and the attribute information in the memory, outputting, by the processor, the nodes stored in the memory and connection states between the nodes.
However, in an analogous art, Russell discloses:
further comprising, after the storing of only the node information or the node information and the attribute information in the memory, outputting, by the processor, the nodes stored in the memory and connection states between the nodes (e.g., Russell, Fig. 2 and associated text, col. 13 ll. 25-26: the system outputs the generated subgraph projection).
It would have been obvious to modify the graph of trainable nodes taught by AOTAutograd/Zhu to include after the storing of only the node information or the node information and the attribute information in the memory, outputting the nodes stored in the memory and connection states between the nodes, as taught by Russell, as Russell would provide the advantage of a means of generating a subgraph of the graph, including leaf nodes, that excludes functions irrelevant to a particular workflow. (See Russell, col. 5 ll. 42-25, col. 9 l. 66 – col. 10 l. 2).
As to claim 6, AOTAutograd/Zhu/Russell discloses the method of claim 5 (see rejection of claim 5 above), but AOTAutograd/Zhu does not explicitly disclose further comprising, after the outputting of the nodes stored in the memory and the connection states between the nodes, outputting, by the processor, actual values of the attribute information of the nodes stored in the memory.
However, in an analogous art, Russell discloses:
further comprising, after the outputting of the nodes stored in the memory and the connection states between the nodes, outputting, by the processor, actual values of the attribute information of the nodes stored in the memory (e.g., Russell, col. 13 ll. 35-37: the generated subgraph projection [generating the subgraph being outputting nodes stored in the memory and the connection states between the nodes, see above] may be deployed to an application server [i.e., its actual values are output to the server])
It would have been obvious to modify the graph of trainable nodes taught by AOTAutograd/Zhu to include after the outputting of the nodes stored in the memory and the connection states between the nodes, outputting actual values of the attribute information of the nodes stored in the memory, as taught by Russell, as Russell would provide the advantage of a means of transmitting the subgraph to another device so that it can be used there. (See Russell, col. 13 ll. 35-37).
As to claim 13, it is a device claim having limitations substantially the same as claim 3. Accordingly, it is rejected for substantially the same reasons.
As to claim 14, it is a device claim having limitations substantially the same as claim 4. Accordingly, it is rejected for substantially the same reasons.
As to claim 15, it is a device claim having limitations substantially the same as claim 5. Accordingly, it is rejected for substantially the same reasons.
As to claim 16, it is a device claim having limitations substantially the same as claim 6. Accordingly, it is rejected for substantially the same reasons.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to TODD AGUILERA whose telephone number is (571)270-5186. The examiner can normally be reached M-F 11AM - 7:30PM EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Hyung S Sough can be reached at (571)272-6799. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/TODD AGUILERA/Primary Examiner, Art Unit 2192
1 See, e.g., Zhao, “Optimisation of a modern numerical library: a bottom-up approach” at p. 20 last par. – p. 21 par. 1.
2 The tensor IR syntax tree of TVM is an abstract syntax tree. See “Coverage-Guided Tensor Compiler Fuzzing with Joint IR-Pass Mutation” at p. 73:8 last par. – p. 73:9 par. 1.
Prosecution Insights