DETAILED ACTION
This non-final office action is responsive to application 18/495,609 as submitted 26 Oct., 2023.
Claim status is currently pending and under examination for claims 1-30 of which independent claims are 1, 9, 17 and 24.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Acknowledgment is made of applicant’s claim for domestic priority based on provisional application 63/467,136 giving the application an effective filing date of 05/17/23.
Information Disclosure Statement
As required by MPEP 609(c), the applicant’s submissions of the Information Disclosure Statements dated 10/26/23 – 07/25/24 are acknowledged by the examiner and the cited references have been considered in the examination of the claims now pending. As required by MPEP 609 C(2), a copy of the PTOL-1449 initialed and dated by the examiner is attached to the instant office action.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
7. This application includes one or more claim limitations that uses the word “means,” and are being interpreted under 35 U.S.C. 112(f) because the claim limitations use a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f). The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f), is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f):
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Such functional language is found in the claims as follows:
Claim 24: An apparatus, comprising:
“means for receiving a graph”
“means for determining retention intervals”
“means for determining a sequence of tasks”
Claim 25: The apparatus of claim 24, further comprising
“means for determining the retention intervals”
Because these claim limitations are being interpreted under 35 U.S.C. 112(f), they are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
The corresponding structure is interpreted in light of the specification. The specification does not make it clear that the units are limited to particular embodiments. The means for language is repeated at Fig 8, [0008], [0131-134] and further states per PG Pub No 2024/0386237A1:
[0140] “any suitable means… The means may include various hardware and/or software component(s) and/or module(s), including, but not limited to” cont’d “operations may have corresponding counterpart means-plus-function components”
If applicant does not intend to have these limitations interpreted under 35 U.S.C. 112(f), applicant may: (1) amend the claim limitations to avoid them being interpreted under 35 U.S.C. 112(f) (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitations recite sufficient structure to perform the claimed function so as to avoid them being interpreted under 35 U.S.C. 112(f).
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
Claims 24-30 are rejected under 35 U.S.C. 112(b), as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor, regards as the invention. Particularly, claims 24-25 recite limitations “means for receiving a graph”, “means for determining retention intervals”, “means for determining a sequence of tasks”, and “means for determining the retention intervals” invoke 35 U.S.C. 112(f). However, the written description fails to disclose the corresponding structure, material, or acts for performing the entire claimed function and to clearly link the structure, material, or acts to the function. The means for language is repeated at Fig 8, [0008], [0131-134] and described at [0140] as non-limiting embodiments that may be used. However, may is not must, and the means are not defined with particularity sufficient to discern what specifically the means is required to include. Accordingly, neither the specification nor the drawings describe sufficient supporting structure for the functional language that clearly links the structure, material, or acts in performance of the entire claimed function. Thus, the claims are indefinite and are rejected under 35 U.S.C. 112(b). The functionality is interpreted as any combination of hardware or software. Further claims 25-30 depend from claim 24 without curing the deficiency. As such, claims 24-30 are rejected under 35 USC 112(b).
Applicant may:
(a) Amend the claim so that the claim limitation will no longer be interpreted as a limitation under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph;
(b) Amend the written description of the specification such that it expressly recites what structure, material, or acts perform the entire claimed function, without introducing any new matter (35 U.S.C. 132(a)); or
(c) Amend the written description of the specification such that it clearly links the structure, material, or acts disclosed therein to the function recited in the claim, without introducing any new matter (35 U.S.C. 132(a)).
If applicant is of the opinion that the written description of the specification already implicitly or inherently discloses the corresponding structure, material, or acts and clearly links them to the function so that one of ordinary skill in the art would recognize what structure, material, or acts perform the claimed function, applicant should clarify the record by either:
(a) Amending the written description of the specification such that it expressly recites the corresponding structure, material, or acts for performing the claimed function and clearly links or associates the structure, material, or acts to the claimed function, without introducing any new matter (35 U.S.C. 132(a)); or
(b) Stating on the record what the corresponding structure, material, or acts, which are implicitly or inherently set forth in the written description of the specification, perform the claimed function. For more information, see 37 CFR 1.75(d) and MPEP §§ 608.01(o) and 2181.
Claim Objections
As a result of the above 112(f) issue, applicant is advised that should claim 1 be found allowable, claim 24 will be objected to under 37 CFR 1.75 as being a substantial duplicate thereof. Since the functional language is not found to specifically include anything which is not already recited in claim 1, then the claims are substantial duplicates. When two claims in an application are duplicates or else are so close in content that they both cover the same thing, despite a slight difference in wording, it is proper after allowing one claim to object to the other as being a substantial duplicate of the allowed claim. See MPEP § 608.01(m).
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1, 4-9, 12-17, 20-24 and 27-30 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. In determining whether the claims are subject matter eligible, the examiner applies guidance set forth under MPEP 2106.
Step 1: Is the claim to a process, machine, manufacture, or composition of matter? Yes—all claims fall within one of the four statutory categories: claims 1, 4-8, 24 and 27-30 are drawn to an apparatus/machine, claims 9 and 12-16 are a method/process, further claims 17 and 20-23 are a computer readable medium/article of manufacture. Therefore, the claims are drawn to at least one of the identified statutory categories and the analysis should proceed per MPEP 2106.03.
Step 2A, prong one: Does the claim recite an abstract idea, law of nature or natural phenomenon? Yes—the claims, under the broadest reasonable interpretation, recites an abstract idea. In this case, claims fall within the enumerated grouping of abstract idea being “Mental Processes”, but for the recitation of general computer components. In particular, claims recite:
“determine retention intervals for outputs of the multiple nodes based on rematerialization constraints and paging constraints, the retention intervals corresponding to a time interval for retaining each node output” (mental determination)
“determine a sequence of tasks for executing the multiple nodes based on the retention intervals” (mental determination)
Focus of the claims concern determining retention intervals corresponding to a time interval. This may be performed by a human as wall-clock time or calendar-based mental determination. The specification does not describe milliseconds or granularity of time beyond mental capacity. For example, a large model might take hours or weeks to train and this length of time is readily discerned by a human developer. While additional constraints and tasks are recited, these are broadly underspecified and do not preclude mental performance. Accordingly, the claims are directed to mental processes as the abstract idea.
Step 2A, prong two: Does the claim recite additional elements that integrate the judicial exception into a practical application? No—a practical application is not integrated by the judicial exception because the additional elements are as follows:
“An apparatus, comprising: at least one local memory; at least one global memory; and at least one processor coupled to the at least one local memory and the at least one global memory” MPEP 2106.05(f) merely uses a computer as a tool to perform an abstract idea
“receive a graph representing an artificial neural network (ANN), the graph including multiple nodes connected by edges and each node represents an operation” MPEP 2106.05(g) adding insignificant extra-solution activity to the judicial exception, e.g. mere data gathering
Balance of the claim concerns computer components and receiving a graph representing a neural network. The neural network represented by graph is described by specification [0067] as a directed acyclic graph (DAG), and which serves to characterize collected/received information. Data gathering and computer implementation do not meaningfully limit the claim such that the claim as a whole is more than a drafting effort designed to monopolize the judicial exception. Nothing responsive to the claim functionalities points to a real-world use-case, but merely uses a computer to carry out the abstract idea. As set forth under MPEP 2106.05(a)(2)(III)(C) “A claim that requires a computer may still recite a mental process.” As such, claims remain drawn to the abstract idea and additional elements fail to further integrate the judicial exception into a practical application.
Step 2B: Does the claim recite additional elements that amount to significantly more than the judicial exception? No—the claims do not include additional elements that amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea in to a practical application, the additional elements are identified with respect to MPEP 2106.05 and do not demonstrate inventive concept. Particularly, additional elements are as follows:
“An apparatus, comprising: at least one local memory; at least one global memory; and at least one processor coupled to the at least one local memory and the at least one global memory” MPEP 2106.05(f) merely uses a computer as a tool to perform an abstract idea. Particularly, the general purpose computer does not qualify as a particular machine under MPEP 2106.05(b). General purpose computing includes local and global memories such that a personal computer has local memory, and a global memory may be cloud based where commodity computer communicates as server via global/world wide web, i.e. Internet.
“receive a graph representing an artificial neural network (ANN), the graph including multiple nodes connected by edges and each node represents an operation” MPEP 2106.05(g) adding insignificant extra-solution activity to the judicial exception, e.g. mere data gathering. Particularly, said extra-solution activity is a well-understood, routine and conventional activity under MPEP 2106.05(d)(II)(i) “Receiving or transmitting data over a network” and/or evidenced by Steiner et al., “OLLA” (detailed below) at [P.2 Sect. 2.1 ¶1] “Deep neural networks can be represented using dataflow graphs, as pioneered by TensorFlow [1]” [1] bibliographic reference showing over 30,000 citations in Google Scholar as it is one of the most fundamental or commonly used tools in the field of endeavor.
Significantly more is not satisfied by the additional elements for at least the reasons above. The significance of receiving a graph representing neural network is shown that this is not new, and the significance of computer implementation is not a particular hardware setup beyond general purpose. Considered as whole, the claim lacks particular transformation with insufficient detail for how or what the invention is to perform, and there is no indication that the combination of elements improves the functioning of a computer or improves any other technology. Claims appear to present at least some risk of pre-emption. Accordingly, claims remain directed to the abstract idea of mental processes and the additional elements fail to integrate the abstract idea into a practical application or amount to significantly more.
For at least the foregoing reasons, the claims are not patent eligible. This rejection applies equally to independent claims 1, 9, 17 and 24 as well to dependent claims 4-8, 12-16, 20-23 and 27-30. Dependent claims when analyzed as a whole are held to be patent ineligible under 35 U.S.C. 101 because the additional limitations fail to establish that the claims are not directed to an abstract idea, or that they include additional elements which integrate the judicial exception into a practical application or amount to significantly more.
Independent claims 9, 17 and 24 recite limitations substantively similar to claim 1 and include general computer components. This includes claim 9 “processor” and claim 17 “non-transitory computer-readable medium having program code recorded thereon, the program code executed by processor” as well as claim 24 “means for” performing the limitations already addressed. These are additional elements which fall under MPEP 2106.05(f) mere instructions to implement an abstract idea on a computer, and more particularly MPEP 2106.05(b) general purpose computer that does not satisfy the test of particular machine. Accordingly, the additional elements do not integrate the abstract idea into a practical application or amount to significantly more.
Dependent claims 4-5, 12-13, 20-21 and 27-28 embellish paging constraints as quantity of times outputs are paged out and whether outputs are paged in or paged out. The quantity of times and whether decision to define paging constraints are considered part of the abstract idea. For example, a simple count for increment of outputs and determining whether to page-in input or page-out output are mental determinations. The memory is an additional element which amounts to mere use of computers to perform the abstract idea under MPEP 2106.05(f) and which does not qualify as a particular machine under MPEP 2106.05(b). Therefore, the additional elements do not integrate the judicial exception into a practical application or amount to significantly more.
Dependent claims 6-7, 14-15, 22-23 and 29-30 embellish rematerialization constraints as a quantity of times a node is permitted to be recomputed, and data dependencies determined based on edges connecting nodes. The limitations are considered part of the abstract idea being mental processes which may mentally determine an allowed number of iterations for repetitive recursion, and data sorting by relationships in a graph representation - for example, a subgraph set membership. There are no additional elements.
Dependent claims 8 and 16 disclose wherein local memory comprises tightly-coupled memory. The claim gives no indication what constitutes “tight” or even what coupling is coupled to. The memory is considered an additional element which falls under MPEP 2106.05(f) adding the words ‘apply-it’ or merely uses a computer as a tool to perform an abstract idea, and which fails the test of particular machine under MPEP 2106.05(b). Therefore, the additional elements do not integrate the abstract idea into a practical application or amount to significantly more.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1, 5, 9, 13, 17, 21, 24 and 28 are rejected under 35 U.S.C. 103 as being unpatentable over:
Steiner et al., “OLLA: Optimizing the Lifetime and Location of Arrays to Reduce the Memory Usage of Neural Networks” hereinafter Steiner (arXiv: 2210.12924v2) in view of
Patil et al., “POET: Training Neural Networks on Tiny Devices with Integrated Rematerialization and Paging” hereinafter Patil (arXiv: 2207.07697v1).
With respect to claim 1, Steiner teaches:
An apparatus, comprising: at least one local memory; at least one global memory; and at least one processor coupled to the at least one local memory and the at least one global memory, the at least one processor configured to: {Steiner discloses [P.8 Sect. 5.1] Experimental Setup, CPU and GPU processor with 40GB memory. Memory that is local is e.g. [P.1 ¶2] “memory constrained edge devices such as smartphones” similar at [P.8 Sect. 5.1] and/or [P.3 ¶3] “resident in memory” in-memory described throughout, and global memory comprises [P.5 ¶2] “shared preallocated buffer”, fanin cache [P.6] Alg/Func. 2 and/or [P.8 Sect. 5.2] “datacenter hardware” introduced Fig 2 “memory capacity of Nvidia datacenter GPUs (in gigabytes)”}
receive a graph representing an artificial neural network (ANN), the graph including multiple nodes connected by edges and each node represents an operation {Steiner discloses [P.3 Sect3.1] “we model a neural network as a directed acyclic graph G = (V, E) with n nodes V = v1, …, vn that represent the operators and the neural network, and m edges E = e1, …, em the encode the tensors” illustrated Figs 3, 5};
determine retention intervals for outputs of the multiple nodes {Steiner discloses [P.5 Sect. 4.1] Eq.12 (Pres)erve is retention, interval is time specified “tensors must be preserved in memory from the time they are created until their last sink node has run” run in node shown Fig 3 as edges resident in memory. Further Eq.12 ASAP (as soon as possible), ALAP (as late as possible) are used in SPAN (range-bound) Eq.10 for Lifetime of Tensor, introduced [P.4 Sect3.1]). Steiner additionally suggests [P.11 ¶2] “Rematerialization, also known as checkpointing… Paging, aka spilling” cont’d “combining several of these techniques has been proposed to increase their effectiveness and mitigate their drawbacks [4,62]” where [62] is bibliography reference Patil’s POET}; and
determine a sequence of tasks for executing the multiple nodes based on the retention intervals {Steiner [P.4 ¶2] “sequence of tensor creations and preservations reflects a valid execution sequence of the neural network corresponding to a feasible topological ordering of the DAG” where preservation is preserved by Pres Eq.12 as retention intervals. See also Alg./Function 1 [P.4-5 Sect3.5] “optimized execution sequence” Figs 3, 5}.
Patil discloses
{Patil discloses [P.4 Sect.4-5] “Integrated paging and rematerialization …jointly considers both rematerialization and paging in an integrated search space” describing Fig 2 shows scheduling over time, contraints e.g. “latency target” Eq.3 deadline where ≤ operator is [P.9 ¶1] “deadline bounds” and/or [P.5 ¶2] “time budget μdeadline” the deadline formulation included in Alg.1 POET optimizer}; and
Patil is directed to memory constrained edge devices with neural network optimization thus being analogous. A person having ordinary skill in the art would have considered it obvious prior to the effective filing date to include rematerialization and paging constraints per Patil as suggested by Steiner in combination to arrive at the invention as claimed for a motivation from Patil being [P.4 Sect. 4-5] “Rematerializtion and paging are two techniques to lower the memory consumption of large, state-of-the-art ML models…major advantage of paging is that depending on how occupied the memory bus is, it can be pipelined to hide latency” cont’d “Rematerialization is most efficient for operations that are cheap-to-compute yet memory intensive. These operations can be recalculated with low energy overhead. Paging, however, is best suited to compute-intensive operations where rematerialization would otherwise incur significant energy overhead. POET jointly considers both.”
With respect to claim 5, the combination of Steiner and Patil teaches the apparatus of claim 1, in which
the paging constraints defines whether node outputs are paged into the at least one local memory from the at least one global memory or paged out from the at least one local memory to the at least one global memory {Patil [P.6 Sect. 5.5] “page-in (Mint,k) and the page-out (Moutt,k) schedule to ensure there is no contention for the memory… in-memory schedule (Sauxt-1,i) to account for the earlier paging-in” Alg.1, e.g. [P.5 Sect. 5.2] “page-in if SRAMt-1,i is resident on flash” if-condition and/or schedule corresponds to whether, describes optimal paging in memories Fig 1-2}.
With respect to claim 9, the rejection of claim 1 is incorporated. The difference in scope being a method implemented by processor to perform limitations of apparatus claim 1. Steiner discloses per [P.1 ¶2] “Our method” as “We present OLLA, an algorithm that optimizes the lifetime and memory location of the tensors used to train neural networks” methodology and experimental setup with processor described [P.8 Sect. 5.1-5.2]. The remainder of this claim is rejected for the same rationale as claim 1.
With respect to claim 13, the combination of Steiner and Patil teaches the processor-implemented method of claim 9, and further teaches the limitation of claim 5. Therefore, the rejection of claim 5 is applied to claim 13.
With respect to claim 17, the rejection of claim 1 is incorporated. The difference in scope being a non-transitory computer-readable medium encoding program code executed by processor to perform limitations of apparatus claim 1. Steiner discloses [P.8 Sect5.1] “implemented OLLA on top of PyTorch” with processor and memory, and further provides [P.2 ¶4] open-source GitHub for implementation of algorithmic functions. The remainder of this claim is rejected for the same rationale as claim 1.
With respect to claim 21, the combination of Steiner and Patil teaches the non-transitory computer-readable medium of claim 17, and further teaches the limitation of claim 5. Therefore, the rejection of claim 5 is applied to claim 21.
With respect to claim 24, the rejection of claim 1 is incorporated. The difference in scope being an apparatus comprising means for performing the limitations of apparatus claim 1. The “means for” is read in light of instant specification [0140] “any suitable means.” Steiner discloses experimental setup [P.8 Sect. 5.1] comprising processor and memory as means for implementation. The remainder of this claim is rejected for the same rationale as claim 1.
With respect to claim 28, the combination of Steiner and Patil teaches the apparatus of claim 24, and further teaches the limitation of claim 5. Therefore, the rejection of claim 5 is applied to claim 28.
Claims 2, 10, 18 and 25 are rejected under 35 U.S.C. 103 as being unpatentable over Steiner and Patil in view of Hu et al., “MegTaiChi: Dynamic Tensor-based Memory Management Optimization for DNN Training” hereinafter Hu.
With respect to claim 2, the combination of Steiner and Patil discloses the apparatus of claim 1. Hu teaches in which
the retention intervals are determined using a joint optimization of communication between the at least one local memory and the at least one global memory based on the rematerialization constraints and the paging constraints to reduce the communication between the at least one local memory and the at least one global memory {Hu discloses [P.6 Sect. 4.2] Eq.2 min is minimization to reduce/decrease memory communicated by evicting tensors, time & memories are jointly optimized as βret(t) recompute/rematerialize time parameterized by β; Cs(t) cost swap time (swap akin to paging), over memory m(t) as local, and global is free memories Mleft(t) and Mright(t), see e.g. [P.3 Sect2.3 ¶1] “host memory as an external memory and copies the intermediate data back and forth between host and on-device memory” describes swapping and tensor rematerialization. See also [Sect. 4.1] Eq.1 argmin and Figs 2, 4-7}.
Hu is directed to tensor rematerialization for neural networks with managed memory devices and operations over dynamic computational graphs thus being analogous. A person having ordinary skill in the art would have considered it obvious prior to the effective filing date to reduce communication with joint optimization as per Hu in combination with Steiner’s retention intervals and Patil’s paging to arrive at the invention as claimed for a motivation “temporal locality, the dynamic tensor evicting is designed to determine whether or when to intercept tensor allocations, accesses, and deallocations by avoiding the release of high frequency used data and balancing the tensor rematerialization time and memory footprint” [P.3 Sect. 3.2 ¶1]. Additional motivation is expressly disclosed [P.5 ¶2], and a goal to enable large model training with parallelization of operations on dynamic computational graph [P.3 ¶1]. See contributions [P.2 Sect.1 Last¶] and challenges addressed [P.4-5 Sect3.3].
With respect to claim 10, the combination of Steiner and Patil teaches the processor-implemented method of claim 9, and further combination with Hu teaches the limitation of claim 2. Therefore, the rejection of claim 2 with equal motivation is applied to claim 10.
With respect to claim 18, the combination of Steiner and Patil teaches the non-transitory computer-readable medium of claim 17, and further combination with Hu teaches the limitation of claim 2. Therefore, the rejection of claim 2 with equal motivation is applied to claim 18.
With respect to claim 25, the combination of Steiner and Patil teaches the apparatus of claim 24, and further combination with Hu teaches the limitation of claim 2. Therefore, the rejection of claim 2 with equal motivation is applied to claim 25.
Claims 3, 11, 19 and 26 are rejected under 35 U.S.C. 103 as being unpatentable over Steiner and Patil in view of Zuo et al., “SEALing Neural Network Models in Secure Deep Learning Accelerators” hereinafter Zuo (arXiv: 2008.03752v1).
With respect to claim 3, the combination of Steiner and Patil discloses the apparatus of claim 2. Zuo teaches in which
the communication between the at least one local memory and the at least one global memory is conducted using a double data rate (DDR) bus {Zuo see Fig 1 [P.3 ¶3,1] “GDDR memory bus” as “accelerator accesses the DRAM through the high-bandwidth GDDR bus” off-chip DRAM is global and local is the on-chip cache of accelerator. See [Abst] “GDDR (graphics double data rate) memory bus that connects the accelerator chip with DRAM memory”}.
Zuo is directed to neural networks and memory optimization thus being analogous. A person having ordinary skill in the art would have considered it obvious prior to the effective filing date to employ GDDR memory bus per Zuo to arrive at the invention as claimed as obvious to try in choosing from a finite number of identified, predictable solutions, with a reasonable expectation of success. Here, the identified solution serves to couple memories using known devices. Additionally, advantage of such arrangement is that “GDDR memory is designed for GPUs to achieve high memory access bandwidth” (Zuo [P.2 ¶2], [P.4 ¶5]).
With respect to claim 11, the combination of Steiner, Patil and Hu teaches the processor-implemented method of claim 10, and further combination with Zuo teaches the limitation of claim 3. Therefore, the rejection of claim 3 with equal motivation is applied to claim 11.
With respect to claim 19, the combination of Steiner, Patil and Hu teaches the non-transitory computer-readable medium of claim 17, and further combination with Zuo teaches the limitation of claim 3. Therefore, the rejection of claim 3 with equal motivation is applied to claim 19.
With respect to claim 26, the combination of Steiner, Patil and Hu teaches the apparatus of claim 24, and further combination with Zuo teaches the limitation of claim 3. Therefore, the rejection of claim 3 with equal motivation is applied to claim 26.
Claims 4, 12, 20 and 27 are rejected under 35 U.S.C. 103 as being unpatentable over Steiner and Patil in view of Liu et al., US PG Pub No 2024/0211409A1 hereinafter Liu.
With respect to claim 4, the combination of Steiner and Patil teaches the apparatus of claim 1. Liu teaches in which
the paging constraints define a quantity of times node outputs are paged out from the at least one local memory to the at least one global memory {Liu [0050] “page-out count when a storage manager writes a page into the system memory from the swap space of the disk… page-out rate”}.
Liu is directed to page allocation for memories and machine learning models thus being analogous. A person having ordinary skill in the art would have considered it obvious prior to the effective filing date to specify page-out count per Liu in combination to arrive at the invention as claimed as applying a known technique to a known device ready for improvement to yield predictable results, and/or for a motivation that [0050] “page-out rate is tracked.”
With respect to claim 12, the combination of Steiner and Patil teaches the processor-implemented method of claim 9, and further combination with Liu teaches the limitation of claim 4. Therefore, the rejection of claim 4 with equal motivation is applied to claim 12.
With respect to claim 20, the combination of Steiner and Patil teaches the non-transitory computer-readable medium of claim 17, and further combination with Liu teaches the limitation of claim 4. Therefore, the rejection of claim 4 with equal motivation is applied to claim 20.
With respect to claim 27, the combination of Steiner and Patil teaches the apparatus of claim 24, and further combination with Liu teaches the limitation of claim 4. Therefore, the rejection of claim 4 with equal motivation is applied to claim 27.
Claims 6, 14, 22 and 29 are rejected under 35 U.S.C. 103 as being unpatentable over Steiner and Patil in view of Tang et al., “DELTA: Dynamically Optimizing GPU Memory beyond Tensor Recomputation” hereinafter Tang (arXiv: 2203.15980v2).
With respect to claim 6, the combination of Steiner and Patil teaches the apparatus of claim 1. Tang teaches in which
the rematerialization constraints define a quantity of times that a node is permitted to be recomputed {Tang [P.4 Sect.4] Alg.1 Line 1 is the quantity of times specified by for-loop (for i = 1, …, T) where T is training iterations, Alg.1 Line 3 Recompute(o) is rematerialize current tensor, e.g. [P.2 Last¶] “We combine dynamic tensor swapping together with dynamic tensor rematerialization” Fig 2 shows GPU which is a compute node. Additionally, Alg.1 Line 8 reload calls Alg.4 prefetcher with tensor queue and reload count. See also [P.6 ¶4] “Repeat the process”}.
Tang is directed to tensor rematerialization for neural network training and memory management thus being analogous. A person having ordinary skill in the art would have considered it obvious prior to the effective filing date to specify a number of recompute times per Tang in combination to arrive at the invention as claimed for a motivation being [P.3 Last¶] “we aim to establish a new GPU memory optimization manager, settling the optimization limitation of dynamic tensor recomputation” and/or [P.6 ¶4] “mitigates the drawbacks of the extra time cost of swapping tensors between GPU and CPU. This is a great improvement.” See contributions [P.1-2 Pg.Brk].
With respect to claim 14, the combination of Steiner and Patil teaches the processor-implemented method of claim 9, and further combination with Tang teaches the limitation of claim 6. Therefore, the rejection of claim 6 with equal motivation is applied to claim 14.
With respect to claim 22, the combination of Steiner and Patil teaches the non-transitory computer-readable medium of claim 17, and further combination with Tang teaches the limitation of claim 6. Therefore, the rejection of claim 6 with equal motivation is applied to claim 22.
With respect to claim 29, the combination of Steiner and Patil teaches the apparatus of claim 24, and further combination with Tang teaches the limitation of claim 6. Therefore, the rejection of claim 6 with equal motivation is applied to claim 29.
Claims 7, 15, 23 and 30 are rejected under 35 U.S.C. 103 as being unpatentable over Steiner and Patil in view of Wen et al., “A Swap Dominated Tensor Re-Generation Strategy for Training Deep Learning Models” hereinafter Wen.
With respect to claim 7, the combination of Steiner and Patil teaches the apparatus of claim 1. Wen teaches in which
the rematerialization constraints include data dependencies determined based on the edges connecting the multiple nodes {Wen [P.998 Last¶] “Edges represent the computational dependencies between tensors” tensor re-generation (i.e. rematerialization) describes tensor graph with nodes, edges representing DNN deep neural network. See Figs 1-4 and Eqs. 1-4}.
Wen is directed to tensor re-generation with graph representation of neural networks and memory optimization thus being analogous. A person having ordinary skill in the art would have considered it obvious prior to the effective filing date to specify edge dependencies per Wen to clarify Steiner’s graph edges as applying as combining prior art elements according to known methods to yield predictable results, and/or so that downstream operations may consider dependencies [P.999 ¶1] with dependency accessible described [P.999 Last2¶] “When computing node j in stage t, dependencies need to reside in GPU memory or be calculated… tensors reside in memory until it is used as a dependency to complete the computations.”
With respect to claim 15, the combination of Steiner and Patil teaches the processor-implemented method of claim 9, and further combination with Wen teaches the limitation of claim 7. Therefore, the rejection of claim 7 with equal motivation is applied to claim 15.
With respect to claim 23, the combination of Steiner and Patil teaches the non-transitory computer-readable medium of claim 17, and further combination with Wen teaches the limitation of claim 7. Therefore, the rejection of claim 7 with equal motivation is applied to claim 23.
With respect to claim 30, the combination of Steiner and Patil teaches the apparatus of claim 24, and further combination with Wen teaches the limitation of claim 7. Therefore, the rejection of claim 7 with equal motivation is applied to claim 30.
Claims 8 and 16 are rejected under 35 U.S.C. 103 as being unpatentable over Steiner and Patil in view of Varia, Meghal, US PG Pub No 2020/0401898A1 hereinafter Varia.
With respect to claim 8, the combination of Steiner and Patil teaches the apparatus of claim 1. Varia teaches in which
the at least one local memory comprises a tightly-coupled memory {Varia discloses [0063] “the local memory may be a tightly-coupled memory (TCM)” again [0059], [0028], see Fig 11}.
Varia is directed to neural network learning with memory constraints thus being analogous. A person having ordinary skill in the art would have considered it obvious prior to the effective filing date to employ tightly-couple local memory per Varia in combination to arrive at the invention as claimed for a motivation [0063] “The benefit of a TCM’s proximity to a processor is that the processor can access the TCM every cycle, and, unlike non-local memory, there is no cache involved which makes all memory accesses predictable. Further, fetching data from the TCM is significantly more power efficient” and/or [0028] “local memory (e.g., a tightly-coupled memory), which improves access speed, lowers system data bus utilization, and reduces power consumption.”
With respect to claim 16, the combination of Steiner and Patil teaches the processor-implemented method of claim 9, and further combination with Varia teaches the limitation of claim 8. Therefore, the rejection of claim 8 with equal motivation is applied to claim 16.
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
Schuler et al., “XEngine: Optimal Tensor Rematerialization for Neural Networks in Heterogeneous Environments” see Figs 1-3, Eq.1 argmin, [Sect4.1] Intel DevCloud setup
Mostafa, Hesham “Sequential Aggregation and Rematerialization: Distributed Full-Batch Training of Graph Neural Networks on Large Graphs” arXiv: 2111.06483v3 Sect.3, Alg.1-2
Kirisame et al., “Dynamic Tensor Rematerialization” arXiv: 2006.09616v4 ICLR, Fig 1
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Chase P Hinckley whose telephone number is (571)272-7935. The examiner can normally be reached M-F 9:00 - 5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Miranda M. Huang can be reached at 571-270-7092. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/CHASE P. HINCKLEY/Examiner, Art Unit 2124
Prosecution Insights