CTFR 18/353,637 CTFR 88484 Notice of Pre-AIA or AIA Status 07-03-aia AIA 15-10-aia The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA. Response to Arguments 07-37 AIA Applicant's arguments filed 5/11/2026 have been fully considered but they are not persuasive. The 112 rejections are not fully resolved by the amendments, see new 112 rejections below. Applicant argues, “These are concrete, machine-implemented memory management operations directed at a specific technical problem: enabling neural network training on memory-scarce hardware by controlling which node outputs are retained in local memory and which are recomputed. These operations are not performable in the human mind and do not fall within any of the three abstract idea groupings identified in the 2019 PEG.” Remarks 8. Determining nodes in a network is a mental concept. Applicant doesn’t claim memory-scarce hardware, and even if that was claimed, all hardware is memory-scarce because that term doesn’t have an accepted meaning. The determining steps in the claims are a mental concept and a mathematical relationship. Applicant argues, “This is a concrete improvement to computer memory management - reducing peak memory consumption during neural network training on resource-constrained hardware - not merely a mathematical relationship linked to a technical field.” Remarks 9. Applicant doesn’t claim anything that improves computer memory. Determining to recompute a node based on some calculation does not actually recompute the node, or limit memory usage, it’s just a determination. Something has to be done in response to the determination for there to be an improvement to memory usage. Because there is no improvement, and because the claims are directed to an abstract idea, the claims are not patent eligible subject matter. Applicant argues, “The schedule of Patil is not the same as ‘retention intervals corresponding to a time interval for retai ning each node output in a local memory, wherein each retention interval is defined by a start time and an end time that respectively correspond to allocation and deallocation of an output of a corresponding node in the local memory.’" The newly claimed start time and end time are taught by the time steps in Patil fig. 2, see rejection below . 07-30-03-h AIA Claim Interpretation The following is a quotation of 35 U.S.C. 112(f): (f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof. 07-30-05 Claims 22-28 use means-for language. The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked. As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph: (A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function; (B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and (C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function. Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function. Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function. Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Claim Rejections - 35 USC § 112 The following is a quotation of 35 U.S.C. 112(b): (b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention. Claims 9, 10, 13, 16, 17, 20, 23, 24 and 27 recite the limitation "the precedence constraint for each of the multiple nodes". There is insufficient antecedent basis for this limitation in the claim. The precedence constraint is introduced as “a” precedence constraint for the multiple nodes, not a precedence constraint for each of the multiple nodes. Claim Rejections - 35 USC § 101 07-04-01 AIA 07-04 35 U.S.C. 101 reads as follows: Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title. Claims 1-28 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea of a mathematical relationship without significantly more. The claims recite receiving a graph NN; determining parameters of the graph and its nodes; and determining to recompute a node. This judicial exception is not integrated into a practical application because this method does not improve the functioning of a computer and is merely linked to the technical field of computing. The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional elements of memory and processors are generic computer parts. Claim Rejections - 35 USC § 102 07-07-aia AIA 07-07 The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action: A person shall be entitled to a patent unless – 07-08-aia AIA (a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention. 07-15 AIA Claim s 1-6, 8-13, 15-20 and 22-27 are rejected under 35 U.S.C. 102( a)(1 ) as being described by POET: Training Neural Networks on Tiny Devices with Integrated Rematerialization and Paging by Patil et al Patil teaches claims 1, 8, 15 and 22. A processor-implemented method, performed by at least one processor, the method comprising: receiving, by the at least one processor, a graph representing an artificial neural network (ANN), the graph including multiple nodes connected by edges and each node represents an operation; (Patil abs. “POET, an algorithm to enable training large neural networks on memory-scarce battery-operated edge devices.” The neural network and its nodes are shown in fig. 1, below.) PNG media_image1.png 184 728 media_image1.png Greyscale determining, by the at least one processor, retention intervals for the multiple nodes based on a precedence constraint for each of the multiple nodes, the retention intervals corresponding to a time interval for retaining each node output in a local memory, each retention interval defined by a start time and an end time that respectively correspond to allocation and deallocation of an output of a corresponding node in the local memory; (Patil sec. 5.2 “consider a graph where the output depends on the result of two operations v1 and v2 where both nodes have equivalent memory costs but v2 is cheaper to evaluate. A paging strategy may evict v2 which would force rematerialization to recompute the more expensive v1 rather than v2. We represent a schedule as a series of nodes that are either being saved SRAM, (re)computed R or paged from secondary storage SAUX.” The schedule is the retention intervals, the precedence is the series of nodes. The schedule and their start time and end time for allocation/deallocation are shown in fig. 2 below as the time slots t1-t18.) PNG media_image2.png 958 674 media_image2.png Greyscale determining, by the at least one processor, a node of the multiple nodes to recompute based on the retention intervals. (Patil sec. 5.2 “A paging strategy may evict v2 which would force rematerialization to recompute the more expensive v1 rather than v2. We represent a schedule as a series of nodes that are either being saved SRAM, (re)computed R or paged from secondary storage SAUX.”) Patil teaches claims 2, 9, 16, 23. The processor-implemented method of claim 1, further comprising determining, by the at least one processor, an order for execution of the multiple nodes based on the precedence constraint for each of the multiple nodes and a memory constraint. (Patil sec. 5.1 “Given a directed acyclic dataflow graph G = (V,E) with n nodes, a topological ordering {v 1 ,...,v n } is computed which constrains execution to that order of instructions. Two key decision variables are introduced: (1) R ∈ {0,1} n×n where r t,i represents the decision to (re)materialize an operation v i at timestep t and (2) S ∈ {0,1} n× n where s t,i represents whether the result of an operation v i is resident in memory at timestep t.”) Patil teaches claims 3, 10, 17, 24. The processor-implemented method of claim 2, further comprising determining, by the at least one processor, the precedence constraint for each of the multiple nodes based on the retention intervals. (Patil sec. 5.1 “Given a directed acyclic dataflow graph G = (V,E) with n nodes, a topological ordering {v 1 ,...,v n } is computed which constrains execution to that order of instructions. Two key decision variables are introduced: (1) R ∈ {0,1} n×n where r t,i represents the decision to (re)materialize an operation v i at timestep t and (2) S ∈ {0,1} n× n where s t,i represents whether the result of an operation v i is resident in memory at timestep t.”) Patil teaches claims 4, 11, 18, 25. The processor-implemented method of claim 2, further comprising determining, by the at least one processor, the memory constraint based on a physical memory capacity of the local memory and the retention intervals. (Patil sec. 5.1 “From the rematerialzation matrix R, and the storage matrix S, we define a series of constraints to maintain graph dependencies. All arguments for an operation j must be resident in memory prior to running that operation, yielding constraint …. Similarly, the result of an operation is only resident in memory in one of the two cases: a) if it was already res ident in memory before, or b) if it was (re)materialized …. To adhere to the strict constraints on the peak memory used during training, an intermediate variable U ∈ R n×n is defined. U t,i is the total memory used by the system during training at timestep t when evaluating operation i. By bounding the maximum value of… to the user-specified memory limit µRAM, we limit the total memory consumption during training.”) Patil teaches claims 5, 12, 19 and 26. The processor-implemented method of claim 1, in which the retention intervals are determined based on a recompute constraint, the recompute constraint defining a number of times that each node of the multiple nodes is permitted to be recomputed. (Patil sec. 5.0 “we do not limit rematerialization to occur once. We assume auxiliary storage (e.g., flash/ SD card) is available. However, if auxiliary storage is not available, the POET optimizer will fall back to only performing rematerialization.” The rest of that last sentence is only performing rematierialization once. This makes sense because the previous sentence states that the default is one rematerialization, and without auxiliary memory, there isn’t time for multiple saves for rematerialization. Patil sec 5.0 “POET finds an energy optimal schedule by choosing to either a) rematerialize or b) page the tensors to/from secondary storage such as an SD card.” Choosing to rematerialize/recompute the tensors only once if there is no SD card, is a schedule based on a recompute limit.) Patil teaches claims 6, 13, 20 and 27. The processor-implemented method of claim 1, in which the precedence constraint is determined based on the edges connecting the multiple nodes. (Patil sec. 4 “When the deleted tensor is needed again, for example, to compute gradients during backpropagation, it is recomputed from the other dependent activations as dictated by the lineage.” The lineage is the string of edges connecting to the other nodes from the deleted tensor.) Claim Rejections - 35 USC § 103 07-20-aia AIA The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action: A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made. 07-21-aia AIA Claim s 7, 14, 21 and 28 is/are rejected under 35 U.S.C. 103 as being unpatentable over POET: Training Neural Networks on Tiny Devices with Integrated Rematerialization and Paging by Patil et al and TensorDIMM: A Practical Near-Memory Processing Architecture for Embeddings and Tensor Operations in Deep Learning by Kwon et al . Patil teaches claims 7, 14, 21 and 28. The processor-implemented method of claim 1, in which the local memory comprises a tightly-coupled memory. (Patil sec 5.0 “POET finds an energy optimal schedule by choosing to either a) rematerialize or b) page the tensors to/from secondary storage such as an SD card.”) Patil doesn’t teach tightly-coupled memory. However, Kwon teaches the local memory comprises a tightly-coupled memory. (Kwon abs “We present our vertically integrated hardware/software co-design, which includes a custom DIMM module enhanced with near-memory processing cores tailored for DL tensor operations.” Near-memory cores make the memory tightly-coupled.) Kwon, Patil and the claims all teach machine learning memory operations. It would have been obvious to a person having ordinary skill in the art, at the time of filing, to use tightly-coupled memory because “tensor reduction operations should be conducted ‘near memory’ using a lightweight NMP core, incurring minimum power and area overheads to the buffered DIMM device.” Kwon sec. 4.2. Conclusion 07-39 AIA THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a). A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action. Any inquiry concerning this communication or earlier communications from the examiner should be directed to Austin Hicks whose telephone number is (571)270-3377. The examiner can normally be reached Monday - Thursday 8-4 PST. Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Mariela Reyes can be reached at (571) 270-1006. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300. Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. /AUSTIN HICKS/Primary Examiner, Art Unit 2142 Application/Control Number: 18/353,637 Page 2 Art Unit: 2142 Application/Control Number: 18/353,637 Page 3 Art Unit: 2142 Application/Control Number: 18/353,637 Page 4 Art Unit: 2142 Application/Control Number: 18/353,637 Page 5 Art Unit: 2142 Application/Control Number: 18/353,637 Page 6 Art Unit: 2142 Application/Control Number: 18/353,637 Page 7 Art Unit: 2142 Application/Control Number: 18/353,637 Page 8 Art Unit: 2142 Application/Control Number: 18/353,637 Page 9 Art Unit: 2142 Application/Control Number: 18/353,637 Page 10 Art Unit: 2142 Application/Control Number: 18/353,637 Page 11 Art Unit: 2142 Application/Control Number: 18/353,637 Page 12 Art Unit: 2142 Application/Control Number: 18/353,637 Page 13 Art Unit: 2142 Application/Control Number: 18/353,637 Page 14 Art Unit: 2142 Application/Control Number: 18/353,637 Page 15 Art Unit: 2142 Application/Control Number: 18/353,637 Page 16 Art Unit: 2142
Prosecution Insights