Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 09/13/2023 is in compliance
with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-13 are rejected under U.S.C 101 for containing an abstract idea without significantly more.
Regarding claim 1:
Step 1 – Is the claim to a process, machine, manufacture or composition of matter?
Yes, the claim is a process.
Step 2A – Prong 1 – Does the claim recite an abstract idea, law of nature, or natural phenomenon?
Yes, the claim recites an abstract idea.
wherein, calculating and analyzing an operator that is in the plurality of operators and whose input and output occupy memory space that can overlap; and This limitation is directed to the abstract idea of a mental process (including an observation, evaluation, judgement, opinion) which can be performed in the human mind, or by a human using pen and paper (see MPEP 2106.04(a)(2) Ill. C.)
determining whether a size of an input of the neural network is a fixed size; and This limitation is directed to the abstract idea of a mental process (including an observation, evaluation, judgement, opinion) which can be performed in the human mind, or by a human using pen and paper (see MPEP 2106.04(a)(2) Ill. C.)
calculating a size of each memory block in an inference process of a neural network model, This limitation is directed to mathematical calculation (see MPEP 2106.04(a)(2) l. C.)
determining a life cycle of each memory block, This limitation is directed to the abstract idea of a mental process (including an observation, evaluation, judgement, opinion) which can be performed in the human mind, or by a human using pen and paper (see MPEP 2106.04(a)(2) Ill. C.)
determining whether the memory block is a memory block that can be overlapped, and This limitation is directed to the abstract idea of a mental process (including an observation, evaluation, judgement, opinion) which can be performed in the human mind, or by a human using pen and paper (see MPEP 2106.04(a)(2) Ill. C.)
Step 2A – Prong 2 – Does the claim recite additional elements that integrate the judicial exception into a practical application?
No, there are no additional elements that integrate the judicial exception into a practical application. The additional elements:
A memory allocation method for an Al processor, comprising: This limitation is directed to a computer merely used as a tool to perform an existing process (see MPEP 2106.05(f) (2)).
obtaining a plurality of operators of a neural network; Adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea [see MPEP 2106.05(f)] and therefore fails to integrate the exception into a practical application.
if yes, determining storage addresses of a plurality of memory blocks by using a static memory pool allocation algorithm; Adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea [see MPEP 2106.05(f)] and therefore fails to integrate the exception into a practical application.
otherwise, requesting memory space for a plurality of memory blocks by using a dynamic memory pool allocation algorithm, Adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea [see MPEP 2106.05(f)] and therefore fails to integrate the exception into a practical application.
wherein the determining storage addresses of a plurality of memory blocks by using a static memory pool allocation algorithm comprises: Adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea [see MPEP 2106.05(f)] and therefore fails to integrate the exception into a practical application.
if the memory block is a memory block that can be overlapped, correcting the size and the life cycle of the memory block, and Adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea [see MPEP 2106.05(f)] and therefore fails to integrate the exception into a practical application
allocating a storage address to each memory block based on the corrected size and life cycle of the memory block. Adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea [see MPEP 2106.05(f)] and therefore fails to integrate the exception into a practical application
Step 2B – Does the claim recite additional elements that amount to significantly more than the judicial exception?
No, there are no additional elements that amount to significantly more than the judicial exception. The additional elements are:
A memory allocation method for an Al processor, comprising: This limitation is directed to a computer merely used as a tool to perform an existing process (see MPEP 2106.05(f) (2)).
obtaining a plurality of operators of a neural network; Adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea [see MPEP 2106.05(f)] and therefore fails to integrate the exception into a practical application.
if yes, determining storage addresses of a plurality of memory blocks by using a static memory pool allocation algorithm; Adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea [see MPEP 2106.05(f)] and therefore fails to integrate the exception into a practical application.
otherwise, requesting memory space for a plurality of memory blocks by using a dynamic memory pool allocation algorithm, Adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea [see MPEP 2106.05(f)] and therefore fails to integrate the exception into a practical application.
wherein the determining storage addresses of a plurality of memory blocks by using a static memory pool allocation algorithm comprises: Adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea [see MPEP 2106.05(f)] and therefore fails to integrate the exception into a practical application.
if the memory block is a memory block that can be overlapped, correcting the size and the life cycle of the memory block, and Adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea [see MPEP 2106.05(f)] and therefore fails to integrate the exception into a practical application
allocating a storage address to each memory block based on the corrected size and life cycle of the memory block. Adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea [see MPEP 2106.05(f)] and therefore fails to integrate the exception into a practical application
Regarding claim 2,
Claim 2 is rejected under 35 U.S.C 101 because the claimed invention is directed to an abstract idea without significantly more. The claim is dependent on claim 1 which includes an abstract idea (see rejection for claim 1). The additional limitations:
the calculating and analyzing an operator that is in the plurality of operators and whose input and output occupy memory space that can overlap comprises: This limitation is directed to the abstract idea of a mental process (including an observation, evaluation, judgement, opinion) which can be performed in the human mind, or by a human using pen and paper (see MPEP 2106.04(a)(2) Ill. C.)
determining whether input and output activations of an operator participate only in calculation of an operator at a current layer; and This limitation is directed to the abstract idea of a mental process (including an observation, evaluation, judgement, opinion) which can be performed in the human mind, or by a human using pen and paper (see MPEP 2106.04(a)(2) Ill. C.)
if the input and output activations of the operator participate only in calculation of the operator at the current layer, determining that memory space occupied by an input and an output of the operator can overlap; This limitation is directed to the abstract idea of a mental process (including an observation, evaluation, judgement, opinion) which can be performed in the human mind, or by a human using pen and paper (see MPEP 2106.04(a)(2) Ill. C.)
otherwise, determining that memory space occupied by an input and an output of the operator cannot overlap. This limitation is directed to the abstract idea of a mental process (including an observation, evaluation, judgement, opinion) which can be performed in the human mind, or by a human using pen and paper (see MPEP 2106.04(a)(2) Ill. C.)
Regarding claim 3,
Claim 3 is rejected under 35 U.S.C 101 because the claimed invention is directed to an abstract idea without significantly more. The claim is dependent on claim 2 which includes an abstract idea (see rejection for claim 2). The additional limitations:
wherein the analyzed operator is an operator that undergoes linear splitting. -This claim merely recites a further limitation on the the calculating and analyzing an operator that is in the plurality of operators and whose input and output occupy memory space that can overlap comprises from Claim 2 which was directed to the abstract idea of a mental process (including an observation, evaluation, judgement, opinion) which can be performed in the human mind, or by a human using pen and paper (see MPEP 2106.04(a)(2) Ill. C.)
Regarding claim 4,
Claim 4 is rejected under 35 U.S.C 101 because the claimed invention is directed to an abstract idea without significantly more. The claim is dependent on claim 1 which includes an abstract idea (see rejection for claim 1). The additional limitations:
wherein the determining a life cycle of each memory block comprises: calculating the life cycle of the memory block based on the first access time and the last access time of an operator stored in the memory block. This limitation is directed to mathematical calculation (see MPEP 2106.04(a)(2) l. C.)
Regarding claim 5,
Claim 5 is rejected under 35 U.S.C 101 because the claimed invention is directed to an abstract idea without significantly more. The claim is dependent on claim 1 which includes an abstract idea (see rejection for claim 1). The additional limitations:
the allocating a storage address to each memory block based on the corrected size and life cycle of the memory block comprises: Adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea [see MPEP 2106.05(f)] and therefore fails to integrate the exception into a practical application.
placing each memory block into a static memory pool based on the corrected size and life cycle of the memory block, and Adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea [see MPEP 2106.05(f)] and therefore fails to integrate the exception into a practical application.
calculating an offset address of each memory block by using a heuristic algorithm. This limitation is directed to mathematical calculation (see MPEP 2106.04(a)(2) l. C.)
Regarding claim 6,
Claim 6 is rejected under 35 U.S.C 101 because the claimed invention is directed to an abstract idea without significantly more. The claim is dependent on claim 5 which includes an abstract idea (see rejection for claim 5). The additional limitations:
wherein before the storage address is allocated to each memory block, a size of the static memory pool is determined: This limitation is directed to the abstract idea of a mental process (including an observation, evaluation, judgement, opinion) which can be performed in the human mind, or by a human using pen and paper (see MPEP 2106.04(a)(2) Ill. C.)
a size of a memory block set at any moment is calculated, and This limitation is directed to mathematical calculation (see MPEP 2106.04(a)(2) l. C.)
a minimum value of a memory block set required at any moment is used as a lower
limit value of the size of the static memory pool. Adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea [see MPEP 2106.05(f)] and therefore fails to integrate the exception into a practical application.
Regarding claim 7,
Claim 7 is rejected under 35 U.S.C 101 because the claimed invention is directed to an abstract idea without significantly more. The claim is dependent on claim 1 which includes an abstract idea (see rejection for claim 1). The additional limitations:
wherein the requesting memory space for a plurality of memory blocks by using a dynamic memory pool allocation algorithm comprises: Adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea [see MPEP 2106.05(f)] and therefore fails to integrate the exception into a practical application.
determining a size of memory space required for calculating a current operator; This limitation is directed to the abstract idea of a mental process (including an observation, evaluation, judgement, opinion) which can be performed in the human mind, or by a human using pen and paper (see MPEP 2106.04(a)(2) Ill. C.)
determining whether an idle memory block that meets a requirement exists in a
memory linked list; and This limitation is directed to the abstract idea of a mental process (including an observation, evaluation, judgement, opinion) which can be performed in the human mind, or by a human using pen and paper (see MPEP 2106.04(a)(2) Ill. C.)
if an idle memory block that meets the requirement exists in the memory linked list, using the idle memory block that meets the requirement as memory required for calculating the current operator, and Adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea [see MPEP 2106.05(f)] and therefore fails to integrate the exception into a practical application.
removing the idle memory block from the memory linked list. Adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea [see MPEP 2106.05(f)] and therefore fails to integrate the exception into a practical application.
Regarding claim 8,
Claim 8 is rejected under 35 U.S.C 101 because the claimed invention is directed to an abstract idea without significantly more. The claim is dependent on claim 7 which includes an abstract idea (see rejection for claim 7). The additional limitations:
wherein after a life cycle of a memory block ends, the memory block is released and inserted into the memory linked list. Adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea [see MPEP 2106.05(f)] and therefore fails to integrate the exception into a practical application.
Regarding claim 9,
Claim 9 is rejected under 35 U.S.C 101 because the claimed invention is directed to an abstract idea without significantly more. The claim is dependent on claim 7 which includes an abstract idea (see rejection for claim 7). The additional limitations:
wherein if no idle memory block that meets the requirement exists in the memory linked list, the memory space required for calculating the current operator is requested. Adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea [see MPEP 2106.05(f)] and therefore fails to integrate the exception into a practical application.
Regarding claim 10,
Claim 10 is rejected under 35 U.S.C 101 because the claimed invention is directed to an abstract idea without significantly more. The claim is dependent on claim 7 which includes an abstract idea (see rejection for claim 7). The additional limitations:
wherein the using the idle memory block that meets the requirement as memory required for calculating the current operator comprises: Adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea [see MPEP 2106.05(f)] and therefore fails to integrate the exception into a practical application.
using, as a memory block corresponding to the current operator, an idle memory block that is in the memory linked list, that meets a requirement of the memory space required for calculating the current operator, and that has minimum memory space. Adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea [see MPEP 2106.05(f)] and therefore fails to integrate the exception into a practical application.
Regarding claim 11,
Claim 11 is rejected under 35 U.S.C 101 because the claimed invention is directed to an abstract idea without significantly more. The claim is dependent on claim 7 which includes an abstract idea (see rejection for claim 7). The additional limitations:
wherein the using the idle memory block that meets the requirement as memory required for calculating the current operator comprises: Adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea [see MPEP 2106.05(f)] and therefore fails to integrate the exception into a practical application.
determining that a ratio between a size of memory space occupied by the current operator and a size of a used memory block is greater than a preset memory usage ratio. This limitation is directed to the abstract idea of a mental process (including an observation, evaluation, judgement, opinion) which can be performed in the human mind, or by a human using pen and paper (see MPEP 2106.04(a)(2) Ill. C.)
Regarding claim 12,
Claim 12 is rejected under 35 U.S.C 101 because the claimed invention is directed to an abstract idea without significantly more. The claim is dependent on claim 1 which includes an abstract idea (see rejection for claim 1). The additional limitations:
A computer apparatus, comprising a processor and a memory, wherein the memory stores a computer program, and the computer program is executed by the processor, steps of the memory allocation method for an Al processor according to claim1 are implemented. This limitation is directed to a computer merely used as a tool to perform an existing process (see MPEP 2106.05(f) (2)).
Regarding claim 13,
Claim 13 is rejected under 35 U.S.C 101 because the claimed invention is directed to an abstract idea without significantly more. The claim is dependent on claim 1 which includes an abstract idea (see rejection for claim 1). The additional limitations:
A computer-readable storage medium that stores a computer program, wherein when the computer program is executed by a processor, steps of the memory allocation method for an Al processor according to claim 1 implemented. This limitation is directed to a computer merely used as a tool to perform an existing process (see MPEP 2106.05(f) (2)).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1, 4, 5, 7-9 and 12-13 are rejected under 35 U.S.C. 103 as being unpatentable over Pisarchyk & Lee (“EFFICIENT MEMORY MANAGEMENT FOR DEEP NEURAL NET INFERENCE”) in view of Goldberg (US 2012/0303927 A1)
Regarding claim 1, Pisarchyk explicitly discloses:
A memory allocation method for an Al processor, comprising:
obtaining a plurality of operators of a neural network; (Pisarchyk, Pg. 2, Col. 1, Section 3, ¶[1]: “Tensor Usage Interval of an intermediate tensor t is defined as the pair {first_opt , last_opt}, where first_opt and last_opt are the indices of the first and the last operator that use t as its input or output, respectively. The indices are from a topological sort of the neural network which is also the operators’ execution order.” )
wherein, calculating and analyzing an operator that is in the plurality of operators and whose input and output occupy memory space that can overlap; and (Pisarchyk, Pg. 3, Col. 1, Section 4.2: “For each operator in this sorted ordering, we assign shared objects to tensors from its profile, but only for those that have not been assigned yet (L.7). If there are several such tensors, we start from the largest by size. A shared object s is suitable for assignment to tensor t, if and only if there is no tensor u, such that s is assigned to the u and usage intervals of t and u overlap (L.18–23)”, Pg. 3, Algorithm 1:
PNG
media_image1.png
133
464
media_image1.png
Greyscale
) [Examiner’s note: Algorithm 1 teaches calculating and analyzing whether tensor memory usage intervals overlap by computing max_first_op and min_last_op for tensor usage records and determining whether a tensor is already assigned to the shared object has an overlapping usage interval, as shown in lines 19-23]
determining whether a size of an input of the neural network is a fixed size; and (Pisarchyk, Pg. 2, Col. 1, Section 3, ¶[1-2]: “…the first and the last operator that use t as its input or output, respectively… Tensor Usage Record of an intermediate tensor t is defined as the triple {first_opt, last_opt, sizet}, where sizet is t’s aligned size in bytes.”, Pg. 3, Col. 1, Section 4.2 :“Shared object assignment (L.12–17, 24–28) can be summarized as: If there are suitable shared objects not smaller than sizet, assign the smallest to t. If all suitable shared objects are smaller than sizet, update the largest size to sizet and assign it to t. If there are no suitable shared objects, create a new shared object with size sizet and assign it to t.”) [Examiner’s note: sizet is being interpreted as the fixed memory size requested by input t]
if yes, determining storage addresses of a plurality of memory blocks by using a static memory pool allocation algorithm; (Pisarchyk, Pg. 4, Col. 2, Section 5, ¶[1]: “We call the other memory sharing approach Offset Calculation where a large chunk of memory is pre-allocated and the intermediate tensors are given parts of the memory by the offsets within the memory block. The main objective is to minimize the size of the allocated memory block”, Pg. 5, Section 5.2: “As Greedy by Size works well for Shared Objects, we employ a similar method for Offsets Calculation (Algorithm 3). We first iterate through tensor usage records in nonincreasing order by their size (L.1,6). For each record, we check already assigned tensors whose usage intervals intersect with that of the current tensor (L.10–13) to find the smallest gap in memory between them such that current tensor fits into that gap (L.9,14–17). If such a gap is found, the current tensor is allocated to this gap.”)
otherwise, requesting memory space for a plurality of memory blocks by using a dynamic memory pool allocation algorithm, (Pisarchyk, Pg. 5, Section 5.2: “Otherwise, we allocate it after the rightmost tensor whose usage interval intersect with that of the current tensor (L.19–20). We assign the corresponding offset to current tensor and the tensor becomes assigned (L.21–23) as shown in Figure 6.”, Pg. 5, Fig. 6:
PNG
media_image2.png
379
563
media_image2.png
Greyscale
)
wherein the determining storage addresses of a plurality of memory blocks by using a static memory pool allocation algorithm comprises: (Pisarchyk, Pg. 4, Col. 2, Section 5, ¶[1]: “We call the other memory sharing approach Offset Calculation where a large chunk of memory is pre-allocated and the intermediate tensors are given parts of the memory by the offsets within the memory block. The main objective is to minimize the size of the allocated memory block”, Pg. 5, Section 5.2: “As Greedy by Size works well for Shared Objects, we employ a similar method for Offsets Calculation (Algorithm 3). We first iterate through tensor usage records in nonincreasing order by their size (L.1,6). For each record, we check already assigned tensors whose usage intervals intersect with that of the current tensor (L.10–13) to find the smallest gap in memory between them such that current tensor fits into that gap (L.9,14–17). If such a gap is found, the current tensor is allocated to this gap.”)
determining a life cycle of each memory block, (Pisarchyk, Pg. 2, Col. 1, Section 3: “Tensor Usage Interval of an intermediate tensor t is defined as the pair {first_opt , last_opt}, where first_opt and last_opt are the indices of the first and the last operator that use t as its input or output, respectively. The indices are from a topological sort of the neural network which is also the operators’ execution order”) [Examiner’s note: This teaches the life cycle of the memory block because it identifies a). when the tensor first becomes needed: first_opt, b). when the tensor is last used: last_opt]
determining whether the memory block is a memory block that can be overlapped, and (Pisarchyk, Pg.3, Algorithm 2, Lines 7-13: “
PNG
media_image3.png
182
448
media_image3.png
Greyscale
”) [Examiner’s note: these lines of code determine whether the current tensor’s lifecycle overlaps with any tensor already assigned to the same shared object. If there is overlap, the object is not suitable. If there is no overlap, the object can be reused. The reference defines the tensor usage interval as {first_opt, last_opt}, where those are the first and last operators that use the tensor as input or output]
if the memory block is a memory block that can be overlapped, correcting the size and the life cycle of the memory block, and (Pisarchyk, Pg. 3, Section 4.2, Col. 1: “Shared object assignment (L.12–17, 24–28) can be summarized as: If there are suitable shared objects not smaller than sizet, assign the smallest to t. If all suitable shared objects are smaller than sizet, update the largest size to sizet and assign it to t. If there are no suitable shared objects, create a new shared object with size sizet and assign it to t.”, Algorithm 1, Lines 19-21:
PNG
media_image4.png
67
471
media_image4.png
Greyscale
Lines 30-3:
PNG
media_image5.png
42
482
media_image5.png
Greyscale
) [Examiner’s note: Lines 19-20 teach updating the lifecycle of the memory blocks. Lines 30-31 teach the shared/ overlapped object’s size is corrected to fit the largest tensor assigned to it.]
allocating a storage address to each memory block based on the corrected size and life cycle of the memory block. (Pisarchyk, Pg. 5, Algorithm 3, Lines 10-23:
PNG
media_image6.png
343
528
media_image6.png
Greyscale
) [Examiner’s note: Lines 10-13 of this algorithm teach comparing the current tensor’s first and last operator indices with another tensor’s first and last operator indices to determine whether their lifecycle overlap. Lines 14-20 use tensor size to decide whether the tensor first in a memory gap. Lines 21-23 teach assigning a storage address. Thus, Pisarchyk teaches allocating a storage address to each block based on size and lifecycle information. ]
Pisarchyk fails to teach:
calculating a size of each memory block in an inference process of a neural network model,
However, Goldberg explicitly teaches:
calculating a size of each memory block in an inference process of a neural network model, (Goldberg, ¶[0044]: “The data word 300 is, in the embodiment shown, a 32-bit word, portions of which are used to monitor an in-use status, an address, and a size of a block associated with the track. In the embodiment shown, a first bit 302 is used to track the in-use status of the track identified by the data word. A second set of bits 304 are used to define a size of the block associated with that track. In the embodiment shown, the second set of bits is 11 bits in length, although in other embodiments, additional bits could be used, depending upon the maximum and minimum sizes of blocks to be addressed within the system. In an embodiment using 11 bits, one possible arrangement allows for allocating memory in sizes between 1-64 tracks.”)
It would have obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Pisarchyk and Goldberg. Pisarchyk teaches various strategies to smartly share memory buffers among intermediate tensors in deep neural nets. Goldberg teaches a method of allocating storage space in a memory of a computing system. One of ordinary skill would have motivation to combine Pisarchyk and Goldberg to improve memory utilization as little memory is wasted and minimal fragmentation occurs across disks or other memory locations.
Regarding claim 4, the combination of Pisarchyk and Goldberg explicitly discloses all the limitations of claim 1 (as shown in the rejections above)
Pisarchyk in view of Goldberg further teaches:
wherein the determining a life cycle of each memory block comprises: calculating the life cycle of the memory block based on the first access time and the last access time of an operator stored in the memory block. (Pisarchyk, Pg.3, Algorithm 2:
PNG
media_image7.png
57
380
media_image7.png
Greyscale
)
Regarding claim 5, the combination of Pisarchyk and Goldberg explicitly discloses all the limitations of claim 1 (as shown in the rejections above)
Pisarchyk in view of Goldberg further teaches:
wherein the allocating a storage address to each memory block based on the corrected size and life cycle of the memory block comprises:
placing each memory block into a static memory pool based on the corrected size and life cycle of the memory block, and (Piasrchyk, Pg. 4, Col. 1, ¶[1]: “If there are no suitable shared objects, create a new shared object with size sizet and assign it to t.”, Pg. 5, Algorithm 3:
PNG
media_image8.png
356
421
media_image8.png
Greyscale
) [Examiner’s note: Line 23 teaches inserting tensor t (i.e., memory block) into an ordered list of pre-allocated memory chunk (i.e., the static memory pool)]
calculating an offset address of each memory block by using a heuristic algorithm. (Pisarchyk, Pg. 5, Col. 1, Section 5.2, ¶[1]: “As Greedy by Size works well for Shared Objects, we employ a similar method for Offsets Calculation (Algorithm 3).”, Pg. 5, Algorithm 3) [Examiner’s note: In this context, the heuristic algorithm refers to a greedy memory-planning algorithm]
Regarding claim 7, the combination of Pisarchyk and Goldberg explicitly discloses all the limitations of claim 1 (as shown in the rejections above)
Pisarchyk in view of Goldberg further teaches:
wherein the requesting memory space for a plurality of memory blocks by using a dynamic memory pool allocation algorithm comprises:
determining a size of memory space required for calculating a current operator;
(Goldberg, ¶[0062]: “An allocation request operation 1004 receives an allocation request that defines a particular size of memory requested, and a requesting process. The allocation request operation 1004 also determines a best-fit memory block size that can be allocated to respond to the request.”)
determining whether an idle memory block that meets a requirement exists in a
memory linked list; and (Goldberg, ¶[0064]: “The operating system performs an availability determination operation 1006 to determine if a best-fit block size is available to fulfill the allocation request. If the memory pool associated with the best-fit block size is empty, operational flow proceeds to a larger block size operation 1008, which adjusts the operating system to assess the availability of the next-larger sized block”) [Examiner’s note: “an idle memory block” is being interpreted as the empty memory block]
if an idle memory block that meets the requirement exists in the memory linked list, using the idle memory block that meets the requirement as memory required for calculating the current operator, and (Goldberg, ¶[0065]: “If the block is in fact the best-fit size, a block allocation operation 1018 allocates the block in response to the request”)
removing the idle memory block from the memory linked list. (Goldberg, ¶[0065]: “buffer pool update operation 1014 adds the upper block to the memory pool of the next smaller size, and removes the block being split from its current memory pool.”)
Regarding claim 8, the combination of Pisarchyk and Goldberg explicitly discloses all the limitations of claim 7 (as shown in the rejections above)
Pisarchyk in view of Goldberg further teaches:
wherein after a life cycle of a memory block ends, the memory block is released and inserted into the memory linked list. (Goldberg, ¶[0057]: “If a deallocation request is received by the operating system, the reverse process is performed, with the memory block 702a being returned to a memory pool 404e (as shown in FIG. 7).”)
Regarding claim 9, the combination of Pisarchyk and Goldberg explicitly discloses all the limitations of claim 7 (as shown in the rejections above)
Pisarchyk in view of Goldberg further teaches:
wherein if no idle memory block that meets the requirement exists in the memory linked list, the memory space required for calculating the current operator is requested. (Goldberg, ¶[0064]: “If no block is available at that next-larger block size, operational flow returns to the larger block size operation 1008 to adjust to a still larger block size. This loop will continue until an available block is found in a buffer pool, up to a maximum block size available as defined by the operating system.”)
Regarding claim 12, the combination of Pisarchyk and Goldberg explicitly discloses all the limitations of claim 1 (as shown in the rejections above)
Pisarchyk in view of Goldberg further teaches:
) A computer apparatus, comprising a processor and a memory, wherein the memory stores a computer program, and the computer program is executed by the processor, steps of the memory allocation method for an Al processor according to claim1 are implemented. (Goldberg, ¶[0040]: “The term computer readable media as used herein may include computer storage media and communication media. As used in this document, a computer storage medium is a device or article of manufacture that stores data and/or computer-executable instructions. Computer storage media may include volatile and nonvolatile, removable and nonremovable devices or articles of manufacture implemented in any method or technology for storage of information, such as computer readable instructions, data structures, program modules, or other data.”, ¶[0033]: “In another example, the processing system 104 can include one or more separate microprocessors.”)
Regarding claim 13, the combination of Pisarchyk and Goldberg explicitly discloses all the limitations of claim 1 (as shown in the rejections above)
Pisarchyk in view of Goldberg further teaches:
A computer-readable storage medium that stores a computer program, wherein when the computer program is executed by a processor, steps of the memory allocation method for an Al processor according to claim 1 implemented. (Goldberg, ¶[0040]: “The term computer readable media as used herein may include computer storage media and communication media. As used in this document, a computer storage medium is a device or article of manufacture that stores data and/or computer-executable instructions. Computer storage media may include volatile and nonvolatile, removable and nonremovable devices or articles of manufacture implemented in any method or technology for storage of information, such as computer readable instructions, data structures, program modules, or other data.”, ¶[0033]: “In another example, the processing system 104 can include one or more separate microprocessors.”)
Claim(s) 2, 3, 6 are rejected under 35 U.S.C. 103 as being unpatentable over Pisarchyk & Lee (“EFFICIENT MEMORY MANAGEMENT FOR DEEP NEURAL NET INFERENCE”) in view of Goldberg (US 2012/0303927 A1) and further in view of Petar Jokic (“Improving Memory Utilization in Convolutional Neural Network Accelerators”)
Regarding claim 2, the combination of Pisarchyk and Goldberg explicitly discloses all the limitations of claim 1 (as shown in the rejections above)
Pisarchyk in view of Goldberg fails to teach:
the calculating and analyzing an operator that is in the plurality of operators and whose input and output occupy memory space that can overlap comprises: determining whether input and output activations of an operator participate only in calculation of an operator at a current layer; and
if the input and output activations of the operator participate only in calculation of the operator at the current layer, determining that memory space occupied by an input and an output of the operator can overlap;
otherwise, determining that memory space occupied by an input and an output of the operator cannot overlap.
However, Jokic explicitly discloses:
wherein the calculating and analyzing an operator that is in the plurality of operators and whose input and output occupy memory space that can overlap comprises: determining whether input and output activations of an operator participate only in calculation of an operator at a current layer; and (Jokic, Pg. 3, Col. 1, ¶[2]: “The maximum overlap of the two activation regions is found by mathematically describing the pointer positions and optimizing their relative offset distance at the beginning of each layer such that the total memory is minimal while constraining the write pointer to be smaller than the read pointer, avoiding any overwriting of still needed data.”)
if the input and output activations of the operator participate only in calculation of the operator at the current layer, determining that memory space occupied by an input and an output of the operator can overlap; (Jokic, Pg. 3, Col. 1, ¶[1-2]: “the activations for each window position are only read from a small, connected region of the input layer which itself is slid over the inputs in a continuous fashion. Because the corresponding memory data is ordered in the same way as they appear in this sliding operation, most parts of the processed input data will never be used again and can thus be overwritten by resulting output activations. The maximum overlap of the two activation regions is found by mathematically describing the pointer positions and optimizing their relative offset distance at the beginning of each layer such that the total memory is minimal while constraining the write pointer to be smaller than the read pointer, avoiding any overwriting of still needed data.”) [Examiner’s note: The highlights explain that input activations are used in the current convolution operation through a sliding window, and after processed portions are no longer needed, they can be overwritten. So, the reference determines whether the activation data is still needed for the current layer or operator computation.]
otherwise, determining that memory space occupied by an input and an output of the operator cannot overlap. (Jokic, Pg. 2, Col. 2, ¶[2]: “If two subsequent layers have overlapping memory regions, the allocation method must avoid that output activations are overwriting data from the preceding layer that is still needed for pending computations.”, Pg. 3, Col. 1, ¶[2]: “…while constraining the write pointer to be smaller than the read pointer, avoiding any overwriting of still needed data.”) [Examiner’s note: Jokic teaches that output activations may overwrite processed input activation data only after the input data is no longer needed. In contrast, where input activation data is still needed for pending computations, the allocation method must avoid overwriting that data by constraining the write pointer relative to the read pointer. Thus, the reference teaches when the input and output activations cannot share memory without overwriting still needed input data, the memory space occupied by the input and output of the operator cannot overlap]
It would have obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Pisarchyk, Goldberg and Jokic. Pisarchyk teaches various strategies to smartly share memory buffers among intermediate tensors in deep neural nets. Goldberg teaches a method of allocating storage space in a memory of a computing system. Jokic teaches the mathematical model to compute the maximum activations memory overlap and thus the lower bound of on-chip memory needed to perform layer-by-layer processing of convolutional neural networks on memory-limited accelerators. One of ordinary skill would have motivation to combine Pisarchyk, Goldberg and Jokic to improve memory utilization in resource-limit CNN accelerators by allowing activation memory regions of subsequent layers to overlap, thereby reducing the on-chip memory footprint required for layer-by-layer processing.
Regarding claim 3, the combination of Pisarchyk, Goldberg and Jokic explicitly discloses all the limitations of claim 2 (as shown in the rejections above)
Pisarchyk in view of Goldberg and Jokic further teaches:
wherein the analyzed operator is an operator that undergoes linear splitting. (Goldberg, ¶0053]: “FIG. 6 represents a first splitting of a 64 track block into two 32 track blocks in response to a memory allocation request defining a size that is equal to or smaller than 32 tracks in memory (in the case of this example, a 16 track block).”)
Regarding claim 6, the combination of Pisarchyk, Goldberg and Jokic explicitly discloses all the limitations of claim 5 (as shown in the rejections above)
Pisarchyk in view of Goldberg further teaches:
wherein before the storage address is allocated to each memory block, a size of the static memory pool is determined: a size of a memory block set at any moment is calculated, and (Pisarchyk, Pg. 5, Col. 1, Section 5.2, ¶[1]: “As Greedy by Size works well for Shared Objects, we employ a similar method for Offsets Calculation (Algorithm 3).”, Pg. 5, Algorithm 3) [Examiner’s note: In this context, the heuristic algorithm refers to a greedy memory-planning algorithm]
Pisarchyk in view of Goldberg fails to teach:
a minimum value of a memory block set required at any moment is used as a lower
limit value of the size of the static memory pool.
However, Jokic explicitly discloses:
a minimum value of a memory block set required at any moment is used as a lower
limit value of the size of the static memory pool. (Jokic, Pg. 3, Col. 2, ¶1]: “From (5), the lower bound of activations memory required for computing the entire CNN, Mmin, can be derived by finding the minimum memory size that supports all layers of the network, as shown in (6).”)
Claim(s) 10-11 are rejected under 35 U.S.C. 103 as being unpatentable over Pisarchyk & Lee (“EFFICIENT MEMORY MANAGEMENT FOR DEEP NEURAL NET INFERENCE”) in view of Goldberg (US 2012/0303927 A1) and further in view of Pang et al. (US 2021/0142154 A1)
Regarding claim 10, the combination of Pisarchyk, Goldberg and Jokic explicitly discloses all the limitations of claim 7 (as shown in the rejections above)
Pisarchyk in view of Goldberg fails to teach:
wherein the using the idle memory block that meets the requirement as memory required for calculating the current operator comprises: using, as a memory block corresponding to the current operator, an idle memory block that is in the memory linked list, that meets a requirement of the memory space required for calculating the current operator, and that has minimum memory space
However, Pang explicitly teaches:
wherein the using the idle memory block that meets the requirement as memory required for calculating the current operator comprises: using, as a memory block corresponding to the current operator, an idle memory block that is in the memory linked list, that meets a requirement of the memory space required for calculating the current operator, and that has minimum memory space. (Pang, ¶[0169]: “…selecting, as the memory block identifier having a mapping relationship with the output, the idle memory block identifier corresponding to the memory block capacity that has the smallest difference from the capacity required by the output in the memory block capacities from the idle memory block identifiers whose corresponding memory block capacities are greater than or equal to the capacity required by the output.”)
It would have obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Pisarchyk, Goldberg and Pang. Pisarchyk teaches various strategies to smartly share memory buffers among intermediate tensors in deep neural nets. Goldberg teaches a method of allocating storage space in a memory of a computing system. Pang teaches a method for memory pre-allocation for forward calculation in a neural network. One of ordinary skill would have motivation to combine Pisarchyk, Goldberg and Pang to improve memory utilization through real-time allocation of memory.
Regarding claim 11, the combination of Pisarchyk, Goldberg and Jokic explicitly discloses all the limitations of claim 7 (as shown in the rejections above)
Pisarchyk in view of Goldberg fails to teach:
wherein the using the idle memory block that meets the requirement as memory required for calculating the current operator comprises: determining that a ratio between a size of memory space occupied by the current operator and a size of a used memory block is greater than a preset memory usage ratio. (Pang, ¶[0051]: “the obtaining a memory block identifier from a list of idle memory block identifiers that is used to record an idle memory block identifier may comprise: if there are a plurality of idle memory block identifiers in the list of idle memory block identifiers, and the memory block capacities corresponding to the plurality of idle memory block identifiers are all less than the capacity required by the output (that is, there is no memory block identifier whose capacity is equal to or greater than the capacity required by the output), selecting, as a memory block identifier having a mapping relationship with the output, a corresponding idle memory block identifier with the largest memory block capacity from the plurality of idle memory block identifiers.”)
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to AMY TRAN whose telephone number is (571)270-0693. The examiner can normally be reached Monday - Friday 7:30 am - 5:00 pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, David Yi can be reached at (571) 270-7519. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/AMY TRAN/Examiner, Art Unit 2126
/DAVID YI/Supervisory Patent Examiner, Art Unit 2126
Prosecution Insights