DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on November 4, 2025 has been entered.
Response to Amendment
The amendment filed November 4, 2025 has been entered. Claims 1-20 remain pending in the application.
Response to Arguments
Applicant’s arguments, see Page 12, filed November 4, 2025, with respect to the rejection(s) of claim(s) 1-20 under 35 USC 103 have been fully considered and are persuasive. Therefore, the rejection has been withdrawn. However, upon further consideration, a new ground(s) of rejection is made in view of Gould et al. (US 20190236749 A1).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-6, 9-14, and 17-20 are rejected under 35 U.S.C. 103 as being unpatentable over Kim et al. (US 20160253257 A1) in view of Smith (US 10380712 B2), Engh-Halstvedt et al. (US 10650580 B2), Gao et al. (Estimating GPU Memory Consumption of Deep Learning Models), and Gould et al. (US 20190236749 A1), hereinafter Kim, Smith, Engh-Halstvedt, Gao, and Gould respectively.
Regarding claim 1, Kim teaches a method (Paragraph 0009 – “an operating method”) comprising:
in response to a write request for memory (Paragraph 0035 – “The processor 211 may control the write operation or read operation of the nonvolatile memory device 220 through a command in response to a write request or a read request from the host device 100. The processor 211 may drive a software program, on the memory 212, for controlling the operation of the data storage device 200”) needed during running of a target task (Paragraph 0022 – “The application unit 110 may generate, change or delete a file in the data storage device 200 when performing an assigned task”; Note: a request may be made when running a task), classifying target data (Paragraph 0026-0027 – “The identifier generator 140 may classify attribute information of data, and select a context identifier corresponding to relevant data among a plurality of context identifiers…When transmitting a write request WTRQ of data DT to the data storage device 200, the host device 100 may transmit a context identifier CTXID assigned to the data DT, together with the write request WTRQ”; Note: data is classified) generated during running of each task (Paragraph 0022, 0024 – “The application unit 110 may generate, change or delete a file in the data storage device 200 when performing an assigned task…The file system 120 may provide the identifier generator 140 with a file, i.e., attribute information of data, to be stored in the data storage device 200. For example, when certain data is stored in the data storage device 200, the file system 120 may determine the attributes of the corresponding data”; Note: the file system generates the attribute data, which occurs while the application unit runs a task. Kim only teaches tasks, but when modified as described below, teaches multiple sub-tasks), wherein the write request is generated (Paragraph 0035 – “The processor 211 may control the write operation or read operation of the nonvolatile memory device 220 through a command in response to a write request or a read request from the host device 100. The processor 211 may drive a software program, on the memory 212, for controlling the operation of the data storage device 200”) during the running of the target task (Paragraph 0022 – “The application unit 110 may generate, change or delete a file in the data storage device 200 when performing an assigned task”; Note: a request may be made when running a task), wherein a type of the target data comprises at least first data (Paragraph 0030 – “when pieces of data having mutually similar update frequencies are stored together in predetermined memory regions, memory regions may have a large deviation in their degree of invalidation. For example, when pieces of data having a first attribute have a relatively high update frequency, a memory region in which pieces of data having the first attribute are classified and stored may have a relatively high invalidation degree”; Note: invalid data is equivalent to the first data), and wherein the first data is not used by a subsequent task (Paragraph 0030 – “when pieces of data having mutually similar update frequencies are stored together in predetermined memory regions, memory regions may have a large deviation in their degree of invalidation. For example, when pieces of data having a first attribute have a relatively high update frequency, a memory region in which pieces of data having the first attribute are classified and stored may have a relatively high invalidation degree”; Note: invalid data is equivalent to the first data because invalid data is frequently replaced, meaning it is not reused);
dividing each target memory pool of the multiple target memory pools into at least one memory block based on a type of the target data (Paragraph 0044, 0046 – “The memory cell array 225 may include a plurality of memory blocks BLK1 to BLKm. Each of the memory blocks BLK1 to BLKm may include a plurality of pages (e.g., P1 to Pn)…The processor 211 may match the respective context identifiers CTXID1 to CTXIDi, which are to be provided from the host device 100, with corresponding memory regions”; Note: the memory cell array, which is divided into memory blocks, is equivalent to the memory pool. Additionally, the memory blocks contain data based on their context identifiers, which is equivalent to the type of data), and wherein the at least one memory block comprises at least a first memory block corresponding to the first data (Paragraph 0030 – “when pieces of data having mutually similar update frequencies are stored together in predetermined memory regions, memory regions may have a large deviation in their degree of invalidation”; Note: memory regions are the equivalent to memory blocks and memory regions may contain invalid data, which is equivalent to the first data, wherein the invalid data is frequently replaced, meaning it is not used for future tasks);
and clearing memory space (Paragraph 0054 – “when the host device 100 assigns a context identifier CTXIDi to metadata and thus an i.sup.th memory block BLK1i stores only metadata, the second memory block BLK1i may have a high invalidation degree due to the characteristics of metadata that is frequently updated, and may be processed at a low copy cost although the i.sup.th memory block BLK1i is selected as a victim region on a garbage collection operation”; Note: the garbage collection operation is the equivalent to clearing the memory space, and the victim region is the equivalent to the memory address corresponding to the first data) corresponding to the same target physical memory address that stored the first data (Paragraph 0036 – “The processor 211 may determine a memory region, in which write-requested data is to be stored, according to a context identifier assigned to the data, and map a physical address of the determined memory region to a logical address provided from the host device 100”; Note: the data has a corresponding physical address).
Kim does not teach a computer-implemented method for graphics memory allocation nor wherein multiple first graphics memory blocks corresponding to the multiple sub-tasks are mapped to a same target physical memory address, wherein only one of the multiple first graphics memory blocks occupies the same target physical memory address at a time. However, Smith teaches a computer-implemented method for graphics memory allocation (Col. 1 lines 5-9, Col. 15 lines 39-41 – “The technology described herein relates to graphics processing methods and systems in which graphics data stored in memory is accessible using virtual memory addresses that map to physical memory addresses in the memory…The technology described herein can be implemented in any suitable system, such as a suitably configured computer or micro-processor based system”; Note: the method can be implemented by a computer, and it involves allocating memory for graphics data). Smith also teaches wherein multiple first graphics memory blocks corresponding to the multiple tasks (Col. 14 lines 15-20 – “Some embodiments may comprise performing further graphics processing, e.g. a texture look up, compositing, etc., using a page of the set of graphics data read from the memory. Other embodiments may comprise outputting a page of the set of graphics data read from the memory, e.g. for display”; Note: the page of graphics data, which is equivalent to the graphics memory block, corresponds to the listed graphics processing tasks, which are equivalent to the sub-tasks) are mapped to a same target physical memory address (Col. 3 lines 31-35 – “When a particular page of the set of graphics data is determined as being similar to a particular stored page of the set of graphics data, a virtual memory address for the particular page is mapped to the same physical memory address as the particular stored page”; Note: a page of graphics data is the equivalent to the graphics memory block, and it gets mapped to the same physical memory address), wherein only one of the multiple first graphics memory blocks occupies the same target physical memory address at a time (Col. 3 lines 31-40, 62-65 – “When a particular page of the set of graphics data is determined as being similar to a particular stored page of the set of graphics data, a virtual memory address for the particular page is mapped to the same physical memory address as the particular stored page, e.g. rather than mapping the virtual memory address for the particular page to a unique physical memory address for the particular page. The particular page and the particular stored page may therefore have different virtual memory addresses that map to the same physical memory address…a single page of graphics data may be read (and, e.g., cached) from the single physical memory address that maps to the virtual memory addresses for the merged pages”; Note: only one page is stored in the physical memory address at a time). While Kim’s method applies to data in general, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Kim to incorporate the teachings of Smith to have a method of memory allocation apply specifically to graphics data because “in graphics processing systems, e.g. of portable devices, it is generally desirable to try to reduce the amount of memory required to store sets of graphics data and/or to try to increase efficiency, e.g. so as to reduce power consumption and/or increase processing speed, but without a corresponding undesirable reduction in the quality or resolution of the graphics data” (Smith: Col. 1 lines 45-51). It also would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Kim to incorporate the teachings of Smith to map graphics memory blocks to a same physical memory address for the benefit of “an efficient use of memory since similar pages of a set of graphics data need not be retained in memory at multiple different memory locations” (Smith: Col. 3 lines 51-54). Finally, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Kim to incorporate the teachings of Smith to have only one memory block/page in a physical memory address at a time because logically, physical memory is limited. To ensure efficient memory usage, it should only store a certain about of data at a time, and multiple virtual addresses can map to it instead. Doing so would “reduce the amount of memory bandwidth (and, e.g., the amount of cache) that is needed in order to read plural pages of similar graphics data from memory, whilst still allowing the virtual memory addresses for those pages to be used” (Smith: Col. 3 lines 55-59).
Kim modified by Smith still does not teach a graphics memory allocation request for graphics memory; wherein the target task comprises multiple serial sub-tasks; nor the “multiple sub-tasks” in the limitation: “allocating multiple target graphics memory pools to the multiple sub-tasks”. However, Engh-Halstvedt teaches a graphics memory allocation request for graphics memory (Col. 4 lines 13-18 – “sending a request for memory space to be allocated for the vertex or group of vertices; and in response to the request from the testing stage, a memory allocator allocating memory space for storing vertex shaded attribute data for the vertex or group of vertices for which the request has been sent”; Note: a graphics memory allocation for graphics memory is made), and wherein the target task comprises multiple serial sub-tasks (Col. 34 lines 15-17 – “A graphics processing job (task) (e.g. defining a set of draw calls/sub tasks to be processed for a frame or render pass) is received at the control unit”; Note: a graphics processing job is the target task and it is comprised of sub-tasks, also called draw calls). Engh-Halstvedt further teaches allocating multiple target graphics memory pools (Col. 12 lines 37-44, 29 lines 21-29 – “a pool of memory space (a set of memory addresses) is set aside (made available) for storing the vertex data, and then when it is determined that memory space should be allocated for storing vertex shaded attributes data for a vertex or group of vertices, a portion of memory (a subset of the memory addresses) of the set aside memory space pool is allocated for storing the vertex data for the vertex or group of vertices…The graphics processing pipeline 20 also has access to suitable memory for storing data to be processed or generated by the various processing stages of the graphics processing pipeline 20. In particular, memory is allocated for: raw position data (at memory block 25), raw varyings data (at block 26), vertex-shaded (transformed) position data (at block 27), vertex-shaded (transformed) varyings data (at block 28), index data (at block 29) and primitive list data (at block 210)”; Note: memory pools are allocated to different sub-tasks in the graphics processing pipeline) to the multiple sub-tasks (Col. 34 lines 15-17 – “A graphics processing job (task) (e.g. defining a set of draw calls/sub tasks to be processed for a frame or render pass) is received at the control unit”; Note: a graphics processing job is the target task and it is comprised of sub-tasks, also called draw calls). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Kim to incorporate the teachings of Engh-Halstvedt to have a graphics memory allocation request for graphics memory because “there is a need to store geometry (and in particular vertex) data for a period of time for use in the later, deferred, rasterising and rendering pass” (Engh-Halstvedt: Col. 3 lines 4-7). Kim already teaches a write request, but it would be beneficial to allow graphics memory allocation requests for storing specific graphics processing operations. It also would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Kim to incorporate the teachings of Engh-Halstvedt to separate tasks into multiple sub-tasks and allocate memory to those sub-tasks because it would help organize memory usage based on the graphics processing pipeline and assist in distinguishing between data that will be used again or not used again in later sub-tasks (Engh-Halstvedt: Col. 5 lines 4-20). Distinguishing between useful and non-useful data would help in reducing memory waste.
While Kim modified by Smith and Engh-Halstvedt does not teach wherein a capacity of each target graphics memory pool of the multiple target graphics memory pools is determined using at least one forward inference, Gao teaches wherein a capacity of each target graphics memory pool of the multiple target graphics memory pools is determined using at least one forward inference (Paragraph 7 in 2nd Col. of Page 8, Paragraph 1 in 1st Col. of Page 9 – “Currently, the three DL frameworks do not provide stand-alone shape inference tools. DNNMem has already implemented our own using the framework APIs for establishing the operator memory cost functions. We query such cost functions for tensors of the initial input, weights, intermediate outputs, and final output under forward propagation, and then add them up as the GPU memory consumption estimated by Shape Inference”; Note: Potential memory consumption, which is the memory capacity required for a deep learning model, is determined using DNNMem, which contains inference techniques). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Kim to incorporate the teachings of Gao to determine memory capacity using forward inference because “since the GPU memory consumed by a DL model is often unknown to developers before the training or inferencing job starts running, an improper model configuration of neural architecture or hyperparameters can cause such a job to run out of the limited GPU memory and fail…Therefore, knowing the accurate GPU memory consumption (aka memory footprint) in advance is very important to reduce OOM failures and save precious platform resources including GPU/CPU/storage, by helping developers choose an optimal model configuration or facilitating DL frameworks to better utilize the mechanisms of dynamic memory management” (Gao: Paragraph 2-3 in 2nd Col. of Page 1). Because many AI models utilize graphics memory, it would be beneficial for graphics memory systems to monitor memory capacity in order to ensure successful AI model operations. But while Gao specifically advocates for knowing memory consumption before running a deep learning model, knowing the memory consumption for any task beforehand, not just deep learning models, is useful in preventing memory failures.
Finally, Kim modified by Smith, Engh-Halstvedt, and Gao still do not teach upon completion of a current sub-task, clearing graphics memory space corresponding to the same target physical memory address that stored the first data of the current sub-task, wherein the graphics memory space is available for storing the first data of a next sub-task. However, Gould teaches upon completion of a current sub-task, clearing graphics memory space corresponding to the same target physical memory address that stored the first data of the current sub-task (Paragraph 0040, 0050, 0064 – “the memory pool 82 may include virtual memory addresses that are at least partially allocated to physical memory in the graphics memory 58… GPU 12 or CPU 34 can manage the pool to allocate/deallocate physical resources when the respective thresholds are achieved…Executing the read operation can also optionally include, at 412, deallocating one or more pages of memory to the memory pool. In an example, GPU 12, e.g., via a compute shader or other thread of execution managing the read threads, can be configured to deallocate the one or more pages of memory from the memory pool (e.g., by using functionality provided by the FIFO queue manager 74). In one example, the GPU 12 can deallocate the one or more pages of memory based on a location of the read done pointer. For example, where the read done pointer achieves a threshold, which may correspond to the end of a page, the GPU 12 can deallocate the page (and optionally some number of preceding pages in memory), back to the memory pool (e.g., memory pool 82, which can add the page or pages (e.g., pointers thereto) to free page list 204)… For example, once all data on a page is consumed, the page may be freed to pool 82 immediately”; Note: after all the data in the page have been read, which indicates that the sub-task was completed, the memory space of the corresponding page is cleared) and wherein the graphics memory space is available for storing the first data of a next sub-task (Paragraph 0064 – “Freeing the memory back into the memory pool in this regard can allow for reuse of the memory for other FIFO queues such to conserve resources over previous memory management for executing graphics processes…For example, once all data on a page is consumed, the page may be freed to pool 82 immediately”; Note: the graphics memory is available for storing data of subsequent FIFO queues of graphics processes after it is freed). Since Kim already teaches clearing memory space (Paragraph 0054 – “when the host device 100 assigns a context identifier CTXIDi to metadata and thus an i.sup.th memory block BLK1i stores only metadata, the second memory block BLK1i may have a high invalidation degree due to the characteristics of metadata that is frequently updated, and may be processed at a low copy cost although the i.sup.th memory block BLK1i is selected as a victim region on a garbage collection operation”), it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Kim to incorporate the teachings of Gould to clear and make available the memory space after the running of a current sub-task ends for the benefit of allowing “for reuse of the memory for other FIFO queues such to conserve resources over previous memory management for executing graphics processes” (Gould: Paragraph 0064). In other words, the release of the memory page helps prevent memory waste.
Regarding claim 2, Kim in view of Smith, Engh-Halstvedt, Gao, and Gould teach the computer-implemented method of claim 1. Kim further teaches during running of each sub-task of multiple sub-tasks: storing the target data in a corresponding graphics memory block based on the type of the target data (Paragraph 0044, 0046 – “The memory cell array 225 may include a plurality of memory blocks BLK1 to BLKm. Each of the memory blocks BLK1 to BLKm may include a plurality of pages (e.g., P1 to Pn)…The processor 211 may match the respective context identifiers CTXID1 to CTXIDi, which are to be provided from the host device 100, with corresponding memory regions”; Note: the memory blocks store data based on their context identifiers, which is equivalent to the type of data. The graphics memory block was previously taught in the rejection of claim 1).
Regarding claim 3, Kim in view of Smith, Engh-Halstvedt, Gao, and Gould teach the computer-implemented method of claim 1. Kim further teaches wherein classifying target data generated during running of each sub-task of multiple sub-tasks, comprises: adding a type label to the target data based on the type of the target data, wherein the type label comprises at least a first label corresponding to the first data (Paragraph 0026, 0054 – “select a context identifier corresponding to relevant data among a plurality of context identifiers…The generated context identifiers are assigned to corresponding data… For example, when the host device 100 assigns a context identifier CTXID2 to journal data and thus a second memory block BLK12 stores only journal data, the second memory block BLK12 may have a high invalidation degree due to the characteristics of journal data that is frequently updated, and may be processed at a low copy cost”; Note: the context identifier is the equivalent to the type label. Context identifier CTXID2 for journal data is a first label corresponding to invalid data, which is the equivalent to first data. Journal data is invalid data because it has a high invalidation degree).
Regarding claim 4, Kim in view of Smith, Engh-Halstvedt, Gao, and Gould teach the computer-implemented method of claim 3. Kim further teaches wherein dividing each target memory pool of the multiple target memory pools into at least one memory block based on a type of the target data, comprises: dividing at least a part of memory space of each target memory pool into the first memory block (Paragraph 0044, 0046 – “The memory cell array 225 may include a plurality of memory blocks BLK1 to BLKm. Each of the memory blocks BLK1 to BLKm may include a plurality of pages (e.g., P1 to Pn)…The processor 211 may match the respective context identifiers CTXID1 to CTXIDi, which are to be provided from the host device 100, with corresponding memory regions”; Note: the memory cell array, which is divided into memory blocks, is equivalent to the memory pool. Additionally, the memory blocks contain data based on their context identifiers, which is equivalent to the type of data); and allocating the first memory block to the first data (Paragraph 0044, 0046, 0054 – “The memory cell array 225 may include a plurality of memory blocks BLK1 to BLKm…The processor 211 may match the respective context identifiers CTXID1 to CTXIDi, which are to be provided from the host device 100, with corresponding memory regions. For example, the processor 211 may match the context identifiers CTXID1 to CTXIDi with memory blocks, respectively… the second memory block BLK12 may have a high invalidation degree due to the characteristics of journal data that is frequently updated”; Note: memory blocks are allocated to data of specific context identifiers. A memory block is allocated to the first data, which is invalid data. While the reference refers to the memory block as a “second memory block”, in the context of the present claim, the second memory block of the reference is equivalent to the first memory block of the present claim). Kim does not teach mapping the multiple first graphics memory blocks to the same physical memory address. However, Smith teaches mapping the multiple graphics memory blocks to the same physical memory address (Col. 3 lines 31-35 – “When a particular page of the set of graphics data is determined as being similar to a particular stored page of the set of graphics data, a virtual memory address for the particular page is mapped to the same physical memory address as the particular stored page”; Note: a page of graphics data is the equivalent to the graphics memory block, and the virtual memory address for the page is mapped to the same physical memory address). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Kim to incorporate the teachings of Smith to map graphics memory blocks to a same physical memory address for the benefit of “an efficient use of memory since similar pages of a set of graphics data need not be retained in memory at multiple different memory locations” (Smith: Col. 3 lines 51-54).
Regarding claim 5, Kim in view of Smith, Engh-Halstvedt, Gao, and Gould teach the computer-implemented method of claim 4. Kim further teaches mapping based on a singleton pattern (Paragraph 0046, 0048 – “The processor 211 may match the respective context identifiers CTXID1 to CTXIDi, which are to be provided from the host device 100, with corresponding memory regions…the processor 211 may match the memory blocks BLK11 to BLK1i, in which write-requested data is to be stored, with the context identifiers CTXID1 to CTXIDi, respectively”; Note: mapping context identifiers with memory blocks is considered a singleton pattern, as memory blocks will have to go through this process before getting placed into virtual memory, and they follow a single pattern to be organized in memory). Kim does not teach wherein the multiple first graphics memory blocks are virtual graphics memories, and mapping the multiple first graphics memory blocks to the same target physical memory address, comprises: mapping the multiple first graphics memory blocks to multiple virtual graphics memories based on a singleton pattern; mapping the multiple virtual graphics memories to the same target physical memory address; and feeding back multiple virtual graphics memory pointers, wherein each virtual graphic memory pointer of the multiple virtual graphics memory pointers comprises a same target physical memory address and a capacity of the first graphics memory block corresponding to each virtual graphics memory pointer. However, Smith teaches wherein the multiple graphics memory blocks are virtual graphics memories (Col. 12 lines 50-57 – “those merged pages are later read from memory using respective virtual memory addresses for those pages… virtual memory address for the pages of the set of graphics data”; Note: pages, which are graphics memory blocks, are represented by virtual graphics memory), and mapping the multiple graphics memory blocks to the same physical memory address, comprises: mapping the multiple graphics memory blocks to multiple virtual graphics memories (Col. 12 lines 50-57 – “those merged pages are later read from memory using respective virtual memory addresses for those pages… virtual memory address for the pages of the set of graphics data”; Note: pages, which are graphics memory blocks, are mapped to virtual graphics memory addresses); mapping the multiple virtual graphics memories to the same physical memory address (Col. 3 lines 31-35 – “When a particular page of the set of graphics data is determined as being similar to a particular stored page of the set of graphics data, a virtual memory address for the particular page is mapped to the same physical memory address as the particular stored page”; Note: virtual memories, represented by virtual memory addresses, are mapped to the same physical memory address); and feeding back multiple virtual graphics memory pointers, wherein each virtual graphic memory pointer of the multiple virtual graphics memory pointers comprises a same physical memory address (Col. 13 lines 42-46, 55-59; Col. 14 lines 58-60 – “the virtual memory addresses within the page table may be unique, whereas at least some of the physical memory addresses (e.g. for merged pages) may not be unique…reading the page of the set of graphics data stored in the memory may comprise using a page table for the set of graphics data that maps virtual memory addresses for pages of the set of graphics data to physical memory addresses for pages of the set of graphics data…the virtual memory address for the particular page may be mapped to the same physical memory address as the particular stored page”; Note: virtual memory addresses are virtual graphics memory pointers, which can point to the same physical memory address). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Kim to incorporate the teachings of Smith to map graphics memory blocks to a same physical memory address for the benefit of “an efficient use of memory since similar pages of a set of graphics data need not be retained in memory at multiple different memory locations” (Smith: Col. 3 lines 51-54). Since the singleton pattern can be used for mapping memory blocks, it also would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Kim to incorporate the teachings of Smith to map graphics memory blocks to virtual memories using a singleton pattern for the benefit of organizing data into virtual memory so that data near each other have similar attributes, allowing for low-cost memory usage (Kim: Paragraph 0054, Smith: Col. 3 lines 51-54). Furthermore, Kim modified by Smith still does not teach wherein each virtual graphic memory pointer of the multiple virtual graphics memory pointers comprises a capacity of the graphics memory block corresponding to each virtual graphics memory pointer. However, Engh-Halstvedt teaches wherein each virtual graphic memory pointer of the multiple virtual graphics memory pointers comprises a capacity of the graphics memory block corresponding to each virtual graphics memory pointer (Col. 12 lines 37-44, Col. 13 lines 13-17, Col. 15 lines 10-12 – “a pool of memory space (a set of memory addresses) is set aside (made available) for storing the vertex data, and then when it is determined that memory space should be allocated for storing vertex shaded attributes data for a vertex or group of vertices, a portion of memory (a subset of the memory addresses) of the set aside memory space pool is allocated for storing the vertex data for the vertex or group of vertices… an amount of memory that is based on an expected amount of memory that may be required for storing vertex shaded data (for visible vertices) for the graphics processing output being generated is set aside as the memory space pool…the memory address for one end (e.g., and in an embodiment the current bottom allocation pointer) for the memory space pool is set”; Note: the memory addresses, which are pointers, correspond to a portion of memory and comprise a capacity of memory that is used to store vertices). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Kim to incorporate the teachings of Engh-Halstvedt to have the pointer comprise a capacity of a corresponding graphics memory block because pointers, by definition, allow access to locations in memory. Logically, it would follow that they correspond to a portion of memory, and the portion of memory has a capacity, as it cannot be infinite, or as Engh-Halstvedt suggests, only a certain amount is “set aside”.
Regarding claim 6, Kim in view of Smith, Engh-Halstvedt, Gao, and Gould teach the computer-implemented method of claim 4. Kim further teaches wherein the type of the target data comprises second data (Paragraph 0030 – “when pieces of data having a second attribute have a relatively low update frequency, a memory region in which pieces of data having the second attribute are classified and stored may have a relatively low invalidation degree”; Note: the second data type is the valid data, which is data with low invalidation degree); the type label comprises a second label corresponding to the second data (Paragraph 0056 – “the data storage device 200 may refer to a context identifier assigned to valid data of a victim memory block BLK, and store the valid data in a memory block corresponding to the referred context identifier”; Note: the valid data, which is the second data, has a corresponding context identifier, which is a label); the at least one memory block comprises a memory block corresponding to the second data (Paragraph 0056 – “the data storage device 200 may refer to a context identifier assigned to valid data of a victim memory block BLK, and store the valid data in a memory block corresponding to the referred context identifier”; Note: there is a second memory block containing valid data, which is the second data); and dividing each target memory pool of the multiple target memory pools into at least one memory block based on a type of the target data, comprises: dividing at least a part of memory space of each target memory pool into the second memory block (Paragraph 0044, 0046 – “The memory cell array 225 may include a plurality of memory blocks BLK1 to BLKm. Each of the memory blocks BLK1 to BLKm may include a plurality of pages (e.g., P1 to Pn)…The processor 211 may match the respective context identifiers CTXID1 to CTXIDi, which are to be provided from the host device 100, with corresponding memory regions”; Note: the memory cell array, which is divided into memory blocks, is equivalent to the memory pool. Additionally, the memory blocks contain data based on their context identifiers, which is equivalent to the type of data. There are multiple memory blocks, and thus there is a second memory block); and allocating the second memory block to the second data (Paragraph 0056 – “the data storage device 200 may refer to a context identifier assigned to valid data of a victim memory block BLK, and store the valid data in a memory block corresponding to the referred context identifier”; Note: the valid data, which is the second data, is stored in a memory block. This implies that the memory block is allocated to the valid data). Kim does not teach that the second data is used by a subsequent sub-task. However, Engh-Halstvedt teaches that the data is used by a subsequent sub-task (Col. 7 lines 16-19 – “the vertex belongs to (is to be used for) a primitive that could be visible in the output of the graphics processing operation”; Note: the vertex, which is data, is used by a subsequent sub-task since it would be needed for the output of the operation). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Kim to incorporate the teachings of Engh-Halstvedt to have the second data be used by a sub-sequent sub-task because distinguishing the required data ensures that “only a reduced amount of memory space for storing vertex shaded attribute data can be (and needs to be) allocated” and it would facilitate “the more efficient use of memory for storing vertex data” (Engh-Halstvedt: Col. 5 lines 12-20).
Regarding claim 9, Kim teaches the operations of:
in response to a write request for memory (Paragraph 0035 – “The processor 211 may control the write operation or read operation of the nonvolatile memory device 220 through a command in response to a write request or a read request from the host device 100. The processor 211 may drive a software program, on the memory 212, for controlling the operation of the data storage device 200”) needed during running of a target task (Paragraph 0022 – “The application unit 110 may generate, change or delete a file in the data storage device 200 when performing an assigned task”; Note: a request may be made when running a task), classifying target data (Paragraph 0026-0027 – “The identifier generator 140 may classify attribute information of data, and select a context identifier corresponding to relevant data among a plurality of context identifiers…When transmitting a write request WTRQ of data DT to the data storage device 200, the host device 100 may transmit a context identifier CTXID assigned to the data DT, together with the write request WTRQ”; Note: data is classified) generated during running of each task (Paragraph 0022, 0024 – “The application unit 110 may generate, change or delete a file in the data storage device 200 when performing an assigned task…The file system 120 may provide the identifier generator 140 with a file, i.e., attribute information of data, to be stored in the data storage device 200. For example, when certain data is stored in the data storage device 200, the file system 120 may determine the attributes of the corresponding data”; Note: the file system generates the attribute data, which occurs while the application unit runs a task. Kim only teaches tasks, but when modified as described below, teaches multiple sub-tasks), wherein the write request is generated (Paragraph 0035 – “The processor 211 may control the write operation or read operation of the nonvolatile memory device 220 through a command in response to a write request or a read request from the host device 100. The processor 211 may drive a software program, on the memory 212, for controlling the operation of the data storage device 200”) during the running of the target task (Paragraph 0022 – “The application unit 110 may generate, change or delete a file in the data storage device 200 when performing an assigned task”; Note: a request may be made when running a task), wherein a type of the target data comprises at least first data (Paragraph 0030 – “when pieces of data having mutually similar update frequencies are stored together in predetermined memory regions, memory regions may have a large deviation in their degree of invalidation. For example, when pieces of data having a first attribute have a relatively high update frequency, a memory region in which pieces of data having the first attribute are classified and stored may have a relatively high invalidation degree”; Note: invalid data is equivalent to the first data), and wherein the first data is not used by a subsequent task (Paragraph 0030 – “when pieces of data having mutually similar update frequencies are stored together in predetermined memory regions, memory regions may have a large deviation in their degree of invalidation. For example, when pieces of data having a first attribute have a relatively high update frequency, a memory region in which pieces of data having the first attribute are classified and stored may have a relatively high invalidation degree”; Note: invalid data is equivalent to the first data because invalid data is frequently replaced, meaning it is not reused);
dividing each target memory pool of the multiple target memory pools into at least one memory block based on a type of the target data (Paragraph 0044, 0046 – “The memory cell array 225 may include a plurality of memory blocks BLK1 to BLKm. Each of the memory blocks BLK1 to BLKm may include a plurality of pages (e.g., P1 to Pn)…The processor 211 may match the respective context identifiers CTXID1 to CTXIDi, which are to be provided from the host device 100, with corresponding memory regions”; Note: the memory cell array, which is divided into memory blocks, is equivalent to the memory pool. Additionally, the memory blocks contain data based on their context identifiers, which is equivalent to the type of data), and wherein the at least one memory block comprises at least a first memory block corresponding to the first data (Paragraph 0030 – “when pieces of data having mutually similar update frequencies are stored together in predetermined memory regions, memory regions may have a large deviation in their degree of invalidation”; Note: memory regions are the equivalent to memory blocks and memory regions may contain invalid data, which is equivalent to the first data, wherein the invalid data is frequently replaced, meaning it is not used for future tasks);
and clearing memory space (Paragraph 0054 – “when the host device 100 assigns a context identifier CTXIDi to metadata and thus an i.sup.th memory block BLK1i stores only metadata, the second memory block BLK1i may have a high invalidation degree due to the characteristics of metadata that is frequently updated, and may be processed at a low copy cost although the i.sup.th memory block BLK1i is selected as a victim region on a garbage collection operation”; Note: the garbage collection operation is the equivalent to clearing the memory space, and the victim region is the equivalent to the memory address corresponding to the first data) corresponding to the same target physical memory address that stored the first data (Paragraph 0036 – “The processor 211 may determine a memory region, in which write-requested data is to be stored, according to a context identifier assigned to the data, and map a physical address of the determined memory region to a logical address provided from the host device 100”; Note: the data has a corresponding physical address).
Kim does not teach a non-transitory, computer-readable medium storing one or more instructions executable by a computer system to perform one or more operations, nor wherein multiple first graphics memory blocks corresponding to the multiple sub-tasks are mapped to a same target physical memory address, wherein only one of the multiple first graphics memory blocks occupies the same target physical memory address at a time. However, Smith teaches a non-transitory, computer-readable medium storing one or more instructions executable by a computer system to perform one or more operations (Col. 16 lines 46-52 – “a series of computer readable instructions either fixed on a tangible, non transitory medium, such as a computer readable medium, for example, diskette, CD, DVD, ROM, RAM, flash memory, or hard disk. It could also comprise a series of computer readable instructions transmittable to a computer system”). Smith also teaches wherein multiple first graphics memory blocks corresponding to the multiple tasks (Col. 14 lines 15-20 – “Some embodiments may comprise performing further graphics processing, e.g. a texture look up, compositing, etc., using a page of the set of graphics data read from the memory. Other embodiments may comprise outputting a page of the set of graphics data read from the memory, e.g. for display”; Note: the page of graphics data, which is equivalent to the graphics memory block, corresponds to the listed graphics processing tasks, which are equivalent to the sub-tasks) are mapped to a same target physical memory address (Col. 3 lines 31-35 – “When a particular page of the set of graphics data is determined as being similar to a particular stored page of the set of graphics data, a virtual memory address for the particular page is mapped to the same physical memory address as the particular stored page”; Note: a page of graphics data is the equivalent to the graphics memory block, and it gets mapped to the same physical memory address), wherein only one of the multiple first graphics memory blocks occupies the same target physical memory address at a time (Col. 3 lines 31-40, 62-65 – “When a particular page of the set of graphics data is determined as being similar to a particular stored page of the set of graphics data, a virtual memory address for the particular page is mapped to the same physical memory address as the particular stored page, e.g. rather than mapping the virtual memory address for the particular page to a unique physical memory address for the particular page. The particular page and the particular stored page may therefore have different virtual memory addresses that map to the same physical memory address…a single page of graphics data may be read (and, e.g., cached) from the single physical memory address that maps to the virtual memory addresses for the merged pages”; Note: only one page is stored in the physical memory address at a time). While Kim’s method applies to data in general, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Kim to incorporate the teachings of Smith to have a method of memory allocation apply specifically to graphics data because “in graphics processing systems, e.g. of portable devices, it is generally desirable to try to reduce the amount of memory required to store sets of graphics data and/or to try to increase efficiency, e.g. so as to reduce power consumption and/or increase processing speed, but without a corresponding undesirable reduction in the quality or resolution of the graphics data” (Smith: Col. 1 lines 45-51). It also would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Kim to incorporate the teachings of Smith to map graphics memory blocks to a same physical memory address for the benefit of “an efficient use of memory since similar pages of a set of graphics data need not be retained in memory at multiple different memory locations” (Smith: Col. 3 lines 51-54). Finally, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Kim to incorporate the teachings of Smith to have only one memory block/page in a physical memory address at a time because logically, physical memory is limited. To ensure efficient memory usage, it should only store a certain about of data at a time, and multiple virtual addresses can map to it instead. Doing so would “reduce the amount of memory bandwidth (and, e.g., the amount of cache) that is needed in order to read plural pages of similar graphics data from memory, whilst still allowing the virtual memory addresses for those pages to be used” (Smith: Col. 3 lines 55-59).
Kim modified by Smith still does not teach a graphics memory allocation request for graphics memory; wherein the target task comprises multiple serial sub-tasks; nor the “multiple sub-tasks” in the limitation: “allocating multiple target graphics memory pools to the multiple sub-tasks”. However, Engh-Halstvedt teaches a graphics memory allocation request for graphics memory (Col. 4 lines 13-18 – “sending a request for memory space to be allocated for the vertex or group of vertices; and in response to the request from the testing stage, a memory allocator allocating memory space for storing vertex shaded attribute data for the vertex or group of vertices for which the request has been sent”; Note: a graphics memory allocation for graphics memory is made), and wherein the target task comprises multiple serial sub-tasks (Col. 34 lines 15-17 – “A graphics processing job (task) (e.g. defining a set of draw calls/sub tasks to be processed for a frame or render pass) is received at the control unit”; Note: a graphics processing job is the target task and it is comprised of sub-tasks, also called draw calls). Engh-Halstvedt further teaches allocating multiple target graphics memory pools (Col. 12 lines 37-44, 29 lines 21-29 – “a pool of memory space (a set of memory addresses) is set aside (made available) for storing the vertex data, and then when it is determined that memory space should be allocated for storing vertex shaded attributes data for a vertex or group of vertices, a portion of memory (a subset of the memory addresses) of the set aside memory space pool is allocated for storing the vertex data for the vertex or group of vertices…The graphics processing pipeline 20 also has access to suitable memory for storing data to be processed or generated by the various processing stages of the graphics processing pipeline 20. In particular, memory is allocated for: raw position data (at memory block 25), raw varyings data (at block 26), vertex-shaded (transformed) position data (at block 27), vertex-shaded (transformed) varyings data (at block 28), index data (at block 29) and primitive list data (at block 210)”; Note: memory pools are allocated to different sub-tasks in the graphics processing pipeline) to the multiple sub-tasks (Col. 34 lines 15-17 – “A graphics processing job (task) (e.g. defining a set of draw calls/sub tasks to be processed for a frame or render pass) is received at the control unit”; Note: a graphics processing job is the target task and it is comprised of sub-tasks, also called draw calls). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Kim to incorporate the teachings of Engh-Halstvedt to have a graphics memory allocation request for graphics memory because “there is a need to store geometry (and in particular vertex) data for a period of time for use in the later, deferred, rasterising and rendering pass” (Engh-Halstvedt: Col. 3 lines 4-7). Kim already teaches a write request, but it would be beneficial to allow graphics memory allocation requests for storing specific graphics processing operations. It also would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Kim to incorporate the teachings of Engh-Halstvedt to separate tasks into multiple sub-tasks and allocate memory to those sub-tasks because it would help organize memory usage based on the graphics processing pipeline and assist in distinguishing between data that will be used again or not used again in later sub-tasks (Engh-Halstvedt: Col. 5 lines 4-20). Distinguishing between useful and non-useful data would help in reducing memory waste.
While Kim modified by Smith and Engh-Halstvedt does not teach wherein a capacity of each target graphics memory pool of the multiple target graphics memory pools is determined using at least one forward inference, Gao teaches wherein a capacity of each target graphics memory pool of the multiple target graphics memory pools is determined using at least one forward inference (Paragraph 7 in 2nd Col. of Page 8, Paragraph 1 in 1st Col. of Page 9 – “Currently, the three DL frameworks do not provide stand-alone shape inference tools. DNNMem has already implemented our own using the framework APIs for establishing the operator memory cost functions. We query such cost functions for tensors of the initial input, weights, intermediate outputs, and final output under forward propagation, and then add them up as the GPU memory consumption estimated by Shape Inference”; Note: Potential memory consumption, which is the memory capacity required for a deep learning model, is determined using DNNMem, which contains inference techniques). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Kim to incorporate the teachings of Gao to determine memory capacity using forward inference because “since the GPU memory consumed by a DL model is often unknown to developers before the training or inferencing job starts running, an improper model configuration of neural architecture or hyperparameters can cause such a job to run out of the limited GPU memory and fail…Therefore, knowing the accurate GPU memory consumption (aka memory footprint) in advance is very important to reduce OOM failures and save precious platform resources including GPU/CPU/storage, by helping developers choose an optimal model configuration or facilitating DL frameworks to better utilize the mechanisms of dynamic memory management” (Gao: Paragraph 2-3 in 2nd Col. of Page 1). Because many AI models utilize graphics memory, it would be beneficial for graphics memory systems to monitor memory capacity in order to ensure successful AI model operations. But while Gao specifically advocates for knowing memory consumption before running a deep learning model, knowing the memory consumption for any task beforehand, not just deep learning models, is useful in preventing memory failures.
Finally, Kim modified by Smith, Engh-Halstvedt, and Gao still do not teach upon completion of a current sub-task, clearing graphics memory space corresponding to the same target physical memory address that stored the first data of the current sub-task, wherein the graphics memory space is available for storing the first data of a next sub-task. However, Gould teaches upon completion of a current sub-task, clearing graphics memory space corresponding to the same target physical memory address that stored the first data of the current sub-task (Paragraph 0040, 0050, 0064 – “the memory pool 82 may include virtual memory addresses that are at least partially allocated to physical memory in the graphics memory 58… GPU 12 or CPU 34 can manage the pool to allocate/deallocate physical resources when the respective thresholds are achieved…Executing the read operation can also optionally include, at 412, deallocating one or more pages of memory to the memory pool. In an example, GPU 12, e.g., via a compute shader or other thread of execution managing the read threads, can be configured to deallocate the one or more pages of memory from the memory pool (e.g., by using functionality provided by the FIFO queue manager 74). In one example, the GPU 12 can deallocate the one or more pages of memory based on a location of the read done pointer. For example, where the read done pointer achieves a threshold, which may correspond to the end of a page, the GPU 12 can deallocate the page (and optionally some number of preceding pages in memory), back to the memory pool (e.g., memory pool 82, which can add the page or pages (e.g., pointers thereto) to free page list 204)… For example, once all data on a page is consumed, the page may be freed to pool 82 immediately”; Note: after all the data in the page have been read, which indicates that the sub-task was completed, the memory space of the corresponding page is cleared) and wherein the graphics memory space is available for storing the first data of a next sub-task (Paragraph 0064 – “Freeing the memory back into the memory pool in this regard can allow for reuse of the memory for other FIFO queues such to conserve resources over previous memory management for executing graphics processes…For example, once all data on a page is consumed, the page may be freed to pool 82 immediately”; Note: the graphics memory is available for storing data of subsequent FIFO queues of graphics processes after it is freed). Since Kim already teaches clearing memory space (Paragraph 0054 – “when the host device 100 assigns a context identifier CTXIDi to metadata and thus an i.sup.th memory block BLK1i stores only metadata, the second memory block BLK1i may have a high invalidation degree due to the characteristics of metadata that is frequently updated, and may be processed at a low copy cost although the i.sup.th memory block BLK1i is selected as a victim region on a garbage collection operation”), it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Kim to incorporate the teachings of Gould to clear and make available the memory space after the running of a current sub-task ends for the benefit of allowing “for reuse of the memory for other FIFO queues such to conserve resources over previous memory management for executing graphics processes” (Gould: Paragraph 0064). In other words, the release of the memory page helps prevent memory waste.
Regarding claim 10, Kim in view of Smith, Engh-Halstvedt, Gao, and Gould teach the non-transitory, computer-readable medium of claim 9. Kim further teaches during running of each sub-task of multiple sub-tasks: storing the target data in a corresponding graphics memory block based on the type of the target data (Paragraph 0044, 0046 – “The memory cell array 225 may include a plurality of memory blocks BLK1 to BLKm. Each of the memory blocks BLK1 to BLKm may include a plurality of pages (e.g., P1 to Pn)…The processor 211 may match the respective context identifiers CTXID1 to CTXIDi, which are to be provided from the host device 100, with corresponding memory regions”; Note: the memory blocks store data based on their context identifiers, which is equivalent to the type of data. The graphics memory block was previously taught in the rejection of claim 9).
Regarding claim 11, Kim in view of Smith, Engh-Halstvedt, Gao, and Gould teach the non-transitory, computer-readable medium of claim 9. Kim further teaches wherein classifying target data generated during running of each sub-task of multiple sub-tasks, comprises: adding a type label to the target data based on the type of the target data, wherein the type label comprises at least a first label corresponding to the first data (Paragraph 0026, 0054 – “select a context identifier corresponding to relevant data among a plurality of context identifiers…The generated context identifiers are assigned to corresponding data… For example, when the host device 100 assigns a context identifier CTXID2 to journal data and thus a second memory block BLK12 stores only journal data, the second memory block BLK12 may have a high invalidation degree due to the characteristics of journal data that is frequently updated, and may be processed at a low copy cost”; Note: the context identifier is the equivalent to the type label. Context identifier CTXID2 for journal data is a first label corresponding to invalid data, which is the equivalent to first data. Journal data is invalid data because it has a high invalidation degree).
Regarding claim 12, Kim in view of Smith, Engh-Halstvedt, Gao, and Gould teach the non-transitory, computer-readable medium of claim 11. Kim further teaches wherein dividing each target memory pool of the multiple target memory pools into at least one memory block based on a type of the target data, comprises: dividing at least a part of memory space of each target memory pool into the first memory block (Paragraph 0044, 0046 – “The memory cell array 225 may include a plurality of memory blocks BLK1 to BLKm. Each of the memory blocks BLK1 to BLKm may include a plurality of pages (e.g., P1 to Pn)…The processor 211 may match the respective context identifiers CTXID1 to CTXIDi, which are to be provided from the host device 100, with corresponding memory regions”; Note: the memory cell array, which is divided into memory blocks, is equivalent to the memory pool. Additionally, the memory blocks contain data based on their context identifiers, which is equivalent to the type of data); and allocating the first memory block to the first data (Paragraph 0044, 0046, 0054 – “The memory cell array 225 may include a plurality of memory blocks BLK1 to BLKm…The processor 211 may match the respective context identifiers CTXID1 to CTXIDi, which are to be provided from the host device 100, with corresponding memory regions. For example, the processor 211 may match the context identifiers CTXID1 to CTXIDi with memory blocks, respectively… the second memory block BLK12 may have a high invalidation degree due to the characteristics of journal data that is frequently updated”; Note: memory blocks are allocated to data of specific context identifiers. A memory block is allocated to the first data, which is invalid data. While the reference refers to the memory block as a “second memory block”, in the context of the present claim, the second memory block of the reference is equivalent to the first memory block of the present claim). Kim does not teach mapping the multiple first graphics memory blocks to the same physical memory address. However, Smith teaches mapping the multiple graphics memory blocks to the same physical memory address (Col. 3 lines 31-35 – “When a particular page of the set of graphics data is determined as being similar to a particular stored page of the set of graphics data, a virtual memory address for the particular page is mapped to the same physical memory address as the particular stored page”; Note: a page of graphics data is the equivalent to the graphics memory block, and the virtual memory address for the page is mapped to the same physical memory address). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Kim to incorporate the teachings of Smith to map graphics memory blocks to a same physical memory address for the benefit of “an efficient use of memory since similar pages of a set of graphics data need not be retained in memory at multiple different memory locations” (Smith: Col. 3 lines 51-54).
Regarding claim 13, Kim in view of Smith, Engh-Halstvedt, Gao, and Gould teach the non-transitory, computer-readable medium of claim 12. Kim further teaches mapping based on a singleton pattern (Paragraph 0046, 0048 – “The processor 211 may match the respective context identifiers CTXID1 to CTXIDi, which are to be provided from the host device 100, with corresponding memory regions…the processor 211 may match the memory blocks BLK11 to BLK1i, in which write-requested data is to be stored, with the context identifiers CTXID1 to CTXIDi, respectively”; Note: mapping context identifiers with memory blocks is considered a singleton pattern, as memory blocks will have to go through this process before getting placed into virtual memory, and they follow a single pattern to be organized in memory). Kim does not teach wherein the multiple first graphics memory blocks are virtual graphics memories; and mapping the multiple first graphics memory blocks to the same target physical memory address, comprises: mapping the multiple first graphics memory blocks to multiple virtual graphics memories based on a singleton pattern; mapping the multiple virtual graphics memories to the same target physical memory address; and feeding back multiple virtual graphics memory pointers, wherein each virtual graphic memory pointer of the multiple virtual graphics memory pointers comprises a same target physical memory address and a capacity of the first graphics memory block corresponding to each virtual graphics memory pointer. However, Smith teaches wherein the multiple graphics memory blocks are virtual graphics memories (Col. 12 lines 50-57 – “those merged pages are later read from memory using respective virtual memory addresses for those pages… virtual memory address for the pages of the set of graphics data”; Note: pages, which are graphics memory blocks, are represented by virtual graphics memory); and mapping the multiple graphics memory blocks to the same physical memory address, comprises: mapping the multiple graphics memory blocks to multiple virtual graphics memories (Col. 12 lines 50-57 – “those merged pages are later read from memory using respective virtual memory addresses for those pages… virtual memory address for the pages of the set of graphics data”; Note: pages, which are graphics memory blocks, are mapped to virtual graphics memory addresses); mapping the multiple virtual graphics memories to the same physical memory address (Col. 3 lines 31-35 – “When a particular page of the set of graphics data is determined as being similar to a particular stored page of the set of graphics data, a virtual memory address for the particular page is mapped to the same physical memory address as the particular stored page”; Note: virtual memories, represented by virtual memory addresses, are mapped to the same physical memory address); and feeding back multiple virtual graphics memory pointers, wherein each virtual graphic memory pointer of the multiple virtual graphics memory pointers comprises a same physical memory address (Col. 13 lines 42-46, 55-59; Col. 14 lines 58-60 – “the virtual memory addresses within the page table may be unique, whereas at least some of the physical memory addresses (e.g. for merged pages) may not be unique…reading the page of the set of graphics data stored in the memory may comprise using a page table for the set of graphics data that maps virtual memory addresses for pages of the set of graphics data to physical memory addresses for pages of the set of graphics data…the virtual memory address for the particular page may be mapped to the same physical memory address as the particular stored page”; Note: virtual memory addresses are virtual graphics memory pointers, which can point to the same physical memory address). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Kim to incorporate the teachings of Smith to map graphics memory blocks to a same physical memory address for the benefit of “an efficient use of memory since similar pages of a set of graphics data need not be retained in memory at multiple different memory locations” (Smith: Col. 3 lines 51-54). Since the singleton pattern can be used for mapping memory blocks, it also would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Kim to incorporate the teachings of Smith to map graphics memory blocks to virtual memories using a singleton pattern for the benefit of organizing data into virtual memory so that data near each other have similar attributes, allowing for low-cost memory usage (Kim: Paragraph 0054, Smith: Col. 3 lines 51-54). Furthermore, Kim modified by Smith still does not teach wherein each virtual graphic memory pointer of the multiple virtual graphics memory pointers comprises a capacity of the graphics memory block corresponding to each virtual graphics memory pointer. However, Engh-Halstvedt teaches wherein each virtual graphic memory pointer of the multiple virtual graphics memory pointers comprises a capacity of the graphics memory block corresponding to each virtual graphics memory pointer (Col. 12 lines 37-44, Col. 13 lines 13-17, Col. 15 lines 10-12 – “a pool of memory space (a set of memory addresses) is set aside (made available) for storing the vertex data, and then when it is determined that memory space should be allocated for storing vertex shaded attributes data for a vertex or group of vertices, a portion of memory (a subset of the memory addresses) of the set aside memory space pool is allocated for storing the vertex data for the vertex or group of vertices… an amount of memory that is based on an expected amount of memory that may be required for storing vertex shaded data (for visible vertices) for the graphics processing output being generated is set aside as the memory space pool…the memory address for one end (e.g., and in an embodiment the current bottom allocation pointer) for the memory space pool is set”; Note: the memory addresses, which are pointers, correspond to a portion of memory and comprise a capacity of memory that is used to store vertices). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Kim to incorporate the teachings of Engh-Halstvedt to have the pointer comprise a capacity of a corresponding graphics memory block because pointers, by definition, allow access to locations in memory. Logically, it would follow that they correspond to a portion of memory, and the portion of memory has a capacity, as it cannot be infinite, or as Engh-Halstvedt suggests, only a certain amount is “set aside”.
Regarding claim 14, Kim in view of Smith, Engh-Halstvedt, Gao, and Gould teach the non-transitory, computer-readable medium of claim 12. Kim further teaches wherein the type of the target data comprises second data (Paragraph 0030 – “when pieces of data having a second attribute have a relatively low update frequency, a memory region in which pieces of data having the second attribute are classified and stored may have a relatively low invalidation degree”; Note: the second data type is the valid data, which is data with low invalidation degree); the type label comprises a second label corresponding to the second data (Paragraph 0056 – “the data storage device 200 may refer to a context identifier assigned to valid data of a victim memory block BLK, and store the valid data in a memory block corresponding to the referred context identifier”; Note: the valid data, which is the second data, has a corresponding context identifier, which is a label); the at least one memory block comprises a memory block corresponding to the second data (Paragraph 0056 – “the data storage device 200 may refer to a context identifier assigned to valid data of a victim memory block BLK, and store the valid data in a memory block corresponding to the referred context identifier”; Note: there is a second memory block containing valid data, which is the second data); and dividing each target memory pool of the multiple target memory pools into at least one memory block based on a type of the target data, comprises: dividing at least a part of memory space of each target memory pool into the second memory block (Paragraph 0044, 0046 – “The memory cell array 225 may include a plurality of memory blocks BLK1 to BLKm. Each of the memory blocks BLK1 to BLKm may include a plurality of pages (e.g., P1 to Pn)…The processor 211 may match the respective context identifiers CTXID1 to CTXIDi, which are to be provided from the host device 100, with corresponding memory regions”; Note: the memory cell array, which is divided into memory blocks, is equivalent to the memory pool. Additionally, the memory blocks contain data based on their context identifiers, which is equivalent to the type of data. There are multiple memory blocks, and thus there is a second memory block); and allocating the second memory block to the second data (Paragraph 0056 – “the data storage device 200 may refer to a context identifier assigned to valid data of a victim memory block BLK, and store the valid data in a memory block corresponding to the referred context identifier”; Note: the valid data, which is the second data, is stored in a memory block. This implies that the memory block is allocated to the valid data). Kim does not teach that the second data is used by a subsequent sub-task. However, Engh-Halstvedt teaches that the data is used by a subsequent sub-task (Col. 7 lines 16-19 – “the vertex belongs to (is to be used for) a primitive that could be visible in the output of the graphics processing operation”; Note: the vertex, which is data, is used by a subsequent sub-task since it would be needed for the output of the operation). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Kim to incorporate the teachings of Engh-Halstvedt to have the second data be used by a sub-sequent sub-task because distinguishing the required data ensures that “only a reduced amount of memory space for storing vertex shaded attribute data can be (and needs to be) allocated” and it would facilitate “the more efficient use of memory for storing vertex data” (Engh-Halstvedt: Col. 5 lines 12-20).
Regarding claim 17, Kim teaches a system (Paragraph 0007 – “a data processing system may include: a host device suitable for assigning a context identifier to data based on attribute information of the data; and a data storage device suitable for performing a garbage collection operation based on the assigned context identifier”) performing operations comprising:
in response to a write request for memory (Paragraph 0035 – “The processor 211 may control the write operation or read operation of the nonvolatile memory device 220 through a command in response to a write request or a read request from the host device 100. The processor 211 may drive a software program, on the memory 212, for controlling the operation of the data storage device 200”) needed during running of a target task (Paragraph 0022 – “The application unit 110 may generate, change or delete a file in the data storage device 200 when performing an assigned task”; Note: a request may be made when running a task), classifying target data (Paragraph 0026-0027 – “The identifier generator 140 may classify attribute information of data, and select a context identifier corresponding to relevant data among a plurality of context identifiers…When transmitting a write request WTRQ of data DT to the data storage device 200, the host device 100 may transmit a context identifier CTXID assigned to the data DT, together with the write request WTRQ”; Note: data is classified) generated during running of each task (Paragraph 0022, 0024 – “The application unit 110 may generate, change or delete a file in the data storage device 200 when performing an assigned task…The file system 120 may provide the identifier generator 140 with a file, i.e., attribute information of data, to be stored in the data storage device 200. For example, when certain data is stored in the data storage device 200, the file system 120 may determine the attributes of the corresponding data”; Note: the file system generates the attribute data, which occurs while the application unit runs a task. Kim only teaches tasks, but when modified as described below, teaches multiple sub-tasks), wherein the write request is generated (Paragraph 0035 – “The processor 211 may control the write operation or read operation of the nonvolatile memory device 220 through a command in response to a write request or a read request from the host device 100. The processor 211 may drive a software program, on the memory 212, for controlling the operation of the data storage device 200”) during the running of the target task (Paragraph 0022 – “The application unit 110 may generate, change or delete a file in the data storage device 200 when performing an assigned task”; Note: a request may be made when running a task), wherein a type of the target data comprises at least first data (Paragraph 0030 – “when pieces of data having mutually similar update frequencies are stored together in predetermined memory regions, memory regions may have a large deviation in their degree of invalidation. For example, when pieces of data having a first attribute have a relatively high update frequency, a memory region in which pieces of data having the first attribute are classified and stored may have a relatively high invalidation degree”; Note: invalid data is equivalent to the first data), and wherein the first data is not used by a subsequent task (Paragraph 0030 – “when pieces of data having mutually similar update frequencies are stored together in predetermined memory regions, memory regions may have a large deviation in their degree of invalidation. For example, when pieces of data having a first attribute have a relatively high update frequency, a memory region in which pieces of data having the first attribute are classified and stored may have a relatively high invalidation degree”; Note: invalid data is equivalent to the first data because invalid data is frequently replaced, meaning it is not reused);
dividing each target memory pool of the multiple target memory pools into at least one memory block based on a type of the target data (Paragraph 0044, 0046 – “The memory cell array 225 may include a plurality of memory blocks BLK1 to BLKm. Each of the memory blocks BLK1 to BLKm may include a plurality of pages (e.g., P1 to Pn)…The processor 211 may match the respective context identifiers CTXID1 to CTXIDi, which are to be provided from the host device 100, with corresponding memory regions”; Note: the memory cell array, which is divided into memory blocks, is equivalent to the memory pool. Additionally, the memory blocks contain data based on their context identifiers, which is equivalent to the type of data), and wherein the at least one memory block comprises at least a first memory block corresponding to the first data (Paragraph 0030 – “when pieces of data having mutually similar update frequencies are stored together in predetermined memory regions, memory regions may have a large deviation in their degree of invalidation”; Note: memory regions are the equivalent to memory blocks and memory regions may contain invalid data, which is equivalent to the first data, wherein the invalid data is frequently replaced, meaning it is not used for future tasks);
and clearing memory space (Paragraph 0054 – “when the host device 100 assigns a context identifier CTXIDi to metadata and thus an i.sup.th memory block BLK1i stores only metadata, the second memory block BLK1i may have a high invalidation degree due to the characteristics of metadata that is frequently updated, and may be processed at a low copy cost although the i.sup.th memory block BLK1i is selected as a victim region on a garbage collection operation”; Note: the garbage collection operation is the equivalent to clearing the memory space, and the victim region is the equivalent to the memory address corresponding to the first data) corresponding to the same target physical memory address that stored the first data (Paragraph 0036 – “The processor 211 may determine a memory region, in which write-requested data is to be stored, according to a context identifier assigned to the data, and map a physical address of the determined memory region to a logical address provided from the host device 100”; Note: the data has a corresponding physical address).
Kim does not teach a computer-implemented system, comprising: one or more computers; and one or more computer memory devices interoperably coupled with the one or more computers and having tangible, non-transitory, machine-readable media storing one or more instructions that, when executed by the one or more computers, perform one or more operations. Kim also does not teach wherein multiple first graphics memory blocks corresponding to the multiple sub-tasks are mapped to a same target physical memory address, wherein only one of the multiple first graphics memory blocks occupies the same target physical memory address at a time. However, Smith teaches a computer-implemented system (Col. 15 lines 39-41 – “The technology described herein can be implemented in any suitable system, such as a suitably configured computer or micro-processor based system”), comprising: one or more computers (Col. 16 lines 43-45 – “The technology described herein may accordingly suitably be embodied as a computer program product for use with a computer system”; Note: a computer system must comprise a computer in it for it to be considered a computer system); and one or more computer memory devices interoperably coupled with the one or more computers (Col. 15 lines 32-41 – “the graphics processing system comprises, and/or is in communication with, one or more memories and/or memory devices that store the data described herein, and/or store software for performing the processes described herein…The technology described herein can be implemented in any suitable system, such as a suitably configured computer or micro-processor based system”) and having tangible, non-transitory, machine-readable media storing one or more instructions that, when executed by the one or more computers, perform one or more operations (Col. 16 lines 46-52 – “a series of computer readable instructions either fixed on a tangible, non transitory medium, such as a computer readable medium, for example, diskette, CD, DVD, ROM, RAM, flash memory, or hard disk. It could also comprise a series of computer readable instructions transmittable to a computer system”). Smith also teaches wherein multiple first graphics memory blocks corresponding to the multiple tasks (Col. 14 lines 15-20 – “Some embodiments may comprise performing further graphics processing, e.g. a texture look up, compositing, etc., using a page of the set of graphics data read from the memory. Other embodiments may comprise outputting a page of the set of graphics data read from the memory, e.g. for display”; Note: the page of graphics data, which is equivalent to the graphics memory block, corresponds to the listed graphics processing tasks, which are equivalent to the sub-tasks) are mapped to a same target physical memory address (Col. 3 lines 31-35 – “When a particular page of the set of graphics data is determined as being similar to a particular stored page of the set of graphics data, a virtual memory address for the particular page is mapped to the same physical memory address as the particular stored page”; Note: a page of graphics data is the equivalent to the graphics memory block, and it gets mapped to the same physical memory address), wherein only one of the multiple first graphics memory blocks occupies the same target physical memory address at a time (Col. 3 lines 31-40, 62-65 – “When a particular page of the set of graphics data is determined as being similar to a particular stored page of the set of graphics data, a virtual memory address for the particular page is mapped to the same physical memory address as the particular stored page, e.g. rather than mapping the virtual memory address for the particular page to a unique physical memory address for the particular page. The particular page and the particular stored page may therefore have different virtual memory addresses that map to the same physical memory address…a single page of graphics data may be read (and, e.g., cached) from the single physical memory address that maps to the virtual memory addresses for the merged pages”; Note: only one page is stored in the physical memory address at a time). While Kim’s method applies to data in general, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Kim to incorporate the teachings of Smith to have a method of memory allocation apply specifically to graphics data because “in graphics processing systems, e.g. of portable devices, it is generally desirable to try to reduce the amount of memory required to store sets of graphics data and/or to try to increase efficiency, e.g. so as to reduce power consumption and/or increase processing speed, but without a corresponding undesirable reduction in the quality or resolution of the graphics data” (Smith: Col. 1 lines 45-51). It also would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Kim to incorporate the teachings of Smith to map graphics memory blocks to a same physical memory address for the benefit of “an efficient use of memory since similar pages of a set of graphics data need not be retained in memory at multiple different memory locations” (Smith: Col. 3 lines 51-54). Finally, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Kim to incorporate the teachings of Smith to have only one memory block/page in a physical memory address at a time because logically, physical memory is limited. To ensure efficient memory usage, it should only store a certain about of data at a time, and multiple virtual addresses can map to it instead. Doing so would “reduce the amount of memory bandwidth (and, e.g., the amount of cache) that is needed in order to read plural pages of similar graphics data from memory, whilst still allowing the virtual memory addresses for those pages to be used” (Smith: Col. 3 lines 55-59).
Kim modified by Smith still does not teach a graphics memory allocation request for graphics memory; wherein the target task comprises multiple serial sub-tasks; nor the “multiple sub-tasks” in the limitation: “allocating multiple target graphics memory pools to the multiple sub-tasks”. However, Engh-Halstvedt teaches a graphics memory allocation request for graphics memory (Col. 4 lines 13-18 – “sending a request for memory space to be allocated for the vertex or group of vertices; and in response to the request from the testing stage, a memory allocator allocating memory space for storing vertex shaded attribute data for the vertex or group of vertices for which the request has been sent”; Note: a graphics memory allocation for graphics memory is made), and wherein the target task comprises multiple serial sub-tasks (Col. 34 lines 15-17 – “A graphics processing job (task) (e.g. defining a set of draw calls/sub tasks to be processed for a frame or render pass) is received at the control unit”; Note: a graphics processing job is the target task and it is comprised of sub-tasks, also called draw calls). Engh-Halstvedt further teaches allocating multiple target graphics memory pools (Col. 12 lines 37-44, 29 lines 21-29 – “a pool of memory space (a set of memory addresses) is set aside (made available) for storing the vertex data, and then when it is determined that memory space should be allocated for storing vertex shaded attributes data for a vertex or group of vertices, a portion of memory (a subset of the memory addresses) of the set aside memory space pool is allocated for storing the vertex data for the vertex or group of vertices…The graphics processing pipeline 20 also has access to suitable memory for storing data to be processed or generated by the various processing stages of the graphics processing pipeline 20. In particular, memory is allocated for: raw position data (at memory block 25), raw varyings data (at block 26), vertex-shaded (transformed) position data (at block 27), vertex-shaded (transformed) varyings data (at block 28), index data (at block 29) and primitive list data (at block 210)”; Note: memory pools are allocated to different sub-tasks in the graphics processing pipeline) to the multiple sub-tasks (Col. 34 lines 15-17 – “A graphics processing job (task) (e.g. defining a set of draw calls/sub tasks to be processed for a frame or render pass) is received at the control unit”; Note: a graphics processing job is the target task and it is comprised of sub-tasks, also called draw calls). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Kim to incorporate the teachings of Engh-Halstvedt to have a graphics memory allocation request for graphics memory because “there is a need to store geometry (and in particular vertex) data for a period of time for use in the later, deferred, rasterising and rendering pass” (Engh-Halstvedt: Col. 3 lines 4-7). Kim already teaches a write request, but it would be beneficial to allow graphics memory allocation requests for storing specific graphics processing operations. It also would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Kim to incorporate the teachings of Engh-Halstvedt to separate tasks into multiple sub-tasks and allocate memory to those sub-tasks because it would help organize memory usage based on the graphics processing pipeline and assist in distinguishing between data that will be used again or not used again in later sub-tasks (Engh-Halstvedt: Col. 5 lines 4-20). Distinguishing between useful and non-useful data would help in reducing memory waste.
While Kim modified by Smith and Engh-Halstvedt does not teach wherein a capacity of each target graphics memory pool of the multiple target graphics memory pools is determined using at least one forward inference, Gao teaches wherein a capacity of each target graphics memory pool of the multiple target graphics memory pools is determined using at least one forward inference (Paragraph 7 in 2nd Col. of Page 8, Paragraph 1 in 1st Col. of Page 9 – “Currently, the three DL frameworks do not provide stand-alone shape inference tools. DNNMem has already implemented our own using the framework APIs for establishing the operator memory cost functions. We query such cost functions for tensors of the initial input, weights, intermediate outputs, and final output under forward propagation, and then add them up as the GPU memory consumption estimated by Shape Inference”; Note: Potential memory consumption, which is the memory capacity required for a deep learning model, is determined using DNNMem, which contains inference techniques). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Kim to incorporate the teachings of Gao to determine memory capacity using forward inference because “since the GPU memory consumed by a DL model is often unknown to developers before the training or inferencing job starts running, an improper model configuration of neural architecture or hyperparameters can cause such a job to run out of the limited GPU memory and fail…Therefore, knowing the accurate GPU memory consumption (aka memory footprint) in advance is very important to reduce OOM failures and save precious platform resources including GPU/CPU/storage, by helping developers choose an optimal model configuration or facilitating DL frameworks to better utilize the mechanisms of dynamic memory management” (Gao: Paragraph 2-3 in 2nd Col. of Page 1). Because many AI models utilize graphics memory, it would be beneficial for graphics memory systems to monitor memory capacity in order to ensure successful AI model operations. But while Gao specifically advocates for knowing memory consumption before running a deep learning model, knowing the memory consumption for any task beforehand, not just deep learning models, is useful in preventing memory failures.
Finally, Kim modified by Smith, Engh-Halstvedt, and Gao still do not teach upon completion of a current sub-task, clearing graphics memory space corresponding to the same target physical memory address that stored the first data of the current sub-task, wherein the graphics memory space is available for storing the first data of a next sub-task. However, Gould teaches upon completion of a current sub-task, clearing graphics memory space corresponding to the same target physical memory address that stored the first data of the current sub-task (Paragraph 0040, 0050, 0064 – “the memory pool 82 may include virtual memory addresses that are at least partially allocated to physical memory in the graphics memory 58… GPU 12 or CPU 34 can manage the pool to allocate/deallocate physical resources when the respective thresholds are achieved…Executing the read operation can also optionally include, at 412, deallocating one or more pages of memory to the memory pool. In an example, GPU 12, e.g., via a compute shader or other thread of execution managing the read threads, can be configured to deallocate the one or more pages of memory from the memory pool (e.g., by using functionality provided by the FIFO queue manager 74). In one example, the GPU 12 can deallocate the one or more pages of memory based on a location of the read done pointer. For example, where the read done pointer achieves a threshold, which may correspond to the end of a page, the GPU 12 can deallocate the page (and optionally some number of preceding pages in memory), back to the memory pool (e.g., memory pool 82, which can add the page or pages (e.g., pointers thereto) to free page list 204)… For example, once all data on a page is consumed, the page may be freed to pool 82 immediately”; Note: after all the data in the page have been read, which indicates that the sub-task was completed, the memory space of the corresponding page is cleared) and wherein the graphics memory space is available for storing the first data of a next sub-task (Paragraph 0064 – “Freeing the memory back into the memory pool in this regard can allow for reuse of the memory for other FIFO queues such to conserve resources over previous memory management for executing graphics processes…For example, once all data on a page is consumed, the page may be freed to pool 82 immediately”; Note: the graphics memory is available for storing data of subsequent FIFO queues of graphics processes after it is freed). Since Kim already teaches clearing memory space (Paragraph 0054 – “when the host device 100 assigns a context identifier CTXIDi to metadata and thus an i.sup.th memory block BLK1i stores only metadata, the second memory block BLK1i may have a high invalidation degree due to the characteristics of metadata that is frequently updated, and may be processed at a low copy cost although the i.sup.th memory block BLK1i is selected as a victim region on a garbage collection operation”), it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Kim to incorporate the teachings of Gould to clear and make available the memory space after the running of a current sub-task ends for the benefit of allowing “for reuse of the memory for other FIFO queues such to conserve resources over previous memory management for executing graphics processes” (Gould: Paragraph 0064). In other words, the release of the memory page helps prevent memory waste.
Regarding claim 18, Kim in view of Smith, Engh-Halstvedt, Gao, and Gould teach the computer-implemented system of claim 17. Kim further teaches during running of each sub-task of multiple sub-tasks: storing the target data in a corresponding graphics memory block based on the type of the target data (Paragraph 0044, 0046 – “The memory cell array 225 may include a plurality of memory blocks BLK1 to BLKm. Each of the memory blocks BLK1 to BLKm may include a plurality of pages (e.g., P1 to Pn)…The processor 211 may match the respective context identifiers CTXID1 to CTXIDi, which are to be provided from the host device 100, with corresponding memory regions”; Note: the memory blocks store data based on their context identifiers, which is equivalent to the type of data. The graphics memory block was previously taught in the rejection of claim 17).
Regarding claim 19, Kim in view of Smith, Engh-Halstvedt, Gao, and Gould teach the computer-implemented system of claim 17. Kim further teaches wherein classifying target data generated during running of each sub-task of multiple sub-tasks, comprises: adding a type label to the target data based on the type of the target data, wherein the type label comprises at least a first label corresponding to the first data (Paragraph 0026, 0054 – “select a context identifier corresponding to relevant data among a plurality of context identifiers…The generated context identifiers are assigned to corresponding data… For example, when the host device 100 assigns a context identifier CTXID2 to journal data and thus a second memory block BLK12 stores only journal data, the second memory block BLK12 may have a high invalidation degree due to the characteristics of journal data that is frequently updated, and may be processed at a low copy cost”; Note: the context identifier is the equivalent to the type label. Context identifier CTXID2 for journal data is a first label corresponding to invalid data, which is the equivalent to first data. Journal data is invalid data because it has a high invalidation degree).
Regarding claim 20, Kim in view of Smith, Engh-Halstvedt, Gao, and Gould teach the computer-implemented system of claim 19. Kim further teaches wherein dividing each target memory pool of the multiple target memory pools into at least one memory block based on a type of the target data, comprises: dividing at least a part of memory space of each target memory pool into the first memory block (Paragraph 0044, 0046 – “The memory cell array 225 may include a plurality of memory blocks BLK1 to BLKm. Each of the memory blocks BLK1 to BLKm may include a plurality of pages (e.g., P1 to Pn)…The processor 211 may match the respective context identifiers CTXID1 to CTXIDi, which are to be provided from the host device 100, with corresponding memory regions”; Note: the memory cell array, which is divided into memory blocks, is equivalent to the memory pool. Additionally, the memory blocks contain data based on their context identifiers, which is equivalent to the type of data); and allocating the first memory block to the first data (Paragraph 0044, 0046, 0054 – “The memory cell array 225 may include a plurality of memory blocks BLK1 to BLKm…The processor 211 may match the respective context identifiers CTXID1 to CTXIDi, which are to be provided from the host device 100, with corresponding memory regions. For example, the processor 211 may match the context identifiers CTXID1 to CTXIDi with memory blocks, respectively… the second memory block BLK12 may have a high invalidation degree due to the characteristics of journal data that is frequently updated”; Note: memory blocks are allocated to data of specific context identifiers. A memory block is allocated to the first data, which is invalid data. While the reference refers to the memory block as a “second memory block”, in the context of the present claim, the second memory block of the reference is equivalent to the first memory block of the present claim). Kim does not teach mapping the multiple first graphics memory blocks to the same physical memory address. However, Smith teaches mapping the multiple graphics memory blocks to the same physical memory address (Col. 3 lines 31-35 – “When a particular page of the set of graphics data is determined as being similar to a particular stored page of the set of graphics data, a virtual memory address for the particular page is mapped to the same physical memory address as the particular stored page”; Note: a page of graphics data is the equivalent to the graphics memory block, and the virtual memory address for the page is mapped to the same physical memory address). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Kim to incorporate the teachings of Smith to map graphics memory blocks to a same physical memory address for the benefit of “an efficient use of memory since similar pages of a set of graphics data need not be retained in memory at multiple different memory locations” (Smith: Col. 3 lines 51-54).
Claims 7-8 and 15-16 are rejected under 35 U.S.C. 103 as being unpatentable over Kim in view of Smith, Engh-Halstvedt, Gao, Gould, Inventor (CN 110458285 B), and Chen et al. (Training Deep Nets with Sublinear Memory Cost), hereinafter Chen.
Regarding claim 7, Kim in view of Smith, Engh-Halstvedt, Gao, and Gould teach the computer-implemented method of claim 6. Kim does not teach wherein the second data comprises input data, output data, and default data; the second label comprises: an input data label, corresponding to the input data; an output data label, corresponding to the output data; and a default data label, corresponding to the default data; and the second graphics memory block comprises: an input/output graphics memory block, corresponding to the input data and the output data; and a default graphics memory block, corresponding to the default data. However, Inventor teaches wherein the second data comprises input data, output data, and default data (Paragraph 0061 – “The operation node includes corresponding parameters, input data nodes and output data nodes (ie, the above-mentioned unclassified data nodes) connected to the operation node”; Note: the operation node is the second data, the input data node is the input data, the output data node is the output data, and the parameters are the default data); the second label comprises: an input data label, corresponding to the input data (Paragraph 0050 – “the data category may include any of the following: instruction, input neuron, output neuron, hidden neuron, constant neuron, input weight, output weight, constant weight, and auxiliary data”; Note: the data category is the equivalent to the label. Specifically, the data category of input neuron corresponds to the input data); an output data label, corresponding to the output data (Paragraph 0050 – “the data category may include any of the following: instruction, input neuron, output neuron, hidden neuron, constant neuron, input weight, output weight, constant weight, and auxiliary data”; Note: the data category is the equivalent to the label. Specifically, the data category of output neuron corresponds to the output data); and a default data label, corresponding to the default data (Paragraph 0050 – “the data category may include any of the following: instruction, input neuron, output neuron, hidden neuron, constant neuron, input weight, output weight, constant weight, and auxiliary data”; Note: the data category is the equivalent to the label. Specifically, the data category of constant weight corresponds to the default data). Since the method of Kim already has data with labels, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Kim to incorporate the teachings of Inventor to specifically have input, output, and default data with corresponding labels because categorizing data within a neural network may help with processing speed since there is a large amount of data and calculations involved (Inventor: Paragraph 0002). Kim modified by Inventor still does not teach that the second graphics memory block comprises: an input/output graphics memory block, corresponding to the input data and the output data; and a default graphics memory block, corresponding to the default data. However, Chen teaches an input/output memory block, corresponding to the input data and the output data (Page 3, Fig. 1 – The boxes “input” and “input-grad” are memory blocks for input data and output data respectively; see modified screenshot 1 below); and a default memory block, corresponding to the default data (Page 3, Fig. 1 – “we omit the weights and their output gradient nodes from the graph and assume that the gradient of weights are also calculated during backward operations”; Note: weights are equivalent to default data. Although the weights are omitted from the figure for clarity, it is implied that the memory blocks for the weights exist). Since the method of Kim modified by Smith already has graphics memory blocks for data, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Kim to incorporate the teachings of Chen to specifically have graphics memory blocks corresponding to the input, output, and default data for the benefit of being able to “assign the least amount of memory to these nodes by sharing memory when possible” (Chen: page 3). Therefore, it would be useful to group the certain types of data together so that memory can be shared, which allows for more efficient memory usage in neural networks.
PNG
media_image1.png
471
951
media_image1.png
Greyscale
Modified screenshot 1 (taken from Fig. 1 of Chen)
Regarding claim 8, Kim in view of Smith, Engh-Halstvedt, Gao, and Gould teach the computer-implemented method of claim 4. Kim does not teach wherein the first data comprises activation value data and workspace data; the first label comprises: an activation value data label, corresponding to the activation value data; and a workspace data label, corresponding to the workspace data; and the first graphics memory block comprises: an activation value graphics memory block, corresponding to the activation value data; and a workspace graphics memory block, corresponding to the workspace data. However, Inventor teaches the first label comprises: an activation value data label, corresponding to the activation value data (Paragraph 0050 – “the data category may include any of the following: instruction, input neuron, output neuron, hidden neuron, constant neuron, input weight, output weight, constant weight, and auxiliary data”; Note: the data category is the equivalent to the label. Specifically, the data category of hidden neuron corresponds to the activation value data); and a workspace data label, corresponding to the workspace data (Paragraph 0050 – “the data category may include any of the following: instruction, input neuron, output neuron, hidden neuron, constant neuron, input weight, output weight, constant weight, and auxiliary data”; Note: the data category is the equivalent to the label. Specifically, the data category of auxiliary data corresponds to the workspace data). Since the method of Kim already has data with labels, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Kim to incorporate the teachings of Inventor to specifically have activation value and workspace data with corresponding labels because categorizing data within a neural network may help with processing speed since there is a large amount of data and calculations involved (Inventor: Paragraph 0002). Kim modified by Inventor still does not teach wherein the first data comprises activation value data and workspace data; and the first graphics memory block comprises: an activation value graphics memory block, corresponding to the activation value data; and a workspace graphics memory block, corresponding to the workspace data. However, Chen teaches wherein the first data comprises activation value data and workspace data (Page 3, Fig. 1 – The boxes below the “input” and “input-grad” all contain activation value and workspace data; see modified screenshot 1 above); and the first memory block comprises: an activation value memory block, corresponding to the activation value data; and a workspace memory block, corresponding to the workspace data (Page 3, Fig. 1 – The boxes below the “input” and “input-grad” are all memory blocks containing activation value and workspace data; see modified screenshot 1 above). Since the method of Kim modified by Smith already has graphics memory blocks for data, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Kim to incorporate the teachings of Chen to specifically have graphics memory blocks corresponding to the activation value and workspace data for the benefit of being able to “assign the least amount of memory to these nodes by sharing memory when possible” (Chen: page 3). Therefore, it would be useful to group the certain types of data together so that memory can be shared, which allows for more efficient memory usage in neural networks.
Regarding claim 15, Kim in view of Smith, Engh-Halstvedt, Gao, and Gould teach the non-transitory, computer-readable medium of claim 14. Kim does not teach wherein the second data comprises input data, output data, and default data; the second label comprises: an input data label, corresponding to the input data; an output data label, corresponding to the output data; and a default data label, corresponding to the default data; and the second graphics memory block comprises: an input/output graphics memory block, corresponding to the input data and the output data; and a default graphics memory block, corresponding to the default data. However, Inventor teaches wherein the second data comprises input data, output data, and default data (Paragraph 0061 – “The operation node includes corresponding parameters, input data nodes and output data nodes (ie, the above-mentioned unclassified data nodes) connected to the operation node”; Note: the operation node is the second data, the input data node is the input data, the output data node is the output data, and the parameters are the default data); the second label comprises: an input data label, corresponding to the input data (Paragraph 0050 – “the data category may include any of the following: instruction, input neuron, output neuron, hidden neuron, constant neuron, input weight, output weight, constant weight, and auxiliary data”; Note: the data category is the equivalent to the label. Specifically, the data category of input neuron corresponds to the input data); an output data label, corresponding to the output data (Paragraph 0050 – “the data category may include any of the following: instruction, input neuron, output neuron, hidden neuron, constant neuron, input weight, output weight, constant weight, and auxiliary data”; Note: the data category is the equivalent to the label. Specifically, the data category of output neuron corresponds to the output data); and a default data label, corresponding to the default data (Paragraph 0050 – “the data category may include any of the following: instruction, input neuron, output neuron, hidden neuron, constant neuron, input weight, output weight, constant weight, and auxiliary data”; Note: the data category is the equivalent to the label. Specifically, the data category of constant weight corresponds to the default data). Since the method of Kim already has data with labels, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Kim to incorporate the teachings of Inventor to specifically have input, output, and default data with corresponding labels because categorizing data within a neural network may help with processing speed since there is a large amount of data and calculations involved (Inventor: Paragraph 0002). Kim modified by Inventor still does not teach that the second graphics memory block comprises: an input/output graphics memory block, corresponding to the input data and the output data; and a default graphics memory block, corresponding to the default data. However, Chen teaches an input/output memory block, corresponding to the input data and the output data (Page 3, Fig. 1 – The boxes “input” and “input-grad” are memory blocks for input data and output data respectively; see modified screenshot 1 above); and a default memory block, corresponding to the default data (Page 3, Fig. 1 – “we omit the weights and their output gradient nodes from the graph and assume that the gradient of weights are also calculated during backward operations”; Note: weights are equivalent to default data. Although the weights are omitted from the figure for clarity, it is implied that the memory blocks for the weights exist). Since the method of Kim modified by Smith already has graphics memory blocks for data, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Kim to incorporate the teachings of Chen to specifically have graphics memory blocks corresponding to the input, output, and default data for the benefit of being able to “assign the least amount of memory to these nodes by sharing memory when possible” (Chen: page 3). Therefore, it would be useful to group the certain types of data together so that memory can be shared, which allows for more efficient memory usage in neural networks.
Regarding claim 16, Kim in view of Smith, Engh-Halstvedt, Gao, and Gould teach the non-transitory, computer-readable medium of claim 12. Kim does not teach wherein the first data comprises activation value data and workspace data; the first label comprises: an activation value data label, corresponding to the activation value data; and a workspace data label, corresponding to the workspace data; and the first graphics memory block comprises: an activation value graphics memory block, corresponding to the activation value data; and a workspace graphics memory block, corresponding to the workspace data. However, Inventor teaches the first label comprises: an activation value data label, corresponding to the activation value data (Paragraph 0050 – “the data category may include any of the following: instruction, input neuron, output neuron, hidden neuron, constant neuron, input weight, output weight, constant weight, and auxiliary data”; Note: the data category is the equivalent to the label. Specifically, the data category of hidden neuron corresponds to the activation value data); and a workspace data label, corresponding to the workspace data (Paragraph 0050 – “the data category may include any of the following: instruction, input neuron, output neuron, hidden neuron, constant neuron, input weight, output weight, constant weight, and auxiliary data”; Note: the data category is the equivalent to the label. Specifically, the data category of auxiliary data corresponds to the workspace data). Since the method of Kim already has data with labels, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Kim to incorporate the teachings of Inventor to specifically have activation value and workspace data with corresponding labels because categorizing data within a neural network may help with processing speed since there is a large amount of data and calculations involved (Inventor: Paragraph 0002). Kim modified by Inventor still does not teach wherein the first data comprises activation value data and workspace data; and the first graphics memory block comprises: an activation value graphics memory block, corresponding to the activation value data; and a workspace graphics memory block, corresponding to the workspace data. However, Chen teaches wherein the first data comprises activation value data and workspace data (Page 3, Fig. 1 – The boxes below the “input” and “input-grad” all contain activation value and workspace data; see modified screenshot 1 above); and the first memory block comprises: an activation value memory block, corresponding to the activation value data; and a workspace memory block, corresponding to the workspace data (Page 3, Fig. 1 – The boxes below the “input” and “input-grad” are all memory blocks containing activation value and workspace data; see modified screenshot 1 above). Since the method of Kim modified by Smith already has graphics memory blocks for data, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Kim to incorporate the teachings of Chen to specifically have graphics memory blocks corresponding to the activation value and workspace data for the benefit of being able to “assign the least amount of memory to these nodes by sharing memory when possible” (Chen: page 3). Therefore, it would be useful to group the certain types of data together so that memory can be shared, which allows for more efficient memory usage in neural networks.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Gutierrez et al. (US 20200098082 A1) teaches a method of memory management involving modifying memory allocation for kernels. Moreton et al. (US 6950107 B1) teaches a method of memory management involving allocating graphics memory space based on thread type.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MICHELLE HAU MA whose telephone number is (571)272-2187. The examiner can normally be reached M-Th 7-5:30.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, King Poon can be reached on (571) 270-0728. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MICHELLE HAU MA/Examiner, Art Unit 2617 /KING Y POON/Supervisory Patent Examiner, Art Unit 2617