DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
The Examiner acknowledges the present application claims the benefit under 35 U.S.C. 119(a)-(d) of Korean Patent Application No. 10-2023-0015834, filed 02/06/2023.
This action is responsive to the Applicant’s amendments filed on 04/02/2026. Claims 1-23 remain pending in the application. Claims 19 and 21 have been amended. Claims 24-34 have been cancelled. Claims 35-42 have been newly added. Any examiner’s note, objection, and rejection not repeated is withdrawn due to Applicant’s amendment.
Priority
Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55.
Information Disclosure Statement
The information disclosure statements (IDS) submitted on 07/27/2023 and 09/04/2024 are in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statements are being considered by the examiner.
Examiner’s Note
The Examiner cites particular columns, paragraphs, figures, and line numbers in the references as applied to the claims below for the convenience of the applicant. Although the specified citations are representative of the teachings in the art and are applied to the specific limitations within the individual claim, other passages and figures may also apply. It is respectfully requested that, in preparing responses, the Applicant fully consider the references in its entirety as potentially teaching all or part of the claimed invention, as well as the context of the passage as taught by the prior art or disclosed by the Examiner.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitations uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitations are:
“a device controller configured to…” (Claim 1);
“a processing engine configured to…” (Claim 1);
“wherein the processing engine is configured to…” (Claim 2);
“further comprising a processing near-memory (PNM) engine configured to…” (Claim 2);
“a processing near-memory (PNM) engine configured to…” (Claim 3);
“a processing in-memory (PIM) engine configured to…” (Claim 3);
“wherein the device controller is configured to: generate a first level instruction…” (Claim 4);
“wherein the device controller is configured to: generate a second level instruction…” (Claim 4);
“the device controller is configured to…” (Claim 6);
“the processing engine is configured to…” (Claim 6);
“an instruction generator configured to…” (Claim 12);
“wherein the instruction generator comprises a predefined instruction table and is configured to…” (Claim 13);
“wherein the instruction generator is configured to…” (Claim 14);
“a logic die in which a processing near-memory (PNM) engine configured to…” (Claim 17);
“wherein an additional PNM engine configured to…” (Claim 18);
“the PIM engine configured to generate a PIM operation result…” (Claim 19);
“wherein the device controller is configured to: receive…” (Claim 20);
“wherein the device controller is configured to: transmit…” (Claim 20);
“the processing engine is configured to…” (Claim 21);
“a processing near-memory (PNM) engine configured to…” (Claim 22).
Because these claim limitations are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, they are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have these limitations interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claim 1-23 are rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention.
Regarding claims 1, 4, 6, and 20, the claims recite a “device controller” configured to: “generate a sub-processing instruction based on a host processing instruction received from a host” (Claim 1), “generate a first level instruction as the sub-processing instruction from the host processing instruction and transmit the first level instruction to the PNM engine; and generate a second level instruction from the first level instruction and transmit the second level instruction to the PIM engine” (Claim 4), “receive, from the host, a plurality of host processing instructions comprising the host processing instruction in a first order” (Claim 6), “receive the host processing instruction from the host; and transmit, to the host, a final operation result according to a series of operations corresponding to the host processing instruction” (Claim 20). The instant specification discloses non-limiting steps the device controller may take; e.g. “device controller 123 may process a host instruction received from a host”, “device controller 123 may produce a sub-processing instruction based on the host processing instruction received from the host”, “device controller 123 may produce a sub-processing instruction to be used for an operation of the processing engine 121 based on the received host processing instruction” (Paragraphs 65-70). The term “device controller” is interpreted under 35 U.S.C. 112(f) because the device controller does not, by itself, recite sufficient structure or algorithm to perform the functionality of controlling a device, thereby corresponding to a generic placeholder that does not connote a definite structure to a person of ordinary skill in the art. There is no use of “means for” language, and the claim language is purely functional. Although the instant specification provides examples of operations to be performed by the device controller, these are explicitly presented as possible embodiments and do not define a concrete implementation of the device controller. Therefore, the disclosure does not provide sufficient description of the specific structure used to implement the device controller nor the specific algorithm(s) used to generate a sub-processing instruction, generate first/second level instructions, receive host processing instructions, or transmit final operation results, beyond the unbounded functional language of “device controller 123 may…” (Paragraphs 65-70).
Regarding claims 1-2, 6, and 21, the claims recite a “processing engine”, “first processing engine”, or “second processing engine” configured to: “perform an operation based on the generated sub-processing instruction” (Claim 1), “generate an intermediate result by performing the operation according to the generated sub-processing instruction” (Claim 2), “perform operations by processing the host processing instructions in a second order that is different from the first order and is determined based on access addresses of the host processing instructions” (Claim 6), “read the weight elements from a memory block based on the address, and generate a partial operation result based on the read the weight elements and the input fragments” (Claim 21). The instant specification discloses non-limiting examples of what the processing engine 121 is capable of, including “processing engine 121 may perform an operation using data stored in a memory under the control of the device controller 123” (Paragraphs 69-70). Further, “the classification of the processing engines is not limited to the above description” (Paragraph 87). The term “processing engine” is interpreted under 35 U.S.C. 112(f) because the processing engine does not, by itself, recite sufficient structure or algorithm to perform the claimed functionality, thereby corresponding to a generic placeholder that does not connote a definite structure to a person of ordinary skill in the art. There is no use of “means for” language, and the claim language is purely functional. Although the instant specification provides examples of operations to be performed by the processing engine, these are explicitly presented as possible embodiments and do not define a concrete implementation of the processing engine. The disclosure does not provide sufficient description of the specific structure used to implement the processing engine nor the specific algorithm(s) used to perform an operation based on the generated instruction, perform operations by processing instructions in a different order based on access addresses, read weight elements from a memory block and generate a partial operation result, perform an operation using data in memory, generate a partial result based on each respective sub-processing instruction, or generate an operation result by processing the generated partial results, beyond the unbound functional language recited in the instant specification.
Regarding claims 2-3, 17-18, and 22, the claims recite a “processing near-memory (PNM) engine” configured to: “in response to obtaining the generated intermediate result, generate an operation result of the operation by processing the generated intermediate result” (Claim 2); “perform a PNM operation according to the generated sub-processing instruction” (Claim 3); “generate an operation result” (Claim 17); “receive a PIM operation result and generate an intermediate result” (Claim 18); “generate a final output vector based on a plurality of partial operation results including the partial operation result” (Claim 22). The instant specification discloses non-limiting examples of what the PNM engine 211 is capable of performing, “PNM engine 211 may perform an operation based on a processing instruction (e.g., the host processing instruction) using data of the memory block 220”, “PNM engine 211 may generate a final computing result by processing (e.g., summing or adding) partial results output from the first level processing engines 212 of the low level” (Paragraphs 80-82). Further, “the classification of the processing engines is not limited to the above description” (Paragraph 87). The term “PNM engine” is interpreted under 35 U.S.C. 112(f) because the PNM engine does not, by itself, recite sufficient structure or algorithm to perform the claimed functionality, thereby corresponding to a generic placeholder that does not connote a definite structure to a person of ordinary skill in the art. There is no use of “means for” language, and the claim language is purely functional. Although the instant specification provides examples of operations to be performed by the PNM engine, these are explicitly presented as possible embodiments and do not define a concrete implementation of the PNM engine. The disclosure does not provide sufficient description of the specific structure used to implement the PNM engine nor the specific algorithm(s) used to generate an operation result by processing the generated intermediate result, perform a PNM operation according to the generated sub-processing instruction, generate an operation result, or generate a final output vector, beyond the unbound functional language recited in the instant specification.
Regarding claims 3 and 19, the claims recite a “processing in-memory (PIM) engine” configured to: “perform a PIM operation according to the generated sub-processing instruction” (Claim 3); “generate a PIM operation result” (Claim 19). The instant specification discloses non-limiting examples of what the PIM engine is capable of performing, “PIM engine (e.g., the first level processing engine 212) may perform a simple operation indicated by the sub-processing instruction produced by the device controller 230. The PIM engine may perform a PIM operation according to the produced sub-processing instruction” (Paragraph 85). Further, “the classification of the processing engines is not limited to the above description” (Paragraph 87). The term “PIM engine” is interpreted under 35 U.S.C. 112(f) because the PIM engine does not, by itself, recite sufficient structure or algorithm to perform the claimed functionality, thereby corresponding to a generic placeholder that does not connote a definite structure to a person of ordinary skill in the art. There is no use of “means for” language, and the claim language is purely functional. Although the instant specification provides examples of operations to be performed by the PIM engine, these are explicitly presented as possible embodiments and do not define a concrete implementation of the PIM engine. The disclosure does not provide sufficient description of the specific structure used to implement the PIM engine nor the specific algorithm(s) used to perform a PIM operation according to a generated sub-processing instruction or generate a PIM operation result, beyond the unbound functional language recited in the instant specification.
Regarding claims 12-14, the claims recite an “instruction generator” configured to: “generate the sub-processing instruction from the received host processing instruction, and transmit the generated sub-processing instruction to a memory scheduler” (Claim 12); “generate the sub-processing instruction according to a result of matching between the instruction table and the host processing instruction” (Claim 13); “in response to the host processing instruction being either one of a sparse lengths sum (SLS) operation and a multiplication and accumulation (MAC) operation, generate the sub-processing instruction for the corresponding either one of the SLS operation and the MAC operation” (Claim 14). The instant specification discloses non-limiting examples of what the instruction generator may perform, “instruction generator may include a predefined instruction table and may be configured to have generate the sub-processing instruction according to a result of matching between the instruction table and the host processing instruction”, “instruction generator may be configured to, in response to the host processing instruction being either one of a sparse lengths sum (SLS) operation and a multiplication and accumulation (MAC) operation, generate the sub-processing instruction for the corresponding either one of the SLS operation and the MAC operation (Paragraphs 17-18). The term “instruction generator” is interpreted under 35 U.S.C. 112(f) because the instruction generator does not, by itself, recite sufficient structure or algorithm to perform the claimed functionality, thereby corresponding to a generic placeholder that does not connote a definite structure to a person of ordinary skill in the art. There is no use of “means for” language, and the claim language is purely functional. Although the instant specification provides examples of operations to be performed by the instruction generator, these are explicitly presented as possible embodiments and do not define a concrete implementation of the instruction generator. However, the disclosure does not provide sufficient description of the specific structure used to implement the instruction generator nor the specific algorithm(s) used to generate the sub-processing instruction from the received host processing instruction, and transmit the generated sub-processing instruction to a memory scheduler, generate the sub-processing instruction according to a result of matching between the instruction table and the host processing instruction, in response to the host processing instruction being either one of a sparse lengths sum (SLS) operation and a multiplication and accumulation (MAC) operation, generate the sub-processing instruction for the corresponding either one of the SLS operation and the MAC operation, beyond the unbound functional language recited in the instant specification.
Any claim not explicitly mentioned is rejected due to dependency on a rejected claim.
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-23 are rejected under 35 U.S.C. 112(b) or pre-AIA 35 U.S.C. 112, second paragraph. Claims 1-23 limitations of “device controller, processing engine, PNM engine, PIM engine, and instruction generator” configured to perform an action invokes 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. However, the written description fails to disclose the corresponding structure, material, or acts for performing the entire claimed function and to clearly link the structure, material, or acts to the function.
The terms “device controller, processing engine, PNM engine, PIM engine, and instruction generator” are nonce words that do not connote specific structure to a person of ordinary skill in the art. In the absence of clear structural limitations or well-defined corresponding structure in the specification, a person of ordinary skill in the art is unable to determine the metes and bounds of the claims. Additionally, because the instant specification only identifies non-limiting examples of the aforementioned terms; i.e. “the classification of the processing engines is not limited to the above description” (Paragraph 87), “device controller may include a memory scheduler configured to schedule access to memory blocks by processing the host processing instructions in an in-order mode, in response to the host processing instruction being requested from the host” (Paragraphs 19-20, 95); it is unclear what specific structure or algorithm is intended to be encompassed by the claims interpreted under 35 U.S.C. 112(f).
Therefore, the claims are indefinite and is rejected under 35 U.S.C. 112(b) or pre-AIA 35 U.S.C. 112, second paragraph.
Any claim not explicitly mentioned is rejected due to dependency on a rejected claim.
Applicant may:
(a) Amend the claim so that the claim limitation will no longer be interpreted as a limitation under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph;
(b) Amend the written description of the specification such that it expressly recites what structure, material, or acts perform the entire claimed function, without introducing any new matter (35 U.S.C. 132(a)); or
(c) Amend the written description of the specification such that it clearly links the structure, material, or acts disclosed therein to the function recited in the claim, without introducing any new matter (35 U.S.C. 132(a)).
If applicant is of the opinion that the written description of the specification already implicitly or inherently discloses the corresponding structure, material, or acts and clearly links them to the function so that one of ordinary skill in the art would recognize what structure, material, or acts perform the claimed function, applicant should clarify the record by either:
(a) Amending the written description of the specification such that it expressly recites the corresponding structure, material, or acts for performing the claimed function and clearly links or associates the structure, material, or acts to the claimed function, without introducing any new matter (35 U.S.C. 132(a)); or
(b) Stating on the record what the corresponding structure, material, or acts, which are implicitly or inherently set forth in the written description of the specification, perform the claimed function. For more information, see 37 CFR 1.75(d) and MPEP §§ 608.01(o) and 2181.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1, 3, 6, 19-20, 23, 35-36, 38-39, and 42 are rejected under 35 U.S.C. 102(a)(1) and 102(a)(2) as being anticipated by Guz et al. (US 20160098200 A1) hereafter Guz.
Regarding claim 1, Guz teaches:
A memory device (Paragraph 8; “the present disclosure is directed to a memory module”) comprising:
a device controller (Paragraph 58; controller 97) configured to generate a sub-processing instruction (Paragraph 58; “bitcount operations for a vector that spans multiple DRAM pages may require multiple steps. In case of the DRAM page size of 1 KB, each step may calculate the bitcount for a 1KB subset of the vector and partial results may be accumulated in a special register within the memory module 12” and Paragraph 80; “implementation of popcounting in the context of FIGS. 4-7 and FIG. 9”, “The controller 97 may then perform the bitwise operation on each group of aligned segments to thereby generate a plurality of partial results”), based on a host processing instruction received from a host (Paragraph 6; “receiving at a memory module an instruction from a host to perform a POPCOUNT operation”);
and a processing engine (Paragraph 58; controller 97) configured to perform an operation based on the generated sub-processing instruction (Paragraph 7; “(i) receiving at a memory module an instruction from a host to perform a logical bitwise operation on two or more bit vectors stored in the memory module; and (ii) executing the logical bitwise operation within the memory module, without transferring the bit vectors to the host for the execution.”, where the memory module corresponds to the applicant’s processing engine).
Claim 36 recites similar limitations as those of claim 1, directed towards an apparatus, additionally reciting one or more controllers comprising one or more logic circuits, without reciting the processing engine. Guz teaches:
one or more controllers comprising one or more logic circuits (Paragraph 91; “the controller or the host processor may include, for example, a general purpose processor, a special purpose processor, a conventional processor, a digital signal processor (DSP), a plurality of microprocessors, one or more microprocessors in association with a DSP core, a microcontroller, Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs) circuits, any other type of integrated circuit (IC), and/or a state machine. Some or all of the functionalities described above in the context of FIGS. 2-4 may be provided in the hardware.”, where an ASIC is a type of logic circuit.).
Claim 36 is rejected for similar reasons as those of claim 1.
Regarding claim 3, Guz teaches the memory device of claim 1. Guz teaches:
wherein the processing engine comprises either one or both of:
a processing near-memory (PNM) engine (Paragraph 80; PIM controller 97) configured to perform a PNM operation according to the generated sub-processing instruction; and a processing in-memory (PIM) engine configured to perform a PIM operation according to the generated sub-processing instruction (Paragraph 27; “in case of the POPCOUNT operation, a Processing-In-Memory (PIM) model according to the teachings of the present disclosure may provide for the following additions to the memory module”, where Paragraph 80 further discloses “The controller 97 may then perform the bitwise operation on each group of aligned segments to thereby generate a plurality of partial results. As before, the partial results may be stored in the PIMResultRegister 99”. The PIM controller, called controller 97, corresponds to the PIM engine that is configured to perform PIM operations on the segmented operations, corresponding to according to the generated sub-processing instruction).
Regarding claim 6, Guz teaches the memory device of claim 1. Guz teaches:
wherein the device controller (Paragraph 58; controller 97) is configured to receive, from the host, a plurality of host processing instructions comprising the host processing instruction in a first order (Paragraph 67; “The PIM Controller 97 may perform a number of tasks to facilitate the in-memory processing discussed herein. Such tasks may include, for example, providing an interface (API) to enable the host 14 to initiate commands and fetch data, interacting with the host 14”, where the API and interaction with the host allow the device controller to receive multiple host initiated commands which are issued by the host in an original host-defined order.);
and the processing engine (Paragraph 80; controller 97) is configured to perform operations by processing the host processing instructions in a second order that is different from the first order (Paragraph 67; “controlling the implemented reduction tree 49 or 50, controlling the operation of the logic structure 78 or 87, handling computation of popcounts over vectors spanning multiple memory pages, accumulating intermediate results of bit-counting in the PIMResultRegister 99” where the controller schedules and controls internal execution of operations independently of the order in which host commands are issued, thereby reordering execution.), and is determined based on access addresses of the host processing instructions (Paragraph 67; “handling computation of popcounts over vectors spanning multiple memory pages” where processing is organized based on memory space boundaries and accessed memory locations, meaning the execution order is determined based on the access addresses associated with the host processing instructions.).
Regarding claim 19, Guz teaches the memory device of claim 1. Guz teaches:
a memory die in which a memory block (Paragraph 27; memory module acts as the corresponding structure) configured to store data and a processing in-memory (PIM) engine of the processing engine are disposed (Paragraph 31; “the reference numeral “12” also refers to a packaging or housing of the memory module. The packaging 12 may include one or more memory chips (not shown), such as, for example, DRAM (or other semiconductor-based memory) chips”; Paragraph 29; “computations related to the popcount and other logical bitwise operations are implemented/executed within a memory module, without shifting the data throughout the entire system. In particular embodiments, the memory module may be a Dynamic Random Access Memory (DRAM) based Three Dimensional Stack (3DS) memory module”, where the PIM model places processing capability in association with the memory, corresponding to being disposed with the memory die), wherein the PIM engine (Paragraph 80; controller 97) is configured to generate a PIM operation result (Paragraph 27; “(i) A hardware-based reduction tree that may calculate the popcount for (up to) a full Dynamic Random Access Memory (DRAM) page at a time. (ii) A hardware logic that may enable traversing vectors of sizes different than one DRAM page. (iii) A special register—referred to herein as “PIMResultRegister”—that may store the final result (i.e., the popcount value) and may be used by the memory's host to retrieve the final popcount value”, where the in-memory processing logic performs PIM operations on data stored in the memory block and from it, generates a PIM operation result).
Regarding claim 20, Guz teaches the memory device of claim 1. Guz teaches:
receive the host processing instruction from the host (Paragraph 80; “PIM Controller 97 may initially receive from the host 14 the physical addresses of memory locations (like the memory cells 79 shown in FIGS. 8A-8B) in the memory module 12 where the respective bit vectors are stored, and then store each received physical address at a PIM-specific address (or storage location) within the memory module 12”, where the PIM controller receives instructions from the host specifying memory locations and operations to perform);
and transmit, to the host, a final operation result according to a series of operations corresponding to the host processing instruction (Paragraphs 78-80; “controller 97 may divide each bit vector into a plurality of bit vector-specific non-overlapping segments and then align corresponding bit vector-specific segments from all bit vectors into a plurality of groups of aligned segments. The controller 97 may then perform the bitwise operation on each group of aligned segments to thereby generate a plurality of partial results. As before, the partial results may be stored in the PIMResultRegister 99. The controller 97 may subsequently combine all partial results in the register 99 to effectuate the execution of the logical bitwise operation. The final outcome from the combination may be initially stored in the PIMResultRegister 99 before eventually storing it at a pre-defined storage location (or PIM-specific address) for future submission to the host 14. The host 14 may access this storage location to read the result”, where the PIM controller performs a series of segmented operations corresponding to the host instruction, stores the final result, and is submitted to the host, corresponding to transmitting the result to the host).
Regarding claim 23, Guz teaches the memory device of claim 1. Guz teaches:
An electronic device (Paragraph 83; “Some examples of the system 10 include a computer system”) comprising:
the memory device of claim 1 (Paragraph 8; “the present disclosure is directed to a memory module”);
and the host, wherein the host is a host processor (Paragraph 33; “In one embodiment, the host 14 may be a CPU, which can be a general purpose microprocessor.”).
Regarding claim 35, Guz teaches:
A processor-implemented method, the method comprising:
receiving, by a device controller of the memory device, a host processing instruction from a host (Paragraph 48; “the memory module 12 may initiate the POPCOUNT operation in response to an instruction from the host to perform the popcounting”, corresponding to receiving a host processing instruction from the host. The device controller of the memory device is further disclosed in Paragraph 67; “PIM Controller 97 may perform a number of tasks... providing an interface (API) to enable the host 14 to initiate commands and fetch data, interacting with the host 14”, teaching the device controller that interfaces with the host and facilitates execution of the host instruction. Paragraph 8 confirms that the invention is performed in a memory device; “the present disclosure is directed to a memory module”, the memory module corresponding to a memory device.);
generating a sub-processing instruction based on the received host processing instruction (Paragraph 48; “memory module 12 may initiate the POPCOUNT operation in response to an instruction from the host 14”, disclosing a host instruction that riggers initiation of a memory operation where the host-requested POPCOUNT operation is broken up into multiple stages within the memory module, the generation of sub-processing instructions further disclosed in Paragraph 80; “controller 97 may initially receive from the host 14 the physical addresses of memory locations... controller 97 may access the appropriate storage location to obtain the corresponding physical address and then retrieve the respective bit vector-specific non-overlapping segments”, where based on the received host instruction and associated data, the controller generates sub processing actions by decomposing the host-requested operation into segments executed in the memory module, establishing that a host instruction initiates a higher level operation, POPCOUNT, and in response, generates sub processing operations that implement the higher operation.);
and performing an operation based on the generated sub-processing instruction (Paragraph 80; “controller 97 may divide each bit vector into a plurality of bit vector-specific non-overlapping segments”, and “may then perform the bitwise operation on each group of aligned segments to thereby generate a plurality of partial results”, which are then combined to generate a final result, corresponding to performance of an operation, the POPCOUNT operation, by performing the sub-processing instructions and generating partial results, and combining the results into a final result.).
Regarding claim 38, Guz teaches the apparatus of claim 36. Guz teaches:
generate a first level instruction as the sub-processing instruction (Paragraph 80; “in case of a logical bitwise operation on multiple vectors, the PIM Controller 97 may initially receive from the host 14 the physical addresses of memory locations (like the memory cells 79 shown in FIGS. 8A-8B) in the memory module 12 where the respective bit vectors are stored” where the division of vectors and performing operations on aligned segments that yield partial results corresponds to a sub-processing instruction operating on a portion of the overall task) from the host processing instruction (Paragraph 67; “providing an interface (API) to enable the host 14 to initiate commands”, where the host-initiated command defines the overall processing operation from which the segment level sub-processing instructions are derived);
and generate a second level instruction from the first level instruction (Paragraph 80; “the partial results may be stored in the PIMResultRegister 99… [and] subsequently combine all partial results” where instructions to accumulate and combine partial results are derived from the earlier segment-level sub-processing instructions).
Regarding claim 39, Guz teaches the apparatus of claim 36. Guz teaches:
wherein the device controller (Paragraph 58; controller 97) is configured to receive, from the host, a plurality of host processing instructions in a first order (Paragraph 67; “The PIM Controller 97 may perform a number of tasks to facilitate the in-memory processing discussed herein. Such tasks may include, for example, providing an interface (API) to enable the host 14 to initiate commands and fetch data, interacting with the host 14”, where the API and interaction with the host allow the device controller to receive multiple host initiated commands which are issued by the host in an first order.);
and perform operations by processing the host processing instructions in a second order that is different from the first order (Paragraph 67; “controlling the implemented reduction tree 49 or 50, controlling the operation of the logic structure 78 or 87, handling computation of popcounts over vectors spanning multiple memory pages, accumulating intermediate results of bit-counting in the PIMResultRegister 99” where the controller schedules and controls internal execution of operations independently of the order in which host commands are issued, thereby reordering execution.), and is determined based on access addresses of the host processing instructions (Paragraph 67; “handling computation of popcounts over vectors spanning multiple memory pages” where processing is organized based on memory space boundaries and accessed memory locations, meaning the execution order is determined based on the access addresses associated with the host processing instructions.).
Regarding claim 42, Guz teaches:
A memory device (Paragraph 8; “the present disclosure is directed to a memory module”), comprising: one or more controllers, comprising one or more logic circuits (Paragraph 91; “the controller or the host processor may include, for example, a general purpose processor, a special purpose processor, a conventional processor, a digital signal processor (DSP), a plurality of microprocessors, one or more microprocessors in association with a DSP core, a microcontroller, Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs) circuits, any other type of integrated circuit (IC), and/or a state machine. Some or all of the functionalities described above in the context of FIGS. 2-4 may be provided in the hardware.”, where an ASIC is a type of logic circuit.), configured to:
generate sub-processing instructions based on a host processing instruction received from a host (Paragraph 48; “memory module 12 may initiate the POPCOUNT operation in response to an instruction from the host 14”, disclosing a host instruction that riggers initiation of a memory operation where the host-requested POPCOUNT operation is broken up into multiple stages within the memory module, the generation of sub-processing instructions further disclosed in Paragraph 80; “controller 97 may initially receive from the host 14 the physical addresses of memory locations... controller 97 may access the appropriate storage location to obtain the corresponding physical address and then retrieve the respective bit vector-specific non-overlapping segments”, where based on the received host instruction and associated data, the controller generates sub processing actions by decomposing the host-requested operation into segments executed in the memory module, establishing that a host instruction initiates a higher level operation, POPCOUNT, and in response, generates sub processing operations that implement the higher operation.);
generate a partial result based on a respective one of the generated sub-processing instructions (Paragraph 80; “The controller 97 may then perform the bitwise operation on each group of aligned segments to thereby generate a plurality of partial results. As before, the partial results may be stored in the PIMResultRegister 99”, where the partial results correspond to the partial results in accordance with the segmented instructions, corresponding to the generated sub-processing instruction);
generate an operation result of the operation by processing the generated partial results (Paragraph 80; “The controller 97 may subsequently combine all partial results in the register 99 to effectuate the execution of the logical bitwise operation. The final outcome from the combination may be initially stored in the PIMResultRegister 99 before eventually storing it at a pre-defined storage location (or PIM-specific address) for future submission to the host 14. The host 14 may access this storage location to read the result.”, which teaches generating an operation result by processing all of the partial results from the bitwise operation).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 2, 4, and 17-18 are rejected under 35 U.S.C. 103 as being unpatentable over Guz in view of Kotra et al. (US 20230393849 A1) hereafter Kotra.
Regarding claim 2, Guz teaches the memory device of claim 1. Guz teaches:
wherein the processing engine (Paragraph 80; PIM controller 80 performs the sub-processing instructions and generates the intermediate result) is configured to generate an intermediate result by performing the operation according to the generated sub-processing instruction (Paragraph 80; “The controller 97 may then perform the bitwise operation on each group of aligned segments to thereby generate a plurality of partial results. As before, the partial results may be stored in the PIMResultRegister 99”, where the partial results correspond to the intermediate results in accordance with the segmented instruction, corresponding to the generated sub-processing instruction);
further comprising a PIM engine (Paragraph 80; controller 97 corresponds to the PIM engine) configured to, in response to obtaining the generated intermediate result, generate an operation result of the operation by processing the generated intermediate result (Paragraph 80; “The controller 97 may subsequently combine all partial results in the register 99 to effectuate the execution of the logical bitwise operation. The final outcome from the combination may be initially stored in the PIMResultRegister 99 before eventually storing it at a pre-defined storage location (or PIM-specific address) for future submission to the host 14. The host 14 may access this storage location to read the result.”, where the PIM controller (called the controller in the quote) corresponds to the PIM engine that generates an operation result by processing the generated intermediate result, the partial results from the bitwise operation).
Guz does not teach that the PIM engine is a PNM engine.
However, Kotra teaches:
A PNM engine (Paragraph 64; “execution unit 150 is a component of a PIM device 280 that is implemented in a processing-near-memory (PNM) fashion. For example, the PIM device 280 can be a memory accelerator that is used to execute memory-intensive operations that have been offloaded to by the host processor 132 to the accelerator.”).
Guz and Kotra are considered to be analogous to the claimed invention because they are in the same field of processing in memory. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Guz to incorporate the teachings of Kotra and utilize a PNM engine in place of a PIM engine. A person of ordinary skill in the art would recognize a PNM engine as a known architecture to offload computation from in-cell logic to near-memory processors to yield the predictable result of reducing complexity, improving programmability, and perform the same operations as a PIM engine with equivalent functional outcomes while trading in-cell execution for near-memory execution to achieve scalability benefits.
Regarding claim 4, Guz teaches the memory device of claim 3. Guz teaches:
generate a first level instruction as the sub-processing instruction (Paragraph 80; “in case of a logical bitwise operation on multiple vectors, the PIM Controller 97 may initially receive from the host 14 the physical addresses of memory locations (like the memory cells 79 shown in FIGS. 8A-8B) in the memory module 12 where the respective bit vectors are stored” where the division of vectors and performing operations on aligned segments that yield partial results corresponds to a sub-processing instruction operating on a portion of the overall task) from the host processing instruction (Paragraph 67; “providing an interface (API) to enable the host 14 to initiate commands”, where the host-initiated command defines the overall processing operation from which the segment level sub-processing instructions are derived) and transmit the first level instruction to the PIM engine (Paragraph 80; “PIM Controller 97 may initially receive from the host 14 the physical addresses…” where the handling of host-derived parameters that define segment-level operations reflects transmission of the sub-processing instruction to an intermediate engine mapped to the PIM engine);
and generate a second level instruction from the first level instruction (Paragraph 80; “the partial results may be stored in the PIMResultRegister 99… [and] subsequently combine all partial results” where instructions to accumulate and combine partial results are derived from the earlier segment-level sub-processing instructions) and transmit the second level instruction to the PIM engine (Paragraph 67; “controlling the operation of the logic structure 78 or 87… accumulating intermediate results of bit-counting in the PIMResultRegister 99” where control of the in-memory logic to combine partial results constitutes transmission of second level instructions to the PIM engine).
Guz does not teach that the PIM engine is a PNM engine.
However, Kotra teaches:
A PNM engine (Paragraph 64; “execution unit 150 is a component of a PIM device 280 that is implemented in a processing-near-memory (PNM) fashion. For example, the PIM device 280 can be a memory accelerator that is used to execute memory-intensive operations that have been offloaded to by the host processor 132 to the accelerator.”).
Guz and Kotra are considered to be analogous to the claimed invention because they are in the same field of processing in memory. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Guz to incorporate the teachings of Kotra and utilize a PNM engine in place of a PIM engine. A person of ordinary skill in the art would recognize a PNM engine as a known architecture to offload computation from in-cell logic to near-memory processors to yield the predictable result of reducing complexity, improving programmability, and perform the same operations as a PIM engine with equivalent functional outcomes while trading in-cell execution for near-memory execution to achieve scalability benefits.
Regarding claim 17, Guz teaches the memory device of claim 1. Guz teaches:
a logic die in which a PIM engine (Paragraph 80; controller 97) configured to generate an operation result and the device controller are disposed (Paragraph 27; “memory module may be configured to include a controller that may comprise the hardware and/or software to support the in-memory implementations of popcounting and logical bitwise operations. In one embodiment, the memory module may be a Three Dimensional Stack (3DS) memory module whose base or logic die may be configured to include the controller. For example, in case of the POPCOUNT operation, a Processing-In-Memory (PIM) model according to the teachings of the present disclosure may provide for the following additions to the memory module (or to the base/logic die of a 3DS memory”, where the logic die includes controller logic and computation hardware generating operation results based on a PIM model, corresponding to a PIM engine and device controller disposed in the logic die.).
Guz does not teach that the PIM engine is a PNM engine.
However, Kotra teaches:
A PNM engine (Paragraph 64; “execution unit 150 is a component of a PIM device 280 that is implemented in a processing-near-memory (PNM) fashion. For example, the PIM device 280 can be a memory accelerator that is used to execute memory-intensive operations that have been offloaded to by the host processor 132 to the accelerator.”).
Guz and Kotra are considered to be analogous to the claimed invention because they are in the same field of processing in memory. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Guz to incorporate the teachings of Kotra and utilize a PNM engine in place of a PIM engine. A person of ordinary skill in the art would recognize a PNM engine as a known architecture to offload computation from in-cell logic to near-memory processors to yield the predictable result of reducing complexity, improving programmability, and perform the same operations as a PIM engine with equivalent functional outcomes while trading in-cell execution for near-memory execution to achieve scalability benefits.
Regarding claim 18, Guz in view of Kotra teach the memory device of claim 17. Guz teaches:
wherein an additional PIM engine (Paragraph 80; controller 97) configured to receive a PIM operation result and generate an intermediate result is additionally disposed in the logic die (Paragraph 27; “(iii) A special register—referred to herein as “PIMResultRegister”—that may store the final result (i.e., the popcount value) and may be used by the memory's host to retrieve the final popcount value. In particular embodiments, this register also may be used to hold intermediate results when the vector size is larger than a DRAM page. In such a situation, this register may accumulate all intermediate results to eventually store the final result for subsequent retrieval by the host”, where the accumulation logic associated with PIMResultRegister is located in the logic die and operates on results produced by PIM computations, thereby receiving PIM operation results and generating intermediate results, corresponding to an additional PIM engine disposed in the logic die.).
Kotra teaches:
A PNM engine (Paragraph 64; “execution unit 150 is a component of a PIM device 280 that is implemented in a processing-near-memory (PNM) fashion. For example, the PIM device 280 can be a memory accelerator that is used to execute memory-intensive operations that have been offloaded to by the host processor 132 to the accelerator.”).
Claim 5 is rejected under 35 U.S.C. 103 as being unpatentable over Guz in view of Grafton et al. (US 20160350240 A1) hereafter Grafton.
Regarding claim 5, Guz teaches the memory device of claim 1. Guz does not teach a device interface configured to receive a packet from the host; or an address decoder configured to identify whether the received packet indicates either a memory access request or a processing request, based on an access address of the received packet.
However, Grafton teaches:
a device interface (Paragraph 31; SPI interface corresponds to a device interface) configured to receive a packet from the host (Paragraph 31; “SPI memory command protocol generally refers to any SPI communication protocol used in SPI memory architectures (in other words, a memory that includes a SPI interface)” and “using SPI memory command protocol, SPI host port 50 can interpret an SPI communication packet received from host 40” where the SPI interface corresponds to the device interface that is configured to receive packets from a host);
and an address decoder (Paragraph 31; SPI host port 50) configured to identify whether the received packet indicates either a memory access request or a processing request, based on an access address of the received packet (Paragraph 31; “SPI host port 50 can store the SPI communication packet (for example, in SPI host port receive buffer 56), and interpret (decode) a payload of the SPI communication packet”, where “SPI host port 50 can interpret an SPI communication packet received from host 40 into an access instruction that specifies an access operation and address/data information associated with the access operation, and then generate signaling necessary to perform the access operation (such as a read, a write, or a prefetch operation) on memory-mapped resources of processing system 10”, where the access operation corresponds to the applicant’s memory access request, and other operations such as write or prefetch correspond to the applicant’s processing request).
Guz and Grafton are considered to be analogous to the claimed invention because they are in the same field of memory mapped processing. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Guz to incorporate the teachings of Grafton and have a device interface receive a packet from a host and an address decoder to identify whether the request indicates a memory access request or processing request based on an access address of the packet. A person of ordinary skill in the art would have recognized the known method of using memory-mapped I/O with address range decoding would have yielded the predictable result of correctly identifying whether a packet corresponds to a memory access request or processing request to route the packet with appropriate logic.
Claim 7 is rejected under 35 U.S.C. 103 as being unpatentable over Guz in view of Grafton, further in view of Underwood et al. (US 20240202118 A1) hereafter Underwood.
Regarding claim 7, Guz teaches the memory device of claim 1. Guz teaches:
A device controller (Paragraph 58; controller 97).
Guz does not teach an instruction buffer configured to store the host processing instruction in an order designated by the host, in response to a received packet being identified as indicating a processing request.
However, Grafton teaches:
in response to a received packet being identified as indicating a processing request (Paragraph 31; “SPI host port 50 can store the SPI communication packet (for example, in SPI host port receive buffer 56), and interpret (decode) a payload of the SPI communication packet”, where “SPI host port 50 can interpret an SPI communication packet received from host 40 into an access instruction that specifies an access operation and address/data information associated with the access operation, and then generate signaling necessary to perform the access operation (such as a read, a write, or a prefetch operation) on memory-mapped resources of processing system 10”, where the other operations such as write or prefetch correspond to the applicant’s processing request.).
Guz and Grafton are considered to be analogous to the claimed invention because they are in the same field of memory mapped processing. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Guz to incorporate the teachings of Grafton and respond upon receiving a packet identified as indicating a processing request. A person of ordinary skill in the art would have recognized the known method of using memory-mapped I/O with address range decoding would have yielded the predictable result of correctly identifying whether a packet corresponds to a memory access request or processing request to perform actions in response to the packet with appropriate logic.
Guz in view of Grafton does not teach an instruction buffer configured to store the host processing instruction in an order designated by the host.
However, Underwood teaches:
an instruction buffer (Paragraph 53; NIC corresponds to the instruction buffer) configured to store the host processing instruction (Paragraph 53; “a buffer-managing unit 704 to store the incoming packet and information related to the corresponding instruction”), in an order designated by the host (Paragraph 53; “generate, based on the precomputed cached or obtained context, a plurality of write requests comprising addresses and lengths, thereby allowing the NIC to process out-of-order packets based on the precomputed cached or obtained context”, where the precomputed cached or obtained context is obtained “from host memory” thereby being designated by the host.).
Guz, Grafton, and Underwood are considered to be analogous to the claimed invention because they are in the same field of memory mapped processing. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Guz in view of Grafton to incorporate the teachings of Underwood and have an instruction buffer store the host processing instruction in an order designated by the host. A person of ordinary skill in the art would recognize the known method of maintaining a sequence of commands issued by a process or host to ensure correct program semantics to yield the predictable result of instructions being processed in the intended order without altering functionality or requiring additional control logic.
Claim 8 is rejected under 35 U.S.C. 103 as being unpatentable over Guz in view of Grafton, further in view of Underwood, further in view of Nag et al. (NPL: OrderLight: Lightweight Memory-Ordering Primitive for Efficient Fine-Grained PIM Computations, 2021), hereafter Nag.
Regarding claim 8, Guz in view of Grafton, further in view of Underwood teach the memory device of claim 7. Grafton teaches:
transmit the stored host processing instruction to an instruction decoder (Paragraph 31; “to perform a read operation, host 40 can send an SPI communication packet that includes a read data instruction along with associated address information. SPI host port 50 can store the SPI communication packet (for example, in SPI host port receive buffer 56), and interpret (decode) a payload of the SPI communication packet”, where the host sends, corresponding to transmitting, the processing instruction to the host port receive buffer which interprets, corresponding to decoding, the payload)
Guz in view of Grafton, further in view of Underwood does not teach traverse and increment an instruction counter; or performing an action in response to the host processing instruction being stored in an entry indicated by the instruction counter.
However, Nag teaches:
traverse and increment an instruction counter (Page 8, left column; “The scheduler is augmented with a request counter and an OrderLiдht flag for each PIM memory-group. The counter associated with a memory-group is incremented when a request to that memory-group is dequeued by the scheduler and decremented when it is scheduled.”);
in response to the host processing instruction being stored in an entry indicated by the instruction counter (Page 8, left column, “The counter associated with a memory-group is incremented when a request to that memory-group is dequeued by the scheduler and decremented when it is scheduled. When the scheduler receives an OrderLiдht packet, the OrderLiдht flag for the appropriate memory-group is set. Any subsequent request to that memory-group is not scheduled until the flag is unset. The flag is unset when the counter for the memory-group is decremented to zero (i.e., all requests preceding the OrderLiдht packet have been scheduled). Once the OrderLiдht flag is reset, the scheduler is free to process subsequent requests”, where the association with the appropriate particular memory group corresponds to being stored in an entry indicated by the instruction counter).
Guz, Grafton, Underwood, and Nag are considered to be analogous to the claimed invention because they are in the same field of memory mapped processing. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Guz in view of Grafton, further in view of Underwood to incorporate the teachings of Nag and traverse and increment an instruction counter in response to storing an instruction in a corresponding entry. A person of ordinary skill in the art would recognize the known method of sequentially tracking instruction storage and ensuring alignment with buffers, yielding the predictable result of storing each instruction within the correct entry and allowing for retrieval and execution in the intended order without missing or overwriting entries.
Claim 9 is rejected under 35 U.S.C. 103 as being unpatentable over Guz in view of Grafton, further in view of Underwood, further in view of Kotra.
Regarding claim 9, Guz teaches the memory device of claim 1. Guz teaches:
A device controller (Paragraph 58; controller 97). Guz does not teach an instruction decoder configured to transmit an operation corresponding to the host processing instruction received from an instruction buffer to either one or both of a processing near-memory (PNM) engine and an instruction generator.
However, Grafton teaches:
an instruction decoder (Paragraph 31; SPI host port 50) configured to transmit an operation corresponding to the host processing instruction (Paragraph 31; “SPI host port 50 can store the SPI communication packet (for example, in SPI host port receive buffer 56), and interpret (decode) a payload of the SPI communication packet to identify the read data instruction and a read address”, where SPI host port performing the interpreting corresponds to the instruction decoder that transmits operations from the host processing instruction).
Guz and Grafton are considered to be analogous to the claimed invention because they are in the same field of memory mapped processing. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Guz to incorporate the teachings of Grafton and transmit operations corresponding to a host processing instruction. A person of ordinary skill in the art would have recognized the known method of using memory-mapped I/O would have yielded the predictable result of transmitting operations in response to the packet with appropriate logic.
Guz in view of Grafton does not teach received from an instruction buffer to either one or both of an engine and an instruction generator.
However, Underwood teaches:
received from an instruction buffer to either one or both of an engine and an instruction generator (Paragraph 53; “a buffer-managing unit 704 to store the incoming packet and information related to the corresponding instruction”, where Paragraph 36 further discloses “the datatype engine can be integrated with both the transmit and the receive logic in, respectively, the outbound packet engine and the inbound packet engine”).
Guz, Grafton, and Underwood are considered to be analogous to the claimed invention because they are in the same field of memory mapped processing. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Guz in view of Grafton to incorporate the teachings of Underwood and receive from an instruction buffer to an engine or instruction generator. A person of ordinary skill in the art would have recognized the known method of request processing to yield the expected result of properly assigning requests to optimal resources.
However, Kotra teaches:
A PNM engine (Paragraph 64; “execution unit 150 is a component of a PIM device 280 that is implemented in a processing-near-memory (PNM) fashion. For example, the PIM device 280 can be a memory accelerator that is used to execute memory-intensive operations that have been offloaded to by the host processor 132 to the accelerator.”).
Guz, Grafton, Underwood, and Kotra are considered to be analogous to the claimed invention because they are in the same field of processing in memory. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Guz in view of Grafton, further in view of Underwood to incorporate the teachings of Kotra and utilize a PNM engine in place of a PIM engine. A person of ordinary skill in the art would recognize a PNM engine as a known architecture to offload computation from in-cell logic to near-memory processors to yield the predictable result of reducing complexity, improving programmability, and perform the same operations as a PIM engine with equivalent functional outcomes while trading in-cell execution for near-memory execution to achieve scalability benefits.
Claim 10 is rejected under 35 U.S.C. 103 as being unpatentable over Guz in view of Grafton, further in view of Underwood, further in view of Kotra, further in view of Zach et al. (US 20200284903 A1) hereafter Zach, further in view of Lea et al. (US 20190065111 A1) hereafter Lea.
Regarding claim 10, Guz in view of Grafton, further in view of Underwood, further in view of Kotra teach the memory device of claim 9. Underwood teaches:
An engine (Paragraph 36; “the datatype engine can be integrated with both the transmit and the receive logic in, respectively, the outbound packet engine and the inbound packet engine.”).
Kotra teaches:
PNM architecture (Paragraph 64; “execution unit 150 is a component of a PIM device 280 that is implemented in a processing-near-memory (PNM) fashion. For example, the PIM device 280 can be a memory accelerator that is used to execute memory-intensive operations that have been offloaded to by the host processor 132 to the accelerator”, where implementation in a PNM fashion necessarily includes PNM architecture).
Guz in view of Grafton, further in view of Underwood, further in view of Kotra does not teach transmit the host processing instruction, in response to the host processing instruction matching with a pre-stored operation identification code for an operation level corresponding to the instruction decoder.
However, Zach teaches:
transmit the host processing instruction, in response to the host processing instruction matching with a pre-stored operation identification code corresponding to the instruction decoder (Paragraph 89; “If the unique identifier decoded in step 1004 matches the identifier of the specific remote transducer system, the latter transmits a locator pulse as well as a digitally encoded unique identifier, and possibly other information, in 1005.”, where the unique identifier is analogous to the host processing instruction, in response to matching an identifier of the specific system, corresponding to matching with a pre-stored operation identifier code corresponding to the instruction decoder, performs the instruction transmittal).
Guz, Grafton, Underwood, Kotra, and Zach are considered to be analogous to the claimed invention because they are in the same field of data-driven control systems. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Guz in view of Grafton, further in view of Underwood, further in view of Kotra to incorporate the teachings of Zach and transmit the host processing instruction in response to matching a pre-stored identification code corresponding to the decoder. A person of ordinary skill in the art would recognize the known technique of processing only when the source is known to the system and has a relevant identifier, and implementing such a system would yield the predictable result of only allowing particular instructions to be processed.
Guz in view of Grafton, further in view of Underwood, further in view of Kotra, further in view of Zach does not teach an operation level.
However, Lea teaches:
An operation level (Paragraph 16; “An operation hierarchy can be used to define levels of PIM operations. For example, a first, e.g., lower, level in the operation hierarchy may include performance of low level bit vector operations, e.g., fundamental and/or individual logical operations, which may be referred to as “primitive” operations. A next, e.g., middle, level in the operation hierarchy may include performance of composite operations”).
Guz, Grafton, Underwood, Kotra, Zach, and Lea are considered to be analogous to the claimed invention because they are in the same field of data-driven control systems. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Guz in view of Grafton, further in view of Underwood, further in view of Kotra, further in view of Zach to incorporate the teachings of Lea and utilize an operation level. A person of ordinary skill in the art would have recognized the known use of thresholding at particular operation levels would yield the predictable result of applying particular methods at given thresholds for optimal operation.
Claim 11 is rejected under 35 U.S.C. 103 as being unpatentable over Guz in view of Grafton, further in view of Underwood, further in view of Kotra, further in view of Hall et al. (US 20220209814 A1) hereafter Hall, further in view of Lea et al. (US 20190065111 A1) hereafter Lea.
Regarding claim 11, Guz in view of Grafton, further in view of Underwood, further in view of Kotra teach the memory device of claim 9. Guz teaches:
transmit the host processing instruction to the instruction generator (Paragraph 80; “PIM Controller 97 may initially receive from the host 14 the physical addresses…” where the handling of host-derived parameters that define segment-level operations reflects transmission of the sub-processing instruction to an intermediate engine mapped to the PNM engine).
Underwood teaches:
An engine (Paragraph 36; “the datatype engine can be integrated with both the transmit and the receive logic in, respectively, the outbound packet engine and the inbound packet engine.”).
Kotra teaches:
PNM architecture (Paragraph 64; “execution unit 150 is a component of a PIM device 280 that is implemented in a processing-near-memory (PNM) fashion. For example, the PIM device 280 can be a memory accelerator that is used to execute memory-intensive operations that have been offloaded to by the host processor 132 to the accelerator.”).
Guz in view of Grafton, further in view of Underwood, further in view of Kotra does not teach in response to the host processing instruction matching with a pre-stored operation identification code for an operation level corresponding to the instruction decoder.
However, Hall teaches:
in response to the host processing instruction not matching with a pre-stored operation identification code corresponding to the instruction decoder (Paragraph 125; “a CSS device may determine that the identifier is a non-relevant identifier (e.g., not matching a preconfigured identifier or attribute indicated by the identifier)”).
Guz, Grafton, Underwood, Kotra, and Hall are considered to be analogous to the claimed invention because they are in the same field of data-driven control systems. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Guz in view of Grafton, further in view of Underwood, further in view of Kotra to incorporate the teachings of Hall and transmit the host processing instruction elsewhere in response to not matching a pre-stored identification code corresponding to the decoder. A person of ordinary skill in the art would recognize the known technique of processing only when the source is known to the system and has a relevant identifier, and implementing a system that sends instructions with unknown identifiers elsewhere would yield the predictable result of only allowing particular instructions to be processed.
Guz in view of Grafton, further in view of Underwood, further in view of Kotra, further in view of Hall does not teach an operation level.
However, Lea teaches:
An operation level (Paragraph 16; “An operation hierarchy can be used to define levels of PIM operations. For example, a first, e.g., lower, level in the operation hierarchy may include performance of low level bit vector operations, e.g., fundamental and/or individual logical operations, which may be referred to as “primitive” operations. A next, e.g., middle, level in the operation hierarchy may include performance of composite operations”).
Guz, Grafton, Underwood, Kotra, Hall, and Lea are considered to be analogous to the claimed invention because they are in the same field of data-driven control systems. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Guz in view of Grafton, further in view of Underwood, further in view of Kotra, further in view of Hall to incorporate the teachings of Lea and utilize an operation level. A person of ordinary skill in the art would have recognized the known use of thresholding at particular operation levels would yield the predictable result of applying particular methods at given thresholds for optimal operation.
Claim 12 is rejected under 35 U.S.C. 103 as being unpatentable over Guz in view of Nag.
Regarding claim 12, Guz teaches the memory device of claim 1. Guz teaches:
an instruction generator (Paragraph 80; controller 97 divides the vector into segments, thereby generating sub-instructions and therefore acting as an instruction generator) configured to generate the sub-processing instruction (Paragraph 80; “the controller 97 may divide each bit vector into a plurality of bit vector-specific non-overlapping segments and then align corresponding bit vector-specific segments from all bit vectors into a plurality of groups of aligned segments. The controller 97 may then perform the bitwise operation on each group of aligned segments to thereby generate a plurality of partial results”, where dividing the host-requested operation into segmented operations that yield partial results corresponds to generating sub-processing instructions), from the received host processing instruction (Paragraph 67; “providing an interface (API) to enable the host 14 to initiate commands and fetch data”, where the host-initiated command constitutes the received host processing instruction from which the sub-processing instructions are derived), and transmit the generated sub-processing instruction to a memory scheduler (Paragraph 67; “controlling the implemented reduction tree 49 or 50, controlling the operation of the logic structure 78 or 87, handling computation of popcounts over vectors spanning multiple memory pages, accumulating intermediate results of bit-counting in the PIMResultRegister 99 and generating the final popcount value for storage in the result register 99.”, where controlling execution across memory pages requires scheduling segment level operations based on memory layout, which corresponds to transmitting the generated sub-processing instructions to a memory scheduler.).
While Guz implies a memory scheduler, Guz does not explicitly teach a memory scheduler.
However, Nag teaches:
A memory scheduler (Page 8, left column; “When the scheduler receives an OrderLiдht packet, the OrderLiдht flag for the appropriate memory-group is set. Any subsequent request to that memory-group is not scheduled until the flag is unset. The flag is unset when the counter for the memory-group is decremented to zero (i.e., all requests preceding the OrderLiдht packet have been scheduled). Once the OrderLiдht flag is reset, the scheduler is free to process subsequent requests.”).
Guz and Nag are considered to be analogous to the claimed invention because they are in the same field of memory-mapped processing. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Guz to incorporate the teachings of Nag and utilize a memory scheduler. A person of ordinary skill in the art would have recognized the known technique of orchestrating the order and timing of memory accesses and processing instructions to yield the predictable result of executing the host/sub-instructions more efficiently and in an order that preserves correctness without additional control complexity.
Claim 13 is rejected under 35 U.S.C. 103 as being unpatentable over Guz in view of Nag, further in view of Lea.
Regarding claim 13, Guz in view of Nag teach the memory device of claim 12. Guz in view of Nag does not teach wherein the instruction generator comprises a predefined instruction table and is configured to generate the sub-processing instruction according to a result of matching between the instruction table and the host processing instruction.
However, Lea teaches:
wherein the instruction generator (Paragraph 112; control logic 131 on host 110 correspond to the instruction generator) comprises a predefined instruction table (Paragraph 112; “fetching, by the control logic 131 on the host 110, the plurality of microcode instructions from the array 130 of memory cells on the PIM capable device 101” where the array of memory cells storing microcode instructions performs the same role as the predefined instruction table containing instruction entries), and is configured to generate the sub-processing instruction according to a result of matching between the instruction table and the host processing instruction (Paragraph 110; “execute a command instruction set associated with bit vector operations and issuing the command instruction set as the compute request to the sequencer 132, e.g., via a sideband channel shown at 157 and described in connection with FIG. 1A. The command instruction set may be executed as individual microcode instructions, e.g., by a VLIW being decoded by a VLIW controller in the sequencer 132” where the host processing instruction, corresponding to the command instruction set, is matched against predefined microcode instructions in the instruction array and decoded by the controller, thereby generating sub-processing instructions in the form of individual microcode instructions corresponding to the matched instruction entries).
Guz, Nag, and Lea are considered to be analogous to the claimed invention because they are in the same field of memory-mapped processing. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Guz in view of Nag to incorporate the teachings of Lea and have the instruction generator comprise a predefined instruction table and generate a sub-processing instruction based on matching the host processing instruction to the table. A person of ordinary skill in the art would recognize the use of known lookup tables and microcode decoding to yield the predictable result of converting each host instruction into corresponding sub-processing instructions without introducing errors or unexpected behavior.
Claim 14 is rejected under 35 U.S.C. 103 as being unpatentable over Guz in view of Nag, further in view of Nagarajan et al. (US 20230153115 A1) hereafter Nagarajan.
Regarding claim 14, Guz in view of Nag teach the memory device of claim 12. Guz teaches:
In response to a host processing instruction (Paragraph 6; “receiving at a memory module an instruction from a host to perform a POPCOUNT operation”), generate a sub-processing instruction (Paragraph 58; “bitcount operations for a vector that spans multiple DRAM pages may require multiple steps. In case of the DRAM page size of 1 KB, each step may calculate the bitcount for a 1KB subset of the vector and partial results may be accumulated in a special register within the memory module 12” and Paragraph 80; “implementation of popcounting in the context of FIGS. 4-7 and FIG. 9”, “The controller 97 may then perform the bitwise operation on each group of aligned segments to thereby generate a plurality of partial results”, where the segmented instruction generation corresponding to the partial results corresponds to the generation of a sub-processing instruction).
Guz in view of Nag does not teach sparse lengths sum.
However, Nagarajan teaches:
Sparse lengths sum (Paragraph 40; “the XPU may perform vector scatter or gather operations, segment sums, and/or partition the sparse feature tensors”).
Guz, Nag, and Nagarajan are considered to be analogous to the claimed invention because they are in the same field of memory-mapped processing. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Guz in view of Nag to incorporate the teachings of Nagarajan and utilize sparse lengths sum (SLS) as a basis for creating a sub-processing instruction matching the corresponding instruction. A person of ordinary skill in the art would have recognized the known use of pre-computed metadata to guide execution of operations in memory would yield the predictable result of the sub-instruction correctly targeting the relevant portion of the data for processing.
Claims 15 and 40 are rejected under 35 U.S.C. 103 as being unpatentable over Guz in view of Underwood.
Regarding claim 15, Guz teaches the memory device of claim 1. Guz teaches:
A device controller (Paragraph 58; controller 97).
Guz does not teach a memory scheduler configured to schedule access by processing host memory instructions in an out-of-order mode, in response to a normal memory access to memory blocks being requested from the host.
However, Underwood teaches:
a memory scheduler (Paragraph 53; NIC acts as the memory scheduler) configured to schedule access by processing host memory instructions in an out-of-order mode, in response to a normal memory access to memory blocks being requested from the host (Paragraph 53; “generate, based on the precomputed cached or obtained context, a plurality of write requests comprising addresses and lengths, thereby allowing the NIC to process out-of-order packets based on the precomputed cached or obtained context”, where the NIC corresponds to the memory scheduler that schedules access by processing instructions out-of-order in response to an incoming packet, corresponding to a request for normal memory access to memory blocks, being requested from the host system.).
Guz and Underwood are considered to be analogous to the claimed invention because they are in the same field of memory-mapped processing. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Guz to incorporate the teachings of Underwood and have the device controller comprise a memory scheduler configured to schedule access by processing host memory instructions in an out-of-order mode, in response to a normal memory access to memory blocks being requested from the host. A person of ordinary skill in the art would recognize the known implementation of out-of-order scheduling improves memory throughput and reduces stalls by allowing memory accesses to proceed when dependencies allow doing so, yielding the predictable result of efficient execution without violating program correctness.
Claim 40 recites similar limitations as those of claim 15. Claim 40 is rejected for similar reasons as those of claim 15.
Claim 16 is rejected under 35 U.S.C. 103 as being unpatentable over Guz in view of Underwood, further in view of Comparan et al. (US 20140281402 A1) hereafter Comparan.
Regarding claim 16, Guz teaches the memory device of claim 1. Guz teaches:
A device controller (Paragraph 58; controller 97).
Guz does not teach a memory scheduler configured to schedule access to memory blocks by processing the host processing instructions in an in-order mode, in response to the host processing instruction being requested from the host.
However, Underwood teaches:
a memory scheduler (Paragraph 53; NIC acts as the memory scheduler) configured to schedule access to memory blocks by processing the host processing instructions in a particular mode, in response to the host processing instruction being requested from the host (Paragraph 53; “generate, based on the precomputed cached or obtained context, a plurality of write requests comprising addresses and lengths, thereby allowing the NIC to process out-of-order packets based on the precomputed cached or obtained context”, where the NIC corresponds to the memory scheduler that schedules access by processing instructions out-of-order in response to an incoming packet, corresponding to a request for normal memory access to memory blocks, being requested from the host system).
Guz and Underwood are considered to be analogous to the claimed invention because they are in the same field of memory-mapped processing. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Guz to incorporate the teachings of Underwood and have the device controller of Guz comprise a memory scheduler configured to schedule access by processing host memory instructions in a particular mode, in response to a normal memory access to memory blocks being requested from the host. A person of ordinary skill in the art would recognize the known implementation of modal memory scheduling improves memory throughput and reduces stalls by allowing memory accesses to proceed when dependencies allow doing so, yielding the predictable result of efficient execution without violating program correctness.
Guz in view of Underwood does not teach that the mode is an in-order mode.
However, Comparan teaches:
in-order mode (Paragraph 88; “Pipeline stage power controller deactivates issue queue 722, load queue 730, store queue 724, and commit stage 716 so that each are bypassed in executing instruction streams in the in-order mode.”).
Guz, Underwood, and Comparan are considered to be analogous to the claimed invention because they are in the same field of memory-mapped processing. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Guz in view of Underwood to incorporate the teachings of Comparan and perform processing in an in-order mode. A person of ordinary skill in the art would have recognized the implementation of known methods of in-order processing would yield the predictable result of preserving program semantics and maintaining predictable execution.
Claims 21 and 41 are rejected under 35 U.S.C. 103 as being unpatentable over Guz in view of Komuravelli et al. (US 20210294875 A1) hereafter Komuravelli.
Regarding claim 21, Guz teaches the memory device of claim 1. Guz teaches:
wherein the sub-processing instruction comprises an instruction comprising input fragments of an input vector (Paragraph 80; “PIM Controller 97 may initially receive from the host 14 the physical addresses of memory locations (like the memory cells 79 shown in FIGS. 8A-8B) in the memory module 12 where the respective bit vectors are stored, and then store each received physical address at a PIM-specific address (or storage location) within the memory module 12. The controller 97 may access the appropriate storage location to obtain the corresponding physical address and then retrieve the respective bit vector from the specified memory location”, where the host-provided addresses identify the input fragments that the controller processes, corresponding to the sub-processing instruction).
Guz does not teach weight elements or a dot product instruction.
However, Komuravelli teaches:
An address of weight elements of a weight matrix (Paragraph 47; “Each pair of input matrices includes a data input matrix and a corresponding channel weight matrix. Each data input matrix corresponds to a particular channel of a portion of an activation data input matrix and is processed by data input unit 403. Each weight input matrix corresponds to the weight matrix to be applied to the channel and is processed by channel weight input unit 405. Data input unit 403, channel weight input unit 405, and output unit 409 may be implemented using hardware registers, such as flip-flop circuits, for transferring multiple input and output elements to/from channel convolution processor unit 407. In some embodiments, elements corresponding to each data input vector are retrieved from memory, such as memory 161 of FIG. 1, and loaded into a corresponding vector unit”, where the weight input vectors correspond to weight elements of a weight matrix and retrieval from memory corresponds to the instruction having a location of the elements, as evidenced by “In some embodiments, the output vector result may be first written to memory, such as memory 161 of FIG. 1, or another memory location before being transmitted to a dot product engine”).
and wherein, for the performing of the operation based on the generated sub-processing instruction, the processing engine (Paragraph 47; processor acts as the processing engine) is configured to read the weight elements from a memory block based on the address (Paragraph 47; “elements corresponding to each weight input vector are retrieved from memory, such as memory 161 of FIG. 1, and loaded into a corresponding vector unit”, where the processor reads weight vectors from memory which correspond to the weight elements of the sub-processing instruction), based on the reading of the weight elements and input fragments (Paragraph 47; “elements corresponding to each weight input vector are retrieved from memory” and used for further processing. Further, “data input unit 403 loads additional needed data elements and generates new data input vectors corresponding to the new portion of the activation data input matrix for determining additional channel convolution results. As the data input vectors change to correspond to new portions of the activation data input matrix, the weight input vectors can remain the same and can be reused” show that the data input vectors correspond to the input fragments of the sub-processing instruction)
Guz and Komuravelli are considered to be analogous to the claimed invention because they are in the same field of memory-resident vector/matrix computation. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Guz to incorporate the teachings of Komuravelli and utilize an address of weight elements and a weight matrix, and have the processing engine read the weight elements from a memory block based on the address to generate a partial operation result based on the read weight elements and input fragments. A person of ordinary skill in the art would recognize the known methods of fetching operands from memory using a specific address for efficient computation, yielding the predictable result of the partial operation being correctly computed from the input fragments and weight elements.
Regarding claim 41, Guz teaches the apparatus of claim 36. Guz teaches:
wherein the sub-processing instruction comprises an instruction comprising input fragments of an input vector (Paragraph 80; “PIM Controller 97 may initially receive from the host 14 the physical addresses of memory locations (like the memory cells 79 shown in FIGS. 8A-8B) in the memory module 12 where the respective bit vectors are stored, and then store each received physical address at a PIM-specific address (or storage location) within the memory module 12. The controller 97 may access the appropriate storage location to obtain the corresponding physical address and then retrieve the respective bit vector from the specified memory location”, where the host-provided addresses identify the input fragments that the controller processes, corresponding to the sub-processing instruction);
and one or more controllers (Paragraph 80; PIM Controller 97).
Guz does not teach weight elements or a dot product instruction.
However, Komuravelli teaches:
An address of weight elements of a weight matrix (Paragraph 47; “Each pair of input matrices includes a data input matrix and a corresponding channel weight matrix. Each data input matrix corresponds to a particular channel of a portion of an activation data input matrix and is processed by data input unit 403. Each weight input matrix corresponds to the weight matrix to be applied to the channel and is processed by channel weight input unit 405. Data input unit 403, channel weight input unit 405, and output unit 409 may be implemented using hardware registers, such as flip-flop circuits, for transferring multiple input and output elements to/from channel convolution processor unit 407. In some embodiments, elements corresponding to each data input vector are retrieved from memory, such as memory 161 of FIG. 1, and loaded into a corresponding vector unit”, where the weight input vectors correspond to weight elements of a weight matrix and retrieval from memory corresponds to the instruction having a location of the elements, as evidenced by “In some embodiments, the output vector result may be first written to memory, such as memory 161 of FIG. 1, or another memory location before being transmitted to a dot product engine”).
and read the weight elements from a memory block based on the address (Paragraph 47; “elements corresponding to each weight input vector are retrieved from memory, such as memory 161 of FIG. 1, and loaded into a corresponding vector unit”, where the processor reads weight vectors from memory which correspond to the weight elements of the sub-processing instruction), based on the reading of weight elements and input fragments (Paragraph 47; “elements corresponding to each weight input vector are retrieved from memory” and used for further processing. Further, “data input unit 403 loads additional needed data elements and generates new data input vectors corresponding to the new portion of the activation data input matrix for determining additional channel convolution results. As the data input vectors change to correspond to new portions of the activation data input matrix, the weight input vectors can remain the same and can be reused” show that the data input vectors correspond to the input fragments of the sub-processing instruction)
Guz and Komuravelli are considered to be analogous to the claimed invention because they are in the same field of memory-resident vector/matrix computation. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Guz to incorporate the teachings of Komuravelli and utilize an address of weight elements and a weight matrix, and have the controller of Guz read the weight elements from a memory block based on the address to generate a partial operation result based on the reading of the weight elements and input fragments. A person of ordinary skill in the art would recognize the known methods of fetching operands from memory using a specific address for efficient computation, yielding the predictable result of the partial operation being correctly computed from the input fragments and weight elements.
Claim 22 is rejected under 35 U.S.C. 103 as being unpatentable over Guz in view of Komuravelli, further in view of Kotra.
Regarding claim 22, Guz in view of Komuravelli teach the memory device of claim 21. Guz teaches:
comprising a PIM engine (Paragraph 80; controller 97) configured to generate a final output vector based on a plurality of partial operation results including the partial operation result (Paragraph 80; “the controller 97 may divide each bit vector into a plurality of bit vector-specific non-overlapping segments and then align corresponding bit vector-specific segments from all bit vectors into a plurality of groups of aligned segments. The controller 97 may then perform the bitwise operation on each group of aligned segments to thereby generate a plurality of partial results. As before, the partial results may be stored in the PIMResultRegister 99. The controller 97 may subsequently combine all partial results in the register 99 to effectuate the execution of the logical bitwise operation”, where the PIM controller acts as the PIM engine and receives the partial results generated from segmented operations and combines them to produce a final output vector.).
Guz in view of Komuravelli does not teach that the PIM engine is a PNM engine.
However, Kotra teaches:
A PNM engine (Paragraph 64; “execution unit 150 is a component of a PIM device 280 that is implemented in a processing-near-memory (PNM) fashion. For example, the PIM device 280 can be a memory accelerator that is used to execute memory-intensive operations that have been offloaded to by the host processor 132 to the accelerator.”).
Guz and Kotra are considered to be analogous to the claimed invention because they are in the same field of processing in memory. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Guz to incorporate the teachings of Kotra and utilize a PNM engine in place of a PIM engine. A person of ordinary skill in the art would recognize a PNM engine as a known architecture to offload computation from in-cell logic to near-memory processors to yield the predictable result of reducing complexity, improving programmability, and perform the same operations as a PIM engine with equivalent functional outcomes while trading in-cell execution for near-memory execution to achieve scalability benefits.
Claim 37 is rejected under 35 U.S.C. 103 as being unpatentable over Guz in view of Aga et al. (US 20230409238 A1) hereafter Aga.
Regarding claim 37, Guz teaches the apparatus of claim 36. Guz teaches:
wherein the one or more controllers is further configured to:
perform a PIM operation according to the generated sub-processing instruction (Paragraph 27; “in case of the POPCOUNT operation, a Processing-In-Memory (PIM) model according to the teachings of the present disclosure may provide for the following additions to the memory module”, where Paragraph 80 further discloses “The controller 97 may then perform the bitwise operation on each group of aligned segments to thereby generate a plurality of partial results. As before, the partial results may be stored in the PIMResultRegister 99”. The PIM controller, called controller 97, corresponds to the PIM engine that is configured to perform PIM operations on the segmented operations, corresponding to according to the generated sub-processing instruction).
Guz does not teach a PNM operation.
However, Aga teaches:
a PNM operation (Paragraphs 46-48; “a near-memory processing element receives a PIM command”, where the PNM element receives a command for executing an operation, and Paragraph 48 discloses “the near-memory processing element processes the PIM command”, corresponding to execution of the PNM operation because the operation being executed is performed by a near-memory processing element, therefore being a near-memory processing operation, even though the command is labeled as a PIM command. Paragraph 25 confirms the PIM command in Aga is treated as a PNM command, as evidenced by “An approach is provided for processing near-memory processing commands, e.g., PIM commands”.).
Guz and Aga are considered to be analogous to the claimed invention because they are in the same field of processing in memory. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Guz to incorporate the teachings of Aga and additionally perform PNM operations. A person of ordinary skill in the art would have recognized that incorporating the near-memory processing command execution techniques of Aga into the memory processing architecture of Guz, thereby implementing the known method of permitting both PIM and PNM operations in a memory processing system, would yield the predictable result of increased processing flexibility and improved execution efficiency.
Response to Arguments
Applicant's arguments filed 04/02/2026 have been fully considered. Applicant’s arguments are summarized below:
Objections to claims 13, 19, and 21 are moot in light of the amendments and should be withdrawn.
Applicant traverses the interpretation of claims 1-23 and 25-34 under 35 U.S.C. 112(f) and associated 35 U.S.C. 112(a) and 35 U.S.C. 112(b) rejections should be withdrawn.
The disclosure of Guz does not describe or anticipate claim 1.
The disclosure of Guz does not describe or anticipate claim 6.
Dependent claims are submitted as allowable for at least the above reasons.
The Examiner respectfully disagrees with B, C, D, and E.
The Examiner agrees that the amendments to claims 13, 19, and 21 remedy the minor informalities previously objected to. Accordingly, the objections to claims 13, 19, and 21 are withdrawn.
Although claim 1 does not use the word “means”, the absence of the word “means” creates only a rebuttable presumption that 35 U.S.C. 112(f) does not apply. The presumption is overcome when the claim term fails to recite sufficiently definite structure and instead functions as a generic placeholder coupled with functional language as described under Claim Interpretation. The recited “device controller” and “processing engine” (Claim 1) are defined solely by the functions they perform. A person of ordinary skill in the art would recognize the claimed “device controller” as any component capable of performing control functions and not a specific structure. A person of ordinary skill in the art would recognize “engine” as any logic configured to process data and not a specific structure. For example, “search engine, execution engine, inference engine, processing engine” are functional labels and not structural components that inform a person of ordinary skill in the art of how the functionality is achieved. The claim does not recite any specific structure or algorithm that would inform one of ordinary skill in the art how the recited functions are achieved, as discussed in the 35 U.S.C. 112 section. Therefore, after applying the framework set forth in MPEP 2181, it is determined that the limitations invoke 35 U.S.C. 112(f) because the recited terms are generic placeholders coupled with functional language without reciting sufficient structure for performing the claimed functions. The instant specification does not disclose a specific structure or algorithm for performing the claimed functions, and as a result, a person of ordinary skill in the art would be unable to determine the metes and bounds of the claim when interpreted under 35 U.S.C. 112(f), resulting in the rejection under 35 U.S.C. 112(b). Applicant’s arguments towards enablement have been considered. However, the rejection under 35 U.S.C. 112(a) is directed towards lack of written description resulting from interpretation under 35 U.S.C. 112(f), as discussed in the non-final rejection, “[t]he disclosure does not provide sufficient description of the specific structure used to implement the device controller nor the specific algorithm(s) used to generate a sub-processing instruction... beyond the unbounded functional language of “device controller 123 may...” (Paragraphs 65-70)” (Page 7, non-final rejection). Accordingly, the interpretation of claim 1 under 35 U.S.C. 112(f) is maintained, and the rejections of claim 1 under 35 U.S.C. 112(a) and 35 U.S.C. 112(b) are maintained.
Claim 1 recites “A memory device comprising: a device controller configured to generate a sub-processing instruction based on a host processing instruction received from a host; and a processing engine configured to perform an operation based on the generated sub- processing instruction.” Applicant’s assertion that Guz is limited to a “flat host memory link” or a particular API structure and is therefore irrelevant to the memory device operations is not supported by the claim language. Guz at Paragraphs 55, 58, and 80 discloses a memory module including a controller (PIM Controller 97) that receives host-provided information and processes the request by dividing each bit vector into non-overlapping segments for processing to generate partial results which are combined to form a final result. Applicant’s argument towards Guz being directed to a “flat host-memory link device, not an in-memory processor” is not persuasive because claim 1 does not require any particular “in-memory processor” architecture. Guz expressly discloses a controller (PIM Controller 97) within the memory module which receives host instructions and performs partitioning and execution of operations “in the memory module” (Paragraph 80), thereby meeting the claimed limitations. Thus, Guz teaches a controller that decomposes a host-requested operation into multiple subordinate processing actions executed within the memory device, corresponding to the claimed generation of a sub-processing instruction based on a host processing instruction, performed on a memory device, under the broadest reasonable interpretation. Accordingly, Guz remains applicable to the limitations of claim 1, and the rejection under 35 U.S.C. 102 is proper and maintained.
Applicant’s arguments towards claim 6 are not persuasive because claim 6 requires receiving a plurality of host processing instructions in a first order and performing operations in a second order different from the first order, determined based on access addresses of the host processing instructions. Guz describes that “the PIM controller may provide an interface (API) to enable the host 14 to initiate commands” (Paragraph 67), and “the controller 97 may initially receive from the host 14 the physical addresses of memory locations” (Paragraph 80) thereby teaching receipt of host processing instructions in an initial order, corresponding to the first order, from the host. Guz further describes that “controller 97 may initially receive from the host 14 the physical addresses of memory locations”, and may “access the appropriate storage location to obtain the corresponding physical address” (Paragraph 80). The retrieval and access of stored physical addresses corresponds to a second order different from the first because execution order is governed by memory address access flow rather than the original host receipt sequence, as discussed in the rejection of claim 6. Therefore, Guz describes the recited features of claim 6. Accordingly, the rejection of claim 6 under 35 U.S.C. 102 is proper and maintained.
Independent claim 1 remains rejected for the reasons stated above. Therefore, contrary to Applicant's arguments, because the dependent claims depend from an unpatentable claim and does not add limitations that overcome the rejection, it likewise remains rejected
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Nagabhushana et al. (US 20190042489 A1) discusses writing data between a source and destination utilizing streaming addresses and splitting the packet before decoding.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to KENNETH P TRAN whose telephone number is (571)272-6926. The examiner can normally be reached M-TH 4:30 a.m. - 12:30 p.m. PT, F 4:30 a.m. - 8:30 a.m. PT, or at Kenneth.Tran@uspto.gov.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, April Blair can be reached at (571) 270-1014. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/KENNETH P TRAN/ Examiner, Art Unit 2196
/APRIL Y BLAIR/ Supervisory Patent Examiner, Art Unit 2196
Prosecution Insights