Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant’s arguments with respect to claim(s) 1-18 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Claim Objections
Claim 4 is objected to as being dependent on claim 3, but would be allowable if rewritten in independent dorm including all of the limitations of the base claim.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claim 18 is rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter. the claimed invention is directed to non-statutory subject matter. The claim(s) does/do not fall within at least one of the four categories of patent eligible subject matter the claim recites “A computer program for controlling a host data processing apparatus to provide an instruction execution environment, comprising:”
The Examiner suggests amending the claim(s) to instead read as a “A non-transitory computer readable storage medium having a computer program storing instruction for controlling a host data processing apparatus to provide an instruction execution environment , comprising:….”
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1-3 and 5-18 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Deming (US 20140281263 A1).
Regarding claim 1, Deming teaches:
An apparatus comprising. (Claim 19. A system configured to process virtual memory transactions, the system comprising: a memory; and a multithreaded processing unit coupled to the memory and configured to:)
processing circuitry configured to perform data processing in response to instructions; and transactional memory support circuitry configured to support execution of a transaction within a thread of data processing by the processing circuitry, the transaction comprising a sequence of instructions executed speculatively and for which the processing circuitry is configured to prevent commitment of results of the speculatively executed instructions until the transaction has reached a transaction end point. ([0091] FIG. 3 is a block diagram illustrating a unified virtual memory system (UVM) 200 configured with replay units 350, according to another embodiment of the present invention. The PPU 202 includes any number N of streaming multiprocessors (SMs) 310 and N replay units 350--one replay unit 350 per SM 310. For instance, if the PPU 202 were to include thirty-two SMs 310(0:31), then the PPU 202 would include thirty-two replay units 350(0:31). Each replay unit 350 enables the PPU 202 to stall the corresponding SM 310 while replaying selected memory transactions, without delaying the other SMs 310.[0092] FIG. 4 is a conceptual diagram illustrating the replay unit 350(0) of FIG. 3, according to one embodiment of the present invention. As shown, the replay unit 350(0) includes, without limitation, a transaction multiplexer (transaction mux) 420, a micro-translation lookaside buffer (uTLB) 430, an in-flight buffer 440, a fault detector 450, and a replay buffer 460. [0093] In general, threads executing within the SM 310(0) each generate a stream of virtual memory transactions from the SM 310(0). After the SM 310(0) issues a particular virtual memory transaction from the SM 310(0), the virtual memory transaction from the SM 310(0) passes through the transaction mux 420 before reaching the uTLB 430 and the in-flight buffer 440.[0094] The uTLB 430 performs one or more lookup operations to map the virtual memory address of the virtual memory transaction from the SM 310(0) to a physical memory address in the PPU memory 204. Notably, the uTLB 340 is configured to cache mappings, which are further represented by a hierarchy of TLB caches. A page table, or global TLB data structure, (not shown) is configured to store all mappings over all virtual address spaces associated with a processor complex comprising one or more PPUs 202, and one or more CPUs 102. See also [0102-0107])
the transactional memory support circuitry comprising abort event detection circuitry configured to cause execution of the transaction to be aborted when an abort event is detected before the transaction has reached the transaction end point, and to cause abort status information to be stored for later reference when determining whether to retry execution of the transaction. ([0096] If the uTLB successfully processes the virtual request from the SM 310(0), then the fault detector 450 routes a corresponding physical memory transaction to the PPU memory 204. In the physical memory transaction to the PPU memory 204, the virtual address included in the virtual memory transaction from the SM 310(0) is replaced with the physical address resulting from the uTLB 430 lookup operations. [0097] By contrast, if uTLB 430 is unable to map the virtual address specified by the virtual memory transaction from the SM 310, or if the virtual address requires a change of disposition of a target page of memory, then the uTLB 430 generates a memory access fault. The fault detector 450 processes the memory access fault--sending a fault signal to the CPU 120 and temporarily inhibiting the SM 310(0) from issuing new virtual memory transactions. Advantageously, the fault detector 450 does not cause any other SMs 310 included in the PPU 202 to cease issuing new virtual memory transactions. [0098] As part of processing the memory access fault, the fault detector 450 causes a fault buffer entry to be written to the fault buffer 216 of FIG. 2. And the fault detector 450 performs write operations that store the faulting virtual memory transaction from the SM 310(0) in the replay buffer 460. Further, the fault detector 450 causes any virtual memory transactions from the SM 310(0) that are queued in the in-flight buffer 440 to finish executing. If any of these virtual memory transactions also fault, then the fault detector 450 performs write operations that store the additional faulting virtual memory transactions in the replay buffer 460. Optionally, but preferentially, the fault detector 450 causes fault buffer entries corresponding to the additional faulting virtual memory transactions to be written to the fault buffer 216.)
wherein when the abort event arises due to a given exception event of a given type, the abort event detection circuitry is configured to cause syndrome information to be captured for use when seeking to resolve the given exception event, and to cause the abort status information to identify that a retry of the transaction is suggested at least in the event that the given exception event is transient and resolved.( [0069] The fault buffer 216 stores fault buffer entries that indicate information related to page faults generated by the PPU 202. Fault buffer entries may include, for example, the type of access that was attempted (e.g., read, write, or atomic), the virtual memory address for which an attempted access caused a page fault, the virtual address space, and an indication of a unit or thread that caused a page fault. In operation, when the PPU 202 causes a page fault, the PPU 202 may write a fault buffer entry into the fault buffer 216 to inform the PPU fault handler 215 about the faulting page and the type of access that caused the fault. The PPU fault handler 215 then performs actions to remedy the page fault. The fault buffer 216 can store multiple faults because the PPU 202 is executing a plurality of threads, where each thread can cause a one or more faults due the pipelined nature of the memory accesses of the PPU 202. [0098] As part of processing the memory access fault, the fault detector 450 causes a fault buffer entry to be written to the fault buffer 216 of FIG. 2. And the fault detector 450 performs write operations that store the faulting virtual memory transaction from the SM 310(0) in the replay buffer 460. Further, the fault detector 450 causes any virtual memory transactions from the SM 310(0) that are queued in the in-flight buffer 440 to finish executing. If any of these virtual memory transactions also fault, then the fault detector 450 performs write operations that store the additional faulting virtual memory transactions in the replay buffer 460. Optionally, but preferentially, the fault detector 450 causes fault buffer entries corresponding to the additional faulting virtual memory transactions to be written to the fault buffer 216. [0099] The PPU fault handler 215 then performs page fault sequences designed to resolve the memory access faults. Upon resolution of one or more memory access faults, the CPU 102 sends a replay signal to the replay unit 350(0). The CPU 102 may generate the replay signal in any technically feasible fashion at any time. Preferentially, the PPU fault hander 215 included in the CPU 102 generates the replay signal, generally via the command queue 214. In this way, access fault resolution processes that have a high overhead cost may be performed together, thereby improving overall performance. Generating the replay signal via the command queue 214 also allows the replay operation to be synchronized with commands that resolve faults, which pipelines fault-resolving operations and replay operations, allowing the PPU fault hander 215 to operate in a fire-and-forget manner. In alternate embodiments, the CPU 102 or the PPU 202 may generate the replay signal in any technically feasible fashion. For example, the PPU 202 may generate the replay signal at predetermined time intervals, causing periodic replays at a fixed frequency.)
Regarding claim 2, Deming teaches:
The apparatus as claimed in Claim 1, wherein: when the abort event arises due to an exception event of a type other than the given type, the abort event detection circuitry is configured to cause the abort status information to identify that a retry of the transaction is not recommended. ([0100] Upon receiving the replay signal, the replay unit 350(0) invalidates the uTLB 430, and the transaction mux 320 routes the faulting virtual memory transactions in the replay buffer 460 to the uTLB 430. For each of these faulting virtual memory transactions, the uTLB 430 attempts to map the virtual memory address to an accessible physical memory address. If, the uTLB 430 successfully maps a virtual memory transaction included in the replay buffer 460, then the fault detector 450 routes a corresponding physical memory transaction to the PPU memory 204. However, if the uTLB 430 is unable to map a particular virtual memory transaction included in the replay buffer 460, then the fault detector 450 performs write operations that re-queues the virtual memory transaction in the replay buffer 460. Notably, as the CPU 102 successfully remedies the reason for each particular page fault, the corresponding virtual memory transaction succeeds, a physical memory transaction is generated, and the virtual memory transaction is removed from the replay buffer 460.)
Regarding claim 3, Deming teaches:
The apparatus as claimed in Claim 1, wherein: the processing circuitry is configured, following abort of the transaction, to reference the abort status information in order to determine whether to retry the transaction, or whether instead to employ a fallback path provided by the software being executed on the processing circuitry, the fallback path providing a non-transaction based mechanism for performing the data processing required by the transaction. ([0104] As shown, a method 500 begins at step 502, where the replay unit 350(0) receives a virtual memory transaction from an SM 310(0). In response to receiving the virtual memory transaction from the SM310(0), the transaction mux 420 included in the replay unit 350(0) routes the virtual memory transaction to the uTLB 430, and queues the virtual memory transaction in the in-flight buffer 440. If, at step 504, if the uTLB 430 successfully processes the virtual memory transaction, then the method 500 proceeds to step 506. At step 506, the fault detector 450 included in the replay unit 350(0) routes a corresponding physical memory transaction to the PPU memory 204, and the method 500 returns to step 502. The replay unit 350(0) cycles through steps 502-506, receiving and processing virtual memory transactions from the SM 310(0), until the uTLB 430 is unable to successfully process a virtual memory transaction from the SM 310(0).)
Regarding claim 5, Deming teaches:
The apparatus as claimed in Claim 3, wherein: the abort event detection circuitry is configured to trigger execution of an exception handling routine by the processing circuitry, during which the syndrome information is captured by passing the syndrome information to the exception handling routine, wherein execution of the exception handling routine by the processing circuitry seeks to resolve the exception using the syndrome information; and the abort event detection circuitry is further configured to cause the abort status information to be stored once the exception handling routine has been completed. ([0068] During a page fault sequence, the CPU 102 or the PPU 202 may write commands into the command queue 214 for execution by the copy engine 212. Such an approach frees up the CPU 102 or the PPU 202 to execute other tasks while the copy engine 212 reads and executes the commands stored in the command queue 214, and allow all the commands for a fault sequence to be queued at one time, thereby avoiding the monitoring of progress of the fault sequence. Commands executed by the copy engine 212 may include, among other things, deleting, creating, or modifying page table entries in the PPU page table 208, reading or writing data from the system memory 104, and reading or writing data to the PPU memory 204.[0069] The fault buffer 216 stores fault buffer entries that indicate information related to page faults generated by the PPU 202. Fault buffer entries may include, for example, the type of access that was attempted (e.g., read, write, or atomic), the virtual memory address for which an attempted access caused a page fault, the virtual address space, and an indication of a unit or thread that caused a page fault. In operation, when the PPU 202 causes a page fault, the PPU 202 may write a fault buffer entry into the fault buffer 216 to inform the PPU fault handler 215 about the faulting page and the type of access that caused the fault. The PPU fault handler 215 then performs actions to remedy the page fault. The fault buffer 216 can store multiple faults because the PPU 202 is executing a plurality of threads, where each thread can cause a one or more faults due the pipelined nature of the memory accesses of the PPU 202.)
Regarding claim 6, Deming teaches:
The apparatus as claimed in Claim 5, wherein the abort status information is configured to identify that a retry of the transaction is suggested when execution of the exception handling routine causes the exception to be resolved. ([0100] Upon receiving the replay signal, the replay unit 350(0) invalidates the uTLB 430, and the transaction mux 320 routes the faulting virtual memory transactions in the replay buffer 460 to the uTLB 430. For each of these faulting virtual memory transactions, the uTLB 430 attempts to map the virtual memory address to an accessible physical memory address. If, the uTLB 430 successfully maps a virtual memory transaction included in the replay buffer 460, then the fault detector 450 routes a corresponding physical memory transaction to the PPU memory 204. However, if the uTLB 430 is unable to map a particular virtual memory transaction included in the replay buffer 460, then the fault detector 450 performs write operations that re-queues the virtual memory transaction in the replay buffer 460. Notably, as the CPU 102 successfully remedies the reason for each particular page fault, the corresponding virtual memory transaction succeeds, a physical memory transaction is generated, and the virtual memory transaction is removed from the replay buffer 460.)
Regarding claim 7, Deming teaches:
The apparatus as claimed in Claim 5, wherein the abort status information is configured to identify that a retry of the transaction is not recommended when execution of the exception handling routine causes the exception to remain unresolved. ([0106] At step 512, the replay unit 350(0) waits for the CPU 102 to signal that one or more faults have been resolved via the replay signal. Upon receiving the replay signal, the replay unit 350(0) invalidates the uTLB 430 and re-executes the virtual memory transactions that are stored in the replay buffer 460. If, the uTLB 430 successfully maps a virtual memory transaction included in the replay buffer 460, then the replay unit 350(0) routes a corresponding physical memory transaction to the PPU memory 204. However, if the uTLB 430 is unable to map a particular virtual memory transaction included in the replay buffer 460, then the fault detector 450 performs a write operation that re-queues the virtual memory transaction in the replay buffer 460. If, at step 514, the replay unit 350(0) determines that the replay buffer 460 is not empty, then the method 500 returns to step 512. The replay unit 350(0) cycles through steps 512-514, re-executing virtual memory transactions included in the replay buffer 460, until the replay unit 350(0) determines that the replay buffer 460 is empty.)
Regarding claim 8, Deming teaches:
The apparatus as claimed in Claim 5, wherein the abort status information is further configured to identify that the abort event arose due to an exception of the given type. ([0069] The fault buffer 216 stores fault buffer entries that indicate information related to page faults generated by the PPU 202. Fault buffer entries may include, for example, the type of access that was attempted (e.g., read, write, or atomic), the virtual memory address for which an attempted access caused a page fault, the virtual address space, and an indication of a unit or thread that caused a page fault. In operation, when the PPU 202 causes a page fault, the PPU 202 may write a fault buffer entry into the fault buffer 216 to inform the PPU fault handler 215 about the faulting page and the type of access that caused the fault. The PPU fault handler 215 then performs actions to remedy the page fault. The fault buffer 216 can store multiple faults because the PPU 202 is executing a plurality of threads, where each thread can cause a one or more faults due the pipelined nature of the memory accesses of the PPU 202.)
Regarding claim 9, Deming teaches:
The apparatus as claimed in Claim 1, wherein the given type of exception is a page fault exception. ([0070] As stated above, in response to receiving a request for translation of a virtual memory address, the CPU MMU 209 generates a page fault if the CPU page table 206 does not include a mapping associated with the requested virtual memory address or does not permit the type of access being requested. Similarly, in response to receiving a request for translation of a virtual memory address, the PPU MMU 213 generates a page fault if the PPU page table 208 does not include a mapping associated with the requested virtual memory address or does not permit the type of access being requested. When the CPU MMU 209 or the PPU MMU 213 generates a page fault, the thread that requested the data at the virtual memory address stalls, and a "local fault handler"--the CPU fault handler 211 for the CPU 102 or the PPU fault handler 215 for the PPU 202--attempts to remedy the page fault by executing a "page fault sequence." As indicated above, a page fault sequence includes a series of operations that enable the faulting unit (i.e., the unit--either the CPU 102 or the PPU 202--that caused the page fault) to access the data associated with the virtual memory address. After the page fault sequence completes, the thread that requested the data via the virtual memory address resumes execution. In some embodiments, fault recovery is simplified by allowing the fault recovery logic to track faulting memory accesses as opposed to faulting instructions. See also [0066-0069])
Regarding claim 10, Deming teaches:
The apparatus as claimed in Claim 9, wherein by causing the abort status information to identify that a retry of the transaction is suggested following abort of the transaction due to the given exception being a page fault exception, the apparatus is enabled to successfully retry the transaction in the event that the page fault exception is a transient exception, thereby avoiding a need to employ a fallback path provided by the software being executed on the processing circuitry, where the fallback path provides a non-transaction based mechanism for performing the data processing required by the transaction. ([0106] At step 512, the replay unit 350(0) waits for the CPU 102 to signal that one or more faults have been resolved via the replay signal. Upon receiving the replay signal, the replay unit 350(0) invalidates the uTLB 430 and re-executes the virtual memory transactions that are stored in the replay buffer 460. If, the uTLB 430 successfully maps a virtual memory transaction included in the replay buffer 460, then the replay unit 350(0) routes a corresponding physical memory transaction to the PPU memory 204. However, if the uTLB 430 is unable to map a particular virtual memory transaction included in the replay buffer 460, then the fault detector 450 performs a write operation that re-queues the virtual memory transaction in the replay buffer 460. If, at step 514, the replay unit 350(0) determines that the replay buffer 460 is not empty, then the method 500 returns to step 512. The replay unit 350(0) cycles through steps 512-514, re-executing virtual memory transactions included in the replay buffer 460, until the replay unit 350(0) determines that the replay buffer 460 is empty.)
Regarding claim 11, Deming teaches:
A apparatus as claimed in Claim 1, wherein the sequence of instructions forming the transaction is delimited by a transaction start instruction and a transaction end instruction. ([0094] The uTLB 430 performs one or more lookup operations to map the virtual memory address of the virtual memory transaction from the SM 310(0) to a physical memory address in the PPU memory 204. Notably, the uTLB 340 is configured to cache mappings, which are further represented by a hierarchy of TLB caches. A page table, or global TLB data structure, (not shown) is configured to store all mappings over all virtual address spaces associated with a processor complex comprising one or more PPUs 202, and one or more CPUs 102. [0095] As persons skilled in the art will recognize, in the event of a cache-miss, the lookup operations performed by the uTLB 430 may be time-consuming. Consequently, the in-flight buffer 440 queues the virtual memory transactions from the SM 310(0) in a first-in-first-out order, thereby retaining the context of each virtual memory transaction from the SM 310(0) with respect to the lookup operations of the uTLB 430.)
Regarding claim 12, Deming teaches:
The apparatus according to Claim 11, wherein the transactional memory support circuitry comprises restoration state storage circuitry to store transaction restoration state data captured in response to the transaction start instruction. ([0093] In general, threads executing within the SM 310(0) each generate a stream of virtual memory transactions from the SM 310(0). After the SM 310(0) issues a particular virtual memory transaction from the SM 310(0), the virtual memory transaction from the SM 310(0) passes through the transaction mux 420 before reaching the uTLB 430 and the in-flight buffer 440.[0094] The uTLB 430 performs one or more lookup operations to map the virtual memory address of the virtual memory transaction from the SM 310(0) to a physical memory address in the PPU memory 204. Notably, the uTLB 340 is configured to cache mappings, which are further represented by a hierarchy of TLB caches. A page table, or global TLB data structure, (not shown) is configured to store all mappings over all virtual address spaces associated with a processor complex comprising one or more PPUs 202, and one or more CPUs 102. [0095] As persons skilled in the art will recognize, in the event of a cache-miss, the lookup operations performed by the uTLB 430 may be time-consuming. Consequently, the in-flight buffer 440 queues the virtual memory transactions from the SM 310(0) in a first-in-first-out order, thereby retaining the context of each virtual memory transaction from the SM 310(0) with respect to the lookup operations of the uTLB 430.)
Regarding claim 13, Deming teaches:
The apparatus according to Claim 12, wherein the transactional memory support circuitry is configured to cause the transaction restoration state data to be restored in response to the abort event. ([0099] The PPU fault handler 215 then performs page fault sequences designed to resolve the memory access faults. Upon resolution of one or more memory access faults, the CPU 102 sends a replay signal to the replay unit 350(0). The CPU 102 may generate the replay signal in any technically feasible fashion at any time. Preferentially, the PPU fault hander 215 included in the CPU 102 generates the replay signal, generally via the command queue 214. In this way, access fault resolution processes that have a high overhead cost may be performed together, thereby improving overall performance. Generating the replay signal via the command queue 214 also allows the replay operation to be synchronized with commands that resolve faults, which pipelines fault-resolving operations and replay operations, allowing the PPU fault hander 215 to operate in a fire-and-forget manner. In alternate embodiments, the CPU 102 or the PPU 202 may generate the replay signal in any technically feasible fashion. For example, the PPU 202 may generate the replay signal at predetermined time intervals, causing periodic replays at a fixed frequency.)
Regarding claim 14, Deming teaches:
The apparatus as claimed in Claim 13, wherein the transactional memory support circuitry is configured to cause the transaction restoration state data to be restored before the exception handling routine is triggered. ([0071] The operations executed during a page fault sequence depend on the change in ownership state or change in access permissions, if any, that the memory page associated with the page fault has to undergo. The transition from a current ownership state to a new ownership state, or a change in access permissions, may be part of the page fault sequence. In some instances, migrating the memory page associated with the page fault from the system memory 104 to the PPU memory 204 is also part of the page fault sequence. In other instances, migrating the memory page associated with the page fault from the PPU memory 204 to the system memory 104 is also part of the page fault sequence. Various heuristics, more fully described herein, may be used to configure UVM system 200 to change memory page ownership state or to migrate memory pages under various sets of operating conditions and patterns. Described in greater detail below are page fault sequences for the following four memory page ownership state transitions: CPU-owned to CPU-shared, CPU-owned to PPU-owned, PPU-owned to CPU-owned, and PPU-owned to CPU-shared.)
Regarding claim 15, Deming teaches:
The apparatus as claimed in Claim 11, wherein the abort event detection circuitry is configured, in response to detection of the abort event, to cause the transaction start instruction to be re-executed by the processing circuitry in order to cause the abort status information to be stored for later reference when determining whether to retry execution of the transaction. ([0106] At step 512, the replay unit 350(0) waits for the CPU 102 to signal that one or more faults have been resolved via the replay signal. Upon receiving the replay signal, the replay unit 350(0) invalidates the uTLB 430 and re-executes the virtual memory transactions that are stored in the replay buffer 460. If, the uTLB 430 successfully maps a virtual memory transaction included in the replay buffer 460, then the replay unit 350(0) routes a corresponding physical memory transaction to the PPU memory 204. However, if the uTLB 430 is unable to map a particular virtual memory transaction included in the replay buffer 460, then the fault detector 450 performs a write operation that re-queues the virtual memory transaction in the replay buffer 460. If, at step 514, the replay unit 350(0) determines that the replay buffer 460 is not empty, then the method 500 returns to step 512. The replay unit 350(0) cycles through steps 512-514, re-executing virtual memory transactions included in the replay buffer 460, until the replay unit 350(0) determines that the replay buffer 460 is empty.)
Regarding claim 16, Deming teaches:
The apparatus according to Claim 11, wherein the processing circuitry is configured to prevent commitment of results of the speculatively executed instructions until the transaction end instruction is reached. ([0100] Upon receiving the replay signal, the replay unit 350(0) invalidates the uTLB 430, and the transaction mux 320 routes the faulting virtual memory transactions in the replay buffer 460 to the uTLB 430. For each of these faulting virtual memory transactions, the uTLB 430 attempts to map the virtual memory address to an accessible physical memory address. If, the uTLB 430 successfully maps a virtual memory transaction included in the replay buffer 460, then the fault detector 450 routes a corresponding physical memory transaction to the PPU memory 204. However, if the uTLB 430 is unable to map a particular virtual memory transaction included in the replay buffer 460, then the fault detector 450 performs write operations that re-queues the virtual memory transaction in the replay buffer 460. Notably, as the CPU 102 successfully remedies the reason for each particular page fault, the corresponding virtual memory transaction succeeds, a physical memory transaction is generated, and the virtual memory transaction is removed from the replay buffer 460.)
Regarding claim 17, Deming teaches:
A method of handling transactions in an apparatus, comprising: (1. A computer-implemented method for processing virtual memory transactions associated with a multithreaded processing unit, the method comprising:)
employing processing circuitry to perform data processing in response to instructions. ([0091] FIG. 3 is a block diagram illustrating a unified virtual memory system (UVM) 200 configured with replay units 350, according to another embodiment of the present invention. The PPU 202 includes any number N of streaming multiprocessors (SMs) 310 and N replay units 350--one replay unit 350 per SM 310. For instance, if the PPU 202 were to include thirty-two SMs 310(0:31), then the PPU 202 would include thirty-two replay units 350(0:31). Each replay unit 350 enables the PPU 202 to stall the corresponding SM 310 while replaying selected memory transactions, without delaying the other SMs 310.[0092] FIG. 4 is a conceptual diagram illustrating the replay unit 350(0) of FIG. 3, according to one embodiment of the present invention. As shown, the replay unit 350(0) includes, without limitation, a transaction multiplexer (transaction mux) 420, a micro-translation lookaside buffer (uTLB) 430, an in-flight buffer 440, a fault detector 450, and a replay buffer 460.)
employing transactional memory support circuitry to support execution of a transaction within a thread of data processing by the processing circuitry, the transaction comprising a sequence of instructions executed speculatively and for which the processing circuitry is configured to prevent commitment of results of the speculatively executed instructions until the transaction has reached a transaction end point .( [0100] Upon receiving the replay signal, the replay unit 350(0) invalidates the uTLB 430, and the transaction mux 320 routes the faulting virtual memory transactions in the replay buffer 460 to the uTLB 430. For each of these faulting virtual memory transactions, the uTLB 430 attempts to map the virtual memory address to an accessible physical memory address. If, the uTLB 430 successfully maps a virtual memory transaction included in the replay buffer 460, then the fault detector 450 routes a corresponding physical memory transaction to the PPU memory 204. However, if the uTLB 430 is unable to map a particular virtual memory transaction included in the replay buffer 460, then the fault detector 450 performs write operations that re-queues the virtual memory transaction in the replay buffer 460. Notably, as the CPU 102 successfully remedies the reason for each particular page fault, the corresponding virtual memory transaction succeeds, a physical memory transaction is generated, and the virtual memory transaction is removed from the replay buffer 460.)
detecting an abort event by the transactional memory support circuitry before the transaction has reached the transaction end point and aborting execution of the transaction; ([0069] The fault buffer 216 stores fault buffer entries that indicate information related to page faults generated by the PPU 202. Fault buffer entries may include, for example, the type of access that was attempted (e.g., read, write, or atomic), the virtual memory address for which an attempted access caused a page fault, the virtual address space, and an indication of a unit or thread that caused a page fault. In operation, when the PPU 202 causes a page fault, the PPU 202 may write a fault buffer entry into the fault buffer 216 to inform the PPU fault handler 215 about the faulting page and the type of access that caused the fault. The PPU fault handler 215 then performs actions to remedy the page fault. The fault buffer 216 can store multiple faults because the PPU 202 is executing a plurality of threads, where each thread can cause a one or more faults due the pipelined nature of the memory accesses of the PPU 202.Page Fault sequences [0070] As stated above, in response to receiving a request for translation of a virtual memory address, the CPU MMU 209 generates a page fault if the CPU page table 206 does not include a mapping associated with the requested virtual memory address or does not permit the type of access being requested. Similarly, in response to receiving a request for translation of a virtual memory address, the PPU MMU 213 generates a page fault if the PPU page table 208 does not include a mapping associated with the requested virtual memory address or does not permit the type of access being requested. When the CPU MMU 209 or the PPU MMU 213 generates a page fault, the thread that requested the data at the virtual memory address stalls, and a "local fault handler"--the CPU fault handler 211 for the CPU 102 or the PPU fault handler 215 for the PPU 202--attempts to remedy the page fault by executing a "page fault sequence." As indicated above, a page fault sequence includes a series of operations that enable the faulting unit (i.e., the unit--either the CPU 102 or the PPU 202--that caused the page fault) to access the data associated with the virtual memory address. After the page fault sequence completes, the thread that requested the data via the virtual memory address resumes execution. In some embodiments, fault recovery is simplified by allowing the fault recovery logic to track faulting memory accesses as opposed to faulting instructions.)
on aborting execution of the transaction, storing abort status information for later reference to determine whether to retry execution of the transaction; and determining that the abort event arises due to a given exception event of a given type, capturing syndrome information to see (Page Fault Sequences[0070] As stated above, in response to receiving a request for translation of a virtual memory address, the CPU MMU 209 generates a page fault if the CPU page table 206 does not include a mapping associated with the requested virtual memory address or does not permit the type of access being requested. Similarly, in response to receiving a request for translation of a virtual memory address, the PPU MMU 213 generates a page fault if the PPU page table 208 does not include a mapping associated with the requested virtual memory address or does not permit the type of access being requested. When the CPU MMU 209 or the PPU MMU 213 generates a page fault, the thread that requested the data at the virtual memory address stalls, and a "local fault handler"--the CPU fault handler 211 for the CPU 102 or the PPU fault handler 215 for the PPU 202--attempts to remedy the page fault by executing a "page fault sequence." As indicated above, a page fault sequence includes a series of operations that enable the faulting unit (i.e., the unit--either the CPU 102 or the PPU 202--that caused the page fault) to access the data associated with the virtual memory address. After the page fault sequence completes, the thread that requested the data via the virtual memory address resumes execution. In some embodiments, fault recovery is simplified by allowing the fault recovery logic to track faulting memory accesses as opposed to faulting instructions. [0071] The operations executed during a page fault sequence depend on the change in ownership state or change in access permissions, if any, that the memory page associated with the page fault has to undergo. The transition from a current ownership state to a new ownership state, or a change in access permissions, may be part of the page fault sequence. In some instances, migrating the memory page associated with the page fault from the system memory 104 to the PPU memory 204 is also part of the page fault sequence. In other instances, migrating the memory page associated with the page fault from the PPU memory 204 to the system memory 104 is also part of the page fault sequence. Various heuristics, more fully described herein, may be used to configure UVM system 200 to change memory page ownership state or to migrate memory pages under various sets of operating conditions and patterns. Described in greater detail below are page fault sequences for the following four memory page ownership state transitions: CPU-owned to CPU-shared, CPU-owned to PPU-owned, PPU-owned to CPU-owned, and PPU-owned to CPU-shared. [0072] A fault by the PPU 202 may initiate a transition from CPU-owned to CPU-shared. Prior to such a transition, a thread executing in the PPU 202 attempts to access data at a virtual memory address that is not mapped in the PPU page table 208. This access attempt causes a PPU-based page fault, which then causes a fault buffer entry to be written to the fault buffer 216. In response, the PPU fault handler 215 reads the PSD 210 entry corresponding to the virtual memory address and identifies the memory page associated with the virtual memory address. After reading the PSD 210, the PPU fault handler 215 determines that the current ownership state for the memory page associated with the virtual memory address is CPU-owned. Based on the current ownership state as well as other factors, such as usage characteristics for the memory page or the type of memory access, the PPU fault handler 215 determines that a new ownership state for the page should be CPU-shared.)
Regarding claim 18, Deming teaches:
A computer program for controlling a host data processing apparatus to provide an instruction execution environment, comprising: (Claim 10. A computer-readable storage medium including instructions that, when executed by a multithreaded processing unit, cause the multithreaded processing unit to process virtual memory transactions by performing the steps of:)
processing program logic to support execution of a transaction within a thread of data processing, the transaction comprising a sequence of instructions executed speculatively and for which the processing program logic is configured to prevent commitment of results of the speculatively executed instructions until the transaction has reached a transaction end point; and ([0091] FIG. 3 is a block diagram illustrating a unified virtual memory system (UVM) 200 configured with replay units 350, according to another embodiment of the present invention. The PPU 202 includes any number N of streaming multiprocessors (SMs) 310 and N replay units 350--one replay unit 350 per SM 310. For instance, if the PPU 202 were to include thirty-two SMs 310(0:31), then the PPU 202 would include thirty-two replay units 350(0:31). Each replay unit 350 enables the PPU 202 to stall the corresponding SM 310 while replaying selected memory transactions, without delaying the other SMs 310.[0092] FIG. 4 is a conceptual diagram illustrating the replay unit 350(0) of FIG. 3, according to one embodiment of the present invention. As shown, the replay unit 350(0) includes, without limitation, a transaction multiplexer (transaction mux) 420, a micro-translation lookaside buffer (uTLB) 430, an in-flight buffer 440, a fault detector 450, and a replay buffer 460. [0093] In general, threads executing within the SM 310(0) each generate a stream of virtual memory transactions from the SM 310(0). After the SM 310(0) issues a particular virtual memory transaction from the SM 310(0), the virtual memory transaction from the SM 310(0) passes through the transaction mux 420 before reaching the uTLB 430 and the in-flight buffer 440.[0094] The uTLB 430 performs one or more lookup operations to map the virtual memory address of the virtual memory transaction from the SM 310(0) to a physical memory address in the PPU memory 204. Notably, the uTLB 340 is configured to cache mappings, which are further represented by a hierarchy of TLB caches. A page table, or global TLB data structure, (not shown) is configured to store all mappings over all virtual address spaces associated with a processor complex comprising one or more PPUs 202, and one or more CPUs 102. See also [0102-0107])
abort event program logic to cause execution of the transaction to be aborted when an abort event is detected before the transaction has reached the transaction end point, and to cause abort status information to be stored for later reference when determining whether to retry execution of the transaction. ([0096] If the uTLB successfully processes the virtual request from the SM 310(0), then the fault detector 450 routes a corresponding physical memory transaction to the PPU memory 204. In the physical memory transaction to the PPU memory 204, the virtual address included in the virtual memory transaction from the SM 310(0) is replaced with the physical address resulting from the uTLB 430 lookup operations. [0097] By contrast, if uTLB 430 is unable to map the virtual address specified by the virtual memory transaction from the SM 310, or if the virtual address requires a change of disposition of a target page of memory, then the uTLB 430 generates a memory access fault. The fault detector 450 processes the memory access fault--sending a fault signal to the CPU 120 and temporarily inhibiting the SM 310(0) from issuing new virtual memory transactions. Advantageously, the fault detector 450 does not cause any other SMs 310 included in the PPU 202 to cease issuing new virtual memory transactions. [0098] As part of processing the memory access fault, the fault detector 450 causes a fault buffer entry to be written to the fault buffer 216 of FIG. 2. And the fault detector 450 performs write operations that store the faulting virtual memory transaction from the SM 310(0) in the replay buffer 460. Further, the fault detector 450 causes any virtual memory transactions from the SM 310(0) that are queued in the in-flight buffer 440 to finish executing. If any of these virtual memory transactions also fault, then the fault detector 450 performs write operations that store the additional faulting virtual memory transactions in the replay buffer 460. Optionally, but preferentially, the fault detector 450 causes fault buffer entries corresponding to the additional faulting virtual memory transactions to be written to the fault buffer 216.)
wherein when the abort event arises due to a given exception event of a given type, the abort event program logic is configured to cause syndrome information to be captured for use when seeking to resolve the given exception event, and to cause the abort status information to identify that a retry of the transaction is suggested at least in the event that the given exception event is transient and resolved.([0069] The fault buffer 216 stores fault buffer entries that indicate information related to page faults generated by the PPU 202. Fault buffer entries may include, for example, the type of access that was attempted (e.g., read, write, or atomic), the virtual memory address for which an attempted access caused a page fault, the virtual address space, and an indication of a unit or thread that caused a page fault. In operation, when the PPU 202 causes a page fault, the PPU 202 may write a fault buffer entry into the fault buffer 216 to inform the PPU fault handler 215 about the faulting page and the type of access that caused the fault. The PPU fault handler 215 then performs actions to remedy the page fault. The fault buffer 216 can store multiple faults because the PPU 202 is executing a plurality of threads, where each thread can cause a one or more faults due the pipelined nature of the memory accesses of the PPU 202. [0098] As part of processing the memory access fault, the fault detector 450 causes a fault buffer entry to be written to the fault buffer 216 of FIG. 2. And the fault detector 450 performs write operations that store the faulting virtual memory transaction from the SM 310(0) in the replay buffer 460. Further, the fault detector 450 causes any virtual memory transactions from the SM 310(0) that are queued in the in-flight buffer 440 to finish executing. If any of these virtual memory transactions also fault, then the fault detector 450 performs write operations that store the additional faulting virtual memory transactions in the replay buffer 460. Optionally, but preferentially, the fault detector 450 causes fault buffer entries corresponding to the additional faulting virtual memory transactions to be written to the fault buffer 216. [0099] The PPU fault handler 215 then performs page fault sequences designed to resolve the memory access faults. Upon resolution of one or more memory access faults, the CPU 102 sends a replay signal to the replay unit 350(0). The CPU 102 may generate the replay signal in any technically feasible fashion at any time. Preferentially, the PPU fault hander 215 included in the CPU 102 generates the replay signal, generally via the command queue 214. In this way, access fault resolution.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to CARLOS A ESPANA whose telephone number is (703)756-1069. The examiner can normally be reached Monday - Friday 8 a.m - 5 p.m EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, LEWIS BULLOCK JR can be reached at (571)272-3759. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/C.A.E./Examiner, Art Unit 2199
/LEWIS A BULLOCK JR/Supervisory Patent Examiner, Art Unit 2199
Prosecution Insights