DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This Office Action is in response to the Applicants’ Election/Restriction filed on March 30, 2026. Applicant elects the invention of Group I, Claims 1-14; claims 15-20 non-elected. Claims 1-6, 8-9 and 11-14 have been amended; claims 1-14 are currently presented in the instant application.
Drawings
The drawings submitted on August 29, 2023. These drawings are reviewed and accepted by the examiner.
Information Disclosure Statement
The information Disclosure Statement (IDS) Forms PTO-1449, filed on November 29, 2023 in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosed therein was considered by the examiner.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 5 and 7 are rejected under 35 U.S.C. 103 as being unpatentable over Choi et al. (US 20220051017 A1) in view of Cowperthwaite et al. (US 20230297440 A1).
Regarding claim 1. (Currently Amended) Choi discloses in a GPU based system that uses unified memory addressing to enable multiple processing cores to have a common unified view into memory and access each other's locally connected memory (Choi, see [0277] As described herein, although various multi-core processors 1405 and GPUs 1410 may be physically coupled to a particular memory 1401, 1420, respectively, and/or a unified memory architecture may be implemented in which a virtual system address space (also referred to as “effective address” space) is distributed among various physical memories. For example, processor memories 1401(1)-1401(M) may each comprise 64 GB of system memory address space and GPU memories 1420(1)-1420(N) may each comprise 32 GB of system memory address space resulting in a total of 256 GB addressable memory when M=2 and N=4. Other values for N and M are possible.);
the GPU based system capable of allowing data storage by an application to range across plural storage components comprising the common unified view into memory (Choi, see at least par. [0171] In at least one embodiment, GPU(s) 908 may include unified memory technology. In at least one embodiment, address translation services (“ATS”) support may be used to allow GPU(s) 908 to access CPU(s) 906 page tables directly. In at least one embodiment, embodiment, when a GPU of GPU(s) 908 memory management unit (“MMU”) experiences a miss, an address translation request may be transmitted to CPU(s) 906. In response, 2 CPU of CPU(s) 906 may look in its page tables for a virtual-to-physical mapping for an address and transmit translation back to GPU(s) 908, in at least one embodiment. In at least one embodiment, unified memory technology may allow a single unified virtual address space for memory of both CPU(s) 906 and GPU(s) 908, thereby simplifying GPU(s) 908 programming and porting of applications to GPU(s) 908.]),
Choi does not disclose GPU address mapping hardware configured to, in response to a request from the application, selectively confine a scope of memory access by at least one function of the application to memory locally connected to a processing core executing the application. However,
Cowperthwaite discloses:
GPU address mapping hardware configured to, in response to a request from the application, selectively confine a scope of memory access by at least one function of the application to memory locally connected to a processing core executing the application (Cowperthwaite, see par. [0085] The instruction cache 252 may receive a stream of instructions to execute from the pipeline manager 232. The instructions are cached in the instruction cache 252 and dispatched for execution by the instruction unit 254. The instruction unit 254 can dispatch instructions as thread groups (e.g., warps), with each thread of the thread group assigned to a different execution unit within GPGPU core 262. An instruction can access any of a local, shared, or global address space by specifying an address within a unified address space. The address mapping unit 256 can be used to translate addresses in the unified address space into a distinct memory address that can be accessed by the load/store units 266. [0086] The register file 258 provides a set of registers for the functional units of the graphics multiprocessor 234. The register file 258 provides temporary storage for operands connected to the data paths of the functional units (e.g., GPGPU cores 262, load/store units 266) of the graphics multiprocessor 234. The register file 258 may be divided between each of the functional units such that each functional unit is allocated a dedicated portion of the register file 258. For example, the register file 258 may be divided between the different warps being executed by the graphics multiprocessor 234. [0087] The GPGPU cores 262 can each include floating point units (FPUs) and/or integer arithmetic logic units (ALUs) that are used to execute instructions of the graphics multiprocessor 234. In some implementations, the GPGPU cores 262 can include hardware logic that may otherwise reside within the tensor and/or ray-tracing cores 263. The GPGPU cores 262 can be similar in architecture or can differ in architecture. For example and in one embodiment, a first portion of the GPGPU cores 262 include a single precision FPU and an integer ALU while a second portion of the GPGPU cores include a double precision FPU. Optionally, the FPUs can implement the IEEE 754-2008 standard for floating point arithmetic or enable variable precision floating point arithmetic. The graphics multiprocessor 234 can additionally include one or more fixed function or special function units to perform specific functions such as copy rectangle or pixel blending operations. One or more of the GPGPU cores can also include fixed or special function logic).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filling date of claimed invention to combine the method and system of Choi, and have GPU address mapping hardware configured to, in response to a request from the application, selectively confine a scope of memory access by at least one function of the application to memory locally connected to a processing core executing the application, as provided by Cowperthwaite. The modification provide an improved system and method to enable multiple processing cores to have a common unified view into memory and access each other's locally connected memory, thereby to efficiency provided by parallel machine learning algorithm implementations allows the use of high-capacity networks and enables those networks to be trained on larger datasets. (Cowperthwaite, see par. [0176]).
Regarding claim 5. Choi in view of Cowperthwaite further discloses the GPU address mapping hardware of claim 1 (as rejected above), and Choi in view of Cowperthwaite further discloses further including a hardware scheduler that selectively restricts confines execution of the application to the processing core (Choi, see par. [0506], [0526]).
Regarding claim 7. Choi in view of Cowperthwaite further discloses the GPU address mapping hardware of claim 1 (as rejected above), and Choi in view of Cowperthwaite further discloses wherein the GPU based system enables access by the multiple processing cores of each other's locally connected memory via chip-to-chip network connectivity (Choi, see at least par. [0114] In at least one embodiment, inference and/or training logic 615 may include, without limitation, code and/or data storage 601 to store forward and/or output weight and/or input/output data, and/or other parameters to configure neurons or layers of a neural network trained and/or used for inferencing in aspects of one or more embodiments. In at least one embodiment, training logic 615 may include, or be coupled to code and/or data storage 601 to store graph code or other software to control timing and/or order, in which weight and/or other parameter information is to be loaded to configure, logic, including integer and/or floating point units (collectively, arithmetic logic units (ALUs). In at least one embodiment, code, such as graph code, loads weight or other parameter information into processor ALUs based on an architecture of a neural network to which such code corresponds. In at least one embodiment, code and/or data storage 601 stores weight parameters and/or input/output data of each layer of a neural network trained or used in conjunction with one or more embodiments during forward propagation of input/output data and/or weight parameters during training and/or inferencing using aspects of one or more embodiments. In at least one embodiment, any portion of code and/or data storage 601 may be included with other on-chip or off-chip data storage, including a processor's L1, L2, or L3 cache or system memory.).
Claim 6 is rejected under 35 U.S.C. 103 as being unpatentable over Choi et al. (US 20220051017 A1) in view of Cowperthwaite et al. (US 20230297440 A1), as applied claim 5 above, and further in view of Wan et al. (US 20090210649 A1).
Regarding claim 6. (Currently Amended) Choi in view of Cowperthwaite discloses the GPU address mapping hardware of claim 5 (as rejected above), but Choi in view of Cowperthwaite does not discloses wherein the hardware scheduler selectively restricts confines execution in response to an affinity mask. However,
Wan, discloses:
wherein the hardware scheduler selectively restricts confines execution in response to an affinity mask (Wan, see at least par. [0126] To change from the first mode to the second mode, the exemplary embodiments perform the actions which are illustrated in FIG. 5. Here, the FUSE 194 requests a current load status from the load monitor 22 as illustrated at {circle around (1)} and the load status is provided as at {circle around (2)}. In response, the system selects one of the processors which is currently lightly loaded and the affinity unit 23 sets affinity for the target code 21, in this case threads T1 and T2, to the selected processor as at {circle around (3)}. For example, program SC-AP1 was executing on processor P1 at the time of the intercepted system call but the current load status indicates that processor “P2” would be most appropriate for future execution. Thus, the affinity unit 23 sets affinity to the indicated processor P2. This is a hard affinity. That is, the affinity set by the affinity unit 23 overrides the soft affinity of the system. As a result, the existing thread T1 and the newly created thread T2 always now execute on the selected processor P2. In Linux-based systems affinity is set by a system command of the form “taskset [options] [mask|list] [pid|command [arg] . . . ].” Similar commands exist on other systems to the same general effect. The result is that the multiple threads of the particular program SC-AP1 all now execute on the same processor. Any further threads initiated by the relevant program SC-AP1 will also have affinity set to the selected processor P2 and in effect are locked to execute together on a single selected processor.).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filling date of claimed invention to combine the method and system of Choi, and have wherein the hardware scheduler selectively restricts confines execution in response to an affinity mask, as provided by Wan. The modification provide an improved system and method to dynamically keep the needed program codes and data on fast-access on-chip memory based on the state of the user programs being processed, therefore to provide a multiprocessor computer system in which memory consistency errors are reduced. Another aim of at least some exemplary embodiments of the present invention is to provide a multiprocessor computer system in which memory consistency errors are reduced when executing code produced by automatic program code conversion such as dynamic binary translation. (Wan, see par. [0009]).
Claims 8 and 14 are rejected under 35 U.S.C. 103 as being unpatentable over Choi et al. (US 20220051017 A1) in view of Sandstrom (US 20140229669 A1).
Regarding claim 8. Choi discloses In a GPU based system that uses unified memory addressing to enable applications executing on multiple processing cores to have a common unified view into memory and access locally connected memory of the multiple processing cores (Choi, at least par. [0158] In at least one embodiment, any number of stereo camera(s) 968 may also be included in a front-facing configuration. In at least one embodiment, one or more of stereo camera(s) 968 may include an integrated control unit comprising a scalable processing unit, which may provide a programmable logic (“FPGA”) and a multi-core micro-processor with an integrated Controller Area Network (“CAN”) or Ethernet interface on a single chip. In at least one embodiment, such a unit may be used to generate a 3D map of an environment of vehicle 900, including a distance estimate for all points in an image. In at least one embodiment, one or more of stereo camera(s) 968 may include, without limitation, compact stereo vision sensor(s) that may include, without limitation, two camera lenses (one each on left and right) and an image processing chip that may measure distance from vehicle 900 to target object and use generated information (e.g., metadata) to activate autonomous emergency braking and lane departure warning functions. In at least one embodiment, other types of stereo camera(s) 968 may be used in addition to, or alternatively from, those described herein.), the GPU based system configured to range data storage across dynamic random access memory connected to different ones of the multiple processing cores (Choi, see at least [0115] In at least one embodiment, any portion of code and/or data storage 601 may be internal or external to one or more processors or other hardware logic devices or circuits. In at least one embodiment, code and/or code and/or data storage 601 may be cache memory, dynamic randomly addressable memory (“DRAM”), static randomly addressable memory (“SRAM”), non-volatile memory (e.g., flash memory), or other storage. In at least one embodiment, a choice of whether code and/or code and/or data storage 601 is internal or external to a processor, for example, or comprising DRAM, SRAM, flash or some other storage type may depend on available storage on-chip versus off-chip, latency requirements of training and/or inferencing functions being performed, batch size of data used in inferencing and/or training of a neural network, or some combination of these factors.), a memory access (Choi, see at least par. [0170] In at least one embodiment, one or more of GPU(s) 908 may include a high bandwidth memory (“HBM) and/or a 16 GB HBM2 memory subsystem to provide, in some examples, about 900 GB/second peak memory bandwidth. In at least one embodiment, in addition to, or alternatively from, HBM memory, a synchronous graphics random-access memory (“SGRAM”) may be used, such as a graphics double data rate type five synchronous random-access memory (“GDDR5”).) method comprising:
launching execution of an application on a processing core (Choi, see at least par. [0554] In at least one embodiment, shared storage may be mounted to AI services 3618 within system 3600. In at least one embodiment, shared storage may operate as a cache (or other storage device type) and may be used to process inference requests from applications. In at least one embodiment, when an inference request is submitted, a request may be received by a set of API instances of deployment system 3506, and one or more instances may be selected (e.g., for best fit, for load balancing, etc.) to process a request. In at least one embodiment, to process a request, a request may be entered into a database, a machine learning model may be located from model registry 3524 if not already in a cache, a validation step may ensure appropriate machine learning model is loaded into a cache (e.g., shared storage), and/or a copy of a model may be saved to a cache. In at least one embodiment, a scheduler (e.g., of pipeline manager 3612) may be used to launch an application that is referenced in a request if an application is not already running or if there are not enough instances of an application. In at least one embodiment, if an inference server is not already launched to execute a model, an inference server may be launched. In at least one embodiment, any number of inference servers may be launched per model. In at least one embodiment, in a pull model, in which inference servers are clustered, models may be cached whenever load balancing is advantageous. In at least one embodiment, inference servers may be statically loaded in corresponding,), and
Choi does not disclose in response to a request from the application, selectively restricting a scope of memory access by the application executing on the processing core to dynamic random access memory locally connected to the processing core. However,
Sandstrom discloses:
in response to a request from the application, selectively restricting a scope of memory access by the application executing on the processing core to dynamic random access memory locally connected to the processing core (Sandstrom, see par. [0046] Note that the terms on-chip and off-chip memories as used herein (incl. reference elements in FIG. 1) are not necessarily in all embodiments of the invention on or off chip, respectively. Rather, the naming conventions of on vs. off chip memory refer to the access latency to the particular memory type: on-chip memory typically provides faster access than off-chip. Hence, the inventive techniques can be used for optimizing memory system performance in various types of scenarios where there are tradeoffs in terms of volumes and access latencies of a given memory class; typically, the higher the storage capacity of a given memory type, the greater its access latency. Thus, despite the references to off-chip memory for convenience in this specification, it shall be understood that e.g. memory 850 can in certain embodiments of the invention also be implemented (at least in part) as on-chip, however still so that its greater storage capacity comes at the expense of higher access latency compared to the memory 550. Also, please note that the term access in this specification generally covers both read and write access. Moreover, the term RAM that is used for convenience in referring to memories herein is not to be understood in a restrictive sense: in various embodiments of the invention, the memories 550 and 850 can encompass memory types also other than random access memory. For instance, certain ranges of these memories can be read-only, etc. What's more, though this and the referenced patent applications mainly disclose processors with multiple processing cores, it shall be understood that the inventive memory access and task scheduling optimization concepts can also be used in cases of a single core processor. Furthermore, references to task instances in this disclosure can refer, depending on the overall system architecture, e.g. the processing system architectures per the referenced patent applications [5] and [4], both to instances of the same task type or different task types..).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filling date of claimed invention to combine the method and system of Choi, and have in response to a request from the application, selectively restricting a scope of memory access by the application executing on the processing core to dynamic random access memory locally connected to the processing core, as provided by Sandstrom. The modification provide an improved system and method to dynamically keep the needed program codes and data on fast-access on-chip memory based on the state of the user programs being processed. As such, the more dynamic the system software tries to make the fast- and slow-access memory content optimizations, the greater the portion of the system processing core capacity that is consumed by the system software tasks (e.g. copying memory contents between memory types of different access latencies and storage capacities) rather than processing user programs. On the other hand, without dynamic optimization of fast-access, fast-access memory contents, the user program processing performance is likely to at least periodically suffer from long access times to instructions/data residing at slow-access off-chip memories.. (Sandstrom, see par. [0006]).
Regarding claim 14. (Currently Amended) Choi in view of Sandstrom discloses the method of claim 8 (as rejected above), and Choi in view of Sandstrom further discloses further including enabling access by the multiple processing cores of each other's locally connected memory via chip-to-chip network connectivity (Choi, see at least par. [0158] In at least one embodiment, any number of stereo camera(s) 968 may also be included in a front-facing configuration. In at least one embodiment, one or more of stereo camera(s) 968 may include an integrated control unit comprising a scalable processing unit, which may provide a programmable logic (“FPGA”) and a multi-core micro-processor with an integrated Controller Area Network (“CAN”) or Ethernet interface on a single chip. In at least one embodiment, such a unit may be used to generate a 3D map of an environment of vehicle 900, including a distance estimate for all points in an image. In at least one embodiment, one or more of stereo camera(s) 968 may include, without limitation, compact stereo vision sensor(s) that may include, without limitation, two camera lenses (one each on left and right) and an image processing chip that may measure distance from vehicle 900 to target object and use generated information (e.g., metadata) to activate autonomous emergency braking and lane departure warning functions. In at least one embodiment, other types of stereo camera(s) 968 may be used in addition to, or alternatively from, those described herein.).
Allowable Subject Matter
Claims 2-4 and 9-13 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Regarding claim 2. Choi in view of Cowperthwaite disclose the GPU address mapping hardware of claim 1 (as reject above). However, the limitation:
“wherein the GPU address mapping hardware is further configured to selectively expand the scope of the memory access to striding memory not locally connected to the processing core executing the application. “, taken as a
whole, render the claims patentably distinct over the prior art.
Regarding claim 3. Choi in view of Cowperthwaite disclose the GPU address mapping hardware of claim 1 (as reject above). However, the limitation:
wherein the GPU address mapping hardware is further configured to selectively confine the scope of the memory access in response to receipt of a localization attribute from the application. taken as a whole, render the claims patentably distinct over the prior art.
Regarding claim 4. is object is because of its dependency.
Regarding claim 9. Choi in view Sandstrom the method of claim 8 (as reject above). However, the limitation:
“further including selectively expanding the scope of the memory access to striding dynamic random access memory that is not locally connected to the processing core.”, taken as a whole, render the claims patentably distinct over the prior art.
Regarding claim 10. Choi in view Sandstrom the method of claim 8 (as reject above). However, the limitation:
“selectively restricting the scope of the memory access is performed in response to receipt of a localization attribute from the application.” taken as a whole, render the claims patentably distinct over the prior art.
Regarding claims 11-13. are object is because of their dependencies.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to KIM THANH THI TRAN whose telephone number is (571)270-1408. The examiner can normally be reached Monday-Friday 8:00am-5:00pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, ALICIA HARRINGTON can be reached at 5712722330. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
KIM THANH T TRAN/Examiner, Art Unit 2615
/JAMES A THOMPSON/Primary Examiner, Art Unit 2615
Prosecution Insights