Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement filed 9/2/2023 fails to comply with 37 CFR 1.98(a)(2), which requires a legible copy of each cited foreign patent document; each non-patent literature publication or that portion which caused it to be listed; and all other information or that portion which caused it to be listed. It has been placed in the application file, but the information referred to therein has not been considered. While the abstract for each of the foreign patent documents was considered, the NPL document is not translated and cannot be considered in review of the application as it is not legible / translated.
Claim Rejections - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claims 4-6 and 17-19 are rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention. Claim 4 and 17 both indicate - in response to a sub-engine sending a data request to another sub-engine, subtracting a size of the data to be requested in the data request from the first counter in the sub-engine sending the data request, and adding the size of the data to be requested in the data request to the second counter in the another sub-engine receiving the data request, wherein the size of the data to be requested in the data request is not greater than the size of the corresponding cache space. Thus the limitation implies the Applicant has possession of the size of the data to be requested in the data request is not greater than (e.g. smaller) than the size of the corresponding cache space. However, the specification indicates:
PNG
media_image1.png
98
583
media_image1.png
Greyscale
PNG
media_image2.png
387
610
media_image2.png
Greyscale
In the highlighted portion of specification amended paragraphs 0088 and 0089, the required cache will be much less than the data block of the sub-task in size wherein the two counters are realized when avoiding an overflow when a small data cache is used to process a large data block. The identified claims are the inverse wherein the data cache is large and the data block is small. Thus, the claims (even in there original form) do not show possession of the claim subject matter as the inverse of the claim is shown in possession in the specification. Claims 5-6 and 18-19 are rejected based on there dependency on claims 4 and 17. Note MPEP 2163, section I and II.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1, 2, 7-15 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over CHEN et al (Publication 2023/0038051) in view of PEERCY et al (Publication 2022/0197698).
As to claim 1, CHEN teaches a task scheduling method, comprising:
in response to receiving an issued task, dividing the task into a plurality of sub-tasks (splitting computing task into N subtasks and storing the divided task), wherein a task parameter corresponding to each of the sub-tasks is recorded, and the task parameter comprises a start phase of a next sub-task ([0086] After obtaining a computing task, the main processor splits the computing task into N subtasks. N is less than or equal to M. In an example, the N subtasks are concurrently executed. In this case, the main processor selects N coprocessors based on the device ID of each coprocessor, and separately distributes the N subtasks to the N coprocessors for processing. In another example, some subtasks in the N subtasks are concurrently executed, and some subtasks need to be executed after other subtasks are executed, that is, serially executed. In this case, several coprocessors may be selected based on a requirement for processing, and the subtasks are separately distributed to the selected coprocessors. [0088] In the process of executing the subtask, the coprocessor stores, in the shared memory, data generated through execution of the subtask. In a process of executing the subtask, if one coprocessor requires data generated in a process in which another coprocessor executes the subtask, the coprocessor may obtain, based on a mapping relationship, a storage address of data of a subtask processed by the device ID of the another coprocessor, and then read the data from a storage space corresponding to the storage address. When the coprocessor reads the data from the storage space corresponding to the storage address, a direct memory access (DMA) technology or another data reading technology may be used for implementation. For related descriptions of the DMA technology, refer to the conventional technology. Details are not described in this application.
[0089] The foregoing mapping relationship includes the device ID of each coprocessor, an identifier of the subtask executed by each coprocessor, and a storage address of a storage space used to store data generated in a process in which each coprocessor executes the subtask. The mapping relationship may be stored in the shared memory. The storage address of the storage space stored in the mapping relationship may be notified to each coprocessor when the mapping relationship is configured. A specific configuration manner and a specific maintenance manner of the mapping relationship are described in detail subsequently, and details are not described herein again.);
sending, by a scheduler (task scheduling engine), the task parameter (start condition) of a sub-task (subtask) to be processed to a corresponding sub-engine (coprocessor / accelerator / first processor / second processor) ([0021] In an embodiment, the first coprocessor receives the first subtask delivered by a main processor, and starts to execute the first subtask. The foregoing coprocessors (including the first coprocessor and the second coprocessor) are included in the AI system. The AI system may further include the main processor. The main processor herein may be a CPU. The main processor may be configured to split a task, and deliver a split subtask to the coprocessor, so that the coprocessor assists the main processor in executing the task, thereby improving computing power of the AI system. In the foregoing design, the main processor delivers the subtask to the coprocessor, that is, triggers the coprocessor to execute the subtask.
[0022] In an embodiment, the main processor may send a start condition of the subtask to the coprocessor when delivering the subtask to the coprocessor. The first coprocessor is used as an example. The first coprocessor receives a start condition of the first subtask and the first subtask delivered by the main processor. When the start condition of the first subtask is met, the first coprocessor starts to execute the first subtask. In the foregoing design, the main processor delivers the start condition to the coprocessor, so that the coprocessor determines whether the start condition is met, and then starts to execute the subtask when determining that the start condition is met.
[0023] The start condition may also be referred to as start time. For example, the start condition may include but is not limited to one or more of the following conditions:
[0024] (1) Duration for receiving the subtask reaches preset duration.
[0025] (2) A preset time point is reached.
[0026] (3) A timer corresponding to starting of the task times out.
[0027] (4) A previous subtask that is serially executed is executed.
[0109] The start condition may also be referred to as start time. For example, the start condition may include but is not limited to one or more of the following conditions:
[0110] (1) Duration for receiving the subtask reaches preset duration.
[0111] (2) A preset time point is reached.
[0112] (3) A timer corresponding to starting of the task times out.
[0113] (4) A previous subtask that is serially executed is executed.; see also [0116-0136]; [0143] After splitting the to-be-processed task into the Task A and the Task B, the TSE service sends the Task A to the TSE 1 of the coprocessor 1, and sends the Task B to the TSE 2 of the coprocessor 2. After receiving the Task A, the TSE 1 sends, to the NPU 1, a trigger instruction used to trigger the NPU 1 to execute the Task A. Specifically, a virtual address (for example, V1) corresponding to the Task A may be carried in the trigger instruction and sent to the NPU 1. The NPU 1 may query, based on V1, the TSE 1 for data of the to-be-executed Task A and a physical address of an execution instruction. The TSE 1 queries, based on an address translation table managed by the TSE 1, the data of the Task A and the physical address of the execution instruction, for example, P1, and sends the data and the physical address to the NPU 1. Therefore, the NPU 1 reads the data of the Task A and the execution instruction from a storage space corresponding to P1, to execute the Task A. Similarly, the NPU 1 reads data of the Task B and the execution instruction, to execute the Task B.
[0144] In a process of executing the Task A, when intermediate data is generated during each round of execution of the Task A, the NPU 1 queries the TSE 1 for a storage address for storing the intermediate data. The TSE 1 queries the address information table to obtain the storage address for storing the intermediate data, for example, P1, and sends P1 to the NPU 1, so that the NPU 1 stores the intermediate data in the storage space corresponding to P1. Similarly, in a process of executing the Task B, when intermediate data is generated during each round of execution of the Task B, the NPU 2 queries the TSE 2 for a storage address for storing the intermediate data. The TSE 2 queries the address information table to obtain the storage address for storing the intermediate data, for example, P4, and sends P4 to the NPU 2, so that the NPU 2 stores the intermediate data in a storage space corresponding to P4.
[0145] If the intermediate data generated when the NPU 1 executes the Task A needs to be used in a process in which the NPU 2 continues to execute the Task B, the NPU 2 queries the address information table by using the TSE 2, to determine an address 1 used to store an address translation table 1. The TSE 2 determines, from the address translation table 1 stored in a first memory space corresponding to the address 1, a physical address used to store the intermediate data generated when the NPU 1 executes the Task A. Then, the TSE 2 reads, from the physical address, the intermediate data generated when the NPU 1 executes the Task A.
[0146] In another possible implementation, an example in which the coprocessor allocates the storage space in the process of executing the subtask and maintains the address information table is used to describe the process in which the NPU 1 and the NPU 2 execute the subtask and obtain the data. For example, the address information table is in a form shown in Table 4 and Table 5.);
executing, by the corresponding sub-engine, a corresponding sub-task to be processed according to a received task parameter (start condition) ([0109-0113; 0116-0136; 0143-0151]);
sending a notification to the scheduler in response to an operating phase when the corresponding sub-engine executes the corresponding sub-task to be processed being the same as the start phase in the received task parameter ([0109-0113; 0116-0136; 0143-0151]);; and
in response to the notification being detected by the scheduler, returning to the step of sending, by a scheduler, the task parameter of a sub-task to be processed to a corresponding sub-engine (sending data necessary for the task to execute / after completion of the task, sending next task to the coprocessor) ([0109-0113; 0116-0136; 0143-0151; 0159-0161]). However, CHEN does not explicitly indicate that the parser is the entity dividing / partitioning the task. CHEN teaches the main processor / TSE service generates the subtask from a computing task is operates in the same mannerism as a parser and therefore it would be obvious to one of ordinary skill in the art before the effective filing of the claimed invention that the main processor / TSE service includes a parser in generating subtask for processing. Further CHEN does not explicitly teach that the task when divided are put into a task list.
PEERCY teaches partitioning task into subtask for execution wherein the task are ordered in a task list ([0038-0047). Therefore, it would be obvious to one of ordinary skill in the art before the effective filing of the claimed invention to combine PEERCY with the teachings of CHEN in order to dynamically subdivide work among the compute engines ([0047]). Further, note Zhong (2023/0124520) which shows a parser is a well-known entity ([0066]).
As to claim 2, CHEN teaches connecting the parser, the scheduler, and a plurality of sub-engines through an interconnection bus ([0087] There is a shared memory between the N coprocessors. The shared memory may be configured to store data generated by the coprocessor in a process of executing the subtask. In a manner, the shared memory may be configured by the main processor for each coprocessor by using an SVM technology. In another manner, there is a shared storage device between the coprocessors. The shared storage device includes the foregoing shared memory. The shared storage device may be implemented by one or more memories. In an example, one or more memories may be disposed in the computer system independently of each coprocessor. In another example, a memory is disposed in each coprocessor, the memories included in the coprocessors are connected to each other through the fast bus that ensures cache coherence, and are shared by using an SVM technology, to form the shared memory.).
As to claim 7, CHEN teaches the sending a notification to the scheduler in response to an operating phase when the corresponding sub-engine executes the corresponding sub-task to be processed being the same as the start phase in the received task parameter further comprises: saving, by the corresponding sub-engine, the start phase in the task parameter; outputting the current operating phase to a comparator in response to the corresponding sub-engine executing the corresponding sub-task to be processed; comparing, by the comparator, the current operating phase with the start phase; and in response to the current operating phase being the same as the start phase, issuing, by the comparator, the notification to the scheduler (based on the start condition, sending data necessary for the task to execute / execute the task and after completion of the task, sending next task to the coprocessor) ([0109-0113; 0116-0136; 0143-0151; 0159-0161]). However, CHEN does not teach that operating parameters for tasks are stored in registers. Official Notice is taken in that it is obvious to one of ordinary skill in the art before the effective filing of the claimed invention that one can store parameters / conditions in registers for triggering subtask execution.
As to claim 8, CHEN teaches the issuing, by the comparator, the notification to the scheduler further comprises: writing, by the comparator, a preset content; and in response to the scheduler detecting an action of the writing, acquiring the content and determining, based on the content whether to return to the step of sending, by a scheduler, the task parameter of a sub-task to be processed in the sub-task list to a corresponding sub-engine, and then sending the task parameter of a next sub-task to be processed to the corresponding sub-engine (based on the start condition, sending data necessary for the task to execute / execute the task and after completion of the task, sending next task to the coprocessor) ([0109-0113; 0116-0136; 0143-0151; 0159-0161]). However, CHEN does not teach that operating parameters for tasks are stored in registers, but does teach using storage space for storing data used by coprocessors (see [0011]). Official Notice is taken in that it is obvious to one of ordinary skill in the art before the effective filing of the claimed invention that one can store parameters / conditions in registers for triggering subtask execution.
As to claim 11, CHEN teaches the interconnection bus is based on a standard protocol or a customized bus protocol, and the interconnection bus is implemented in a topology of Crossbar, Mesh or Ring ([0084] FIG. 4 is a schematic diagram of an architecture of a computer system according to an embodiment of this application. The computer system includes a main processor and M coprocessors. M is an integer greater than 1. The coprocessor in this embodiment of this application includes at least an accelerator. The main processor and the coprocessor are located in different chips. The M coprocessors may be deployed in different chips, or may be deployed in a same chip, or may be deployed in X chips. X is an integer less than M and greater than 1. The main processor and the coprocessor are connected through a system bus. The system bus is responsible for inter-device communication. For example, the system bus may be a PCIe bus, or may be another type of bus. This is not specifically limited in this embodiment of this application. For example, in FIG. 4, the main processor is connected to the coprocessor, and the coprocessors are connected by using a PCIe switch chip. The PCIe switch chip is a switch chip that provides a PCIe connection. The processor and the coprocessor may be connected through a fast bus that ensures cache coherence. The fast bus can be a Huawei cache coherence system (HCCS), an advanced extensible interface (AXI), a hyper transport and quick path interconnect (Hyper transport and QPI), or an advanced microcontroller bus architecture (AMBA). In FIG. 4, for example, M=4, to be specific, four coprocessors are respectively a coprocessor 1 to a coprocessor 4). Further, Crossbar, Mesh or Ring type of buses are well known types of bus configurations and obvious to one of ordinary skill in the art before the effective filing of the claimed invention.
As to claim 12, CHEN teaches the interconnection bus is used to carry both a control flow comprising a sub-engine scheduling command and a data flow between the sub-engines ([0084] FIG. 4 is a schematic diagram of an architecture of a computer system according to an embodiment of this application. The computer system includes a main processor and M coprocessors. M is an integer greater than 1. The coprocessor in this embodiment of this application includes at least an accelerator. The main processor and the coprocessor are located in different chips. The M coprocessors may be deployed in different chips, or may be deployed in a same chip, or may be deployed in X chips. X is an integer less than M and greater than 1. The main processor and the coprocessor are connected through a system bus. The system bus is responsible for inter-device communication. For example, the system bus may be a PCIe bus, or may be another type of bus. This is not specifically limited in this embodiment of this application. For example, in FIG. 4, the main processor is connected to the coprocessor, and the coprocessors are connected by using a PCIe switch chip. The PCIe switch chip is a switch chip that provides a PCIe connection. The processor and the coprocessor may be connected through a fast bus that ensures cache coherence. The fast bus can be a Huawei cache coherence system (HCCS), an advanced extensible interface (AXI), a hyper transport and quick path interconnect (Hyper transport and QPI), or an advanced microcontroller bus architecture (AMBA). In FIG. 4, for example, M=4, to be specific, four coprocessors are respectively a coprocessor 1 to a coprocessor 4).
As to claim 13, CHEN does not explicitly teach the claim limitations. PEERCY teaches a task process of the sub-engine is defined as a plurality of phases, and a quantity of phases and a time length of each phase vary according to an engine type and the task (claim 9; [0024] The present disclosure relates to systems and methods for subdividing an unknown list for execution of operations by multiple computer engines. The system and methods may list objects in order on separate computer engines using common object store mechanisms, divide the lists into tasks using information from the objects in the list, and handle those tasks on the separate computer engines. The system and method may capture requests to subdivide the tasks into a number of smaller subtasks and handle those subtasks on separate computer engines. The system and method may divide the lists into tasks and subtasks using a hash on each object name to distribute the work evenly and without bias from any characteristic of the object name or object. The system and method may divide the list into tasks and subtasks using a hash on each object prefix name to distribute the work evenly by prefix so all objects within each prefix may be handled by the same computer engine. The system and method may divide the list into tasks and subtasks using a hash on each object prefix name to distribute the work by a first phase that handles prefixes only, then divides the list into tasks and subtasks using a hash on each object name to distribute the work in a second phase that handles objects only, then again divides the list by prefix to distribute the work in a third phase that handles prefixes only again. The system and method may divide the lists into tasks and subtasks using size of each object or alternatively, or in addition to, using the most recent create, modify, or access time of each object.; [0053] Another aspect of an embodiment of the present invention divides the list in phases. For example, the list may be divided into tasks and subtasks using a hash on each object prefix name to distribute the work by a first phase that handles prefixes only, then divides the list into tasks and subtasks using a hash on each object name to distribute the work in a second phase that handles objects only, then again divides the list by prefix to distribute the work in a third phase that handles prefixes only again. This may allow all preceding operations on a per-prefix basis to occur before any object content handling occurs and all succeeding operations on a per-prefix basis to occur after all object content handling occurs. [0061] This embodiment may allow stripes of work based on the dimension of the subdivision function of the original list to be assigned to the worker computer 12A so higher priority sublists of the whole list may be completed before lower priority sublists. In accordance with one embodiment of the present invention, and without loss of generality, this may allow objects with more recent access, modify, or create time to be operated on before objects with less recent access, modify, or create time. Also, without loss of generality, the next subtask from the reservoir of subtasks could be assigned to a worker computer 12A from the reservoir of subtasks randomly or uniformly without regard to priority at all. In another embodiment of the present invention, the reservoir of subtasks may be held centrally or it may be distributed among the several worker computers 12A in advance and redistributed as each worker computer 12A complete the work on their current subtasks.; It would be obvious that since the phases uses different prefix / conditions for processing that they vary according to task length.).
As to claim 14, PEERCY teaches each phase corresponds to a different stage of the task (claim 9; [0024] The present disclosure relates to systems and methods for subdividing an unknown list for execution of operations by multiple computer engines. The system and methods may list objects in order on separate computer engines using common object store mechanisms, divide the lists into tasks using information from the objects in the list, and handle those tasks on the separate computer engines. The system and method may capture requests to subdivide the tasks into a number of smaller subtasks and handle those subtasks on separate computer engines. The system and method may divide the lists into tasks and subtasks using a hash on each object name to distribute the work evenly and without bias from any characteristic of the object name or object. The system and method may divide the list into tasks and subtasks using a hash on each object prefix name to distribute the work evenly by prefix so all objects within each prefix may be handled by the same computer engine. The system and method may divide the list into tasks and subtasks using a hash on each object prefix name to distribute the work by a first phase that handles prefixes only, then divides the list into tasks and subtasks using a hash on each object name to distribute the work in a second phase that handles objects only, then again divides the list by prefix to distribute the work in a third phase that handles prefixes only again. The system and method may divide the lists into tasks and subtasks using size of each object or alternatively, or in addition to, using the most recent create, modify, or access time of each object.; [0053] Another aspect of an embodiment of the present invention divides the list in phases. For example, the list may be divided into tasks and subtasks using a hash on each object prefix name to distribute the work by a first phase that handles prefixes only, then divides the list into tasks and subtasks using a hash on each object name to distribute the work in a second phase that handles objects only, then again divides the list by prefix to distribute the work in a third phase that handles prefixes only again. This may allow all preceding operations on a per-prefix basis to occur before any object content handling occurs and all succeeding operations on a per-prefix basis to occur after all object content handling occurs. [0061] This embodiment may allow stripes of work based on the dimension of the subdivision function of the original list to be assigned to the worker computer 12A so higher priority sublists of the whole list may be completed before lower priority sublists. In accordance with one embodiment of the present invention, and without loss of generality, this may allow objects with more recent access, modify, or create time to be operated on before objects with less recent access, modify, or create time. Also, without loss of generality, the next subtask from the reservoir of subtasks could be assigned to a worker computer 12A from the reservoir of subtasks randomly or uniformly without regard to priority at all. In another embodiment of the present invention, the reservoir of subtasks may be held centrally or it may be distributed among the several worker computers 12A in advance and redistributed as each worker computer 12A complete the work on their current subtasks.; It would be obvious that since the phases uses different prefix / conditions for processing that they vary according to task length.). Refer to claim 13 for the motivation to combine.
As to claim 9, reference is made to a chip comprising a digital logic circuit that corresponds to the method of claim 1 and is therefore met by the rejection of claim 1 above ([0028] According to a second aspect, this application provides a coprocessor (for example, the first coprocessor or the second coprocessor in the first aspect). The coprocessor may also be considered as an independent chip or a partial structure in a chip. The coprocessor may also be referred to as a co-processing device, an acceleration device, or the like in some scenarios.; [0046] The first coprocessor and the second coprocessor may be deployed in one chip, or may be deployed in different chips. When the first coprocessor and the second coprocessor are deployed in a same chip, the computer system may also be understood as a chip system.).
As to claims 10, 15 and 20, reference is made to a device comprising a memory and a processor, that corresponds to the method of claims 1, 2 and 7 and is therefore met by the rejection of claims 1, 2 and 7 above ([0084] FIG. 4 is a schematic diagram of an architecture of a computer system according to an embodiment of this application. The computer system includes a main processor and M coprocessors. M is an integer greater than 1. The coprocessor in this embodiment of this application includes at least an accelerator. The main processor and the coprocessor are located in different chips. The M coprocessors may be deployed in different chips, or may be deployed in a same chip, or may be deployed in X chips. X is an integer less than M and greater than 1. The main processor and the coprocessor are connected through a system bus. The system bus is responsible for inter-device communication. For example, the system bus may be a PCIe bus, or may be another type of bus. This is not specifically limited in this embodiment of this application. For example, in FIG. 4, the main processor is connected to the coprocessor, and the coprocessors are connected by using a PCIe switch chip. The PCIe switch chip is a switch chip that provides a PCIe connection. The processor and the coprocessor may be connected through a fast bus that ensures cache coherence. The fast bus can be a Huawei cache coherence system (HCCS), an advanced extensible interface (AXI), a hyper transport and quick path interconnect (Hyper transport and QPI), or an advanced microcontroller bus architecture (AMBA). In FIG. 4, for example, M=4, to be specific, four coprocessors are respectively a coprocessor 1 to a coprocessor 4).
Claims 3 and 16 are rejected under 35 U.S.C. 103 as being unpatentable over CHEN in view of PEERCY as applied to claim 1 above, and further in view of MOYER (Publication 2016/0011907) and CATRON (Publication 2024/0054384).
As to claim 3, CHEN and PEERCY do not teach the recited limitations.
MOYER teaches partitioning operations for execution on different accelerator devices (abstract) by initializing counters in each sub engine respectively; and setting an initial value of the counters to a respective value associated with each sub-engine or 0 (abstract; [0011] In operation, the disclosed embodiments provide configurable counting mechanisms for measuring the amount of time individual tasks spend in various local scheduling states on a processing core or combinations of these local scheduling states. These resulting per-task state count values from these task state counters are then reported back for use in assigning new tasks to the processing cores, such as through a global work scheduler. In example embodiments below, task state information is communicated to a task lifetime counter mechanism by individual task state tracker (TST) state machines located in a core task scheduler within each processing core. Each individual task state tracker (TST) state machine tracks the state of a given task, from the time the task is initially assigned, until the time the task completes and returns to the unassigned state, awaiting reassignment for new work. When a task completes, the processing returns to an initial unassigned task state, and the task lifetime counter value is reported to the global work scheduler for the multi-tasking processing system. This reporting of the count value can be implemented using a hard-wired port or interface that connects the processing core to the global work scheduler. The lifetime counter value from a completing task can then be used by the work scheduler to influence future task assignments to processing cores made by the work scheduler. Further, states to be monitored are individually selectable through configuration settings, and the selected states can be varied over time based upon system events, work scheduler status events, performance feedback, and/or other desired operating parameters. Advantageously, software maintenance of the task lifetime counter logic is not required as the states to be monitored are initially selected and tracked with each processing core.
[0012] The disclosed embodiments, therefore, provide a flexible hardware mechanism for accumulating task lifetime execution profiles and for providing feedback to a global work scheduler. Unlike traditional software-based task lifetime timers that rely upon a free-running counter, the disclosed embodiments provide selectable low-level processing state tracking information including local and global hardware task scheduling decisions and conflicts (e.g., waiting states, inhibit states). This state tracking information is then provided directly back to the global work scheduler on task completion, and the global work scheduler can use this per-task processing state information from each processing core to make dynamic adjustments with respect to how new tasks are allocated to the pool of processing cores.
[0013] With respect to tracked states and as described below, a task state counter for each assigned task within each processing core is configured to provide information regarding the number of cycles a task spends in particular processing states during its lifetime from initiation to termination by the processing core. In particular, selective control is provided over which states the task state tracker tracks using the task state counter, and this state selection causes the task lifetime counter to accumulate cycles spent in the selected states. The per-task lifetime counters accumulate the number of cycles spent in particular states (e.g., active execution, active acceleration, waiting for execution, execution inhibited, etc.). At the time the task is initiated, the task state counter for the task is initialized (e.g., cleared). At the time the task completes or is terminated, the counter value is published or reported to the global work scheduler, for example, through a hardware interface between the processing core and the work scheduler. The work scheduler then uses the task lifetime counter values from the processing cores within the multi-tasking processing system to determine how to efficiently assign new tasks to the processing cores based upon the task state counter operational history for the processing cores. As such, the disclosed embodiments advantageously provide dedicated per-task state counters, configurable selection of states to be counted, and automatic publishing of the task state lifetime counter values to the global work scheduler upon task termination. This counting of selected task states during task processing and reporting of the resulting counter values provides more specific state information that can be used to more efficiently schedule new tasks as compared to the overall elapsed time provided by prior solutions.
[0014] FIG. 1 is a block diagram of an example embodiment for a multi-tasking processing system 100 including a task state tracker 126 and task state counters 128A, 128B . . . 128C for each processing core 122A, 122B, 122C . . . within the multi-core processor 118. For the example multi-tasking processing system 100, the multi-core processor 118 includes multiple processing core clusters 120A, 120B, 120C . . . with each processing core cluster 120A, 120B, 120C . . . further including multiple processing cores 122A, 122B, 122C . . . . Each processing core 122A, 122B, 122C . . . further includes a core processing unit 130 and a core task scheduler 124 that manages the tasks processed by the core processing unit 130 through interface 134. The core task scheduler (CTS) 124 further includes task state counters 128A, 128B . . . 128C and a task state tracker 126 that tracks processing states for each task currently assigned to a processing core. Specific states selected by the CTS register 400 are monitored for performing updates to the task state counters 128A, 128B . . . 128C to count states for each task assigned to the processing core, as described in more detail below. When a task is completed, the resulting per-task state count values 132A, 132B . . . 132C are reported by the task state tracker 126. As one further example, four (4) processing core clusters can be provided with each including four (4) processing cores to provide a total of sixteen (16) processing cores within the multi-core processor 118, and each processing core can be configured to have sixteen (16) concurrently assigned tasks to be processed.). Therefore, it would be obvious to one of ordinary skill in the art before the effective filing of the claimed invention to combine MOYER to the teachings of CHEN and PEERCY in order to use performance counters in order to more efficiently schedule new tasks as compared to overall elapsed time provided by prior solutions ([0013-0014]). Further, it would be obvious to one of ordinary skill in the art that counter values can be set to any respective value deemed necessary by one of ordinary skill in the art before the effective filing of the claimed invention.
CHEN, PEERCY and MOYER does not teach the monitoring of a cache space of a preset size.
CATRON teaches partitioning operations for execution on different accelerator devices (abstract) by initializing counters and a cache space of a preset size in each sub engine respectively ([0046] At 404, performance associated with the initial partitioning is tracked. In various embodiments, the plurality of different machine learning accelerator hardware units is used to track costs associated with different portions of the machine learning model network. In various embodiments, compute time and data transfer time are costs that are tracked. In various embodiments, the different portions of the machine learning model network correspond to different machine learning operators of the machine learning model network. Thus, in various embodiments, compute times and data transfer times of the different machine learning operators of the machine learning model network are tracked by the plurality of different machine learning accelerator hardware units. Data may be tracked by utilizing counters and/or hardware clocks of the machine learning accelerator hardware units. Another cost that may be tracked is data size. The amount of data loaded into various partitions can be measured. In some embodiments, a cost function that incorporates several types of costs is utilized. In various embodiments, the costs are tracked during one or more inference executions of the machine learning model network. Stated alternatively, costs are tracked while a trained machine learning model is being utilized in inference mode, e.g., to make personalized recommendations, classify images, detect objects, recognize speech, process natural language data, or perform any other task for which the machine learning model is trained. Multiple samples are taken. Multiple samples (e.g., taken across multiple inference runs or multiple days of operation of the machine learning model) are valuable for more accurately determining costs (e.g., compute times and data transfer times of operators). Tracking costs by collecting actual data is valuable because many costs are difficult to predict without actual collected data. For example, compute times can be hardware dependent and difficult to predict until operators are run on the specific hardware.). It would be obvious that based on tracking the amount and size of data loaded into various partitions / accelerator hardware units by CATRON and that the data is loaded into cache as outlined in CHEN would based on the combination the cache size is tracked for the purpose of subtask assignment. Therefore, it would be obvious to combine the teachings of CHEN, PEERCY and MOYER with the teachings of CATRON in order to track performance associated with an initial partition for future optimization ([0046]).
As to claim 16, reference is made to a device comprising a memory and a processor, that corresponds to the method of claim 3 and is therefore met by the rejection of claim 3 above ([0084] FIG. 4 is a schematic diagram of an architecture of a computer system according to an embodiment of this application. The computer system includes a main processor and M coprocessors. M is an integer greater than 1. The coprocessor in this embodiment of this application includes at least an accelerator. The main processor and the coprocessor are located in different chips. The M coprocessors may be deployed in different chips, or may be deployed in a same chip, or may be deployed in X chips. X is an integer less than M and greater than 1. The main processor and the coprocessor are connected through a system bus. The system bus is responsible for inter-device communication. For example, the system bus may be a PCIe bus, or may be another type of bus. This is not specifically limited in this embodiment of this application. For example, in FIG. 4, the main processor is connected to the coprocessor, and the coprocessors are connected by using a PCIe switch chip. The PCIe switch chip is a switch chip that provides a PCIe connection. The processor and the coprocessor may be connected through a fast bus that ensures cache coherence. The fast bus can be a Huawei cache coherence system (HCCS), an advanced extensible interface (AXI), a hyper transport and quick path interconnect (Hyper transport and QPI), or an advanced microcontroller bus architecture (AMBA). In FIG. 4, for example, M=4, to be specific, four coprocessors are respectively a coprocessor 1 to a coprocessor 4).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to LEWIS ALEXANDER BULLOCK JR whose telephone number is (571)272-3759. The examiner can normally be reached Monday-Friday, 9:00-5:00 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Cordelia Zecher can be reached at 571-272-7771. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/LEWIS A BULLOCK JR/Supervisory Patent Examiner, Art Unit 2199