DETAILED ACTION
The Office Action is in response to claims filed 1/24/2024.
Claims 1-20 are pending.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1 and 3 are rejected under 35 U.S.C. 103 as being unpatentable over US 20250004900 A1 hereinafter “Xu” in view of US 20150355898 A1 hereinafter “Zhang”.
With regards to claim 1, Xu teaches
A method for executing a program, the method comprising: writing a program value to a corresponding accelerator program register of accelerator program registers of an accelerator circuit [according to accelerator program difference information] (Xu [0031], “The command matcher 164 may receive information of the addresses of the registers to be accessed and data (to be referred to as data to be updated below) to be written by a command (for example, the command CMD in FIG. 3) of the predetermined task [writing a program value to a corresponding accelerator program register of accelerator program registers of an accelerator circuit]. The command matcher 164 may search the memory 162 according to the addresses of the registers to be accessed by the command CMD, so as to read the previous data in the registers. The command matcher 164 may accordingly determine whether the previous data is the same as the data to be updated. If the previous data is the same as the data to be updated, it means that the data previously stored in the registers is the same as the new data to be written by the command CMD. In this case, the command matcher 164 does not transmit the addresses of the registers or the data to be updated to the command executor 166. Or, if the previous data is different from the data to be updated, it means that the data previously stored in the registers is different from the new data to be written by the command CMD. In this case, the command matcher 164 stores the data to be updated to the memory 162 to update the memory 162 by replacing the previous data corresponding to the registers with the data to be updated, and provides the addresses of the registers and the data to be updated to the command executor 166.”) and
executing an accelerator program by the accelerator circuit according to program values of the accelerator program registers of the accelerator circuit. (Xu [0017-20], “The processor 101 is capable of executing various types of application software, executes a predetermined task (for example but not limited to, processing image data) in response to a request of the application software, and accordingly applies for a temporary storage space in the memory 102 so as to store one or more commands corresponding to the predetermined task. In some embodiments, the processor 101 may be, for example but not limited to, a central processor [executing an accelerator program by the accelerator circuit]. In some embodiments, the memory 102 may be, for example but not limited to, a dynamic random access memory (DRAM)… In response to the predetermined task assigned by the processor 101, the parameter data of the multiple registers in one corresponding image processing circuit that is selected from the multiple image processing circuits 103[1] to 103[n] to execute the predetermined task can be accordingly updated, so that the corresponding image processing circuit may execute the predetermined task by means of accessing the parameter data in these registers [according to program values of the accelerator program registers of the accelerator circuit].”)
Xu does not explicitly teach: […] according to program difference information
However, in an analogous art Zhang teaches […] according to program difference information (Zhang [0030], “In order to reduce the amount of data to be transferred, the update assembly and existing assembly can be compared (wherein, identical metadata in the update assembly and existing assembly have identical identifiers), for example, by utilizing a ‘xdelta’ tool for comparing differences of binary files, to generate binary delta data comprising only the parts of the update assembly which are different from the existing assembly, and the generated binary delta data is sent to the running program as the update assembly. For example, the source codes modified in accordance with the hot-update requirements can be written in various programming languages, such as C#, Visual Basic .NET, C++/CLI Managed, F#, J#, Jscript .NET, Windows PowerShell, etc., and the modification thus made can include adding a new class, modifying a function, changing the access level of a function or field, adding a field to an existing object, etc. The update assembly compiled from the source codes can be of a human-readable common intermediate language (CIL). The update assembly can exist in the form of a dll or exe file.”)
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have incorporated the teachings of Zhang into the teachings of Xu. This combination of teachings would have resulted a method configured for rewriting the register values only if there are differences between the target and the current data, as in Xu, after generating difference data from modifying the assembly structure, as in Zhang. One of ordinary skill in the art would have been motivated to combine these teachings for the purpose of loading the update into memory in order to obtain a difference description in which it will define the updated version required to run (Zhang [0007]).
With regards to claim 3, the rejection of claim 1 is incorporated.
Xu further teaches: wherein the accelerator program difference information causes the processing device to write only accelerator program registers having program values being updated from prior program values written to the accelerator program registers by an immediately preceding accelerator program in a sequence of recorded accelerator programs, and
wherein the processing device does not write other accelerator program registers in response to the accelerator program difference information. (Xu [0025], “The filter control circuit 160 is coupled to the decoder circuit 140 to receive the commands corresponding to the first task, so as to obtain information of the addresses of the registers to be accessed and the data to be written by the first task. The filter control circuit 160 may record address information of multiple registers of each of the multiple processing circuits 130[1] to 130[n] and data previously stored in each of these registers [from prior program values written to the accelerator program registers]. The filter control circuit 160 may compare whether the data previously stored in the registers to be accessed by the first task is the same as the data to be written by the first task. If the two are the same, the filter control circuit 160 does not write the data to be written by the first task to the registers [wherein the processing device does not write other accelerator program registers in response to the accelerator program difference information]. If the data previously stored in the registers to be accessed by the first task is different from the data to be written by the first task, the filter control circuit 160 writes the data to be written by the first task to the registers [wherein the accelerator program difference information causes the processing device to write only accelerator program registers having program values being updated].”) [Examiner’s Note: Previously stored register values can be associated with previously executed tasks from previously recorded accelerator programs]
Claim 2 is rejected under 35 U.S.C. 103 as being unpatentable over Xu in view of Zhang, as applied to claim 1, in view of US 20240104361 A1 hereinafter “Mehendale” and further in view of US 20220197615 A1 hereinafter “Kinsner”.
With regards to claim 2, the rejection of claim 1 is incorporated.
Xu does not teach: searching metadata of the compiled program for compiled data associated with each layer of the machine learning model
the accelerator program difference information being included in the metadata of the compiled program;
However, in an analogous art Zhang teaches searching metadata of the compiled program […] (Zhang [0047-48], “a receiving unit 310 configured to obtain, in response to receiving a hot-update instruction by a running program, an update assembly and difference descriptions for describing differences between the update assembly and an existing assembly of the running program, wherein, identical metadata in the update assembly and the existing assembly have identical identifiers; a modification determining unit 320 configured to load the update assembly into memory, and find, in the update assembly, functions which have been modified with respect to the existing assembly according to the difference descriptions;”)
the accelerator program difference information being included in the metadata of the compiled program; (Zhang [0029-30], “Further, the method can also include, prior to receiving the hot-update instruction by the running program, compiling source codes modified according to requirements of the hot-update to generate the update assembly; performing structural adjustments to the update assembly, causing identical metadata in the adjusted update assembly and existing assembly of the running program to have identical identifiers [of the compiled program];… In order to reduce the amount of data to be transferred, the update assembly and existing assembly can be compared (wherein, identical metadata in the update assembly and existing assembly have identical identifiers), for example, by utilizing a ‘xdelta’ tool for comparing differences of binary files, to generate binary delta data comprising only the parts of the update assembly which are different from the existing assembly, and the generated binary delta data is sent to the running program as the update assembly [the accelerator program difference information being included in the metadata].”)
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have incorporated the teachings of Zhang into the teachings of Xu. This combination of teachings would have resulted in a method configured for rewriting the register values only if there are differences between the target and the current data, as in Xu, after generating difference data from modifying the assembly structure, as in Zhang. One of ordinary skill in the art would have been motivated to combine these teachings for the purpose of loading the update into memory in order to obtain a difference description in which it will define the updated version required to run (Zhang [0007]).
The combination of Xu and Zhang does not teach: wherein the compiled program corresponds to a machine learning model, the machine learning model having at least one layer and the method further comprises:
[searching metadata of the compiled program for] compiled data associated with each layer of the machine learning model
However, in an analogous art Mehendale teaches wherein the compiled program corresponds to a machine learning model, the machine learning model having at least one layer and the method further comprises: (Mehendale [0033], “The processing nodes of a neural network can be divided into layers including, for example, an input layer, a number of intermediate layers (e.g., hidden layers), and an output layer. The input layer and the intermediate layers can each be a convolution layer forming CNN, whereas the output layer can be a fully-connected layer, and the input layer, intermediate layer, and the output layer together form a DNN [wherein the compiled program corresponds to a machine learning model, the machine learning model having at least one layer]”)
[…] compiled data associated with each layer of the machine learning model (Mehendale [0034], “The neural network processor can be programmed based on a sequence of instructions that include computation operations (e.g., adding, multiplication, processing of activation function, etc.) associated with the model. The instructions may also access internal and external memory devices to obtain and store data. A compiler may receive information about the neural network model, the input data, and the available memory and computation resources, and generate the set of instructions to indicate, for example, when to access the internal and external memory devices for the data, which component of the neural network processor to perform computations on the data based on the neural network model, etc., to perform the neural network processing.”) [Examiner’s Note: A compiler can receive (searched) instructions according to several different kinds of computations including neural network layers]
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have incorporated the teachings of Mehendale into the teachings of Xu in view of Zhang This combination of teachings would have resulted in a method configured for rewriting the register values only if there are differences between the target and the current data, as in Xu, after generating difference data from modifying the assembly structure, as in Zhang, and further applicable to a machine learning application, as in Mehendale. One of ordinary skill in the art would have been motivated to combine these teachings for the purpose of associating APIs with neural network layers to fetch data, perform inferences, and generate controls accordingly (Mehendale [0047]).
The combination of Xu, Zhang and Mehendale does not teach: executing a reference kernel for each layer of the machine learning model having no corresponding compiled data in the metadata of the compiled program.
However, in an analogous art Kinsner teaches executing a reference kernel for each layer of the machine learning model having no corresponding compiled data in the metadata of the compiled program. (Kinsner [0062-63], “In one implementation, the offline telemetry harvesting is referred to herein as an offline harvesting session. At block 520, the processing device may extract device telemetry data corresponding to kernel invocations of the application on one or more accelerator devices… At decision block 530, the processing device determines whether sufficient telemetry data has been collected. This determination may be based on a threshold condition of telemetry data being gathered. If it is determined that insufficient telemetry data has been gathered [having no corresponding compiled data in the metadata of the compiled program], then method 500 proceeds to block 540 where the processing device may run additional sequence of kernels/data copies and return to block 520. [executing a reference kernel for each layer of the machine learning model].”)
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have incorporated the teachings of Kinsner into the teachings of Xu in view of Zhang and further in view of Mehendale This combination of teachings would have resulted in a method configured for rewriting the register values only if there are differences between the target and the current data, as in Xu, after generating difference data from modifying the assembly structure, as in Zhang, and further applicable to a machine learning application, as in Mehendale, and executing a kernel or unit of processing when there is no corresponding data, as in Kinsner. One of ordinary skill in the art would have been motivated to combine these teachings for the purpose of enabling software that can be eventually transitioned into execution on accelerator hardware (Kinsner [0014]).
Claim 4 is rejected under 35 U.S.C. 103 as being unpatentable over Xu in view of Zhang, as applied to claim 1, and further in view of Kinsner.
With regards to claim 4, the rejection of claim 1 is incorporated.
Xu does not teach: storing the accelerator program difference information in metadata of the compiled program.
However, in an analogous art Zhang teaches storing the accelerator program difference information in metadata of the compiled program. (Zhang [0029-30], “Further, the method can also include, prior to receiving the hot-update instruction by the running program, compiling source codes modified according to requirements of the hot-update to generate the update assembly; performing structural adjustments to the update assembly, causing identical metadata in the adjusted update assembly and existing assembly of the running program to have identical identifiers [of the compiled program]; … In order to reduce the amount of data to be transferred, the update assembly and existing assembly can be compared (wherein, identical metadata in the update assembly and existing assembly have identical identifiers), for example, by utilizing a ‘xdelta’ tool for comparing differences of binary files, to generate binary delta data comprising only the parts of the update assembly which are different from the existing assembly, and the generated binary delta data is sent to the running program as the update assembly [storing the accelerator program difference information in metadata].”)
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have incorporated the teachings of Zhang into the teachings of Xu. This combination of teachings would have resulted a method configured for rewriting the register values only if there are differences between the target and the current data, as in Xu, after generating difference data from modifying the assembly structure, as in Zhang. One of ordinary skill in the art would have been motivated to combine these teachings for the purpose of loading the update into memory in order to obtain a difference description in which it will define the updated version required to run (Zhang [0007]).
The combination of Xu and Zhang does not teach: executing the compiled program on a host device and recording all accelerator programs generated thereby;
compressing a recorded accelerator program to generate the accelerator program difference information;
However, in an analogous art Kinsner teaches executing the compiled program on a host device and recording all accelerator programs generated thereby; (Kinsner [0033], “In some implementations, although host system 100 is depicted as implementing a virtualization system to virtualize its resources (e.g., memory resources and processing resources), some implementations may execute applications and/or workload on host system 100 by directly utilizing the resources of host system 100 [executing the compiled program on a host device]”) (Kinsner [0085], “For example, the machine readable instructions may be stored in multiple parts [recording all accelerator programs], which are individually compressed, encrypted, and stored on separate computing devices, wherein the parts when decrypted, decompressed, and combined form a set of executable instructions that implement a program such as that described herein [generated thereby].”)
compressing a recorded accelerator program to generate [the accelerator program difference information;] (Kinsner [0085], “The machine readable instructions described herein may be stored in one or more of a compressed format, an encrypted format, a fragmented format, a compiled format, an executable format, a packaged format, etc. Machine readable instructions as described herein may be stored as data (e.g., portions of instructions, code, representations of code, etc.) that may be utilized to create, manufacture, and/or produce machine executable instructions. For example, the machine readable instructions may be fragmented and stored on one or more storage devices and/or computing devices (e.g., servers). The machine readable instructions may utilize one or more of installation, modification, adaptation, updating, combining, supplementing, configuring, decryption, decompression, unpacking, distribution, reassignment, compilation, etc. in order to make them directly readable, interpretable, and/or executable by a computing device and/or other machine.”)
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have incorporated the teachings of Kinsner into the teachings of Xu in view of Zhang This combination of teachings would have resulted in a method configured for rewriting the register values only if there are differences between the target and the current data, as in Xu, after generating difference data from modifying the assembly structure, as in Zhang, and executing a the corresponding software to generate said difference data for the compression, as in Kinsner. One of ordinary skill in the art would have been motivated to combine these teachings for the purpose of enabling software that can be eventually transitioned into execution on accelerator hardware (Kinsner [0014]).
Claim 5 is rejected under 35 U.S.C. 103 as being unpatentable over Xu in view of Zhang in view of Kinsner, as applied to claim 4, in view of US 20220138151 A1 hereinafter “Yelheri” and further in view of US 20120198427 A1 hereinafter “Schmidt”.
With regards to claim 5, the rejection of claim 4 is incorporated.
Xu does not teach: writing an indication of the corresponding accelerator program register to the accelerator program difference information, the program value being updated from a prior program value written to the corresponding accelerator program register by an immediately preceding accelerator program in a sequence of recorded accelerator programs;
However, in an analogous art Zhang teaches writing an indication of the corresponding accelerator program register to the accelerator program difference information, the program value being updated from a prior program value written to the corresponding accelerator program register by an immediately preceding accelerator program in a sequence of recorded accelerator programs; (Zhang [0049], “a structural adjusting unit 351 configured to perform structural adjustments to the update assembly, causing identical metadata in the adjusted update assembly and the existing assembly of the running program to have identical identifiers; a comparing unit 352 configured to compare the update assembly with the existing assembly to generate a difference description file [writing an indication of the corresponding accelerator program register to the accelerator program difference information, the program value being updated], wherein, the difference description file contains the difference descriptions for describing the differences between the update assembly and the existing assembly [by an immediately preceding accelerator program in a sequence of recorded accelerator programs]; an instruction sending unit 353 configured to send the hot-update instruction to the running program, causing the running program to obtain the difference descriptions from the difference description file.”) [Examiner’s Note: Due to the previously recorded accelerator program existing as an assembly on the device one of ordinary skill in the art could reasonably understand that the update difference is in respect to previously executed software]
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have incorporated the teachings of Zhang into the teachings of Xu. This combination of teachings would have resulted a method configured for rewriting the register values only if there are differences between the target and the current data, as in Xu, after generating difference data from modifying the assembly structure, as in Zhang. One of ordinary skill in the art would have been motivated to combine these teachings for the purpose of loading the update into memory in order to obtain a difference description in which it will define the updated version required to run (Zhang [0007]).
The combination of Xu, Zhang, and Kinsner does not teach: omitting from the accelerator program difference information, an indication of any accelerator program register of the accelerator program registers unchanged by the immediately preceding accelerator program in the sequence of recorded accelerator programs, any unused accelerator program registers, and any uninitialized arithmetic logic unit registers.
However, in an analogous art Yelheri teaches omitting from the accelerator program difference information, an indication of any accelerator program register of the accelerator program registers unchanged by the immediately preceding accelerator program in the sequence of recorded accelerator programs (Yelheri [0040], “While doing incremental snapshot copy, changed blocks between two snapshots may be copied to the object store, and unchanged blocks will be shared with previous snapshots as opposed to being redundantly stored in the object store. In this way, deduplication is provided for and between snapshot data stored within objects of the object store. As will be described later, an embodiment of a snapshot file system in the object store is illustrated by FIG. 5B.”), any unused accelerator program registers (Yelheri [0159], “The fragmented object is compacted to retain the in-use data and exclude the freed data (the unused data) as a written object. Because compacting may store the in-use data in new slots, an object header of the object is updated with new locations of the in-use data within the rewritten object. In this way, defragmentation is performed for objects within the object store 509.”) [Examiner’s Note: An object in storage can be analogous to a register]
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have incorporated the teachings of Yelheri into the teachings of Xu in view of Zhang and further in view of Kinsner. This combination of teachings would have resulted in a method configured for rewriting the register values only if there are differences between the target and the current data, as in Xu, after generating difference data from modifying the assembly structure, as in Zhang, executing a the corresponding software to generate said difference data for the compression, as in Kinsner, and the difference information not including unchanged and unused registers for updating general storage, as in Yelheri. One of ordinary skill in the art would have been motivated to combine these teachings for the purpose of ensuring proper determination/comparison of data so that only data that doesn’t already exist in the data store is transmitted (Yelheri [0164]).
The combination of Xu, Zhang, Kinsner, and Yelheri does not teach: and any uninitialized arithmetic logic unit registers.
However, in an analogous art Schmidt teaches and any uninitialized arithmetic logic unit registers. (Schmidt [0038], “Execution unit 211 comprises a set of general purpose registers 212 for storing data and a scalar arithmetic logic unit (ALU) 213 for performing arithmetic and logical operations on data in general purpose (GP) registers 212 responsive to instructions decoded by instruction unit 201.”) (Schmidt [0058], “The compiler further determines whether it should reserve any general purpose registers for use by a dynamic binary optimizer in addressing a context save area (block 503). Preferably, reservation of one or more registers is performed responsive to a compiler directive. The compiler directive could be a statement in source code file 311 or intermediate code file 312, but in the preferred embodiment it is an optional compiler parameter which is specified by the user at the time the compiler is invoked... It would alternatively be possible for reservation of one or more registers for the dynamic binary optimizer to be a default option, which is automatically selected unless the user directs the compiler not to reserve a register. Alternatively, the compiler might be written to always reserve at least one register, although this is considered undesirable because there may be circumstances where it is desirable to use all registers.”) [Examiner’s Note: A compiler directive could omit instructions in order to leave a register reserved/unchanged. The execution unit will apply these changes to the ALU from received instructions accordingly.]
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have incorporated the teachings of Schmidt into the teachings of Xu in view of Zhang in view of Kinsner and further in view of Yelheri. This combination of teachings would have resulted in a method configured for rewriting the register values only if there are differences between the target and the current data, as in Xu, after generating difference data from modifying the assembly structure, as in Zhang, executing a the corresponding software to generate said difference data for the compression, as in Kinsner, the difference information not including unchanged and unused registers for updating general storage, as in Yelheri, and further leaving registers reserved and unchanged including in an uninitialized state by compiler directives accordingly, as in Schmidt. One of ordinary skill in the art would have been motivated to combine these teachings for the purpose of binary optimization of transitions between previously compiled object code modules and programs of a system (Schmidt [0046]).
Claims 6-8 and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Zhang in view of Xu.
With regards to claim 6, Zhang teaches
memory configured to store a compiled program, the compiled program including a plurality of [accelerator] programs and corresponding [accelerator] program difference information; (Zhang [0035], “It should be noted that, when the update assembly is loaded into memory, the existing assembly can either be retained in the memory or be removed therefrom, which can be specifically determined according to whether there are data in the existing assembly which are still in use.”) (Zhang [0047], “a receiving unit 310 configured to obtain, in response to receiving a hot-update instruction by a running program, an update assembly [the compiled program including a plurality of … programs] and difference descriptions [and corresponding program difference information] for describing differences between the update assembly and an existing assembly of the running program, wherein, identical metadata in the update assembly and the existing assembly have identical identifiers; a modification determining unit 320 configured to load the update assembly into memory, and find, in the update assembly, functions which have been modified with respect to the existing assembly according to the difference descriptions;”)
[…] and the corresponding accelerator program difference information; (Zhang [0030], “For example, the hot-update instruction and difference description file can be sent to the running program, and after the difference description file is parsed by the running program to determine the hot-update assembly corresponding to the difference description file, the running program correspondingly invokes a hot-update interface provided by a running environment, and then reads this update assembly at the running environment (e.g., Mono, cross-platform .NET running environment) level as well as performs subsequent hot-update operations”)
Zhang does not teach: a processing device configured to execute the compiled program using the plurality of accelerator programs
an accelerator circuit configured to execute the plurality of accelerator programs according to program values of accelerator program registers.
However, in an analogous art Xu teaches a processing device configured to execute the compiled program using the plurality of accelerator programs (Xu [0018], “an image processing algorithm to be used and a configuration of registers, based on a task that an upper-layer application (for example, the application software above) requests to be executed. The processor 101 may encode addresses of the registers to be accessed by the task and the data to be written into multiple command in a predetermined format, for the image processing accelerator device 100 to analyze (or decode) these commands so as to obtain information related the addresses of the registers to be accessed and the data to be written.”)
an accelerator circuit configured to execute the plurality of accelerator programs according to program values of accelerator program registers. (Xu [0017-20], “The processor 101 is capable of executing various types of application software, executes a predetermined task (for example but not limited to, processing image data) in response to a request of the application software, and accordingly applies for a temporary storage space in the memory 102 so as to store one or more commands corresponding to the predetermined task. In some embodiments, the processor 101 may be, for example but not limited to, a central processor. In some embodiments, the memory 102 may be, for example but not limited to, a dynamic random access memory (DRAM) The image processing accelerator device 100 may read the command corresponding to the predetermined task from the memory 102, and decode the command to determine whether the predetermined task is to be executed by a corresponding image processing circuit among multiple image processing circuits 103[1] to 103[n], wherein the value n may be a positive integer greater than 1[executing an accelerator program by the accelerator circuit]. … In response to the predetermined task assigned by the processor 101, the parameter data of the multiple registers in one corresponding image processing circuit that is selected from the multiple image processing circuits 103[1] to 103[n] to execute the predetermined task can be accordingly updated, so that the corresponding image processing circuit may execute the predetermined task by means of accessing the parameter data in these registers [according to program values of the accelerator program registers of the accelerator circuit].”)
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have incorporated the teachings of Xu into the teachings of Zhang. This combination of teachings would have resulted in a method configured to generate difference data with the purpose of modifying assembly structure, as in Zhang, thereby rewriting the register values only if there are differences between the target and the current data, as in Xu. One of ordinary skill in the art would have been motivated to combine these teachings for the purpose of updating registers with data changes instead of all registers to save processing time in a data update (Xu [0038]).
With regards to claim 7, the rejection of claim 6 is incorporated.
Zhang does not teach: wherein the corresponding [accelerator program difference] information causes the processing device to write only accelerator program registers having program values being updated from prior program values written to corresponding accelerator program registers by an immediately preceding accelerator program in a sequence of recorded accelerator programs, and
wherein the processing device does not write other accelerator program registers [in response to the corresponding accelerator program difference information.]
However, in an analogous art Xu teaches wherein the corresponding [accelerator program difference] information causes the processing device to write only accelerator program registers having program values being updated from prior program values written to corresponding accelerator program registers by an immediately preceding accelerator program in a sequence of recorded accelerator programs, and
wherein the processing device does not write other accelerator program registers [in response to the corresponding accelerator program difference information.] (Xu [0025], “The filter control circuit 160 is coupled to the decoder circuit 140 to receive the commands corresponding to the first task, so as to obtain information of the addresses of the registers to be accessed and the data to be written by the first task. The filter control circuit 160 may record address information of multiple registers of each of the multiple processing circuits 130[1] to 130[n] and data previously stored in each of these registers [from prior program values written to the accelerator program registers]. The filter control circuit 160 may compare whether the data previously stored in the registers to be accessed by the first task is the same as the data to be written by the first task. If the two are the same, the filter control circuit 160 does not write the data to be written by the first task to the registers [wherein the processing device does not write other accelerator program registers in response to corresponding accelerator program difference information]. If the data previously stored in the registers to be accessed by the first task is different from the data to be written by the first task, the filter control circuit 160 writes the data to be written by the first task to the registers [wherein the accelerator program difference information causes the processing device to write only accelerator program registers having program values being updated].”) [Examiner’s Note: Previously stored register values can be associated with previously executed tasks from previously recorded accelerator programs]
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have incorporated the teachings of Xu into the teachings of Zhang. This combination of teachings would have resulted in a method configured to generate difference data with the purpose of modifying assembly structure, as in Zhang, thereby rewriting the register values only if there are differences between the target and the current data, as in Xu. One of ordinary skill in the art would have been motivated to combine these teachings for the purpose of updating registers with data changes instead of all registers to save processing time in a data update (Xu [0038]).
With regards to claim 8, the rejection of claim 6 is incorporated.
Zhang does not teach: wherein the accelerator program registers are memory-mapped and are accessible using relative addressing
However, in an analogous art Xu teaches wherein the accelerator program registers are memory-mapped and are accessible using relative addressing (Xu [0018], “The processor 101 may encode addresses of the registers to be accessed by the task and the data to be written into multiple command in a predetermined format, for the image processing accelerator device 100 to analyze (or decode) these commands so as to obtain information related the addresses of the registers to be accessed and the data to be written [and are accessible using relative addressing].”) (Xu [0023], “The decoder circuit 140 is coupled to the memory 120, and decodes the commands corresponding to the first task to obtain addresses of registers to be accessed and data to be written by the first task [wherein the accelerator program registers are memory-mapped], so as to determine whether the first task is to access the registers of one corresponding among the multiple image processing circuits 130[1] to 130[n] so as to write first data to the multiple registers.”)
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have incorporated the teachings of Xu into the teachings of Zhang. This combination of teachings would have resulted in a method configured to generate difference data with the purpose of modifying assembly structure, as in Zhang, thereby rewriting the register values only if there are differences between the target and the current data, as in Xu. One of ordinary skill in the art would have been motivated to combine these teachings for the purpose of updating registers with data changes instead of all registers to save processing time in a data update (Xu [0038]).
With regards to claim 15, the rejection of claim 6 is incorporated.
Zhang does not teach: wherein program values of the accelerator program registers do not change during execution of each accelerator program of the plurality of accelerator programs.
However, in an analogous art Xu teaches wherein program values of the accelerator program registers do not change during execution of each accelerator program of the plurality of accelerator programs. (Xu [0034], “On the other hand, during a period in which the image processing circuit 103[1] executes the task T1, the processor 101 additionally applies for another temporary storage space B2 (operation S406) to store multiple commands corresponding to the task T2. The processor 101 further stores a wait event command following the multiple commands corresponding to the task T2 to the temporary storage space B2 (operation S407), and notifies the image processing accelerator device 100 after storing the wait event command (operation S408). Once it is determined that the identifier E[1] the same as the event identifier EID is received, the image processing accelerator device 100 starts decoding the multiple commands corresponding to the task T2, so as to learn about 100 registers in the image processing circuit 103[1] to which second data of the task T2 is to be written.”) [Examiner’s Note: Change doesn’t occur until the first task is completed with execution. The comparison of registers happen during a wait period afterwards]
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have incorporated the teachings of Xu into the teachings of Zhang. This combination of teachings would have resulted in a method configured to generate difference data with the purpose of modifying assembly structure, as in Zhang, thereby rewriting the register values only if there are differences between the target and the current data, as in Xu. One of ordinary skill in the art would have been motivated to combine these teachings for the purpose of updating registers with data changes instead of all registers to save processing time in a data update (Xu [0038]).
Claim 9 is rejected under 35 U.S.C. 103 as being unpatentable over Zhang in view of Xu, as applied to claim 6, in view of Mehendale and further in view of US 20220147810 A1 hereinafter “Zaidy”.
With regards to claim 9, the rejection of claim 6 is incorporated.
The combination of Zhang and Xu does not teach: wherein the compiled program corresponds to a machine learning model, the machine learning model having at least one layer
However, in an analogous art Mehendale teaches wherein the compiled program corresponds to a machine learning model, the machine learning model having at least one layer (Mehendale [0033], “The processing nodes of a neural network can be divided into layers including, for example, an input layer, a number of intermediate layers (e.g., hidden layers), and an output layer. The input layer and the intermediate layers can each be a convolution layer forming CNN, whereas the output layer can be a fully-connected layer, and the input layer, intermediate layer, and the output layer together form a DNN [wherein the compiled program corresponds to a machine learning model, the machine learning model having at least one layer]”) [Examiner’s Note: A program can have compiled instructions that control memory sources or manage neural network layers (See [0046-47])]
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have incorporated the teachings of Mehendale into the teachings of Zhang in view of Xu. This combination of teachings would have resulted in a method configured to generate difference data with the purpose of modifying assembly structure, as in Zhang, thereby rewriting the register values only if there are differences between the target and the current data, as in Xu, and further applying the updates to an accelerator layer in a machine learning application, as in Mehendale. One of ordinary skill in the art would have been motivated to combine these teachings for the purpose of associating APIs with neural network layers to fetch data, perform inferences, and generate controls accordingly (Mehendale [0047]).
The combination of Zhang, Xu, and Mehendale does not teach: wherein the plurality of accelerator programs correspond to at least one kernel of the at least one layer.
However, in an analogous art Zaidy teaches wherein the plurality of accelerator programs correspond to at least one kernel of the at least one layer. (Zaidy [0082], “According to the DLA instructions (205), the Deep Learning Accelerator (103) loads matrix operands into the kernel buffers (131 to 133) [wherein the plurality of accelerator programs correspond to at least one kernel] and maps banks (151 to 153) of its matrix-matrix unit (121). The matrix-matrix unit (121) performs the matrix computation on the matrix operands. For example, the DLA instructions (205) break down matrix computations of the trained Artificial Neural Network (201) according to the computation granularity of the Deep Learning Accelerator (103) (e.g., the sizes/dimensions of matrices that loaded as matrix operands in the matrix-matrix unit (121)) and applies the input feature maps to the kernel of a layer of artificial neurons [of the at least one layer]”)
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have incorporated the teachings of Zaidy into the teachings of Zhang in view of Xu and further in view of Mehendale. This combination of teachings would have resulted in a method configured to generate difference data with the purpose of modifying assembly structure, as in Zhang, thereby rewriting the register values only if there are differences between the target and the current data, as in Xu, and further applying the updates to an accelerator layer in a machine learning application, as in Mehendale, wherein the machine learning model incorporates an associated processing unit for accelerator program computations, as in Zaidy. One of ordinary skill in the art would have been motivated to combine these teachings for the purpose of performing computations on matrices of an artificial neural network to determine properties and the state of the system (Zaidy [0089-92]).
Claim 10 is rejected under 35 U.S.C. 103 as being unpatentable over Zhang in view of Xu, as applied to claim 6, in view of Mehendale and further in view of Kinsner.
With regards to claim 10, the rejection of claim 6 is incorporated.
Zhang further teaches wherein the corresponding accelerator program difference information is included in metadata of the compiled program (Zhang [0029-30], “Further, the method can also include, prior to receiving the hot-update instruction by the running program, compiling source codes modified according to requirements of the hot-update to generate the update assembly; performing structural adjustments to the update assembly, causing identical metadata in the adjusted update assembly and existing assembly of the running program to have identical identifiers [of the compiled program];… In order to reduce the amount of data to be transferred, the update assembly and existing assembly can be compared (wherein, identical metadata in the update assembly and existing assembly have identical identifiers), for example, by utilizing a ‘xdelta’ tool for comparing differences of binary files, to generate binary delta data comprising only the parts of the update assembly which are different from the existing assembly, and the generated binary delta data is sent to the running program as the update assembly [the accelerator program difference information being included in the metadata].”)
The combination of Zhang and Xu does not teach: wherein the compiled program corresponds to a machine learning model, the machine learning model having at least one layer,
However, in an analogous art Mehendale teaches wherein the compiled program corresponds to a machine learning model, the machine learning model having at least one layer, (Mehendale [0033], “The processing nodes of a neural network can be divided into layers including, for example, an input layer, a number of intermediate layers (e.g., hidden layers), and an output layer. The input layer and the intermediate layers can each be a convolution layer forming CNN, whereas the output layer can be a fully-connected layer, and the input layer, intermediate layer, and the output layer together form a DNN [wherein the compiled program corresponds to a machine learning model, the machine learning model having at least one layer]”) [Examiner’s Note: A program can have compiled instructions that control memory sources or manage neural network layers (See [0046-47])]
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have incorporated the teachings of Mehendale into the teachings of Zhang in view of Xu. This combination of teachings would have resulted in a method configured to generate difference data with the purpose of modifying assembly structure, as in Zhang, thereby rewriting the register values only if there are differences between the target and the current data, as in Xu, and further applying the updates to an accelerator layer in a machine learning application, as in Mehendale. One of ordinary skill in the art would have been motivated to combine these teachings for the purpose of associating APIs with neural network layers to fetch data, perform inferences, and generate controls accordingly (Mehendale [0047]).
The combination of Zhang, Xu, Mehendale does not teach: [wherein the corresponding accelerator program difference information is included in metadata of the compiled program and the corresponding accelerator program difference information] causes the processing device to execute a reference kernel for each layer of the machine learning model having no corresponding compiled data in the metadata of the compiled program.
However, in an analogous art Kinsner teaches […] causes the processing device to execute a reference kernel for each layer of the machine learning model having no corresponding compiled data in the metadata of the compiled program. (Kinsner [0062-63], “In one implementation, the offline telemetry harvesting is referred to herein as an offline harvesting session. At block 520, the processing device may extract device telemetry data corresponding to kernel invocations of the application on one or more accelerator devices… At decision block 530, the processing device determines whether sufficient telemetry data has been collected. This determination may be based on a threshold condition of telemetry data being gathered. If it is determined that insufficient telemetry data has been gathered [having no corresponding compiled data in the metadata of the compiled program], then method 500 proceeds to block 540 where the processing device may run additional sequence of kernels/data copies and return to block 520. [to execute a reference kernel for each layer of the machine learning model].”)
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have incorporated the teachings of Kinsner into the teachings of Zhang in view of Xu and further in view of Mehendale. This combination of teachings would have resulted in a method configured to generate difference data with the purpose of modifying assembly structure, as in Zhang, thereby rewriting the register values only if there are differences between the target and the current data, as in Xu, further applying the updates to an accelerator layer in a machine learning application, as in Mehendale, and executing a kernel or unit of processing when there is no corresponding data, as in Kinsner. One of ordinary skill in the art would have been motivated to combine these teachings for the purpose of enabling software that can be eventually transitioned into execution on accelerator hardware (Kinsner [0014]).
Claims 11-12 are rejected under 35 U.S.C. 103 as being unpatentable over Zhang in view of Xu, as applied to claim 6, and further in view of Kinsner.
With regards to claim 11, the rejection of claim 6 is incorporated.
Zhang further teaches [host] memory configured to store the compiled program; (Zhang [0033], “loading the update assembly into memory, and finding, in the update assembly, functions which have been modified with respect to the existing assembly according to the difference descriptions.,”) [Examiner’s Note: an update assembly is generated by compiling software]
[a host processing system configured] to generate the corresponding accelerator program difference information (Zhang [0049], “According to some embodiments of the invention, the hot-update apparatus can also include: a new assembly generating unit 350 configured to compile source codes modified according to requirements of the hot-update to generate the update assembly; a structural adjusting unit 351 configured to perform structural adjustments to the update assembly, causing identical metadata in the adjusted update assembly and the existing assembly of the running program to have identical identifiers;”)
The combination of Zhang and Xu does not teach: a host processing system configured [to generate the corresponding accelerator program difference information by] executing the compiled program and recording accelerator programs generated by execution of the compiled program.
However, in an analogous art Kinsner teaches a host processing system configured […]by executing the compiled program and recording accelerator programs generated by execution of the compiled program. (Kinsner [0033], “In some implementations, although host system 100 is depicted as implementing a virtualization system to virtualize its resources (e.g., memory resources and processing resources), some implementations may execute applications and/or workload on host system 100 by directly utilizing the resources of host system 100 [executing the compiled program on a host device]”) (Kinsner [0085], “For example, the machine readable instructions may be stored in multiple parts [recording all accelerator programs], which are individually compressed, encrypted, and stored on separate computing devices, wherein the parts when decrypted, decompressed, and combined form a set of executable instructions that implement a program such as that described herein [generated by execution of the compiled program].”)
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have incorporated the teachings of Kinsner into the teachings of Zhang in view of Xu. This combination of teachings would have resulted in a method configured to generate difference data with the purpose of modifying assembly structure, as in Zhang, thereby rewriting the register values only if there are differences between the target and the current data, as in Xu, and executing the corresponding software to generate said difference data for the compression, as in Kinsner. One of ordinary skill in the art would have been motivated to combine these teachings for the purpose of enabling software that can be eventually transitioned into execution on accelerator hardware (Kinsner [0014]).
With regards to claim 12, the rejection of claim 11 is incorporated.
Zhang further teaches wherein the [host] processing system is further configured to store the corresponding accelerator program difference information in metadata of the compiled program. (Zhang [0029-30], “Further, the method can also include, prior to receiving the hot-update instruction by the running program, compiling source codes modified according to requirements of the hot-update to generate the update assembly; performing structural adjustments to the update assembly, causing identical metadata in the adjusted update assembly and existing assembly of the running program to have identical identifiers [of the compiled program]; … In order to reduce the amount of data to be transferred, the update assembly and existing assembly can be compared (wherein, identical metadata in the update assembly and existing assembly have identical identifiers), for example, by utilizing a ‘xdelta’ tool for comparing differences of binary files, to generate binary delta data comprising only the parts of the update assembly which are different from the existing assembly, and the generated binary delta data is sent to the running program as the update assembly [to store the corresponding accelerator program difference information in metadata of the compiled program].”)
Claim 13 is rejected under 35 U.S.C. 103 as being unpatentable over Zhang in view of Xu in view of Kinsner, as applied to claim 11, in view of Yelheri and further in view of Schmidt.
With regards to claim 13, the rejection of claim 11 is incorporated.
Zhang further teaches wherein the [host] processing system is further configured to generate the corresponding accelerator program difference information by:
writing indications of accelerator program registers being updated from prior values of an immediately preceding accelerator program of the plurality of accelerator programs to the corresponding accelerator program difference information, (Zhang [0049], “a structural adjusting unit 351 configured to perform structural adjustments to the update assembly, causing identical metadata in the adjusted update assembly and the existing assembly of the running program to have identical identifiers; a comparing unit 352 configured to compare the update assembly with the existing assembly to generate a difference description file [writing indications of accelerator program registers being updated from prior values], wherein, the difference description file contains the difference descriptions for describing the differences between the update assembly and the existing assembly [immediately preceding accelerator program of the plurality of accelerator programs to the corresponding accelerator program difference information]; an instruction sending unit 353 configured to send the hot-update instruction to the running program, causing the running program to obtain the difference descriptions from the difference description file.”) [Examiner’s Note: Due to the previously recorded accelerator program existing as an assembly on the device one of ordinary skill in the art could reasonably understand that the update difference is in respect to previously executed software]
The combination of Zhang, Xu and Kinsner does not teach: wherein writes to unchanged accelerator program registers, unused accelerator program registers, and uninitialized arithmetic logic unit registers are omitted from the corresponding accelerator program difference information.
However, in an analogous art Yelheri teaches wherein writes to unchanged accelerator program registers (Yelheri [0040], “While doing incremental snapshot copy, changed blocks between two snapshots may be copied to the object store, and unchanged blocks will be shared with previous snapshots as opposed to being redundantly stored in the object store. In this way, deduplication is provided for and between snapshot data stored within objects of the object store. As will be described later, an embodiment of a snapshot file system in the object store is illustrated by FIG. 5B.”), unused accelerator program registers […] are omitted from the corresponding accelerator program difference information. (Yelheri [0159], “The fragmented object is compacted to retain the in-use data and exclude the freed data (the unused data) as a written object. Because compacting may store the in-use data in new slots, an object header of the object is updated with new locations of the in-use data within the rewritten object. In this way, defragmentation is performed for objects within the object store 509.”)
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have incorporated the teachings of Yelheri into the teachings of Zhang in view of Xu and further in view of Kinsner. This combination of teachings would have resulted in a method configured to generate difference data with the purpose of modifying assembly structure, as in Zhang, thereby rewriting the register values only if there are differences between the target and the current data, as in Xu, executing a the corresponding software to generate said difference data for the compression, as in Kinsner, and the difference data not including unchanged and unused registers for updating general storage, as in Yelheri. One of ordinary skill in the art would have been motivated to combine these teachings for the purpose of ensuring proper determination/comparison of data so that only data that doesn’t already exist in the data store is transmitted (Yelheri [0164]).
The combination of Zhang, Xu, Kinsner, and Yelheri does not teach: and uninitialized arithmetic logic unit registers are omitted from the corresponding accelerator program difference information.
However, in an analogous art Schmidt teaches and uninitialized arithmetic logic unit registers are omitted from the corresponding accelerator program difference information. (Schmidt [0038], “Execution unit 211 comprises a set of general purpose registers 212 for storing data and a scalar arithmetic logic unit (ALU) 213 for performing arithmetic and logical operations on data in general purpose (GP) registers 212 responsive to instructions decoded by instruction unit 201.”) (Schmidt [0058], “The compiler further determines whether it should reserve any general purpose registers for use by a dynamic binary optimizer in addressing a context save area (block 503). Preferably, reservation of one or more registers is performed responsive to a compiler directive. The compiler directive could be a statement in source code file 311 or intermediate code file 312, but in the preferred embodiment it is an optional compiler parameter which is specified by the user at the time the compiler is invoked... It would alternatively be possible for reservation of one or more registers for the dynamic binary optimizer to be a default option, which is automatically selected unless the user directs the compiler not to reserve a register. Alternatively, the compiler might be written to always reserve at least one register, although this is considered undesirable because there may be circumstances where it is desirable to use all registers.”) [Examiner’s Note: To reserve a register means to avoid using said register. A compiler directive can invoke this action by omitting information which includes actions taken by a register associated with ALUs.]
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have incorporated the teachings of Schmidt into the teachings of Zhang in view of Xu in view of Kinsner and further in view of Yelheri. This combination of teachings would have resulted in a method configured to generate difference data with the purpose of modifying assembly structure, as in Zhang, thereby rewriting the register values only if there are differences between the target and the current data, as in Xu, executing a the corresponding software to generate said difference data for the compression, as in Kinsner, the difference data not including unchanged and unused registers for updating general storage, as in Yelheri, and further leaving registers reserved and unchanged including in an uninitialized state by compiler directives accordingly, as in Schmidt. One of ordinary skill in the art would have been motivated to combine these teachings for the purpose of binary optimization of transitions between previously compiled object code modules and programs of a system (Schmidt [0046]).
Claim 14 is rejected under 35 U.S.C. 103 as being unpatentable over Zhang in view of Xu in view of Kinsner, as applied to claim 11, and further in view of Schmidt.
With regards to claim 14, the rejection of claim 11 is incorporated.
Zhang does not teach: wherein the [host] processing system is further configured to record accelerator program registers used by the compiled program and to identify any initialized and constant [arithmetic logic unit] registers.
However, in an analogous art Xu teaches wherein the [host] processing system is further configured to record accelerator program registers used by the compiled program and to identify any initialized and constant [arithmetic logic unit] registers. (Xu [0018], “The processor 101 may encode addresses of the registers to be accessed by the task and the data to be written into multiple command in a predetermined format, for the image processing accelerator device 100 to analyze (or decode) these commands so as to obtain information related the addresses of the registers to be accessed and the data to be written.”) [Examiner’s Note: Writing data to an ALU register can comprise initialization which can be determined with the information obtained.]
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have incorporated the teachings of Xu into the teachings of Zhang. This combination of teachings would have resulted in a method configured to generate difference data with the purpose of modifying assembly structure, as in Zhang, thereby rewriting the register values only if there are differences between the target and the current data, as in Xu. One of ordinary skill in the art would have been motivated to combine these teachings for the purpose of updating registers with data changes instead of all registers to save processing time in a data update (Xu [0038]).
The combination of Zhang and Xu does not teach: the host processing system
However, in an analogous art Kinsner teaches the host processing system (Kinsner [0033], “In some implementations, although host system 100 is depicted as implementing a virtualization system to virtualize its resources (e.g., memory resources and processing resources), some implementations may execute applications and/or workload on host system 100 by directly utilizing the resources of host system 100 [executing the compiled program on a host device]”)
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have incorporated the teachings of Kinsner into the teachings of Zhang in view of Xu. This combination of teachings would have resulted in a method configured to generate difference data with the purpose of modifying assembly structure, as in Zhang, thereby rewriting the register values only if there are differences between the target and the current data, as in Xu, and executing the corresponding software to generate said difference data for the compression, as in Kinsner. One of ordinary skill in the art would have been motivated to combine these teachings for the purpose of enabling software that can be eventually transitioned into execution on accelerator hardware (Kinsner [0014]).
The combination of Zhang, Xu, and Kinsner does not teach: arithmetic logic units
However, in an analogous art Schmidt teaches arithmetic logic units (Schmidt [0038], “Execution unit 211 comprises a set of general purpose registers 212 for storing data and a scalar arithmetic logic unit (ALU) 213 for performing arithmetic and logical operations on data in general purpose (GP) registers 212 responsive to instructions decoded by instruction unit 201.”)
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have incorporated the teachings of Schmidt into the teachings of Zhang in view of Xu and further in view of Kinsner. This combination of teachings would have resulted in a method configured to generate difference data with the purpose of modifying assembly structure, as in Zhang, thereby rewriting the register values only if there are differences between the target and the current data, as in Xu, executing the corresponding software to generate said difference data for the compression, as in Kinsner, and further identifying the state of registers associated with arithmetic logic units, as in Schmidt. One of ordinary skill in the art would have been motivated to combine these teachings for the purpose of binary optimization of transitions between previously compiled object code modules and programs of a system (Schmidt [0046]).
Claim 16 is rejected under 35 U.S.C. 103 as being unpatentable over Kinsner in view of Zhang.
With regards to claim 16, Kinsner teaches
A method for executing a program, the method comprising: executing a compiled program on a host device and recording all accelerator programs generated thereby; (Kinsner [0033], “In some implementations, although host system 100 is depicted as implementing a virtualization system to virtualize its resources (e.g., memory resources and processing resources), some implementations may execute applications and/or workload on host system 100 by directly utilizing the resources of host system 100 [executing the compiled program on a host device]”) (Kinsner [0085], “For example, the machine readable instructions may be stored in multiple parts [recording all accelerator programs], which are individually compressed, encrypted, and stored on separate computing devices, wherein the parts when decrypted, decompressed, and combined form a set of executable instructions that implement a program such as that described herein [generated thereby].”)
Kinsner does not teach: storing accelerator program difference information for corresponding accelerator programs in metadata of the compiled program.
However, in an analogous art Zhang teaches storing accelerator program difference information for corresponding accelerator programs in metadata of the compiled program. (Zhang [0029-30], “Further, the method can also include, prior to receiving the hot-update instruction by the running program, compiling source codes modified according to requirements of the hot-update to generate the update assembly; performing structural adjustments to the update assembly, causing identical metadata in the adjusted update assembly and existing assembly of the running program to have identical identifiers [of the compiled program]; … In order to reduce the amount of data to be transferred, the update assembly and existing assembly can be compared (wherein, identical metadata in the update assembly and existing assembly have identical identifiers), for example, by utilizing a ‘xdelta’ tool for comparing differences of binary files, to generate binary delta data comprising only the parts of the update assembly which are different from the existing assembly, and the generated binary delta data is sent to the running program as the update assembly [the accelerator program difference information being included in the metadata].”)
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have incorporated the teachings of Zhang into the teachings of Kinsner. This combination of teachings would have resulted in a method configured to execute the program on a host device to generate recorded accelerator software, as in Kinsner, and storing difference data according to metadata associated with accelerator software, as in Zhang. One of ordinary skill in the art would have been motivated to combine these teachings for the purpose of loading the update into memory in order to obtain a difference description in which it will define the updated version required to run (Zhang [0007]).
Claim 17 is rejected under 35 U.S.C. 103 as being unpatentable over Kinsner in view of Zhang, as applied to claim 16, in view of Yelheri and further in view of Schmidt.
With regards to claim 17, the rejection of claim 16 is incorporated.
Kinsner further teaches compressing an accelerator program recorded by the host device to generate the accelerator program difference information, (Kinsner [0085], “The machine readable instructions described herein may be stored in one or more of a compressed format, an encrypted format, a fragmented format, a compiled format, an executable format, a packaged format, etc. Machine readable instructions as described herein may be stored as data (e.g., portions of instructions, code, representations of code, etc.) that may be utilized to create, manufacture, and/or produce machine executable instructions. For example, the machine readable instructions may be fragmented and stored on one or more storage devices and/or computing devices (e.g., servers). The machine readable instructions may utilize one or more of installation, modification, adaptation, updating, combining, supplementing, configuring, decryption, decompression, unpacking, distribution, reassignment, compilation, etc. in order to make them directly readable, interpretable, and/or executable by a computing device and/or other machine.”)
Kinsner does not teach: wherein the compressing comprises writing values of accelerator program registers being updated from corresponding prior values of an immediately preceding accelerator program in a sequence of recorded accelerator programs to the accelerator program difference information;
However, in an analogous art Zhang teaches wherein the compressing comprises writing values of accelerator program registers being updated from corresponding prior values of an immediately preceding accelerator program in a sequence of recorded accelerator programs to the accelerator program difference information; (Zhang [0049], “a structural adjusting unit 351 configured to perform structural adjustments to the update assembly, causing identical metadata in the adjusted update assembly and the existing assembly of the running program to have identical identifiers; a comparing unit 352 configured to compare the update assembly with the existing assembly to generate a difference description file [wherein the compressing comprises writing values of accelerator program registers being updated], wherein, the difference description file contains the difference descriptions for describing the differences between the update assembly and the existing assembly [of an immediately preceding accelerator program in a sequence of recorded accelerator programs]; an instruction sending unit 353 configured to send the hot-update instruction to the running program, causing the running program to obtain the difference descriptions from the difference description file.”) [Examiner’s Note: Due to the previously recorded accelerator program existing as an assembly on the device one of ordinary skill in the art could reasonably understand that the update difference is in respect to previously executed software]
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have incorporated the teachings of Zhang into the teachings of Kinsner. This combination of teachings would have resulted in a method configured to execute the program on a host device to generate recorded accelerator software, as in Kinsner, and storing difference data according to metadata associated with accelerator software, as in Zhang. One of ordinary skill in the art would have been motivated to combine these teachings for the purpose of loading the update into memory in order to obtain a difference description in which it will define the updated version required to run (Zhang [0007]).
The combination of Kinsner and Zhang does not teach: wherein program values and corresponding register addresses of unchanged accelerator program registers, unused accelerator program registers and uninitialized arithmetic logic unit registers are omitted from the corresponding accelerator program difference information.
However, in an analogous art Yelheri teaches wherein program values and corresponding register addresses of unchanged accelerator program registers, (Yelheri [0040], “While doing incremental snapshot copy, changed blocks between two snapshots may be copied to the object store, and unchanged blocks will be shared with previous snapshots as opposed to being redundantly stored in the object store. In this way, deduplication is provided for and between snapshot data stored within objects of the object store. As will be described later, an embodiment of a snapshot file system in the object store is illustrated by FIG. 5B.”), unused accelerator program registers (Yelheri [0159], “The fragmented object is compacted to retain the in-use data and exclude the freed data (the unused data) as a written object. Because compacting may store the in-use data in new slots, an object header of the object is updated with new locations of the in-use data within the rewritten object. In this way, defragmentation is performed for objects within the object store 509.”)
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have incorporated the teachings of Yelheri into the teachings of Kinsner in view if Zhang. This combination of teachings would have resulted in a method configured to execute the program on a host device to generate recorded accelerator software, as in Kinsner, and storing difference data according to metadata associated with accelerator software, as in Zhang, and the difference information not including unchanged and unused registers for updating general storage, as in Yelheri. One of ordinary skill in the art would have been motivated to combine these teachings for the purpose of ensuring proper determination/comparison of data so that only data that doesn’t already exist in the data store is transmitted (Yelheri [0164]).
The combination of Kinsner, Zhang, and Yelheri does not teach: […]and uninitialized arithmetic logic unit registers are omitted from the corresponding accelerator program difference information.
However, in an analogous art Schmidt teaches and uninitialized arithmetic logic unit registers are omitted from the corresponding accelerator program difference information. (Schmidt [0038], “Execution unit 211 comprises a set of general purpose registers 212 for storing data and a scalar arithmetic logic unit (ALU) 213 for performing arithmetic and logical operations on data in general purpose (GP) registers 212 responsive to instructions decoded by instruction unit 201.”) (Schmidt [0058], “The compiler further determines whether it should reserve any general purpose registers for use by a dynamic binary optimizer in addressing a context save area (block 503). Preferably, reservation of one or more registers is performed responsive to a compiler directive. The compiler directive could be a statement in source code file 311 or intermediate code file 312, but in the preferred embodiment it is an optional compiler parameter which is specified by the user at the time the compiler is invoked... It would alternatively be possible for reservation of one or more registers for the dynamic binary optimizer to be a default option, which is automatically selected unless the user directs the compiler not to reserve a register. Alternatively, the compiler might be written to always reserve at least one register, although this is considered undesirable because there may be circumstances where it is desirable to use all registers.”)
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have incorporated the teachings of Schmidt into the teachings of Kinsner in view of Zhang and further in view of Yelheri. This combination of teachings would have resulted in a method configured to execute the program on a host device to generate recorded accelerator software, as in Kinsner, and storing difference data according to metadata associated with accelerator software, as in Zhang, the difference information not including unchanged and unused registers for updating general storage, as in Yelheri, and further leaving registers reserved and unchanged including in an uninitialized state by compiler directives accordingly, as in Schmidt. One of ordinary skill in the art would have been motivated to combine these teachings for the purpose of binary optimization of transitions between previously compiled object code modules and programs of a system (Schmidt [0046]).
Claims 18-19 are rejected under 35 U.S.C. 103 as being unpatentable over Kinsner in view of Zhang, as applied to claim 16, and further in view of Mehendale.
With regards to claim 18, the rejection of claim 16 is incorporated.
Kinsner further teaches: wherein executing the compiled program on the processing device comprises:
executing a reference kernel for each layer of the machine learning model having no corresponding compiled data in the metadata of the compiled program. (Kinsner [0062-63], “In one implementation, the offline telemetry harvesting is referred to herein as an offline harvesting session. At block 520, the processing device may extract device telemetry data corresponding to kernel invocations of the application on one or more accelerator devices… At decision block 530, the processing device determines whether sufficient telemetry data has been collected. This determination may be based on a threshold condition of telemetry data being gathered. If it is determined that insufficient telemetry data has been gathered [having no corresponding compiled data in the metadata of the compiled program], then method 500 proceeds to block 540 where the processing device may run additional sequence of kernels/data copies and return to block 520. [to execute a reference kernel for each layer of the machine learning model].”)
The combination of Kinsner and Zhang does not teach: executing the compiled program on a processing device,
wherein the compiled program corresponds to a machine learning model, the machine learning model having at least one layer,
However, in an analogous art Mehendale teaches executing the compiled program on a processing device,
wherein the compiled program corresponds to a machine learning model, the machine learning model having at least one layer, (Mehendale [0033], “The processing nodes of a neural network can be divided into layers including, for example, an input layer, a number of intermediate layers (e.g., hidden layers), and an output layer. The input layer and the intermediate layers can each be a convolution layer forming CNN, whereas the output layer can be a fully-connected layer, and the input layer, intermediate layer, and the output layer together form a DNN [wherein the compiled program corresponds to a machine learning model, the machine learning model having at least one layer]”)
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have incorporated the teachings of Mehendale into the teachings of Kinsner in view of Zhang. This combination of teachings would have resulted in a method configured to execute the program on a host device to generate recorded accelerator software, as in Kinsner, storing difference data according to metadata associated with accelerator software, as in Zhang, and further applying the updates to an accelerator layer in a machine learning application, as in Mehendale. One of ordinary skill in the art would have been motivated to combine these teachings for the purpose of associating APIs with neural network layers to fetch data, perform inferences, and generate controls accordingly (Mehendale [0047]).
With regards to claim 19, the rejection of claim 16 is incorporated.
Kinsner further teaches executing the compiled program on a processing device, (Kinsner [0093], “The apparatus of Example 1 comprises a processor to: receive, from a compiler, compiled code generated from source code of an application, the compiled code to support a workload of the application;”)
Kinsner does not teach: wherein executing the compiled program on the processing device comprises:
searching the metadata of the compiled program
wherein executing the compiled program on the processing device comprises:
searching the metadata of the compiled program [for compiled data associated with each layer of the machine learning model.]
However, in an analogous art Zhang teaches wherein executing the compiled program on the processing device comprises:
searching the metadata of the compiled program
wherein executing the compiled program on the processing device comprises:
searching the metadata of the compiled program [for compiled data associated with each layer of the machine learning model.] (Zhang [0047-48], “a receiving unit 310 configured to obtain, in response to receiving a hot-update instruction by a running program, an update assembly and difference descriptions for describing differences between the update assembly and an existing assembly of the running program, wherein, identical metadata in the update assembly and the existing assembly have identical identifiers; a modification determining unit 320 configured to load the update assembly into memory, and find, in the update assembly, functions which have been modified with respect to the existing assembly according to the difference descriptions;”)
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have incorporated the teachings of Zhang into the teachings of Kinsner. This combination of teachings would have resulted in a method configured to execute the program on a host device to generate recorded accelerator software, as in Kinsner, and storing difference data according to metadata associated with accelerator software, as in Zhang. One of ordinary skill in the art would have been motivated to combine these teachings for the purpose of loading the update into memory in order to obtain a difference description in which it will define the updated version required to run (Zhang [0007]).
The combination of Kinsner and Zhang does not teach: wherein the compiled program corresponds to a machine learning model, the machine learning model having at least one layer, and
wherein executing the compiled program on the processing device comprises:
[…] for compiled data associated with each layer of the machine learning model.
However, in an analogous art Mehendale teaches wherein the compiled program corresponds to a machine learning model, the machine learning model having at least one layer, (Mehendale [0033], “The processing nodes of a neural network can be divided into layers including, for example, an input layer, a number of intermediate layers (e.g., hidden layers), and an output layer. The input layer and the intermediate layers can each be a convolution layer forming CNN, whereas the output layer can be a fully-connected layer, and the input layer, intermediate layer, and the output layer together form a DNN [wherein the compiled program corresponds to a machine learning model, the machine learning model having at least one layer]”) [Examiner’s Note: A compiler can receive (searched) instructions according to several different kinds of computations including neural network layers] and
[…] for compiled data associated with each layer of the machine learning model. (Mehendale [0034], “The neural network processor can be programmed based on a sequence of instructions that include computation operations (e.g., adding, multiplication, processing of activation function, etc.) associated with the model. The instructions may also access internal and external memory devices to obtain and store data. A compiler may receive information about the neural network model, the input data, and the available memory and computation resources, and generate the set of instructions to indicate, for example, when to access the internal and external memory devices for the data, which component of the neural network processor to perform computations on the data based on the neural network model, etc., to perform the neural network processing.”)
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have incorporated the teachings of Mehendale into the teachings of Kinsner in view of Zhang. This combination of teachings would have resulted in a method configured to execute the program on a host device to generate recorded accelerator software, as in Kinsner, storing difference data according to metadata associated with accelerator software, as in Zhang, and further applying the updates to an accelerator layer in a machine learning application, as in Mehendale. One of ordinary skill in the art would have been motivated to combine these teachings for the purpose of associating APIs with neural network layers to fetch data, perform inferences, and generate controls accordingly (Mehendale [0047]).
Claim 20 is rejected under 35 U.S.C. 103 as being unpatentable over Kinsner in view of Zhang, as applied to claim 16, in view of Mehendale and further in view of Xu.
With regards to claim 20, the rejection of claim 16 is incorporated.
Kinsner further teaches executing the compiled program on a processing device, (Kinsner [0093], “The apparatus of Example 1 comprises a processor to: receive, from a compiler, compiled code generated from source code of an application, the compiled code to support a workload of the application;”)
The combination of Kinsner and Zhang does not teach: wherein the compiled program corresponds to a machine learning model, the machine learning model having at least one layer,
wherein executing the compiled program on the processing device comprises:
for each layer of the machine learning model:
However, in an analogous art Mehendale teaches wherein the compiled program corresponds to a machine learning model, the machine learning model having at least one layer,
wherein executing the compiled program on the processing device comprises:
for each layer of the machine learning model: (Mehendale [0033], “The processing nodes of a neural network can be divided into layers including, for example, an input layer, a number of intermediate layers (e.g., hidden layers), and an output layer. The input layer and the intermediate layers can each be a convolution layer forming CNN, whereas the output layer can be a fully-connected layer, and the input layer, intermediate layer, and the output layer together form a DNN [wherein the compiled program corresponds to a machine learning model, the machine learning model having at least one layer] … The neural network processor of data processor 106 can be programmed to perform computations based on an artificial neural network model. The neural network processor can be programmed based on a sequence of instructions that include computation operations (e.g., adding, multiplication, processing of activation function, etc.) associated with the model. The instructions may also access internal and external memory devices to obtain and store data.”)
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have incorporated the teachings of Mehendale into the teachings of Kinsner in view of Zhang. This combination of teachings would have resulted in a method configured to execute the program on a host device to generate recorded accelerator software, as in Kinsner, storing difference data according to metadata associated with accelerator software, as in Zhang, and further applying the updates to an accelerator layer in a machine learning application, as in Mehendale. One of ordinary skill in the art would have been motivated to combine these teachings for the purpose of associating APIs with neural network layers to fetch data, perform inferences, and generate controls accordingly (Mehendale [0047]).
The combination of Kinsner, Zhang, and Mehendale does not teach: writing changed accelerator program register values [for the layer], to corresponding accelerator program registers of a plurality of accelerator program registers, [according to the accelerator program difference information;] and
executing an accelerator program of the corresponding accelerator programs [associated with the layer] according to contents of the plurality of accelerator program registers.
However, in an analogous art Xu teaches writing changed accelerator program register values [for the layer], to corresponding accelerator program registers of a plurality of accelerator program registers, [according to the accelerator program difference information;] (Xu [0031], “The command matcher 164 may receive information of the addresses of the registers to be accessed and data (to be referred to as data to be updated below) to be written by a command (for example, the command CMD in FIG. 3) of the predetermined task [writing a program value to a corresponding accelerator program register of accelerator program registers of an accelerator circuit]. The command matcher 164 may search the memory 162 according to the addresses of the registers to be accessed by the command CMD, so as to read the previous data in the registers. The command matcher 164 may accordingly determine whether the previous data is the same as the data to be updated. If the previous data is the same as the data to be updated, it means that the data previously stored in the registers is the same as the new data to be written by the command CMD. In this case, the command matcher 164 does not transmit the addresses of the registers or the data to be updated to the command executor 166. Or, if the previous data is different from the data to be updated, it means that the data previously stored in the registers is different from the new data to be written by the command CMD. In this case, the command matcher 164 stores the data to be updated to the memory 162 to update the memory 162 by replacing the previous data corresponding to the registers with the data to be updated, and provides the addresses of the registers and the data to be updated to the command executor 166. [according to accelerator program difference information]”) and
executing an accelerator program of the corresponding accelerator programs [associated with the layer] according to contents of the plurality of accelerator program registers. (Xu [0017-20], “The processor 101 is capable of executing various types of application software, executes a predetermined task (for example but not limited to, processing image data) in response to a request of the application software, and accordingly applies for a temporary storage space in the memory 102 so as to store one or more commands corresponding to the predetermined task. In some embodiments, the processor 101 may be, for example but not limited to, a central processor [executing an accelerator program by the accelerator circuit]. In some embodiments, the memory 102 may be, for example but not limited to, a dynamic random access memory (DRAM)… In response to the predetermined task assigned by the processor 101, the parameter data of the multiple registers in one corresponding image processing circuit that is selected from the multiple image processing circuits 103[1] to 103[n] to execute the predetermined task can be accordingly updated, so that the corresponding image processing circuit may execute the predetermined task by means of accessing the parameter data in these registers [according to program values of the accelerator program registers of the accelerator circuit].”)
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have incorporated the teachings of Xu into the teachings of Kinsner in view of Zhang and further in view of Mehendale. This combination of teachings would have resulted in a method configured to execute the program on a host device to generate recorded accelerator software, as in Kinsner, storing difference data according to metadata associated with accelerator software, as in Zhang, and further applying the updates to an accelerator layer in a machine learning application, as in Mehendale, thereby rewriting the register values only if there are differences between the target and the current data, as in Xu. One of ordinary skill in the art would have been motivated to combine these teachings for the purpose of updating registers with data changes instead of all registers to save processing time in a data update (Xu [0038]).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to TRAVIS VIET TRAN whose telephone number is (571)272-3720. The examiner can normally be reached Monday-Friday 8:30AM-5PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Wei Mui can be reached at 571-272-3708. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/T.V.T./Examiner, Art Unit 2191 /WEI Y MUI/Supervisory Patent Examiner, Art Unit 2191