Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
The text of those sections of Title 35, U.S. Code not included in this action can be found in a prior Office action.
Claim(s) 1-3, 9-11, and 17-19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Cambricon Technologies Co. (FOR from IDS: CN109740751A, earliest publication May 2019, hereinafter “Cambricon Tech”) in view of Jittor (NPL from IDS: “Meta operator: Implement your own convolutional layer through meta operators”, published March 17 2020, hereinafter “Jittor”), and further in view of Brady et al. (US Pub. No. 2019/0392296, published Dec. 2019, hereinafter “Brady”) .
Regarding claim 1, Cambricon Tech teaches a neural network model processing method, comprising:
receiving, by a computer device, an operation process of a neural network model (Cambricon Tech, Page 4, Paragraphs 12-13 – “Step 101: The architecture fusion device of the neural network model extracts an operation flow of the first neural network model. The first neural network model includes a multi-layer structure, and the operation flow includes a plurality of operators of the multi-layer structure and a connection relationship between the plurality of operators, and each layer structure corresponds to at least one operator.” – teaches obtaining an operation process of a neural network model, represented as a plurality of operators. In addition to the previously cited passages, Cambricon Tech further teaches in Pg. 2, Paragraphs 14-15 – “In a second aspect, the embodiment of the present application provides an architecture fusion device of a neural network model, where the device includes: An extracting unit, configured to extract an operation flow of the first neural network model;” and in Pg. 11 Paragraph 5 – “The embodiment of the present application provides a card. Referring to FIG. 5, the card may include other supporting components… the components include, but are not limited to, a storage device 590 and an interface device 591.” – teaches receiving, by a computer device, an operation process of a neural network model (extraction unit of architecture fusion device extracts operation process of a neural network model, system includes an interface device)),
compiling, by the computer device, the operation process of the neural network model to generate a first computation graph of the neural network model based on the operation process (Cambricon Tech, Page 4, Paragraph 16 (last paragraph) – “Step 102: The architecture fusion device of the neural network model generates a calculation graph of the first neural network model according to the operation flow, where the calculation graph is a directed acyclic graph, and the directed acyclic graph includes a plurality of nodes.” – teaches compiling the operation process to generate a first computation graph of the neural network model based on the operation process (generates a calculation graph according to the operation flow). In addition to the previously cited passage, Cambricon Tech further teaches in Page 4, Paragraphs 12-14– “Step 101: The architecture fusion device of the neural network model extracts an operation flow of the first neural network model. The first neural network model includes a multi-layer structure, and the operation flow includes a plurality of operators of the multi-layer structure and a connection relationship between the plurality of operators, and each layer structure corresponds to at least one operator. The operation flow includes at least two processing unit information corresponding to the plurality of operators, and the processing unit information includes a central processing unit (CPU), a graphic processing unit (GPU), and an image processing unit (Image Processing). Unit, IPU).” – teaches compiling, by the computer device, the operation process of the neural network model (extracts operation flow of neural network model to generate a calculation graph as in step 102, thus compiling the operation flow of the neural network process to generate a computation graph)).
Cambricon Tech fails to explicitly teach wherein the operation process is represented by at least one first-type operator and a plurality of second-type operators, wherein, in the operation process, the first-type operator comprises a boundary identifier, and computational logic of the first-type operator is represented by a group of second-type operators in the plurality of second-type operators, and wherein, for any first-type operator, a range of a corresponding group of second-type operators comprised in the any first-type operator is indicated by a boundary identifier in the any first-type operator.
However, analogous to the field of the claimed invention, Jittor teaches:
wherein the operation process is represented by at least one first-type operator and a plurality of second-type operators (Jittor, Fig. 1 Description – “Hierarchical structure of meta-operators. Meta operators include three types of operators: reindex operators, reindex simplification operators, and element-level operators. The backpropagation operator of the meta operator is also a meta operator. Meta operators can form commonly used deep learning operators. These deep learning operators can further compose a deep learning model” – teaches wherein the operation process is represented by at least one first-type operator and a plurality of second-type operators (meta operators comprising three types of operators))[[;]],
wherein, in the operation process, the first-type operator comprises a boundary identifier, and computational logic of the first-type operator is represented by a group of second-type operators in the plurality of second-type operators (Jittor, Fig. 1 – describes first-type operators comprising a boundary identifier (meta-operators bounded by their respective sub-operators) and describes a group of second-type operators in the plurality of second-type operators that represent the computational logic of the first-type operator (sub-operators represent computation logic of their respective meta-operator)), and
wherein, for any first-type operator, a range of a corresponding group of second-type operators comprised in the any first-type operator is indicated by a boundary identifier in the any first-type operator (Jittor, Fig. 1 – describes a range of a corresponding group of second-type operators comprised in the any first type-operator indicated by a boundary identifier in the any first-type operator (the meta-operator “Reindex” comprises common reindex operators such as Broadcast, Pad, and Slice, thus the sub-operators are within a range of a corresponding group defined by the boundary of the meta-operator));
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to incorporate the first-type operator, second-type operators, and boundary identifiers of Jittor to the computation graph representing an operation process of a neural network model of Cambricon Tech in order to represent the operation process of neural networks as graphs of first and second-type operators. Doing so would provide an optimization technique for operators, resulting in much faster implementations of equivalent operations (Jittor, Paragraphs 8 & 13).
The combination of Cambricon Tech and Jittor fails to explicitly teach: receiving an operation process of a neural network model input by a user.
However, analogous to the field of the claimed invention, Brady teaches:
receiving, by a computer device, an operation process of a neural network model input by a user (Brady, [0024] – “FIG. 1 is a simplified block diagram 100 showing an example compiler adapted to generate executable code from machine learning models in a manner adapted to optimize, or efficiently and intelligently utilize, the processing, memory, and interconnect resources of particular target machine learning hardware to be utilized in consuming and executing the machine learning model. For instance, a machine learning model, such as a graph definition 110 of an example neural network model (or other deep learning model) may be provided as an input for consumption by an example neural network compiler 105… The compiler 125 may utilize these inputs to generate an intermediate representation (IR) 140, which includes multiple models 145 to represent the manageable resources provided by processing device 125. Such resources may include memory resources 130 and computation resources 135 (among other resources, such as communication or interconnect resources).” and in [0029] – “Interfaces (e.g., 220) of an example system may be implemented in hardware or software. The interfaces 220 can be used to receive both data and control information from the network as well as local sources, such as a remote control to a television. The electronic device can also provide a variety of user interfaces such as a keyboard, a touch screen, a trackball, a touch pad, and/or a mouse.” – teaches receiving, by a computer device (user interface, or interfaces 220, are of the computing system 205 as in Brady at [0026]), an operation process of a neural network model input by a user (user interface enables users to input data and control information of the network, system may receive neural network model as input and extracts operation processes and resources from the network input by user through user interface))
compiling, by the computer device, the operation process of the neural network model to generate a first computation graph of the neural network model based on the operation process (Brady, [0024] – “FIG. 1 is a simplified block diagram 100 showing an example compiler adapted to generate executable code from machine learning models in a manner adapted to optimize, or efficiently and intelligently utilize, the processing, memory, and interconnect resources of particular target machine learning hardware to be utilized in consuming and executing the machine learning model. For instance, a machine learning model, such as a graph definition 110 of an example neural network model (or other deep learning model) may be provided as an input for consumption by an example neural network compiler 105… Specific models 145 within the IR 140 may provide views of the memory resources 130 (e.g., through a data model) and computation resources 135 (e.g., a control model), among other example models provided within the generated IR to provide views for use in generating, through a set of compilation passes, code 150 (e.g., a binary), which is generated automatically by the compiler 105 as code optimized to the architecture and resources of the processing device 125.” and in [0028] – “In some embodiments, an example compiler (e.g., 105), such as an example neural network compiler such as discussed herein, as well as other components, may be implemented in software stored in memory 215, and operate on the processor 210.” – teaches compiling, by the computer device (compiler may be implemented in memory and operate on processor stored in computing system of Brady at [0026]), the operation process of the neural network model to generate a first computation graph of the neural network based on the operation process (compiler adapted to generate executable code from machine learning models, generates graph definition 110 of example neural network model based on the model’s resources and processes, thus generating a first computation graph of the neural network model based on the operation process))
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention to incorporate the user interface and compiler of Brady to the interface and first and second-type operators of Cambricon Tech and Jittor. Doing so would allow users to author and generate graphs of particular neural network models (Brady, [0058]) and provide a well-suited compiler for generating code for deep-learning applications involving dense and sparse linear algebraic operations (Brady, [0025]).
Claims 9 and 17 incorporate substantively all the limitations of claim 1 in a device and non-transitory computer-readable medium, and are rejected on similar grounds as above.
Regarding claim 2, the combination of Cambricon Tech, Jittor, and Brady teaches the processing method according to claim 1,
wherein the first computation graph comprises a main graph and a subgraph (Cambricon Tech, Page 5, Paragraph 10 – “The architecture fusion device of the neural network model performs the merging segmentation of the N fused nodes to obtain a directed acyclic graph after the merging segment is divided, and the directed acyclic graph after the merging segment is divided into P fused segments.” – teaches wherein the first computation graph comprises a main graph and a subgraph (divides graph into segments, thus resulting in a main graph and subgraphs)), and
wherein compiling the operation process comprises: determining the main graph and the subgraph of the neural network model based on the operation process (Cambricon Tech, Page 5, Paragraph 10 – “The architecture fusion device of the neural network model performs the merging segmentation of the N fused nodes to obtain a directed acyclic graph after the merging segment is divided, and the directed acyclic graph after the merging segment is divided into P fused segments.” – teaches wherein compiling the operation process comprises determining the main graph and subgraph of the neural network model based on the operation process (divides graph according to operation flow into segments, thus resulting in a main graph and subgraphs). In addition to the previously cited passage, Cambricon Tech further teaches in Pg. 4, Paragraph 16 (last paragraph) – “Step 102: The architecture fusion device of the neural network model generates a calculation graph of the first neural network model according to the operation flow, where the calculation graph is a directed acyclic graph, and the directed acyclic graph includes a plurality of nodes.” – teaches determining the main graph and subgraphs of the neural network model based on the operation process (graph generated according to operation flow))
the main graph is used to output a result of the operation process (Cambricon Tech, Page 4, Paragraph 16 – “Step 102: The architecture fusion device of the neural network model generates a calculation graph of the first neural network model according to the operation flow, where the calculation graph is a directed acyclic graph, and the directed acyclic graph includes a plurality of nodes.” – teaches wherein the main graph (calculation graph) is used to output a result of the operation process (calculation graph represents operation flow of the neural network model from input to output)),
the subgraph is used to output a result of a first-type operator (Cambricon Tech, Page 5, Paragraph 6 – “For example, as shown in FIG. 1B, in (a), operator 1 is the target operator, and operator 2 to operator 9 are the lower-level operators of operator 1. The connection relationship of the lower-level operator is: operator 2 And operator 3 are connected with operator 1, the connection direction is operator 1 points to operator 2 and operator 3 respectively;” – teaches the subgraph used to output a result of a first-type operator (lower-level operators 2 to 9 output the result of operator 1)),
Cambricon Tech fails to explicitly teach wherein the first-type operator in the main graph is indicated by the boundary identifier, the second-type operator in the main graph is indicated by a name of the second-type operator, the subgraph comprises a name of the second-type operator that is comprised in the any first-type operator.
However, analogous to the field of the claimed invention, Jittor teaches:
wherein the first-type operator in the main graph is indicated by the boundary identifier (Jittor, Fig. 1 – teaches the first-type operator indicated by the boundary identifier (meta-operator contains three types of operators: reindex, reindex reduce, and element-wise)),
the second-type operator in the main graph is indicated by a name of the second-type operator (Jittor, Fig. 1 – teaches second-type operators indicated by a name of the second type operator (second-type operator names being: broadcast, pad, slice, reduce, product, sum, etc.)),
the subgraph comprises a name of the second-type operator that is comprised in the any first-type operator (Jittor, Fig. 1 – teaches the name of the second-type operators comprised in the any first-type operator).
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to incorporate the first and second-type operators of Jittor to further modify the main graph and subgraph of Cambricon Tech, Jittor, and Brady in order to represent the operation process of a neural network model as a plurality of first and second-type operators represented in a main graph comprising sub-graphs. Doing so would improve the speed of the neural network model, and at the same time, can optimize different neural network models (Cambricon Tech, Page 6) and use operators to compose a deep learning model (Jittor, Fig. 1 Description).
Claim 10 and 18 are similar to claim 2, hence similarly rejected.
Regarding claim 3, the combination of Cambricon Tech, Jittor, and Brady teaches the processing method according to claim 1, wherein the method further comprises:
performing optimization processing on the first computation graph by using the first-type operator as a processing granularity to obtain a second computation graph (Cambricon Tech, Page 5, Paragraph 2 – “The architecture fusion device of the neural network model selects a target operator from the plurality of operators, and the target operator is a starting node of the directed acyclic graph;” – teaches using an operator (target operator) as a processing granularity to perform optimization processing on the computation graph to obtain a second computation graph, as in Page 7, Paragraph 9 – “It can be seen that, in the embodiment of the present application, the architecture fusion device of the neural network model extracts the operation flow of the first neural network model, generates a calculation graph of the first neural network model according to the operation flow, and calculates the graph as a directed acyclic graph. The directed acyclic graph includes a plurality of nodes, and at least two nodes in the directed acyclic graph are fused to obtain a fused directed acyclic graph, and a second neural network model is generated according to the fused directed acyclic graph. . This can improve the speed of the neural network model, and at the same time, can optimize different neural network models.” – teaches performing optimization processing on the first computation graph to obtain a second computation graph (fused directed acyclic graph)).
Claim 11 and 19 are similar to claim 3, hence similarly rejected.
Claim(s) 4-6, 8, 12-14, 16, and 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Cambricon Tech, Jittor, and Brady as applied to claims 1, 9, and 17 above, and further in view of Sui et al. (From IDS: US Pub. No. 2019/0303762, published Oct. 2019, hereinafter “Sui”).
Regarding claim 4, the combination of Cambricon Tech, Jittor, and Brady teaches the processing method according to claim 3.
The combination of Cambricon Tech, Jittor, and Brady fails to explicitly teach wherein the first-type operator comprises a third operator and a fourth operator, and the third operator and the fourth operator comprise same computational logic, and wherein [[the]] performing the optimization processing on the first computation graph by using the first-type operator as [[a]] the processing granularity comprises: fusing a subgraph corresponding to the third operator and a subgraph corresponding to the fourth operator to obtain a fused subgraph, wherein the second computation graph comprises the fused subgraph.
However, analogous to the field of the claimed invention, Sui teaches:
wherein the first-type operator comprises a third operator and a fourth operator, and the third operator and the fourth operator comprise same computational logic (Sui, [0073] – “FIG. 5 shows horizontal fusion of convolutions according to an embodiment of the present invention. The horizontal layer fusion can be understood as several operations sharing a same input feature map, it may not be needed to load a feature map on the chip for each layer, instead, be written back to corresponding position of the off-chip memory respectively after calculation is directly finished on the chip.” – teaches wherein a third and fourth operator comprise the same computational logic (two convolution operations are fused, operations share the same feature map and thus the same computational logic)), and
wherein [[the]] performing the optimization processing on the first computation graph by using the first-type operator as [[a]] the processing granularity comprises: fusing a subgraph corresponding to the third operator and a subgraph corresponding to the fourth operator to obtain a fused subgraph, wherein the second computation graph comprises the fused subgraph (Sui, [0073] – “FIG. 5 shows horizontal fusion of convolutions according to an embodiment of the present invention. The horizontal layer fusion can be understood as several operations sharing a same input feature map, it may not be needed to load a feature map on the chip for each layer, instead, be written back to corresponding position of the off-chip memory respectively after calculation is directly finished on the chip.” – teaches fusing sub-graphs operators having the same computational logic, wherein the second computation graph comprises the fused subgraph, as in [0108] – “In step S1530, according to the preset rules, at least one subgraph matching strategy for the computational graph is obtained. In step S1540, based on the subgraph matching strategy, the computational graph is reconstructed to form a computational graph through layer fusion.” – teaches a second computation graph comprises the fused subgraph).
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to incorporate the optimization processing on the first computation graph by fusing subgraphs corresponding to operators of Sui to the first and second-type operators, and main graph of Cambricon Tech, Jittor, and Brady, in order to fuse subgraphs of operators having the same computational logic. Doing so would allow for operators to be fused according to optimization rules, and reduces the frequency of data exchange in network operation (Sui, [0069-0070]).
Claim 12 and 20 are similar to claim 4, hence similarly rejected.
Regarding claim 5, the combination of Cambricon Tech, Jittor, and Brady teaches the processing method according to claim 3.
The combination of Cambricon Tech, Jittor, and Brady fails to explicitly teach wherein the first-type operator comprises a fifth operator and a sixth operator, and an intermediate computation result of the fifth operator is the same as an intermediate computation result of the sixth operator, and wherein [[the]] performing the optimization processing on the first computation graph by using the first- type operator as [[a]] the processing granularity comprises: using the intermediate computation result of the fifth operator as an input parameter of the sixth operator.
However, analogous to the field of the claimed invention, Sui teaches:
wherein the first-type operator comprises a fifth operator and a sixth operator, and an intermediate computation result of the fifth operator is the same as an intermediate computation result of the sixth operator (Sui, [0072] – “longitudinally adjacent layers where output feature maps of a previous layer are at least part of the input of a next layer.” and in [0073] – “FIG. 4 shows longitudinal fusion of convolutions according to an embodiment of the present invention. The longitudinal layers of two successive convolution operations are fused, all needed parameters can be loaded into on-chip buffer at one time, thus, the two convolution operations are fused. The intermediate result between the two operations does not need to be written back to off-chip memory, instead, is directly cached on the chip, and is written back after the calculation is finished.” – teaches wherein an intermediate computation result of a fifth operator is the same as an intermediate computation result of the sixth operator (two convolution operators fused)), and
wherein [[the]] performing the optimization processing on the first computation graph by using the first- type operator as [[a]] the processing granularity comprises: using the intermediate computation result of the fifth operator as an input parameter of the sixth operator (Sui, [0072] – “longitudinally adjacent layers where output feature maps of a previous layer are at least part of the input of a next layer.” and in [0073] – “FIG. 4 shows longitudinal fusion of convolutions according to an embodiment of the present invention. The longitudinal layers of two successive convolution operations are fused, all needed parameters can be loaded into on-chip buffer at one time, thus, the two convolution operations are fused. The intermediate result between the two operations does not need to be written back to off-chip memory, instead, is directly cached on the chip, and is written back after the calculation is finished.” – teaches using an intermediate computation result of the fifth operator as an input parameter of the sixth operator (feature maps of a previous layer are a part of the input of a next layer, intermediate result directly cached on chip and written after calculation is finished)).
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to incorporate the use of the intermediate computation result of a fifth operator as an input of a sixth operator of Sui to the first and second-type operators of Cambricon Tech, Jittor, and Brady in order to perform optimization processing by using the intermediate computation result of the fifth operator as an input parameter of the sixth operator. Doing so would improve the execution efficiency of hardware by eliminating the influence of data rearrangement and removing operations which have no influence on the calculation result (Sui, [0083])
Claim 13 is similar to claim 5, hence similarly rejected.
Regarding claim 6, the combination of Cambricon Tech, Jittor, Brady, and Sui teaches the processing method according to claim 4, wherein:
the third operator is a forward operator, and the fourth operator is a backpropagation operator corresponding to the third operator; or the fourth operator is a forward operator, and the third operator is a backpropagation operator corresponding to the fourth operator (Jittor, Fig. 1 & Fig. 1 description – “Hierarchical structure of meta-operators. Meta operators include three types of operators: reindex operators, reindex simplification operators, and element-level operators. The backpropagation operator of the meta operator is also a meta operator. Meta operators can form commonly used deep learning operators. These deep learning operators can further compose a deep learning model” – teaches a third operator (backpropagation operator) corresponding to a fourth operator (forward propagation operator - Reindex, Reindex Reduce, Element-Wise operators are forward propagation operators, consistent with the definition of forward-propagation in the specification of the claimed invention at [0015])).
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to incorporate the third and fourth operators performing forward and backward propagation of Jittor to further modify the first-type operator of Cambricon Tech, Jittor, Brady, and Sui in order to represent forward and backward propagation as operators in a computation graph. Doing so would provide faster implementations of deep learning operators (Jittor, Paragraphs 13-14).
Claim 14 is similar to claim 6, hence similarly rejected.
Regarding claim 8, the combination of Cambricon Tech, Jittor, and Brady teaches the processing method according to claim 1.
The combination of Cambricon Tech, Jittor, and Brady fails to explicitly teach wherein an input of the operation process is tensor data, and the tensor data is used to describe a feature of data in at least one of the following scenarios: speech recognition, computer vision (CV), video processing, image recognition, and natural language processing (NLP).
However, analogous to the field of the claimed invention, Sui teaches:
wherein an input of the operation process is tensor data (Sui, [0083] – “As mentioned above, in the execution process for the whole computational graph, the way how data is stored in hardware can be abstracted into a one-dimensional data tensor which is formed by unfolding the feature map according to a certain rule, thus, as to operations which doesn't change data value and which only changes data dimension or arrange mode, deleting from the computational graph can be carried out through pruning.” – teaches wherein an input of the operation process is tensor data (data stored can be abstracted into a one-dimensional data tensor formed by unfolding an input feature map)), and the tensor data is used to describe a feature of data in at least one of the following scenarios: speech recognition, computer vision (CV), video processing, image recognition, and natural language processing (NLP) (Sui, [0069] – “A computational platform involved in the present invention needs to read needed data, such as an image that needs to be classified, from off-chip memory. “ – teaches the tensor data used to describe a feature of data in at least one of the following scenarios: image recognition (reads data of an image to be classified, and data may be abstracted into a one-dimensional data tensor, as in Sui, [0083])).
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to incorporate the tensor data describing features of data in the image recognition scenario of Sui to the method of Cambricon Tech, Jittor, and Brady in order to utilize image data for recognition using the obtained computation graph. Doing so would improve the task execution efficiency of the computational platform (Sui, [0078]) and provide an accelerator for neural network algorithms performing in domains such as image classification, recognition, detection, tracking, etc. (Sui, [0043]).
Claim 16 is similar to claim 8, hence similarly rejected.
Claim(s) 7 and 15 is/are rejected under 35 U.S.C. 103 as being unpatentable over Cambricon Tech, Jittor, and Brady as applied to claims 1, 9, and 17 above, and further in view of Zhao et al. (US Pub. No. 2020/0133735, published April 2020, hereinafter “Zhao”) .
Regarding claim 7, the combination of Cambricon Tech, Jittor, and Brady teaches the processing method according to claim 1, wherein the method further comprises:
determining a second intermediate representation (IR) of the first-type operator based on a first IR of the second-type operator and the computational logic of the first-type operator (Brady, [0052] – “In one example, the IR 140 may be constructed by the compiler by parsing the neural network model 110 to identify the respective operations and data flow used to implement the neural network. Further, the compiler 105 may identify, from a target descriptor file 120, the memory and compute resources (and other resources (e.g., communication resources)) available on the target hardware device (e.g., and store this information in the IR (e.g., in structural model 1020)). A set of sub-models (e.g., 1005, 1010, 1015) may be generated and encapsulated within the intermediate representation 140 to provide a configurable representation of a mathematical structure (e.g., the computation model of the intermediate representation) of the neural network described in graph 110, for instance, in the form of one or more computation graphs from which a binary may be constructed, among other example implementations. The sub-models may each provide distinct views, but refer to the same underlying structure, the computation model of the intermediate representation.” – teaches determining a second intermediate representation of the first-type operator based on a first intermediate representation of the second-type operator and the computational logic of the first-type operator (a set of sub-models encapsulated within intermediate representation 140 to provide a configurable representation of a computation model, the configurable representation being in the form of one or more computational graphs, thus determining an intermediate representation of the operator based on the intermediate representations of the sub-models));
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention to incorporate the determination of the second intermediate representation based on a first intermediate representation of Brady to further modify the first and second-type operators of Cambricon Tech, Jittor, and Brady. Doing so would allow the overall complexity of the intermediate representation to be simplified to address compilation issues in isolation while sustaining the coherence of the logical space, which allows efficient processing of mutual relations between all types of entities considered (Brady, [0052]).
The combination of Cambricon Tech, Jittor, and Brady fails to explicitly teach determining, based on the second IR, a kernel function corresponding to the first-type operator.
However, analogous to the field of the claimed invention, Zhao teaches:
determining, based on the second IR, a kernel function corresponding to the first-type operator (Zhao, [0031] – “For example, the dynamic optimizer and compiler 120 compiles a first task (such as a subtask of the entire high-performance computing task, which may be a function or kernel codes to be executed on the GPU) according to the type of GPU 145, and compiles the second task according to the type of GPU 155.” and in [0047] – “Referring back to FIG. 3, at 308, operators are compiled based on hardware information to generate subtasks (such as kernel code) that are executed on respective accelerator devices. For example, it is possible to use an intermediate representation (IR) to extract and then compile the IR into the kernel codes optimized by the backend device.” – teaches determining, based on the intermediate representation, a kernel function (subtask which may be a function or kernel codes) corresponding to the first-type operator (operators are compiled based on hardware information to generate kernel codes)).
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to incorporate the kernel functions corresponding to operators based on an intermediate representation of Zhao to the second intermediate representation determination and first and second-type operators of Cambricon Tech, Jittor, and Brady in order to determine a kernel function corresponding to the first-type operator based on the determined intermediate representation. Doing so would lower a high-level specification of a deep learning program down to low-level optimized codes for any hardware back-end (Zhao, [0040]).
Claim 15 is similar to claim 7, hence similarly rejected.
Response to Arguments
Applicant’s arguments, see pp. 2-4, filed 8 January 2026, with respect to the rejection(s) of claim(s) 1, 9, and 17 under 35 U.S.C. 103 have been fully considered and are persuasive. Therefore, the rejection has been withdrawn. However, upon further consideration, a new ground(s) of rejection is made over Cambricon Tech in view of Jittor, and further in view of Brady et al. (US Pub. No. 2019/0392296, published Dec. 2019). Brady teaches the amended limitations of claim 1 regarding “receiving, by the computer device, an operation process of a neural network model input by a user” and “compiling, by the computer device, the operation process of the neural network to generate…”.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Rotem et al. (NPL: Glow: Graph Lowering Compiler Techniques for Neural Networks, published Apr. 2019) teaches a machine learning compiler for heterogenous hardware, which lowers traditional neural network dataflow graphs into intermediate representations. Teaches compiling a neural network operation process to obtain a first computation graph and determining intermediate representations from the operation process.
Cyphers et al. (NPL: Intel nGraph, published Jan 2018) teaches a compiler for converting framework specific computation definitions into intermediate representations that can be compiled and executed on the backend. Teaches transformers that compiles or interprets intermediate representations and provides allocation and execution API.
Chen et al. (NPL: TVM: An Automated End-to-End Optimizing Compiler for Deep Learning) teaches a compiler that exposes graph-level and operator-level optimizations to provide performance portability to deep learning workloads. The compiler takes high-level specifications of deep learning programs from existing frameworks and generates low-level optimized code for hardware backends. Teaches compiling an operation process of a neural network to obtain a computational graph, generated operators, and module parameters.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to LOUIS C NYE whose telephone number is 571-272-0636. The examiner can normally be reached Monday - Friday 9:00AM - 5:00PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, MATT ELL can be reached at 571-270-3264. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/LOUIS CHRISTOPHER NYE/Examiner, Art Unit 2141
/MATTHEW ELL/Supervisory Patent Examiner, Art Unit 2141
Prosecution Insights