DETAILED ACTION
This Office Action is sent in response to the Applicant’s Communication received on 08/06/2025 for application number 17/572,740. The Office hereby acknowledges receipt of the following and placed of record in file: Specification, Drawings, Abstract, Oath/Declaration, IDS, and Claims.
Claims 1, 9, 14, and 15 are amended.
Claims 16-19 are new.
Claims 1-19 are pending.
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
35 USC 101
The Applicant argues that amended claims require to analyze runtime data sources to identify different parts of a computation graph that are dependent and independent of input data, which is a step that cannot be practically performed in the human mind and should not be directed to a mental process. Additionally, Applicant further states that the human mind cannot analyze runtime data sources, split a computation graph, nor generate and apply a wrapper that performs a transparent mapping of a neural network.
The Examiner respectfully disagrees. Analyzing runtime data sources and splitting the computation graph are limitations that are recited in a high level of generality, without further defining the steps of how analyzing and splitting are explicitly performed. Therefore, analyzing runtime data source and splitting the computation graph are actions that can be reasonably performed mentally with the aid of pen and paper, and are therefore mental processes. Furthermore, the claimed limitation “generating and applying a wrapper that performs a transparent mapping of a neural network” is not an abstract idea, but rather an additional element analyzed under 2A prong 2 and 2B, mere instructions to implement an abstract idea on a computer recited at a high level of generality (MPEP 2106.05(f)).
Applicant further argues that the wrapper is fundamentally rooted in computer technology to perform particular computer functions, and that these features cannot be considered mere instructions to implement an abstract idea on a computer.
The Examiner respectfully disagrees. The claimed limitation “generating and applying a wrapper that performs a transparent mapping of a neural network,” although rooted in computer technology, is an additional element analyzed under 2A prong 2 and 2B, mere instructions to implement an abstract idea on a computer recited at a high level of generality (MPEP 2106.05(f)). The alleged argument that the wrapper being “particular computer functions” are recited in a high level of generality and therefore are considered mere instructions to implement an abstract idea on a computer.
The Applicant further argues that claims 1-15 recite features which, when considered as a whole, provide for improvements over existing technology by “enabling the pre-evaluable layers to be executed only once and not within every iteration along with saving computational resources, costs, and accuracy, while providing for additional computations to be performed” in order to reduce execution time, as well as “enabling the transparent reconfiguration of the number, shape, padding, data type and data layout of the parameters within the neural network” in order to improve computational runtime without risking a loss of accuracy. Additionally, Applicant cites paragraphs 0014, 0049, and 0050 to be the support for the alleged improvement steps.
The Examiner respectfully finds the Applicant’s argument unpersuasive. The Applicant’s alleged “enabling the pre-evaluable layers to be executed only once and not within every iteration” and “enabling the transparent reconfiguration of the number, shape, padding, data type and data layout of the parameters within the neural network” appear to be disclosed invention. However, such disclosed invention as improvements to the computers of Al frameworks are not explicitly reflected in the claimed invention. Although, instant application paragraphs 0014, 0049, and 0050 have detailed descriptions of implementing steps that provide technological improvement, these details are not explicitly present in the current claim. Specifically, epochs * len(dataset) iterations of the model, moving all layers that are not dependent on runtime input data into the partial evaluation graph, and hiding changes to the neural network from the user, are not explicitly stated in the claim. Therefore, the claimed elements or combination of elements do not reflect a technical improvement as alleged.
Finally, Applicant argues that embodiments of the present invention provide an unconventional solution to overcome the technical problem of AI frameworks only being able to support generic data layouts and require data conversion to a different layout, before the execution of each layer, by identifying and splitting a computation graph into a pre- evaluation part independent of the input data, and into a computation part dependent on the input data, and then generating and applying a wrapper that performs a transparent mapping of data layouts of the pre-evaluation part, as recited in the claims. This results in the technical improvements of reducing execution time and saving computational resources (without any negative impact on accuracy or peak memory consumption) and enabling transparent reconfiguration of neural network parameters, among other technical improvements discussed above. See paragraphs [0014] and [0049] of the published application.
Examiner respectfully disagrees. The specific steps for reducing execution time and saving computational resources without any negative impact on accuracy or peak memory consumption, and their improvement to computers of Al frameworks, are not explicitly reflected in the claimed invention. Although, instant application paragraphs 0014, 0049, and 0050 have detailed descriptions of implementing steps that provide technological improvement, these details are not explicitly present in the current claim. Specifically, epochs * len(dataset) iterations of the model, moving all layers that are not dependent on runtime input data into the partial evaluation graph, and hiding changes to the neural network from the user, are not explicitly stated in the claim. Therefore, the claimed elements or combination of elements do not reflect a technical improvement as alleged.
For at least the above reasons, it is respectfully submitted that claims 1-15 are not directed to patent-eligible subject matter pursuant to the Subject Matter Eligibility Test.
35 USC 103
Applicant argues that Yang and Zishan, alone or in combination, fail to disclose or suggest identifying parts of a computation graph of a neural network that depend on input data and parts that are independent of the input data, much less splitting the computation graph into these parts as claimed.
Examiner respectfully disagrees, Yang and Zishan do indeed teach the cited limitation. Specifically, under the Broadest Reasonable Interpretation (BRI), Yang teaches identifying parts of the neural network that depend on input data, when Yang teaches in Col 13, lines 47-50: adaptor module…may determine kernel size information tailored or, alternatively, optimized for the NPU based on static and/or dynamic information regarding the NPU. Yang also teaches parts that are independent of the input data in Col 13, lines 41-43: an input neural network model performs an operation involving a kernel having a size of 5*5. Additionally, Yang teaches in Col 13, lines 44-47: A neural network adaptor module may parse information regarding the N*N kernel structure from the input neural network model. The initial kernel size 5*5, prior to optimization, is independent of the parsed information. Furthermore, in regards to splitting the computation graph, Zishan teaches, splitting the computation graph into the pre-evaluation part and the computation part in Para 0144: the preset operators in the first computational graph are traversed, and each preset operator is… converted into a target operator subgraph.
Applicant further argues that merely changing the kernel size of the neural network model does not involve differentiating between parts of a computation graph dependent on input data and parts independent of input data, much less splitting a computation graph based thereon. Therefore, Yang fails to disclose any dependency of any part of the kernel structure on the input data of the neural network.
Examiner respectfully disagrees. “-differentiating between parts of a computation graph dependent on input data and parts independent of input data” is not part of the explicit claimed language. The amended claim states “analyzing runtime data sources to identify parts… of the neural network that depend on input data as a computation part,” which is met by Yang Col 12, lines 42-49: Referring to FIG. 9, a neural network model is received by HW which will execute a neural network operation in operation S31. The HW may perform an operation involving a kernel having a first size according to the neural network model in operation S32. The neural network adaptor module may receive dynamic information during runtime in operation S33 and may determine whether the HW executing the neural network model is changed in operation S34. Additionally, Yang Col 13, lines 47-50: adaptor module…may determine kernel size information tailored or, alternatively, optimized for the NPU based on static and/or dynamic information regarding the NPU. In other words, kernel size (parts of the neural network) is determined in the adaptor module (computation part) when dynamic information (input data based on runtime data sources) is received. Additionally, Zishan was brought in to teach identifying parts of a computation graph as a computation part. In this regard, Zishan teaches, in paragraph 0142: preset operators represented by a directed acyclic graph data structure according to the parsing results (identifying parts of a computation graph) provided by the model parsing module. Additionally, paragraph 0144, Zishan teaches: the preset operators in the first computational graph are traversed, and each preset operator is… converted into a target operator subgraph (computation part).
Applicant further argues that Zishan discloses single target operators and preset operators. Zishan does not disclose or suggest that the single target operators depend on the input data of the model as asserted by the Office. Nor does Zishan disclose or suggest that the preset operators are independent of the input data. Rather, Zishan teaches that each preset operator is converted into a single target operator according to the architecture of the processor. The resulting target operators are also not indicated as having dependence on the input data or not.
Examiner respectfully disagrees with the argument. “Depend on the input data” and “independent of the input data” are not taught by Zishan, but rather taught by Yang. In response to applicant's arguments against the references individually, one cannot show nonobviousness by attacking references individually where the rejections are based on combinations of references. See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981); In re Merck & Co., 800 F.2d 1091, 231 USPQ 375 (Fed. Cir. 1986).
Applicant further argues that the 'parameters' asserted to be taught by Yang are in reference to hyper parameters that impact how a computation is executed and how weights are shaped (e.g., options that influence the behavior of a layer, such as what kind of activation function is applied, or the kernel size, stride or dilation of a pooling or convolutions) in contrast to model parameters (e.g., the 'knowledge of the model, the numerical values that actually used in the computation). Therefore, Applicant respectfully further submits that Yang fails to disclose or suggest the features of independent claims 1, 14, and 15 recited above.
Examiner respectfully disagrees. First, 'parameters' are not part of the amended claims 1, 14, and 15. Second, model parameters being the knowledge of the model and the numerical values that actually used in the computation are not part of the claimed language. Although the claims are interpreted in light of the specification, limitations from the specification are not read into the claims. See In re Van Geuns, 988 F.2d 1181, 26 USPQ2d 1057 (Fed. Cir. 1993).
Applicant further argues that Zishan fails to disclose splitting a computation graph into a pre- evaluation part and a computation part. Rather, Zishan merely discloses to convert the preset operators into the single target operators, such that the preset operators (the asserted pre- evaluation part) and the single target operators (the asserted computation part) are not both split from a same computation graph. Merely converting the preset operators into target operators does not involve any splitting of a computation graph. Therefore, Applicant respectfully submits that Zishan fails to disclose the features of independent claims 1, 14, and 15 recited above.
Examiner respectfully disagrees. Zishan does indeed teach the claimed limitations. Specifically, Zishan teaches in paragraph 0144: the preset operators (pre- evaluation part) in the first computational graph (computation graph) are traversed, and each preset operator is… converted into (split) a target operator subgraph (computation part).
Applicant further argues that the performance of the dedicated processor as the "parameter ... depend[ent] on input data" such that the kernel size is the input data. See the Detailed Action, page 28. However, the Office's later assertion that the kernel size is a parameter independent of input data (see the Detailed Action, page 29) results in the Office asserting on the hand that the kernel size is the input data and on the other hand the kernel size is the parameters that are independent of the input data. Because the kernel size cannot be both the input data and parameters that are independent of the input data, Applicant respectfully submits that this is an improper reading of Yang.
Examiner respectfully disagrees. In regards to the claimed limitation “parts of the neural network that depend on input data,” Yang teaches in Col 12, lines 42-49: The neural network adaptor module may receive dynamic information (input data) during runtime in operation S33 and may determine whether the HW executing the neural network model is changed in operation S34. Furthermore, in Col 13, lines 35-37, Yang teaches: when N has a particular value in an N*N kernel structure, the NPU may operate at improved or, alternatively, optimal performance. In detail, when N is a multiple of 4 or a value a little less than a multiple of 4 in the N*N kernel structure, the NPU may operate at improved or, alternatively, optimal performance. Additionally, in Col 13, lines 47-50, Yang teaches adaptor module…may determine kernel size (parts of the neural network) information tailored or, alternatively, optimized for the NPU based on (depend on) static and/or dynamic information regarding the NPU. In regards to the claimed limitation “parts that are independent of the input data,” Yang teaches in Col 13, lines 41-43: an input neural network model performs an operation involving a kernel having a size of 5*5. Furthermore, in Col 13, lines 44-47, Yang teaches: A neural network adaptor module may parse information regarding the N*N kernel structure from the input neural network model. Therefore, the initial kernel size 5*5, prior to optimization, is independent of the parsed information.
Applicant further argues that Zishan’s preset operators disclose the claimed pre-evaluation part and Zishan's single target operators disclose the claimed computation part. However, Zishan's preset operators are converted into the single target operators, meaning that the parameters asserted in the Office Action as being dependent on the input data (single target operators) are converted from the parameters asserted in the Office Action as being independent of the input data (present operators), which Applicant respectfully submits is an improper reading of Zishan. Therefore, Applicant respectfully submits that because the Office's interpretations of both Yang and Zishan are improper, and because Yang and Zishan fail to disclose the features of claim 1 discussed above, both Yang and Zishan, alone or in combination, fail to disclose or suggest the features of the independent claims.
Examiner respectfully disagrees. The claimed limitation “parts of a computation graph of the neural network that depend on input data as a computation part, and parts of the computation graph that are independent of the input data as a pre- evaluation part” are taught by combination of Yang and Zishan, not Zishan alone. Specifically, Yang teaches parts of the neural network that depend on input data as a computation part and parts that are independent of the input data as a pre-evaluation part, while Zishan teaches parameters of a computation graph as a computation part and pre-evaluation part. Therefore, applicant’s alleged argument related to data being “dependent” or “independent” of input data is taught by Yang, not Zishan. In response to applicant's arguments against the references individually, one cannot show nonobviousness by attacking references individually where the rejections are based on combinations of references. See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981); In re Merck & Co., 800 F.2d 1091, 231 USPQ 375 (Fed. Cir. 1986).
In page 9 of the Remarks section, the Applicant states that dependent claim 6 recites features which further distinguish over the cited prior art. Claim 6 recites:
performing the transparent mapping of data layouts of the pre-evaluation part;
executing the neural network; and applying a gradient update to the transparently mapped data layout of the pre- evaluation part. For example, when training the model, the user expects the gradient update not in the precomputed data layouts, but in the original data layout, and reversing the pre-evaluation graph for the gradients satisfies this expectation. Accordingly, the wrapper allows for the abstraction of the automatic optimization and seamlessly integrates it into the code of the user, so the user does not need to manually modify their code to match. In contrast to the claimed features above, (1) Yang's disclosure of reshaping a neural network simply fails to disclose a gradient update, as Yang is silent as to updating any gradients and (2) Yang fails to disclose which part of the input neural network model is reshaped, much less that the reshaping is done to the pre-evaluation part of the input neural network model.
Examiner respectfully disagrees. First, it is to be noted that the claimed limitation states “applying a gradient update to the transparently mapped data layout of the pre- evaluation part,” which is not the same as the Applicant’s alleged argument “updating any gradients.” Yang discloses in Col 9-10, lines 66-1: information regarding an accuracy loss tolerance (a gradient) during execution of an application may be provided (update) to the reshaper module 330 (the transparently mapped data layout of the pre- evaluation part). Second, in response to applicant's argument that the references fail to show certain features of the invention, it is noted that the features upon which applicant relies (i.e., (I) when training the model, the user expects the gradient update not in the precomputed data layouts, but in the original data layout, and reversing the pre-evaluation graph for the gradients satisfies this expectation. Accordingly, the wrapper allows for the abstraction of the automatic optimization and seamlessly integrates it into the code of the user, so the user does not need to manually modify their code to match. (II) part of the input neural network model is reshaped, that the reshaping is done to the pre-evaluation part of the input neural network model) are not recited in the rejected claim(s). Although the claims are interpreted in light of the specification, limitations from the specification are not read into the claims. See In re Van Geuns, 988 F.2d 1181, 26 USPQ2d 1057 (Fed. Cir. 1993).
In pages 9-10 of the remarks section, Applicant states that dependent claim 7 also recites further features which further distinguish over the cited prior art. Claim 7 recites: performing the transparent mapping of data layouts of the pre-evaluation part; receiving a request to export the neural network from a current data layout to a subsequent data layout; and executing the transparent mapping of data layouts of the pre-evaluation part backwards. The Office asserts that Dennison discloses the features of "executing the transparent mapping of data layouts of the pre-evaluation part backwards." However, Dennison performs batch normalization in a distributed environment. Batch normalization training requires computing the mean and variance across all samples, which when distributed, means that all parallel computers need to synchronize their results to compute the correct values, but when speculative, means the normalization does not exactly compute these values, but rather guesses at a "speculative" mean and variance. Applicant respectfully submits that this normalization does not include converting the gradients from optimal data layout to original data layout, as claimed according to above-recited features of claim 7.
Examiner respectfully disagrees. First, Dennison was brought in only to teach the limitation executing data layouts backwards. In contrast, the limitation “the transparent mapping of data layouts of the pre-evaluation part” is taught by Yang in Col 13, lines 56-67 (see claim 1). In response to applicant's arguments against the references individually, one cannot show nonobviousness by attacking references individually where the rejections are based on combinations of references. See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981); In re Merck & Co., 800 F.2d 1091, 231 USPQ 375 (Fed. Cir. 1986). Furthermore, Applicant’s alleged “converting the gradients from optimal data layout to original data layout” is not recited in claim 7. Although the claims are interpreted in light of the specification, limitations from the specification are not read into the claims. See In re Van Geuns, 988 F.2d 1181, 26 USPQ2d 1057 (Fed. Cir. 1993).
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-19 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Claims 1-13 are directed towards a method. Claim 14 is directed towards a system. Claim 15 is directed towards a tangible, non-transitory computer-readable medium. Therefore, all claims are directed towards one of the four statutory categories of patent eligible subject matter.
Claim 1
Step 2A Prong 1:
Claim 1 recites:
“analyzing runtime data sources to identify parts of a computation graph of the neural network that depend on input data as a computation part, and parts of the computation graph that are independent of the input data as a pre-evaluation part;” Analyzing runtime data sources to identify parts graph of the neural network that depend on input data as a computation part, and parts of the computation graph that are independent of the input data as a pre-evaluation part is an action that can be performed mentally with the aid of pen and paper, and is therefore a mental process.
“splitting the computation graph into the pre-evaluation part and the computation part;” Splitting the computation graph into the pre-evaluation part and the computation part is an action that can be performed mentally with the aid of pen and paper, and is therefore a mental process.
Step 2A Prong Two
This judicial exception is not integrated into a practical application because the additional elements are as follows:
“generating and applying a wrapper that performs a transparent mapping of data layouts of the pre-evaluation part;” Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)).
Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional elements are as follows:
“generating and applying a wrapper that performs a transparent mapping of data layouts of the pre-evaluation part;” Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)) which cannot provide an inventive concept.
Even when considered in combination, these additional elements represent mere instructions to apply an exception and therefore do not provide an inventive concept. The claim is ineligible.
Claim 2
Step 2A Prong 1:
Claim 2 recites:
“computes the transparent mapping between a default artificial intelligence (AI) framework layout and a compute library layout of the neural network;” Computing the transparent mapping between a default artificial intelligence (AI) framework layout and a compute library layout of the neural network is a claim that merely uses textual replacements for particular equations, and is therefore a mathematical concept.
Step 2A Prong Two
This judicial exception is not integrated into a practical application because the additional elements are as follows:
“generates code implementing the transparent mapping between the default Al framework layout and the compute library layout;” Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)).
“generates a new neural network from the neural network by injecting the code into an execution of the neural network;” The limitation amounts to merely indicating a field of use or technological environment in which to apply a judicial exception. This does not amount to significantly more than the exception itself (MPEP 2106.05(h)).
Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional elements are as follows:
“generates code implementing the transparent mapping between the default Al framework layout and the compute library layout;” Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)) which cannot provide an inventive concept.
“generates a new neural network from the neural network by injecting the code into an execution of the neural network;” The limitation amounts to merely indicating a field of use or technological environment in which to apply a judicial exception. This does not amount to significantly more than the exception itself (MPEP 2106.05(h)) and cannot provide an inventive concept.
Even when considered in combination, these additional elements represent mere instructions to apply an exception and therefore do not provide an inventive concept. The claim is ineligible.
Claim 3
Step 2A Prong Two
This judicial exception is not integrated into a practical application because the additional elements are as follows:
“executing the new neural network;” Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)).
Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional elements are as follows:
“executing the new neural network;” Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)) which cannot provide an inventive concept.
Even when considered in combination, these additional elements represent mere instructions to apply an exception and therefore do not provide an inventive concept. The claim is ineligible.
Claim 4
Step 2A Prong Two
This judicial exception is not integrated into a practical application because the additional elements are as follows:
“Exporting and Deploying the neural network;” Insignificant extra-solution as the limitation amounts to necessary data outputting (MPEP 2106.05(g)(3)).
“Storing the neural network;” This limitation is merely a post-solution step of storing the data—a nominal addition to the claim that does not meaningfully limit the claim. The method storing is recited at a high level of generality. Simply implementing the abstract idea in a generic method is not a practical application of the abstract idea. Therefore, storing step is an insignificant extra-solution activity. See MPEP 2106.05(g).
“reversing, by the wrapper, the transparent mapping back to the default Al framework layout;” Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)).
Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional elements are as follows:
“Exporting and Deploying the neural network;” Insignificant extra-solution as the limitation amounts to necessary data outputting (MPEP 2106.05(g)(3)). This falls under Well-Understood, Routine, Conventional activity -see MPEP 2106.05(d)(II)(vi).
“Storing the neural network;” These elements amount to storing… information in memory, Versata Dev. Group, Inc. v. SAP Am., Inc., 793 F.3d 1306, 1334, 115 USPQ2d 1681, 1701 (Fed. Cir. 2015); OIP Techs., 788 F.3d at 1363, 115 USPQ2d at 1092-93; See MPEP 2106.05(d) (II)(iv). The courts have recognized the computer functions of storing as well‐understood, routine, and conventional function when they are claimed in a merely generic manner (e.g., at a high level of generality) or as insignificant extra-solution activity.
“reversing, by the wrapper, the transparent mapping back to the default Al framework layout;” Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)) which cannot provide an inventive concept.
Even when considered in combination, these additional elements represent mere instructions to apply an exception and therefore do not provide an inventive concept. The claim is ineligible.
Claim 5
Step 2A Prong Two
This judicial exception is not integrated into a practical application because the additional elements are as follows:
“the transparent mapping of data layouts of the pre- evaluation part includes a parameter update;” The limitation amounts to merely indicating a field of use or technological environment in which to apply a judicial exception. This does not amount to significantly more than the exception itself (MPEP 2106.05(h)).
Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional elements are as follows:
“the transparent mapping of data layouts of the pre- evaluation part includes a parameter update;” The limitation amounts to merely indicating a field of use or technological environment in which to apply a judicial exception. This does not amount to significantly more than the exception itself (MPEP 2106.05(h)) and cannot provide an inventive concept.
Even when considered in combination, these additional elements represent mere instructions to apply an exception and therefore do not provide an inventive concept. The claim is ineligible.
Claim 6
Step 2A Prong 1:
Claim 1 recites:
“performing the transparent mapping of data layouts of the pre-evaluation part;” Performing the transparent mapping of data layouts of the pre-evaluation part is a claim that merely uses textual replacements for particular equations, and is therefore a mathematical concept.
“applying a gradient update to the transparently mapped data layout of the pre- evaluation part;” Applying a gradient update to the transparently mapped data layout of the pre- evaluation part is a claim that merely uses textual replacements for particular equations, and is therefore a mathematical concept.
Step 2A Prong Two
This judicial exception is not integrated into a practical application because the additional elements are as follows:
“executing the neural network;” Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)).
Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional elements are as follows:
“executing the neural network;” Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)) which cannot provide an inventive concept.
Even when considered in combination, these additional elements represent mere instructions to apply an exception and therefore do not provide an inventive concept. The claim is ineligible.
Claim 7
Step 2A Prong One
Claim 7 recites:
“performing the transparent mapping of data layouts of the pre-evaluation part;” Performing the transparent mapping of data layouts of the pre-evaluation part is an action that can be performed mentally with the aid of pen and paper, and is therefore a mental process.
Step 2A Prong Two
This judicial exception is not integrated into a practical application because the additional elements are as follows:
“executing the transparent mapping of data layouts of the pre-evaluation part backwards;” Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)).
“receiving a request to export the neural network from a current data layout to a subsequent data layout;” Mere data gathering recited at a high level of generality, and thus are insignificant extra-solution activity (MPEP 2106.05(g)).
Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional elements are as follows:
“executing the transparent mapping of data layouts of the pre-evaluation part backwards;” Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)) which cannot provide an inventive concept.
“receiving a request to export the neural network from a current data layout to a subsequent data layout;” Mere data gathering recited at a high level of generality, and thus are insignificant extra-solution activity. See MPEP 2106.05(g). The additional element of “receiving” does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. As discussed above with respect to integration of the abstract idea into a practical application, the additional element of receiving steps amounts to no more than mere data gathering. This element amount to receiving data over a network and are well-understood, routine, conventional activity. See MPEP 2106.05(d), subsection II (i). This cannot provide an inventive concept.
Even when considered in combination, these additional elements represent mere instructions to apply an exception and therefore do not provide an inventive concept. The claim is ineligible.
Claim 8
Step 2A Prong One
Claim 8 recites:
“performing the transparent mapping of data layouts of the pre-evaluation part;” Performing the transparent mapping of data layouts of the pre-evaluation part is an action that can be performed mentally with the aid of pen and paper, and is therefore a mental process.
Step 2A Prong Two
This judicial exception is not integrated into a practical application because the additional elements are as follows:
“storing an output of the pre-evaluation part in the neural network;” This limitation is merely a post-solution step of storing the data—a nominal addition to the claim that does not meaningfully limit the claim. The method storing is recited at a high level of generality. Simply implementing the abstract idea in a generic method is not a practical application of the abstract idea. Therefore, storing step is an insignificant extra-solution activity. See MPEP 2106.05(g).
“wherein the pre-evaluation part comprises a generative layer;” The limitation amounts to merely indicating a field of use or technological environment in which to apply a judicial exception. This does not amount to significantly more than the exception itself (MPEP 2106.05(h)).
Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional elements are as follows:
“storing an output of the pre-evaluation part in the neural network;”These elements amount to storing… information in memory, Versata Dev. Group, Inc. v. SAP Am., Inc., 793 F.3d 1306, 1334, 115 USPQ2d 1681, 1701 (Fed. Cir. 2015); OIP Techs., 788 F.3d at 1363, 115 USPQ2d at 1092-93; See MPEP 2106.05(d) (II)(iv). The courts have recognized the computer functions of storing as well‐understood, routine, and conventional function when they are claimed in a merely generic manner (e.g., at a high level of generality) or as insignificant extra-solution activity.
“wherein the pre-evaluation part comprises a generative layer;” The limitation amounts to merely indicating a field of use or technological environment in which to apply a judicial exception. This does not amount to significantly more than the exception itself (MPEP 2106.05(h)) and cannot provide an inventive concept.
Even when considered in combination, these additional elements represent mere instructions to apply an exception and therefore do not provide an inventive concept. The claim is ineligible.
Claim 9
Step 2A Prong Two
This judicial exception is not integrated into a practical application because the additional elements are as follows:
“wherein handling the transparent mapping of the data layouts by the wrapper comprises: generating a new neural network with a new parameter;” “performing the transparent mapping of data layouts of the pre-evaluation part using the parameter of the neural network as an input and the new parameter of the new neural network as an output;” “and replacing the neural network with the new neural network;” Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)).
Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional elements are as follows:
“wherein handling the transparent mapping of the data layouts by the wrapper comprises: generating a new neural network with a new parameter;” “performing the transparent mapping of data layouts of the pre-evaluation part using the parameter of the neural network as an input and the new parameter of the new neural network as an output;” “and replacing the neural network with the new neural network;” Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)) which cannot provide an inventive step.
Even when considered in combination, these additional elements represent mere instructions to apply an exception and therefore do not provide an inventive concept. The claim is ineligible.
Claim 10
Step 2A Prong One
Claim 10 recites:
“detecting a data layout of the neural network;” Detecting a data layout of the neural network is an action that can be performed mentally with the aid of pen and paper, and is therefore a mental process.
“detecting a data layout of a target device that will deploy the neural network;” Detecting a data layout of a target device that will deploy the neural network is an action that can be performed mentally with the aid of pen and paper, and is therefore a mental process.
Step 2A Prong Two
This judicial exception is not integrated into a practical application because the additional elements are as follows:
“wherein handling the transparent mapping of the data layouts by the wrapper comprises: creating a new neural network with the data layout of the target device;” “and replacing the neural network with the new neural network;” Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)).
Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional elements are as follows:
“wherein handling the transparent mapping of the data layouts by the wrapper comprises: creating a new neural network with the data layout of the target device;” “and replacing the neural network with the new neural network;” Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)) which cannot provide an inventive concept.
Even when considered in combination, these additional elements represent mere instructions to apply an exception and therefore do not provide an inventive concept. The claim is ineligible.
Claim 11
Step 2A Prong Two
This judicial exception is not integrated into a practical application because the additional elements are as follows:
“wherein the wrapper detects the data layout of the neural network and detects the data layout of the target device that will deploy the neural network in response to a user execution of the neural network;” Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)).
Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional elements are as follows:
“wherein the wrapper detects the data layout of the neural network and detects the data layout of the target device that will deploy the neural network in response to a user execution of the neural network;” Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)) which cannot provide an inventive concept.
Even when considered in combination, these additional elements represent mere instructions to apply an exception and therefore do not provide an inventive concept. The claim is ineligible.
Claim 12
Step 2A Prong One
Claim 12 recites:
“detecting a data layout of the neural network;” Detecting a data layout of the neural network is an action that can be performed mentally with the aid of pen and paper, and is therefore a mental process.
“detecting a data layout of a target device that will deploy the neural network;” Detecting a data layout of a target device that will deploy the neural network is an action that can be performed mentally with the aid of pen and paper, and is therefore a mental process.
“performing the transparent mapping of data layouts of the pre-evaluation part;” Performing the transparent mapping of data layouts of the pre-evaluation part is an action that can be performed mentally with the aid of pen and paper, and is therefore a mental process.
Step 2A Prong Two
This judicial exception is not integrated into a practical application because the additional elements are as follows:
“replacing the neural network with a neural network that utilizes a data layout of the target device;” The limitation amounts to merely indicating a field of use or technological environment in which to apply a judicial exception. This does not amount to significantly more than the exception itself (MPEP 2106.05(h)).
Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional elements are as follows:
“replacing the neural network with a neural network that utilizes a data layout of the target device;” The limitation amounts to merely indicating a field of use or technological environment in which to apply a judicial exception. This does not amount to significantly more than the exception itself (MPEP 2106.05(h)) and cannot provide an inventive concept.
Even when considered in combination, these additional elements represent mere instructions to apply an exception and therefore do not provide an inventive concept. The claim is ineligible.
Claim 13
Step 2A Prong Two
This judicial exception is not integrated into a practical application because the additional elements are as follows:
“further comprising removing, by the wrapper, a parameter of the neural network in response to a user input;” Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)).
Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional elements are as follows:
“further comprising removing, by the wrapper, a parameter of the neural network in response to a user input;” Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)) which cannot provide an inventive concept.
Even when considered in combination, these additional elements represent mere instructions to apply an exception and therefore do not provide an inventive concept. The claim is ineligible.
Claim 14
Step 2A Prong 1:
Claim 14 recites:
“analyzing runtime data sources to identify parts of a computation graph of the neural network that depend on input data as a computation part, and parts of the computation graph that are independent of the input data as a pre-evaluation part;” Analyzing runtime data sources to identify parts of a computation graph of the neural network that depend on input data as a computation part, and parts of the computation graph that are independent of the input data as a pre-evaluation part is an action that can be performed mentally with the aid of pen and paper, and is therefore a mental process.
“splitting the computation graph into the pre-evaluation part and the computation part;” Splitting the computation graph into the pre-evaluation part and the computation part is an action that can be performed mentally with the aid of pen and paper, and is therefore a mental process.
Step 2A Prong Two
This judicial exception is not integrated into a practical application because the additional elements are as follows:
“A system for optimizing computation graphs of a neural network comprising one or more hardware processors which, alone or in combination, are configured to provide for execution of the following steps;” Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)).
“generating and applying a wrapper that performs a transparent mapping of data layouts of the pre-evaluation part;” Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)).
Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional elements are as follows:
“A system for optimizing computation graphs of a neural network comprising one or more hardware processors which, alone or in combination, are configured to provide for execution of the following steps;” Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)) which cannot provide an inventive concept.
“generating and applying a wrapper that performs a transparent mapping of data layouts of the pre-evaluation part;” Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)) which cannot provide an inventive concept.
Even when considered in combination, these additional elements represent mere instructions to apply an exception and therefore do not provide an inventive concept. The claim is ineligible.
Claim 15
Step 2A Prong 1:
Claim 15 recites:
“analyzing runtime data sources to identify parts of a computation graph of the neural network that depend on input data as a computation part, and parts of the computation graph that are independent of the input data as a pre-evaluation part;” Analyzing runtime data sources to identify parts of a computation graph of the neural network that depend on input data as a computation part, and parts of the computation graph that are independent of the input data as a pre-evaluation part is an action that can be performed mentally with the aid of pen and paper, and is therefore a mental process.
“splitting the computation graph into the pre-evaluation part and the computation part;” Splitting the computation graph into the pre-evaluation part and the computation part is an action that can be performed mentally with the aid of pen and paper, and is therefore a mental process.
Step 2A Prong Two
This judicial exception is not integrated into a practical application because the additional elements are as follows:
“A tangible, non-transitory computer-readable medium having instructions thereon which, upon being executed by one or more hardware processors, alone or in combination, provide for execution of the following steps;” Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)).
“generating and applying a wrapper that performs a transparent mapping of data layouts of the pre-evaluation part;” Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)).
Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional elements are as follows:
“A tangible, non-transitory computer-readable medium having instructions thereon which, upon being executed by one or more hardware processors, alone or in combination, provide for execution of the following steps;” Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)) which cannot provide an inventive concept.
“generating and applying a wrapper that performs a transparent mapping of data layouts of the pre-evaluation part;” Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)) which cannot provide an inventive concept.
Even when considered in combination, these additional elements represent mere instructions to apply an exception and therefore do not provide an inventive concept. The claim is ineligible.
Claim 16
Step 2A Prong 1:
Claim 16 recites:
“performing the transparent mapping of the data layouts of the pre-evaluation part based on reshaping an output channel of the pre-evaluation part from a first number of channel dimensions into a second number of channel dimensions;” Performing the transparent mapping of the data layouts is an action that can be performed mentally with the aid of pen and paper, and is therefore a mental process.
Step 2A Prong Two and Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception under step 2B. Thus, the judicial exception is not integrated into a practical application (see MPEP 2106.04(d) I.), failing step 2A prong 2. The claim is ineligible.
Claim 17
Step 2A Prong Two
This judicial exception is not integrated into a practical application because the additional elements are as follows:
“wherein the parts of the computation graph of the neural network that depend on the input data are model parameters, and wherein the first number of channel dimensions is associated with a batchsize, channels, height, width (NCHW) format and the second number of channel dimensions is associated with an batchsize, height, width, channels (NHWC) format;” The limitation amounts to merely indicating a field of use or technological environment in which to apply a judicial exception. This does not amount to significantly more than the exception itself (MPEP 2106.05(h)).
Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional elements are as follows:
“wherein the parts of the computation graph of the neural network that depend on the input data are model parameters, and wherein the first number of channel dimensions is associated with a batchsize, channels, height, width (NCHW) format and the second number of channel dimensions is associated with an batchsize, height, width, channels (NHWC) format;” The limitation amounts to merely indicating a field of use or technological environment in which to apply a judicial exception. This does not amount to significantly more than the exception itself (MPEP 2106.05(h)) and cannot provide an inventive concept.
Even when considered in combination, these additional elements represent mere instructions to apply an exception and therefore do not provide an inventive concept. The claim is ineligible.
Claim 18
Step 2A Prong Two
This judicial exception is not integrated into a practical application because the additional elements are as follows:
“wherein executing the neural network further comprises executing the transparently mapped pre-evaluation part and the computation part, and wherein the method further comprises: subsequently executing the neural network, comprising executing the computation part and not executing the transparently mapped pre-evaluation part;” The limitation amounts to merely indicating a field of use or technological environment in which to apply a judicial exception. This does not amount to significantly more than the exception itself (MPEP 2106.05(h)).
Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional elements are as follows:
“wherein executing the neural network further comprises executing the transparently mapped pre-evaluation part and the computation part, and wherein the method further comprises: subsequently executing the neural network, comprising executing the computation part and not executing the transparently mapped pre-evaluation part;” The limitation amounts to merely indicating a field of use or technological environment in which to apply a judicial exception. This does not amount to significantly more than the exception itself (MPEP 2106.05(h)) and cannot provide an inventive concept.
Even when considered in combination, these additional elements represent mere instructions to apply an exception and therefore do not provide an inventive concept. The claim is ineligible.
Claim 19
Step 2A Prong Two
This judicial exception is not integrated into a practical application because the additional elements are as follows:
“wherein a compute library provides a plurality of transformation functions that are each associated with a respective layer of the neural network, and wherein the wrapper further performs preprocessing of the plurality of transformation functions before execution of the neural network;” The limitation amounts to merely indicating a field of use or technological environment in which to apply a judicial exception. This does not amount to significantly more than the exception itself (MPEP 2106.05(h)).
Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional elements are as follows:
“wherein a compute library provides a plurality of transformation functions that are each associated with a respective layer of the neural network, and wherein the wrapper further performs preprocessing of the plurality of transformation functions before execution of the neural network;” The limitation amounts to merely indicating a field of use or technological environment in which to apply a judicial exception. This does not amount to significantly more than the exception itself (MPEP 2106.05(h)) and cannot provide an inventive concept.
Even when considered in combination, these additional elements represent mere instructions to apply an exception and therefore do not provide an inventive concept. The claim is ineligible.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1, 5, 6, 10-16 are rejected under 35 U.S.C. 103 as being unpatentable over Yang (U.S. 11803756 B2), hereinafter Yang, in view of Zishan et al. (CN 112947935 A, see attached translation), hereinafter Zishan.
Regarding claim 1, Yang teaches,
A method for optimizing a neural network [Abstract, A method of operating a neural network system includes… generating, by the processor, a reshaped neural network model by changing information of the input neural network model according to a result of determining the information of the at least one dedicated hardware device such that the reshaped neural network model is tailored for execution by the dedicated hardware device], the method comprising: analyzing runtime data sources to identify parts of the neural network that depend on input data as a computation part [Col 12, lines 42-49, The neural network adaptor module may receive dynamic information during runtime in operation S33 and may determine whether the HW executing the neural network model is changed in operation S34; Col 13, lines 35-37, when N has a particular value in an N*N kernel structure, the NPU may operate at improved or, alternatively, optimal performance. In detail, when N is a multiple of 4 or a value a little less than a multiple of 4 in the N*N kernel structure, the NPU may operate at improved or, alternatively, optimal performance; Col 13, lines 47-50, adaptor module…may determine kernel size information tailored or, alternatively, optimized for the NPU based on static and/or dynamic information regarding the NPU], and
parts (Col 13, lines 41-43, a kernel having a size of 5*5) that are independent of the input data as a pre-evaluation part [Col 13, lines 41-43, an input neural network model performs an operation involving a kernel having a size of 5*5; Col 13, lines 44-47, A neural network adaptor module may parse information regarding the N*N kernel structure from the input neural network model (note: the initial kernel size 5*5, prior to optimization, is independent of the parsed information)];
generating [Col 5, lines 53-58, a processor (e.g., processor 110, application processor (AP) 400, processor 410, and/or CPU 540) may implement the neural network adaptor module by executing the computer-executable instructions corresponding to any or all operations described in the present disclosure as being performed by a neural network adaptor module] and applying a wrapper [Col 13, line 44, neural network adaptor module] that performs a transparent mapping (Col 13, lines 56-67, the neural network adaptor module may approximate the 5*5 kernel to two 3*3 kernels) of data layouts of the pre-evaluation part [Col 13, lines 56-67, a calculation result of using the two 3*3 kernels may be identical, or, alternatively, similar, to a calculation result of using the 5*5 kernel. For example, according to at least some example embodiments of the inventive concepts, the two 3*3 kernels may be chosen in such a manner that a result (e.g., a classification result, a recognition result, etc.) of a neural network model performing a desired function (e.g., a classification function, a recognition function, etc.) while using the two 3*3 kernels is identical, or, alternatively, similar, to a result of the neural network model performing the same function while using the 5*5 kernel.].
Yang does not teach identifying parts of a computation graph as a computation part and pre-evaluation part; splitting the computation graph into the pre-evaluation part and the computation part.
Zishan teaches,
identifying (Para 0142, according to the parsing results) parts of a computation graph as a computation part (Para 0144, a single target operator) and pre-evaluation part (Para 0142, preset operators) [Para 0142, preset operators represented by a directed acyclic graph data structure according to the parsing results provided by the model parsing module (such as operator parameters, connection methods, and weight data, etc.)] and splitting (Para 0144, the preset operators in the computational graph can be… split) the computation graph into the pre-evaluation part (Para 0144, the preset operators) and the computation part (Para 0144, a single target operator) [Para 0144, the preset operators in the first computational graph are traversed, and each preset operator is… converted into a target operator subgraph].
Zishan is analogous to the claimed invention as they both relate to optimizing neural networks. Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Yang’s teachings to incorporate the teachings of Zishan and provide distinctions between the pre-evaluation part and computation part [Zishan, para 0144] so as to maximize the computing performance on the target NPU.
Regarding claim 5, Yang-Zishan teach all the limitations of claim 1 including the transparent mapping of data layouts of the of the pre- evaluation part (as in claim 1).
Yang further teaches,
a parameter update [Col 13, lines 50-53, the neural network adaptor module may approximate the 5*5 kernel to two 3*3 kernels in the input neural network model].
Regarding claim 6, Yan-Zishan teach all the limitations of claim 1 including performing the transparent mapping of data layouts of the pre-evaluation part (as in claim 1).
Yang further teaches,
Executing the neural network [Col 13, lines 54-55, The reshaped neural network model may be executed by the NPU].
applying a gradient update to the transparently mapped data layout of the pre- evaluation part [Col 9-10, lines 66-1, information regarding an accuracy loss tolerance during execution of an application may be provided to the reshaper module 330].
Regarding claim 10, Yang-Zishan teaches all the limitations of claim 1.
Yang teaches,
detecting a data layout (Abstract, one item of information) of the neural network [Abstract, parsing, by a processor, at least one item of information related to a neural network operation from an input neural network model];
detecting a data layout (Abstract, information of at least one dedicated hardware device) of a target device that will deploy the neural network [Abstract, determining, by the processor, information of at least one dedicated hardware device];
creating a new neural network with the data layout (Abstract, the information of the at least one dedicated hardware device) of the target device; and replacing the neural network with the new neural network [Abstract, generating, by the processor, a reshaped neural network model by changing information of the input neural network model according to a result of determining the information of the at least one dedicated hardware device such that the reshaped neural network model is tailored for execution by the dedicated hardware device].
Regarding claim 11, Yang-Zishan teach all the limitations of claims 1 and 10 including the wrapper (as in claim 1).
Yang further teaches,
detects the data layout (Abstract, one item of information) of the neural network and detects the data layout (Abstract, information of at least one dedicated hardware device) of the target device that will deploy the neural network [Abstract, parsing, by a processor, at least one item of information related to a neural network operation from an input neural network model…determining, by the processor, information of at least one dedicated hardware device]
in response to a user execution of the neural network [Col 9, lines 41-43 Methods of setting hardware executing a neural network according to the selection information input through the user API 320].
Regarding claim 12, Yang-Zishan teach all the limitations of claim 1 including performing the transparent mapping of data layouts of the pre-evaluation part (as in claim 1).
Yang further teaches,
detecting a data layout of the neural network [Abstract, parsing, by a processor, at least one item of information related to a neural network operation from an input neural network model];
detecting a data layout of a target device that will deploy the neural network [Abstract, determining, by the processor, information of at least one dedicated hardware device];
replacing the neural network with a neural network that utilizes a data layout of the target device [Abstract, generating, by the processor, a reshaped neural network model by changing information of the input neural network model according to a result of determining the information of the at least one dedicated hardware device such that the reshaped neural network model is tailored for execution by the dedicated hardware device].
Regarding claim 13, Yang-Zishan teach all the limitations of claim 1 including the wrapper.
Yang further teaches,
removing a parameter (Col 9, lines 50-58, not using user's selection) of the neural network (Col 9, lines 50-58, arbitrarily performed in the electronic system regardless of the user's selection) in response to a user input [Col 9, lines 50-58, in the implicit hardware fitting, various kinds of preference information regarding a neural network operation may be provided by a user, but setting of a hardware device to execute a neural network model may be arbitrarily performed in the electronic system regardless of the user's selection. In the implicit hardware fitting, hardware to execute a neural network model and the processing speed and power consumption of the hardware may be set by an OS].
Regarding claim 14, Yang further teaches,
A system for optimizing computation graphs of a neural network comprising one or more hardware processors [Abstract, A method of operating a neural network system includes parsing, by a processor, at least one item of information related to a neural network operation from an input neural network model] which, alone or in combination, are configured to provide for execution of the following steps:
analyzing runtime data sources to identify parts of the neural network that depend on input data as a computation part [Col 12, lines 42-49, Referring to FIG. 9, a neural network model is received by HW which will execute a neural network operation in operation S31. The HW may perform an operation involving a kernel having a first size according to the neural network model in operation S32. The neural network adaptor module may receive dynamic information during runtime in operation S33 and may determine whether the HW executing the neural network model is changed in operation S34; Col 13, lines 35-37, when N has a particular value in an N*N kernel structure, the NPU may operate at improved or, alternatively, optimal performance. In detail, when N is a multiple of 4 or a value a little less than a multiple of 4 in the N*N kernel structure, the NPU may operate at improved or, alternatively, optimal performance; Col 13, lines 47-50, adaptor module…may determine kernel size information tailored or, alternatively, optimized for the NPU based on static and/or dynamic information regarding the NPU], and
parts (Col 13, lines 41-43, a kernel having a size of 5*5) that are independent of the input data as a pre-evaluation part [Col 13, lines 41-43, an input neural network model performs an operation involving a kernel having a size of 5*5; Col 13, lines 44-47, A neural network adaptor module may parse information regarding the N*N kernel structure from the input neural network model];
generating [Col 5, lines 53-58, a processor (e.g., processor 110, application processor (AP) 400, processor 410, and/or CPU 540) may implement the neural network adaptor module by executing the computer-executable instructions corresponding to any or all operations described in the present disclosure as being performed by a neural network adaptor module] and applying a wrapper [Col 13, line 44, neural network adaptor module] that performs a transparent mapping (Col 13, lines 56-67, the neural network adaptor module may approximate the 5*5 kernel to two 3*3 kernels) of data layouts of the pre-evaluation part [Col 13, lines 56-67, a calculation result of using the two 3*3 kernels may be identical, or, alternatively, similar, to a calculation result of using the 5*5 kernel. For example, according to at least some example embodiments of the inventive concepts, the two 3*3 kernels may be chosen in such a manner that a result (e.g., a classification result, a recognition result, etc.) of a neural network model performing a desired function (e.g., a classification function, a recognition function, etc.) while using the two 3*3 kernels is identical, or, alternatively, similar, to a result of the neural network model performing the same function while using the 5*5 kernel.].
Yang does not teach identifying parts of a computation graph as a computation part and pre-evaluation part; splitting the computation graph into the pre-evaluation part and the computation part.
Zishan teaches,
identifying (Para 0142, according to the parsing results) parts of a computation graph as a computation part (Para 0144, a single target operator) and pre-evaluation part (Para 0142, preset operators) [Para 0142, preset operators represented by a directed acyclic graph data structure according to the parsing results provided by the model parsing module (such as operator parameters, connection methods, and weight data, etc.)] and splitting (Para 0144, the preset operators in the computational graph can be… split) the computation graph into the pre-evaluation part (Para 0144, the preset operators) and the computation part (Para 0144, a single target operator) [Para 0144, the preset operators in the first computational graph are traversed, and each preset operator is… converted into a target operator subgraph].
Zishan is analogous to the claimed invention as they both relate to optimizing neural networks. Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Yang’s teachings to incorporate the teachings of Zishan and provide distinctions between the pre-evaluation part and computation part [Zishan, para 0144] so as to maximize the computing performance on the target NPU.
Regarding claim 15, Yang further teaches,
A tangible, non-transitory computer-readable medium having instructions thereon which, upon being executed by one or more hardware processors, alone or in combination, provide for execution of the following steps [Col 1, lines 52-57, According to at least some example embodiments of the inventive concepts, an application processor includes memory storing computer-executable instructions; and a processor configured to execute the computer-executable instructions such that the processor is configured to perform operations including, determining information of at least one dedicated hardware device]:
analyzing runtime data sources to identify parts of the neural network that depend on input data as a computation part [Col 12, lines 42-49, Referring to FIG. 9, a neural network model is received by HW which will execute a neural network operation in operation S31. The HW may perform an operation involving a kernel having a first size according to the neural network model in operation S32. The neural network adaptor module may receive dynamic information during runtime in operation S33 and may determine whether the HW executing the neural network model is changed in operation S34; Col 13, lines 35-37, when N has a particular value in an N*N kernel structure, the NPU may operate at improved or, alternatively, optimal performance. In detail, when N is a multiple of 4 or a value a little less than a multiple of 4 in the N*N kernel structure, the NPU may operate at improved or, alternatively, optimal performance; Col 13, lines 47-50, adaptor module…may determine kernel size information tailored or, alternatively, optimized for the NPU based on static and/or dynamic information regarding the NPU], and
parts (Col 13, lines 41-43, a kernel having a size of 5*5) that are independent of the input data as a pre-evaluation part [Col 13, lines 41-43, an input neural network model performs an operation involving a kernel having a size of 5*5; Col 13, lines 44-47, A neural network adaptor module may parse information regarding the N*N kernel structure from the input neural network model];
generating [Col 5, lines 53-58, a processor (e.g., processor 110, application processor (AP) 400, processor 410, and/or CPU 540) may implement the neural network adaptor module by executing the computer-executable instructions corresponding to any or all operations described in the present disclosure as being performed by a neural network adaptor module] and applying a wrapper [Col 13, line 44, neural network adaptor module] that performs a transparent mapping (Col 13, lines 56-67, the neural network adaptor module may approximate the 5*5 kernel to two 3*3 kernels) of data layouts of the pre-evaluation part [Col 13, lines 56-67, a calculation result of using the two 3*3 kernels may be identical, or, alternatively, similar, to a calculation result of using the 5*5 kernel. For example, according to at least some example embodiments of the inventive concepts, the two 3*3 kernels may be chosen in such a manner that a result (e.g., a classification result, a recognition result, etc.) of a neural network model performing a desired function (e.g., a classification function, a recognition function, etc.) while using the two 3*3 kernels is identical, or, alternatively, similar, to a result of the neural network model performing the same function while using the 5*5 kernel.].
Yang does not teach identifying parts of a computation graph as a computation part and pre-evaluation part; splitting the computation graph into the pre-evaluation part and the computation part.
Zishan teaches,
identifying (Para 0142, according to the parsing results) parts of a computation graph as a computation part (Para 0144, a single target operator) and pre-evaluation part (Para 0142, preset operators) [Para 0142, preset operators represented by a directed acyclic graph data structure according to the parsing results provided by the model parsing module (such as operator parameters, connection methods, and weight data, etc.)] and splitting (Para 0144, the preset operators in the computational graph can be… split) the computation graph into the pre-evaluation part (Para 0144, the preset operators) and the computation part (Para 0144, a single target operator) [Para 0144, the preset operators in the first computational graph are traversed, and each preset operator is… converted into a target operator subgraph].
Zishan is analogous to the claimed invention as they both relate to optimizing neural networks. Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Yang’s teachings to incorporate the teachings of Zishan and provide distinctions between the pre-evaluation part and computation part [Zishan, para 0144] so as to maximize the computing performance on the target NPU.
Regarding claim 16, Yang-Zishan teach all the limitations of claim 1.
Yang further teaches,
performing the transparent mapping of the data layouts of the pre-evaluation part based on reshaping an output channel of the pre-evaluation part from a first number of channel dimensions into a second number of channel dimensions [Col 7, paras 2-6, a first feature map FM1 may be an input feature map and a second feature map FM2 may be an output feature map… The first and second feature maps FM1 and FM2 may have a form of a two-dimensional matrix or a form of a three-dimensional matrix… The first and second feature maps FM1 and FM2 have a width (or a column) W, a height (or a row) H, and a depth D, which may respectively correspond to the x-axis, the y-axis, and the z-axis in a coordinate system. The depth D may be referred to as the number of channels… a convolution of the first feature map FM1 and a weight map WM may be performed and the second feature map FM2 may be generated as the convolution result… while convolution is performed based on the weight map WM having a size of N*N in an input neural network model, convolution may be performed based on the weight map WM having a changed size in a reshaped neural network model. The weight map WM having the size of N*N may be changed into at least one or two weight maps. For example, the size of the weight map WM may be changed into M*M, N*1, 1*N, M*1, 1*M, etc.].
Claim(s) 2-4 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Yang in view of Zishan, and in further view of Cyphers et al. (Intel nGraph: An Intermediate Representation, Compiler, and Executor for Deep Learning, published 2018), hereinafter Cyphers, and Hammond et al. (U.S. 11842172 B2), hereinafter Hammond.
Regarding claim 2, Yang-Zishan teach all the limitations of claim 1 including the neural network and the wrapper (as in claim 1).
Yang-Zishan do not teach computes the transparent mapping between a default artificial intelligence (AI) framework layout and a compute library layout; generates code implementing the transparent mapping between the default Al framework layout and the compute library layout; and generates a new neural network by injecting the code into an execution of the neural network.
Cyphers teaches,
Computes (Sect 1, computation) the transparent mapping (Sect 1, Backends’ implementations are encapsulated so that the same computation can execute on multiple backends, such as CPUs and GPUs) between a default artificial intelligence (AI) framework layout (Sect 4, para 1, IR generated by the nGraph library) and a compute library layout (Sect 1, an allocation and execution API… to implement the framework’s API) [Sect 1, Deep learning frameworks are libraries that provide domain-specific languages for defining deep learning computations and APIs for managing data and executing computations. Backends’ implementations are encapsulated so that the same computation can execute on multiple backends, such as CPUs and GPUs. We adopt the organization of compilers such as LLVM[7] by converting framework specific computation definitions into a framework-independent intermediate representation (IR) that we compile into a form that can execute on the backend. An nGraph framework bridge acts as a framework backend. Each nGraph backend has a transformer that compiles or interprets the IR and provides an allocation and execution API that the framework bridges use to implement the framework’s API; Sect 4, para 1, The IR generated by the nGraph library is passed to a transformer for the generation of code optimized specifically for the selected backend. These newly-optimized backends provide facilities for… the combining of tensor-element layout;];
generates code (Sect 1, provides an allocation and execution API that the framework bridges use to implement the framework’s API; Sect 4, para 1, generation of code) implementing the transparent mapping (Sect 1, Backends’ implementations are encapsulated so that the same computation can execute on multiple backends, such as CPUs and GPUs) between the default Al framework layout (Sect 4, para 1, IR generated by the nGraph library) and the compute library layout (Sect 1, an allocation and execution API… to implement the framework’s API) [Sect 1, Deep learning frameworks are libraries that provide domain-specific languages for defining deep learning computations and APIs for managing data and executing computations. Backends’ implementations are encapsulated so that the same computation can execute on multiple backends, such as CPUs and GPUs. We adopt the organization of compilers such as LLVM[7] by converting framework specific computation definitions into a framework-independent intermediate representation (IR) that we compile into a form that can execute on the backend. An nGraph framework bridge acts as a framework backend. Each nGraph backend has a transformer that compiles or interprets the IR and provides an allocation and execution API that the framework bridges use to implement the framework’s API; Sect 4, para 1, The IR generated by the nGraph library is passed to a transformer for the generation of code optimized specifically for the selected backend. These newly-optimized backends provide facilities for… the combining of tensor-element layout];
Cyphers is analogous to the claimed invention as they both relate to optimizing neural networks. Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Yang and Zishan’s teachings to incorporate the teachings of Cyphers and provide generating code linking an AI framework and a compute library [Cyphers, Sect 4, para 1] in order to optimize the backend and provide facilities for pattern matching, liveness analysis, memory management, and the combining of tensor-element layout and shape management with backend kernel selection.
Yang-Zishan-Cyphers does not teach generating a new neural network by injecting the code into an execution of the neural network.
Hammond teaches,
and generates a new neural network (Col 16, lines 48-52, intelligence models) by injecting the code (Col 4, lines 5-20, assembly code) into an execution of the neural network (Col 4, lines 5-20, learner module) [Col 16, lines 48-52, Each program developed in a pedagogical programming language can be fed into the AI engine in order to generate and train appropriate intelligence models, which can be referred to as Basic Recurrent Artificial Intelligence Networks (“BRAINs”); Col 4, lines 5-20, The AI engine can include one or more AI-engine modules including an architect module, an instructor module, and a learner module. A source code written in a pedagogical programming language can be received through an API exposed to the GUI and an assembly code can be subsequently generated from the source code… The architect module can be configured to propose a neural-network layout with one or more neural-network layers from the assembly code. The learner module can be configured to build the AI model with the one or more neural-network layers from the neural-network layout proposed by the architect module.].
Hammond is analogous to the claimed invention as they both relate to improving construction of neural networks. Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Yang, Zishan, and Cyphers’s teachings to incorporate the teachings of Hammond and provide generates a new neural network by injecting the code into an execution of the neural network [Hammond, col 3, lines 63-64] in order to make interaction with AI models more accessible to AI developers and data scientists at the lowest level.
Regarding claim 3, Yang-Zishan-Cyphers-Hammond teach all the limitations of claims 1 and 2.
Hammond further teaches,
further comprising executing the new neural network [Col 16, lines 37-40, the AI engine… executes a trained AI model containing one, two, or more neural networks.].
Hammond is analogous to the claimed invention as they both relate to improving construction of neural networks. Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Yang, Zishan, and Cyphers’s teachings to incorporate the teachings of Hammond and provide execution of the neural network [Hammond, col 3, lines 63-64] in order to make interaction with AI models more accessible to AI developers and data scientists at the lowest level.
Regarding claim 4, Yang-Zishan-Cyphers-Hammond teach all the limitations of claims 1 and 2 including the wrapper.
Hammond further teaches,
further comprising exporting, storing or deploying the neural network, [Col 4, lines 36-38, and deploy the trained neural network 106 as a deployed neural network 108 in any of a number of desired ways.]
and reversing the transparent mapping back to the default Al framework layout [Col 27, lines 11-15, the BRAIN-server is configured to support versioning of BRAINs so that the user can preserve (and possibly revert to) the current state of a BRAIN while refining the trained state of the BRAIN until a new, more satisfactory state is reached].
Hammond is analogous to the claimed invention as they both relate to improving construction of neural networks. Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Yang, Zishan, and Cyphers’s teachings to incorporate the teachings of Hammond and provide exporting, storing or deploying the neural network and reversing the transparent mapping back to the default Al framework layout [Hammond, col 27, lines 11-15] in order to refine and reuse the trained state of a network and reach a new, more satisfactory state.
Regarding claim 18, Yang-Zishan-Cyphers-Hammond teach the limitations of claim 3.
Yang further teaches,
wherein executing the neural network further comprises executing the transparently mapped pre-evaluation part and executing the computation part [Col 8, lines 50-53, The reshaper module 220 may change at least one item of information of an input neural network model so that hardware executes the neural network model with better acceleration; Col 8 lines, 59-66, Referring to FIG. 5, the neural network adaptor module 300 may include… a reshaper module 330… a computing resource module 350, and a computing abstract layer (AL) 360. A hardware block 301 including various kinds of hardware devices executing various kinds of neural network models is also illustrated in FIG. 5; Col 13, lines 56-67, a calculation result of using the two 3*3 kernels may be identical, or, alternatively, similar, to a calculation result of using the 5*5 kernel. For example, according to at least some example embodiments of the inventive concepts, the two 3*3 kernels may be chosen in such a manner that a result (e.g., a classification result, a recognition result, etc.) of a neural network model performing a desired function (e.g., a classification function, a recognition function, etc.) while using the two 3*3 kernels is identical, or, alternatively, similar, to a result of the neural network model performing the same function while using the 5*5 kernel]. and
wherein the method further comprises: subsequently executing the neural network, comprising executing the computation part [Col 6, lines 28-40, The neural network adaptor module 140 may include a reshaper module (or a reshaper) 141. The reshaper module 141 may perform reshaping using the input DL model and various kinds of information and generate a reshaped neural network model based on the reshaping result. For example, the neural network adaptor module 140 may receive at least one kind of information among static information and dynamic information and the reshaper module 141 may reshape the input DL model using the at least one kind of information from among the static information and the dynamic information. In other words, the input DL model may be reshaped taking account of hardware capacity, a desired or, alternatively, optimal operation method, etc.] and not executing the transparently mapped pre-evaluation part [Col 7, lines 24-39, The weight map WM may filter the first feature map FM1 and may be referred to as a filter or a kernel… The weight map WM shifts by traversing the first feature map FM1 as a sliding window… During a shift, each weight included in the weight map WM may be multiplied by and added to all feature values in an area where the weight map WM overlaps the first feature map FM1 (Examiner interprets the FM1 features outside of the WM sliding window as “not executing the transparently mapped pre-evaluation part”)].
Claim 7 is rejected under 35 U.S.C. 103 as being unpatentable over Yang in view of Zishan and in further view of Hammond and Dennison et al. (U.S. 11170263 B2), hereinafter Dennison.
Regarding claim 7, Yang-Zishan teach all the limitations of claim 1 including performing the transparent mapping of data layouts of the pre-evaluation part (as in claim 1).
Hammond further teaches,
receiving a request (Col 27, lines 6-15, user decides) to export the neural network (Col 27, lines 6-15, versioning of BRAINs) from a current data layout to a subsequent data layout [Col 27, lines 6-15, If the user decides further refinement of a BRAIN is needed, be it through additional training with existing data, additional training with new, supplemental data, or additional training with a modified version of the mental model or curricula used for training, the BRAIN-server is configured to support versioning of BRAINs so that the user can preserve (and possibly revert to) the current state of a BRAIN while refining the trained state of the BRAIN until a new, more satisfactory state is reached.];
Hammond is analogous to the claimed invention as they both relate to improving construction of neural networks. Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Yang and Zishan’s teachings to incorporate the teachings of Hammond and provide receiving a request to export the neural network from a current data layout to a subsequent data layout [Hammond, col 27, lines 11-15] in order to refine the trained state of a network and reach a new, more satisfactory state.
Yang-Zishan-Hammond do not teach executing data layouts backwards.
Dennison teaches,
executing data layouts backwards [Col 03, lines 60-64, reverting the speculative normalization operation to a pre-normalization state].
Dennison is analogous to the claimed invention as they both relate to optimizing neural networks. Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Yang, Zishan, and Hammond’s teachings to incorporate the teachings of Dennison and provide executing the transparent mapping of data layouts of the pre-evaluation part backwards [Hammond, col 27, lines 11-15] in order to refine and reuse the trained state of a network and reach a new, more satisfactory state.
Claim 8 is rejected under 35 U.S.C. 103 as being unpatentable over Yang in view of Zishan, and in further view of Hammond.
Regarding claim 8, Yang-Zishan teach all the limitations of claim 1 including performing the transparent mapping of data layouts of the pre-evaluation part (as in claim 1).
Yang further teaches,
storing an output of the pre-evaluation part in the neural network [Col 13, lines 52-54, generating a reshaped neural network model having two 3*3 kernels resulting from the changing of the 5*5 kernel; Col 6, lines 59-60, store the reshaped neural network model in the electronic system 100].
Yang-Zishan do not teach comprises a generative layer.
Hammond further teaches,
a generative layer [Col 28, lines 44-47, A method of an AI engine include… generating an assembly code, proposing a neural-network layout, building an AI model, and training the AI model.].
Hammond is analogous to the claimed invention as they both relate to improving construction of neural networks. Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Yang and Zishan’s teachings to incorporate the teachings of Hammond and provide a generative layer [Hammond, col 3, lines 55-67] in order to make interaction with AI models more accessible to data scientists without the need for complex toolkits and advanced knowledge of programming.
Claim 9 is rejected under 35 U.S.C. 103 as being unpatentable over Yang in view of Zishan, and in further view of Weber (Sol: Transparent Neural Network Acceleration Platform, published 2018), hereinafter Weber.
Regarding claim 9, Yang-Zishan teach all the limitations of claim 1 including performing the transparent mapping of data layouts of the pre-evaluation part.
Yang-Zishan do not teach, wherein handling the transparent mapping of the data layouts by the wrapper comprises: generating a new neural network with a new parameter; using a parameter of the neural network as an input and the new parameter of the new neural network as an output; and replacing the neural network with the new neural network.
Weber teaches,
generating a new neural network (Sect 2, pg 2, col 1, para 5, optimized network) with a new parameter (Sect 2, pg 2, col 1, para 4, hardware characteristics) [Sect 2, pg 2, col 1, para 5, After all groups have been optimized and specialized implementations have been compiled, we generate a new network description for the framework, that is returned to the user as executable neural network object. This optimized network behaves identical to the original, but uses our optimized implementations; Sect 2, pg 2, col 1, para 4, we use hardware characteristics (number of cores, SIMD units per core and cache sizes) to generate specific mappings of loops onto compute resources];
using a parameter (Sect 1, pg 1, col 2, para 3, the data input size) of the neural network (Sect 1, pg 1, col 2, para 3, myNN) as an input and the new parameter (Sect 2, pg 2, col 1, para 4, hardware characteristics) of the new neural network (Sect 2, pg 2, col 1, para 5, optimized network) as an output (Sect 2, pg 2, col 1, para 5, specific mappings of loops) [Sect 1, pg 1, col 2, para 3, The user only needs to execute optimizedNN = sol.optimize(myNN, [0, 3, 224, 224]), where the first parameter is the neural network and the second the data input size; Sect 2, pg 2, col 1, para 5, After all groups have been optimized and specialized implementations have been compiled, we generate a new network description for the framework, that is returned to the user as executable neural network object. This optimized network behaves identical to the original, but uses our optimized implementations; Sect 2, pg 2, col 1, para 4, hardware characteristics we use hardware characteristics (number of cores, SIMD units per core and cache sizes) to generate specific mappings of loops onto compute resources.];
and replacing the neural network with the new neural network [Sect 2, pg 2, col 1, para 5, we generate a new network description for the framework, that is returned to the user as executable neural network object. This optimized network behaves identical to the original, but uses our optimized implementations.].
Weber is analogous to the claimed invention as they both relate to optimizing neural networks. Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Yang and Zishan’s teachings to incorporate the teachings of Weber and provide generating a new, optimized neural network [Weber, Sect 1, pg 1, col 2, para 2] in order to interface seamlessly into frameworks and transparently accelerates neural networks on various types of hardware.
Claim 17 is rejected under 35 U.S.C. 103 as being unpatentable over Yang in view of Zishan, and in further view of Jakubiuk (US 20210294625 A1), hereinafter Jakubiuk.
Regarding claim 17, Yang-Zishan teach the limitations of claim 16 including the parts of the neural network that depend on the input data (claim 1: Yang, Col 13, lines 35-37), parts of a computation graph (claim 1, Zishan, Paras 0142 and 0144), the first number of channel dimensions and the first number of channel dimensions (claim 16: Yang, Col 7, paras 2-6).
Yang further teaches.
wherein input data are model parameters [Col 12, lines 42-49, The neural network adaptor module may receive dynamic information during runtime in operation S33 and may determine whether the HW executing the neural network model is changed in operation S34].
Yang-Zishan do not teach wherein first number of channel dimensions is associated with a batchsize, channels, height, width (NCHW) format and second number of channel dimensions is associated with an batchsize, height, width, channels (NHWC) format.
Jakubiuk teaches,
wherein first number of channel dimensions is associated with a batchsize, channels, height, width (NCHW) format and second number of channel dimensions is associated with an batchsize, height, width, channels (NHWC) format [Para 0045, FIG. 3 illustrates an implementation of a portion of the AI network 200 in which one or more transpose modules 302 are used to convert input/output data formats for some kernels, so those kernels can operate more efficiently. For example, if kernel-2 can be written to operate more efficiently if it can operate in NHWC format, then transpose modules 302 can convert the input and output of kernel-2, so kernel-2 can receive input in NHWC format and output in NHWC format while the rest of the network 200 can feed into and from the kernel-2 in NCHW format].
Jakubiuk is analogous to the claimed invention as they both relate to optimizing artificial intelligence networks. Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Yang and Zishan’s teachings to incorporate the teachings of Jakubiuk and provide a first number of channel dimensions associated with NCHW format and a second number of channel dimensions associated NHWC format [Jakubiuk, Para 0045] in order to have kernels operate more efficiently.
Claim 19 is rejected under 35 U.S.C. 103 as being unpatentable over Yang in view of Zishan, and in further view of S Nayar et al. (US 20210365775 A1), hereinafter Nayar, and Kaku et al. (US 20220269891 A1), hereinafter Kaku.
Regarding claim 19, Yang-Zishan teach the limitations of claim 1 including the wrapper (Yang, Col 13, line 44).
Zishan further teaches,
wherein a compute library provides a plurality of transformation functions of neural network [Para 0022, the target model file generation module is further used to: transform some preset operators in the first computation graph into target operators in the target operator library as a transformation result],
Zishan is analogous to the claimed invention as they both relate to optimizing neural networks. Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Yang’s teachings to incorporate the teachings of Zishan and provide a compute library providing a plurality of transformation functions in order to modify data suitably for the machine learning model.
Yang-Zishan do not teach transformation functions that are each associated with a respective layer of neural network.
Nayar teaches,
transformation functions that are each associated with a respective layer of neural network [Para 0094, The processing may include transforming the formatted dataset at each layer of the plurality of layers of the deep neural networking component based on at least one of a transformation function].
Nayar is analogous to the claimed invention as they both relate to multi-layer transformation functions of a neural network. Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Yang and Zishan’s teachings to incorporate the teachings of Nayar and provide transformation functions at each layer of a neural network in order to modify data suitably for the machine learning model.
Yang-Zishan-Nayar teach the plurality of transformation functions (Nayar, Para 0094).
Yang-Zishan-Nayar do not teach preprocessing transformation functions before execution of neural network.
Kaku teaches,
preprocessing transformation functions before execution of neural network [Para 0065, the preliminary processing module 206 can further include instructions that function to control the processor 202 to preprocess, prior to an execution of the neural network 228, the image 216 and the feature map 218].
Kaku is analogous to the claimed invention as they both relate to preprocessing function prior to execution in machine learning systems. Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Yang, Zishan, and Nayar’s teachings to incorporate the teachings of Kaku and provide preprocessing transformation functions before execution of neural network [Kaku, para 0066] to manipulate data, before being input to a neural network, to optimize model output.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SYED RAYHAN AHMED whose telephone number is (571)270-0286. The examiner can normally be reached Mon-Fri ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, David Yi can be reached at (571) 270-7519. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SYED RAYHAN AHMED/Examiner, Art Unit 2126
/DAVID YI/Supervisory Patent Examiner, Art Unit 2126
Prosecution Insights