DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS), submitted on November 13, 2023, was considered by the examiner. The submission is in compliance with the provisions of 37 CFR 1.97.
Claim Objections
Claim 3 is objected to because of the following informalities: “The high-efficient quantization method for the deep probabilistic network according to claim 2, wherein step 2) comprises…”, and the examiner finds it unclear what ‘step 2)’ is referring to from claim 3.
Claim 4 is objected to because of the following informalities: “The high-efficient quantization method for the deep probabilistic network according to claim 3, wherein step 3) comprises…”, and it is unclear what ‘step 3)’ is referring to from claim 4. Appropriate correction of the claims is required.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-5 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea (mental process or math concept) without significantly more.
Claim 1:
Regarding claim 1, in step 1 of the 101-analysis set forth in MPEP 2106, the claim recites
“1. A high-efficient quantization method for a deep probabilistic network, comprising the following steps: …, and a method is one of the four statutory categories of invention.
In step 2A prong 1 of the 101-analysis set forth in the MPEP 2106, the examiner has determined that the following limitations recite a process that, under the broadest reasonable interpretation, covers a mental process but for recitation of generic computer components:
1. A high-efficient quantization method for a deep probabilistic network, comprising the following steps: 1) when a structure of the deep probabilistic network is a directed acyclic graph (DAG), clustering each node in the DAG to obtain each cluster, and assigning an arithmetic type with different precision based on a characteristic of a clustering category of each cluster,… (mental process, a person can mentally evaluate and cluster each node and assign an arithmetic type with different precision for each cluster, see MPEP 2106.04(a)(2)(III)),
… and preliminarily quantizing each node by using the assigned arithmetic type, to obtain a preliminarily quantized deep probabilistic network; (mental process, a person can mentally evaluate and quantize each node with an assigned arithmetic type using pen and paper, see MPEP 2106.04(a)(2)(III)),
and 3) optimizing a quantization scheme by using an arithmetic type search method based on an optimization strategy, (mental process, a person can mentally evaluate and optimize a quantization scheme using pen and paper, see MPEP 2106.04(a)(2)(III)),
If claim limitations, under their broadest reasonable interpretation, covers performance of the limitations as a mental process but for the recitation of generic computer components, then it falls within the mental process grouping of abstract ideas. Accordingly, the claim “recites” an abstract idea.
In step 2A prong 2 of the 101-analysis set forth in MPEP 2106, the examiner has determined that the following additional elements do not integrate this judicial exception into a practical application:
2) reformulating a structure of a multi-in node for the preliminarily quantized deep probabilistic network by reformulating, based on an input weight, the multi-in node into a binary tree network containing only two input nodes to achieve branch clustering and reformulation of each cluster; and adjusting a weight parameter of the reformulated binary tree network to achieve parameter reformulation; (This recites mere instructions to apply an exception using generic computer – see MPEP 2106.05(f)),
Since the claim as a whole, looking at the additional elements individually and in combination, does not contain any other additional elements that are indicative of integration into a practical application, the claim is “directed” to an abstract idea.
In step 2B of the 101-analysis set forth in the 2019 PEG, the examiner has determined that the claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
As discussed above, additional element iv recites mere instructions to apply the judicial exception using generic computer components, which is not indicative of significantly more.
Considering the additional elements individually and in combination, and the claim as a whole, the additional elements do not provide significantly more than the abstract idea. Therefore, the claim is not patent eligible.
Claim 2:
Regarding claim 2, it is dependent upon claim 1, and thereby incorporates the limitations of, and corresponding analysis applied to claim 1. Further, claim 2 recites the following abstract ideas:
1.1)... and dividing the deep probabilistic network into a plurality of clusters; (This recites a mental process, a person can mentally evaluate and divide a deep probabilistic network into clusters, see MPEP 2106.04(a)(2)(III)),
1.2… and then performing statistical analysis on a data distribution of each cluster, (This recites a mental process, a person can mentally evaluate and run statistical analysis for each cluster, see MPEP 2106.04(a)(2)(III)),
1.3) dynamically adjusting a cluster affiliation of each node based on an overall data range of the cluster and a data range of each node to reduce a data distribution range of each cluster; (This recites a mental process, a person can mentally evaluate and dynamically adjust (or update) a cluster affiliation of each node based on an overall data range of the cluster and a data range of each node using pen and paper, see MPEP 2106.04(a)(2)(III)),
1.4) specifying an appropriate arithmetic type for each cluster based on an adjusted data distribution characteristic of the cluster; (This recites a mental process, a person can mentally evaluate and specify an arithmetic type for each cluster, see MPEP 2106.04(a)(2)(III)),
1.5) preliminarily quantizing each node based on the specified arithmetic type, (This recites a mental process, a person can mentally evaluate and quantize each node based on a specified arithmetic type using pen and paper, see MPEP 2106.04(a)(2)(III)),
If claim limitations, under their broadest reasonable interpretation, covers performance of the limitations as a mental process but for the recitation of generic computer components, then it falls within the mental process grouping of abstract ideas. Accordingly, the claim “recites” an abstract idea.
Further, claim 2 recites the following additional elements:
2. The high-efficient quantization method for the deep probabilistic network according to claim 1, wherein step 1) comprises: 1.1) layering all nodes based on a depth of each node in the deep probabilistic network, (In step 2A, prong 2, this is considered mere instructions to apply an exception using generic computer – see MPEP 2106.05(f)), (In step 2B, this is also considered mere instructions to apply an exception using generic computer – see MPEP 2106.05(f)),
1.2) performing model inference by using data in a dataset based on a double-precision floating-point arithmetic type, (In step 2A, prong 2, this is considered mere instructions to apply an exception using generic computer – see MPEP 2106.05(f)), (In step 2B, this is also considered mere instructions to apply an exception using generic computer – see MPEP 2106.05(f)),
…recording a dynamic data range of each cluster in the deep probabilistic network, (In step 2A, prong2, this recites mere data gathering, which is considered insignificant extra-solution activity – see MPEP 2106.05(g),). In step 2B, this insignificant extra-solution activity is well understood routine and conventional activity which includes receiving or transmitting data over a network from court case Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information); TLI Communications LLC v. AV Auto. LLC, 823 F.3d 607, 610, 118 USPQ2d 1744, 1745 (Fed. Cir. 2016), – see MPEP 2106.05(d) (II)(i)).
Since the claim does not recite additional elements that either integrate the judicial exception into a practical application, nor provide significantly more than the judicial exception, the claim is not patent eligible.
Claim 3:
Regarding claim 3, it is dependent upon claim 2, and thereby incorporates the limitations of, and corresponding analysis applied to claim 2. Further, claim 3 recites the following abstract ideas:
3. The high-efficient quantization method for the deep probabilistic network according to claim 2, wherein step 2) comprises: 2.1) taking a logarithm with two as a base for weights of all input branches of the multi-in node to obtain a result, rounding the result down to obtain an indicator, (This is considered a mathematical relationship, mathematical formula or equation, or mathematical calculation, see specification in paragraphs [0050] state “Finally, a parameter reformulation method is proposed, which can adjust a weight parameter of the binary tree network to reduce an accuracy loss in a calculation process. A specific implementation method is as follows: 2.1. As shown in FIG. 3A, a logarithm with two as a base is taken for weights of all input branches of the multi-in node, a result is rounded down”, see MPEP 2106.04(a)(2), subsection I),
…dividing the input branches into a plurality of clusters based on the indicator, and marking the indicator as In and a corresponding cluster as Cn; (This recites a mental process, a person can mentally evaluate and divide the input branches into clusters based on an indicator using pen and paper, see MPEP 2106.04(a)(2)(III)),
2.2) sorting each cluster based on a size of In, organizing the cluster into a form of the binary tree network, marking a newly generated input branch as B, and setting an initial weight of the input branch to 1, wherein a cluster Cn with a larger In is closer to a root node; (This recites a mental process, a person can mentally evaluate and sort each cluster based on a size, set initial weight of input branch to 1 using pen and paper, see MPEP 2106.04(a)(2)(III)),
2.3) randomly arranging a node in each cluster to obtain a binary tree, such that the structure of the deep probabilistic network is reformulated; (This recites a mental process, a person can mentally evaluate and randomly arrange a node in each cluster to get a binary tree with pen and paper, see MPEP 2106.04(a)(2)(III)),
2.4) amplifying weight parameters of all input branches of each cluster in a same proportion to reduce an impact of accuracy underflow; (This recites a mental process, a person can mentally evaluate and increase or amplify weight parameters of all input branches of each cluster, see MPEP 2106.04(a)(2)(III)),
and 2.5) adjusting a weight coefficient of the input branch B to offset the impact in step 2.4) to restore a calculation result to a normal value. (This recites a mental process, a person can mentally evaluate and adjust a weight coefficient of an input branch and later restore a result back to normal value using pen and paper, see MPEP 2106.04(a)(2)(III)),
If claim limitations, under their broadest reasonable interpretation, covers performance of the limitations as a mental process or math concept, but for the recitation of generic computer components, then it falls within the mental process or math concept grouping of abstract ideas. Accordingly, the claim “recites” an abstract idea.
Since the claim does not recite additional elements that either integrate the judicial exception into a practical application, nor provide significantly more than the judicial exception, the claim is not patent eligible.
Claim 4:
Regarding claim 4, it is dependent upon claim 3, and thereby incorporates the limitations of, and corresponding analysis applied to claim 3. Further, claim 4 recites the following abstract ideas:
4. The high-efficient quantization method for the deep probabilistic network according to claim 3, wherein step 3) comprises: 3.1) analyzing an arithmetic type used in a preliminary quantization scheme to construct larger-range arithmetic type selection space as search space, (This recites a mental process, a person can mentally evaluate and analyze an arithmetic type for a quantization scheme, see MPEP 2106.04(a)(2)(III)),
and sorting the search space based on an expression capability of the arithmetic type in an ascending order; (Mental process, a person can mentally evaluate and sort a search space by arithmetic type in ascending order, see MPEP 2106.04(a)(2)(III)),
3.2) evaluating importance of each cluster in an initial network for overall model accuracy, and setting a priority of the cluster based on an evaluation indicator; (This recites a mental process, a person can mentally evaluate and label importance of each cluster and set a priority per cluster by an indicator, see MPEP 2106.04(a)(2)(III)),
and 3.3) determining the arithmetic type of each cluster in order based on the priority. (This recites a mental process, a person can mentally evaluate and determine the arithmetic type of each cluster, see MPEP 2106.04(a)(2)(III)),
If claim limitations, under their broadest reasonable interpretation, covers performance of the limitations as a mental process, but for the recitation of generic computer components, then it falls within the mental process grouping of abstract ideas. Accordingly, the claim “recites” an abstract idea.
Since the claim does not recite additional elements that either integrate the judicial exception into a practical application, nor provide significantly more than the judicial exception, the claim is not patent eligible.
Claim 5:
Regarding claim 5, it is dependent upon claim 1, and thereby incorporates the limitations of, and corresponding analysis applied to claim 1. Further, claim 5 recites the following additional element:
5. The high-efficient quantization method for the deep probabilistic network according to claim 1, wherein the arithmetic type search method based on the optimization strategy in step 3) is an arithmetic type search method based on power consumption analysis and network accuracy analysis, and dynamically adjusts the arithmetic type of each cluster based on specified power consumption and accuracy requirements to obtain an optimized network configuration, (In step 2A, prong 2, this recites mere instructions to apply an exception using generic computer – see MPEP 2106.05(f)), (In step 2B, this also recites mere instructions to apply an exception using generic computer – see MPEP 2106.05(f)),
Since the claim does not recite additional elements that either integrate the judicial exception into a practical application, nor provide significantly more than the judicial exception, the claim is not patent eligible.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1, 2, and 5 are rejected under 35 U.S.C. 103 over Shawahna A. et al., in “FxP-QNet: a post-training quantizer for the design of mixed low-precision DNNs with dynamic fixed-point representation,” published on March 22, 2022, available at https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=9730831, (hereafter, SHAWAHNA), in view of Shah, N., et al. in “Problp: A framework for low-precision probabilistic inference,” published on June 2, 2019, available in the IDS and at https://dl.acm.org/doi/pdf/10.1145/3316781.3317885, (hereafter, SHAH), further in view of Wu, B., et al., in “Mixed precision quantization of convnets via differentiable neural architecture search,” published on November 30, 2018, available at https://arxiv.org/pdf/1812.00090, (hereafter, WU).
Claim 1:
Regarding claim 1, SHAWAHNA teaches “1) when a structure of the deep probabilistic network is a directed acyclic graph (DAG), clustering each node in the DAG to obtain each cluster, and assigning an arithmetic type with different precision based on a characteristic of a clustering category of each cluster, and preliminarily quantizing each node by using the assigned arithmetic type, to obtain a preliminarily quantized deep probabilistic network;”
See SHAWAHNA in page 30207, section C Mixed-precision quantization, describe " represents the network as a computational directed acyclic graph (DAG). The nodes of the DAG represent activations while edges represent a parameterized convolution operation on the input node and its weights. Cascading nodes are connected using several edges each of which with a manually configured wordlengths. The SGD is used to select the edges, based on system requirements, as well as to optimize the weights of these edges. Note that DNAS follows PACT to quantize weights and activations" Here, SHAWAHNA mentions using a directed acyclic graph (DAG) to represent each node's parametrized convolution operation and the nodes are connected to several edges to optimize edge weights.
Further, see SHAWAHNA in page 30216, section 3) Intra-cluster bit-precision level reduction describes "At this stage, the framework receives a set data-structures, each in fixed-point representation, for the currently visited Level-2clusterasG(t) νj ={(d1, 1),··· ,(dU, U)}, and then, a data-structure, say(du, u), is chosen and quantized to one of the next bit-precision levels" . Here, SHAWAHNA shows quantizing a data structure represented by a point representation (i.e. node), by the next bit-precision level (i.e. assigned arithmetic type).
Further, see SHAWAHNA in page 30215, before section 1, describe "As illustrated in Algorithm 2, the proposed framework starts with an initial quantized DNN where the data-structures are in fixed point representation, each of which with possibly different wordlength from 32 to 8. Thereafter, it progressively quantizes the data-structures until it reaches the maximum compression rate under a given accuracy degradation constraint as elaborated in the three steps given below."
See SHAWAHNA in page 30215, in section 1) Data-structure clustering, second to last paragraph, describe "as it is evident from the given example, the contribution of learnable parameters in deeper layers to the compression rate is much more than that of the shallower layers when both are quantized to the next bit-precision level. Accordingly, the proposed framework adopts the two-level clustering approach [95] to represent the data-structures for wordlength reduction based on their type and size." Here, see SHAWAHNA describe that the parameter information is quantized and the clustering is categorizing data structures based on their type and size.
Further, SHAWAHNA teaches “and adjusting a weight parameter of the reformulated binary tree network to achieve parameter reformulation;”
See SHAWAHNA in page 30207, third paragraph mention “In this paper, we adopt the dynamic fixed-point format, depicted in Figure 1, to represent the data-structures of deep neural networks (DNNs). Therefore, we use a configurable β to increase the flexibility as in the floating-point format but at a lower associated hardware cost because it is shared by a group of numbers. Specifically, given n numbers to be represented, i.e., the weights or the activations of a layer, β is tuned with the aim of minimizing a predefined cost function”. Here, SHAWAHNA shows adjusting a weight parameter. See SHAWAHNA in page 30211, second to last paragraph describe “A MAC tree of integer arithmetic can be used to convolve an activation window Ai-1,j with a weight kernel Wi,m each of N integers. To avoid the overflow, the partial results of the MAC tree in first and last layers are computed with 32 bits, while on the other hand, MAC tree partial results in other layers are computed with 16 bits. This is because these data-structures have a low bit-precision level during deployment and a limited data range with weights in bell-curve distribution.” This shows adjusting a tree computing in different bits.
However, SHAWAHNA did not teach “reformulating a structure of a multi-in node for the preliminarily quantized deep probabilistic network by reformulating, based on an input weight, the multi-in node into a binary tree network containing only two input nodes to achieve branch clustering and reformulation of each cluster…; 3) optimizing a quantization scheme by using an arithmetic type search method based on an optimization strategy”
In an analogous field, SHAH teaches “reformulating a structure of a multi-in node for the preliminarily quantized deep probabilistic network by reformulating, based on an input weight, the multi-in node into a binary tree network containing only two input nodes to achieve branch clustering and reformulation of each cluster…”
See SHAH in page 5 section 3.4 Automatic hardware generation, describe " There are two major stages in the hardware generation process. In the first stage, all AC operators with more than two inputs are decomposed into a tree of 2-input operators. An example of such decomposition is shown in figure 4, wherein the F operator is decomposed into a tree of F1, F2 and F3. " Here, SHAH shows reformulating a structure of more than two inputs (i.e. multi-in node) into a tree of 2-input operators (i.e. into two input nodes)
Further, see SHAH in page 2, section 2, Background and previous work, describe "Bayesian networks (BN) are directed acyclic graphs that compactly encode a joint probability distribution over a set of random variables {X1, ...,Xn} ... In the graphical component of BNs, the variables are represented as nodes and their probabilistic or causal relationships are indicated by the direction of the edges among them, as depicted in Figure 1a." Here, SHAH shows the concept of reformulation of multiple inputs into two inputs is applied to a Bayesian network, which is a binary tree network. See figure 4 in SHAH for details.
PNG
media_image1.png
395
653
media_image1.png
Greyscale
See page 5, section 4, where SHAH shows “we trained Naive Bayes classifier on 60% of the data and used the rest for testing. The testing dataset for Alarm is generated by sampling 1000 instances from the trained network. In all the experiments, the leaf nodes of the BN were used as evidence nodes e and one of the root nodes in the BN (the class node in the case of the classifiers) as a query node q,” where SHAH applies this concept to clustering where the leaf nodes of binary network are viewed to be branch clustering and reformulation of each cluster.
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to combine the base reference of SHAWAHNA and incorporate into the teachings of SHAH because both references teach reformulating a binary network.
One of ordinary skill in the art would be motivated to do so because “probabilistic inference on a BN can be made efficient by compiling it to an Arithmetic circuit, which consists only of multiplications and addition,” (see SHAH, page 2, section 2).
However, SHAWAHNA in view of SHAH did not teach “3) optimizing a quantization scheme by using an arithmetic type search method based on an optimization strategy.”
In an analogous field, WU teaches “3) optimizing a quantization scheme by using an arithmetic type search method based on an optimization strategy.”
See WU in page 9, section 7, Conclusion, describe "mixed precision quantization of a ConvNet to determine its layer-wise bit-widths. We formulate this problem as a neural architecture search (NAS) problem and propose a novel, efficient, and effective differentiable neural architecture search (DNAS) framework to solve it. Under the DNAS framework, we efficiently explore the exponential search space of the NAS problem through gradient based optimization (SGD)." Here, WU shows using a neural architecture search (NAS) that is optimized with gradient based optimization (i.e. search method based on an optimization strategy) to optimize mixed precision quantization, where precision and layer-wise bit-widths relate to arithmetic type. See WU in page 3, first half paragraph from section 2. Related work, for more details.
Further, see Wu in page 2, last paragraph of Introduction section, describe "We apply the DNAS framework to solve the mixed precision quantization problem, by constructing a super net whose macro architecture (number of layers, filter size of each layer, etc.) is the same as the target network. Each layer of the super net contains several parallel edges representing convolution operators with quantized weights and activations with different precisions. We show that using DNAS to search for layer-wise precision assignments for ResNet models on CIFAR10 and ImageNet, we surpass the state-of-the-art compression. Our quantized models with 21.1x smaller model size or 103.9x smaller computational cost can still outperform baseline quantized or even full precision models." Here, Wu describes using an architectural search method and apply this to a quantization problem.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the references of SHAWAHNA and SHAH, and incorporate with the teachings of WU by using the teachings of SHAWAHNA and SHAH of obtaining a quantized deep probabilistic network that is reformulated, with WU’s teaching of optimizing an arithmetic type search method.
One of ordinary skill in the art would be motivated to do so because by integrating WU’s framework into the methods of SHAWAHNA and SHAH, one with ordinary skill in the art would achieve the goal of “using this technique, the stochastic super net becomes fully differentiable and can be effectively and efficiently trained by SGD” (see WU in page 2, paragraph 2) .
Claim 2:
Regarding claim 2, SHAWAHNA in view of SHAH, further in view of WU, teaches the limitations of claim 1.
Further, SHAWAHNA teaches “1.1) … and dividing the deep probabilistic network into a plurality of clusters;”
See SHAWAHNA in page 30207, column 1, second and third paragraphs, describe “employ the weight factorization to make DNNs more robust to layer-wise post-training quantization... KDE-KM [79] creates a quantization codebook for the weights of each layer from the center values of the clusters identified through the k-means clustering algorithm. Huang et al. [80] proposed a mixed quantization framework, de noted as MXQN, for quantizing activations either to fixed point numbers or quantizing them in their log-space based on the signal-to-quantization-noise ratio. On the other hand, the MXQN quantizes weights to fixed-point values.” Here, SHAWAHNA describes clustering a deep neural network (i.e. deep probabilistic network). Specifically, SHAWAHNA refers to clustering the network's weights to compress the model. First, the algorithm groups similar weight values into clusters. Then, the method replaces many unique weights with a few cluster centers. Later, the method stores a small index (codebook) instead of large, unique numbers, and overall divides the neural network into clusters. See SHAWAHNA in page 30208, section IV. Designing Mixed Low-Precision Deep Neural Networks For Integer-Only Deployment for details.
Further, SHAWAHNA teaches “and 1.5) preliminarily quantizing each node based on the specified arithmetic type”
See SHAWAHNA describe in 30207, section C. Mixed Precision quantization, column 2, paragraph 2, mentions “ADMM [82] performs an iterative optimization to prune and quantize the weights in a layer-wise manner. The bit-precision of each layer is determined manually and the quantization scalars are defined so as to minimize the overall MSE. On each iteration, after pruning the weights, the ADMM quantizes a portion of FP32 weights that are closest to their discrete bins, and remaining weights are fine-tuned.” Node is construed to mean any unit or part of a neural network, including its layers or weights. Here, SHAWHANA describes a layer-wise pruning and quantization strategy where a model's weights are iteratively compressed using the ADMM (Alternating Direction Method of Multipliers) method. This process involves Layer-wise Execution where the compression (pruning and quantization) is applied to each layer of the neural network individually, rather than modifying the whole network at once (i.e. preliminarily quantizing each node). There is also a step that involves the bit-precision of each layer is determined manually (i.e. based on the specified arithmetic type) and is adapted and adjusted across different layers.
However, SHAWAHNA did not teach “The high-efficient quantization method for the deep probabilistic network according to claim 1, wherein step 1) comprises: 1.1) layering all nodes based on a depth of each node in the deep probabilistic network, … 1.2) performing model inference by using data in a dataset based on a double-precision floating-point arithmetic type, recording a dynamic data range of each cluster in the deep probabilistic network, and then performing statistical analysis on a data distribution of each cluster; 1.3) dynamically adjusting a cluster affiliation of each node based on an overall data range of the cluster and a data range of each node to reduce a data distribution range of each cluster; 1.4) specifying an appropriate arithmetic type for each cluster based on an adjusted data distribution characteristic of the cluster, ”
Further, SHAH teaches “1.2) performing model inference by using data in a dataset based on a double-precision floating-point arithmetic type, recording a dynamic data range of each cluster in the deep probabilistic network, and then performing statistical analysis on a data distribution of each cluster;”
See SHAH in page 5, section 4.2 Overall performance, describe “In this experiment, the complete ProbLP framework is deployed to choose an appropriate arithmetic representation and generate hardware for different ACs and for given user requirements.” This shows SHAH describes dynamically adjusting arithmetic representation and hardware types based on user requirements, and the choosing an appropriate arithmetic representation shows performing model inference based on an arithmetic type. See SHAH in page 4, section 3.2 for details on “As shown in figure 2, ProbLP aims to estimate the optimal fixed pt and float-pt bit width for a given type of probabilistic query and error tolerance” applying this to any floating point types.
Further, see SHAH in page 5, section 3.3 mention “a method to evaluate error bounds for a given AC in terms of number of bits. Next, ProbLP finds the least number of fixed-pt and float-pt bits needed for given requirements. For this, it evaluates the bounds starting with 2 fraction bits and 2 mantissa bits, and increments them until the error-requirement is satisfied.” SHAH mentions dynamically adjusting nodes to satisfy error requirements (i.e. based on overall data range or reduce data distribution of each cluster), and finds and records the errors. Evaluate error bounds is considered performing statistical analysis on a data distribution, since errors are part of the data distribution.
Further, SHAH teaches “1.3) dynamically adjusting a cluster affiliation of each node based on an overall data range of the cluster and a data range of each node to reduce a data distribution range of each cluster;”
See SHAH in page 5, section 3.4 Automatic hardware generation, describe “ProbLP suggests the most-appropriate low-precision representation for the AC, but this may not translate to energy savings unless hardware has custom arithmetic operators. To address this, ProbLP has an integrated hardware generator that generates custom parallel hardware that is fully-pipelined and consists of arithmetic operators of the exact precision that is required to meet the user requirements.” SHAH shows the process of dynamically adjusting operations that help meet user requirements. The term dynamically is construed to mean adaptable or flexible adjusting a method to satisfy or meet demands or conditions such as user requirements. See figure 2 in SHAH, where SHAH describes HW or hardware generation from selected representation of arithmetic quantities, which describes quantization.
PNG
media_image2.png
517
706
media_image2.png
Greyscale
Further, see SHAH in page 5, section 4.2 Overall performance, describe “In this experiment, the complete ProbLP framework is deployed to choose an appropriate arithmetic representation and generate hardware for different ACs and for given user requirements.” This shows SHAH describes dynamically adjusting arithmetic representation and hardware types based on user requirements. See SHAH in page 5, section 3.3 mention “a method to evaluate error bounds for a given AC in terms of number of bits. Next, ProbLP finds the least number of fixed-pt and float-pt bits needed for given requirements. For this, it evaluates the bounds starting with 2 fraction bits and 2 mantissa bits, and increments them until the error-requirement is satisfied.” SHAH mentions dynamically adjusting nodes to satisfy error requirements (i.e. based on overall data range or reduce data distribution of each cluster).
PNG
media_image3.png
494
819
media_image3.png
Greyscale
See SHAH in figure 4, where SHAH shows grouping a network tree into nodes, and clusters denoted F1, F2, and F3, and shows a cluster affiliation of each node. Node is construed to mean any type of information or point.
See SHAH in page 4, section 3.1.3, where SHAH mentions “The error propagation in fixed-pt arithmetic produces a bound of the form ∆f ≤ c, where ∆f is the absolute error in the output node, and c is a constant that depends on the size and structure of the AC, its parameters, and the number of fixed-pt bits.” The term data range or data distribution is construed to include errors. The spread or shape of your data not only includes values, but this also reflects the combined effect of variance and all the measurement, systematic, or recording errors present in the dataset. Here, SHAH teaches in page 4 and from figure 2 of adjusting group or cluster affiliation of each node from error models, where the error models relate to an overall data range of the cluster.
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to combine the base reference of SHAWAHNA and incorporate into the teachings of SHAH because both references teach reformulating a binary network.
One of ordinary skill in the art would be motivated to do so because “probabilistic inference on a BN can be made efficient by compiling it to an Arithmetic circuit, which consists only of multiplications and addition,” (see SHAH, page 2, section 2).
However, SHAWAHNA in view of SHAH did not teach “The high-efficient quantization method for the deep probabilistic network according to claim 1, wherein step 1) comprises: 1.1) layering all nodes based on a depth of each node in the deep probabilistic network, … 1.4) specifying an appropriate arithmetic type for each cluster based on an adjusted data distribution characteristic of the cluster ”
Further, WU teaches “The high-efficient quantization method for the deep probabilistic network according to claim 1, wherein step 1) comprises: 1.1) layering all nodes based on a depth of each node in the deep probabilistic network, …”
See WU in page 2, Introduction, describe “we apply the DNAS framework to solve the mixed precision quantization problem, by constructing a super net whose macro architecture (number of layers, filter size of each layer, etc.) is the same as the target network. Each layer of the super net contains several parallel edges representing convolution operators with quantized weights and activations with different precisions. We show that using DNAS to search for layer-wise precision assignments for ResNet models on CIFAR10 and ImageNet, we surpass the state-of-the-art compression”. Here, WU teaches the depth, which is construed to also mean the number of layers of a neural network, of each node (or neuron or unit of the neural network model).
Further, WU teaches “1.4) specifying an appropriate arithmetic type for each cluster based on an adjusted data distribution characteristic of the cluster,”
See WU in page 2, Introduction, describe “we apply the DNAS framework to solve the mixed precision quantization problem, by constructing a super net whose macro architecture (number of layers, filter size of each layer, etc.) is the same as the target network. Each layer of the super net contains several parallel edges representing convolution operators with quantized weights and activations with different precisions. We show that using DNAS to search for layer-wise precision assignments for ResNet models on CIFAR10 and ImageNet, we surpass the state-of-the-art compression.” WU here describes using DNAS method to specify layer-wise precision assignments of different precisions (i.e. appropriate arithmetic type) for each layer of the super net that contains several parallel edges (i.e. cluster).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the references of SHAWAHNA and SHAH, and incorporate with the teachings of WU by using the teachings of SHAWAHNA and SHAH of obtaining a quantized deep probabilistic network that is reformulated, with WU’s teaching of layering all nodes based on a depth, and specifying an appropriate arithmetic type for each cluster.
One of ordinary skill in the art would be motivated to do so because by integrating WU’s framework into the methods of SHAWAHNA and SHAH, one with ordinary skill in the art would achieve the goal of “using this technique, the stochastic super net becomes fully differentiable and can be effectively and efficiently trained by SGD” (see WU in page 2, paragraph 2) .
Claim 5:
Regarding claim 5, SHAWAHNA in view of SHAH, further in view of WU, teaches the limitations in claim 1.
Further, SHAH teaches “The high-efficient quantization method for the deep probabilistic network according to claim 1, wherein the arithmetic type search method based on the optimization strategy in step 3) is an arithmetic type search method based on power consumption analysis and network accuracy analysis, and dynamically adjusts the arithmetic type of each cluster based on specified power consumption and accuracy requirements to obtain an optimized network configuration.”
See SHAH in page 5, section 3.3, Selecting optimal representation, describe “ProbLP finds the least number of fixed-pt and float-pt bits needed for given requirements. For this, it evaluates the bounds starting with 2 fraction bits and 2 mantissa bits, and increments them until the error-requirement is satisfied. Then, it estimates the least number of integer and exponent bits required by the min and max analysis explained in section 3.1.4. In this way, ProbLP comes up with the optimal fixed-pt and float-pt representation shown in figure 2.
Subsequently, the framework has to select among fixed-pt and float-pt. ProbLP selects the one with the lowest energy-consumption, estimated using operator-level energy models.” Here, SHAH shows that the search type of finding the least number of bits needed, is based on lowest energy-consumption (i.e. power consumption). Here, SHAH shows finding the method with the lowest -energy consumption, and by incrementing the bits ‘until the error-requirement is satisfied’ shows dynamically adjusting the arithmetic type.
Further, see SHAH in page 5, section 4, where SHAH shows “we trained Naive Bayes classifier on 60% of the data and used the rest for testing. The testing dataset for Alarm is generated by sampling 1000 instances from the trained network. In all the experiments, the leaf nodes of the BN were used as evidence nodes e and one of the root nodes in the BN (the class node in the case of the classifiers) as a query node q,” where SHAH mentions this method is class or cluster based.
Further, see SHAH in page 2, section 3.1, where SHAH mentions “The aim of error analysis is to estimate the minimum number of bits required to achieve the user-specified error tolerance. For this, it has to take into account the impact of reducing the number of bits on the error in the AC output probability.” SHAH mentions error analysis, which relates to network accuracy analysis since reducing the error helps detect accuracy.
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to combine the base reference of SHAWAHNA and incorporate into the teachings of SHAH because the references mention a method based on power consumption analysis.
One of ordinary skill in the art would be motivated to do so because “probabilistic inference on a BN can be made efficient by compiling it to an Arithmetic circuit, which consists only of multiplications and addition,” (see SHAH, page 2, section 2).
Allowable Over Prior Art
Claims 3 and 4:
Claim 3 is objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims. Claim 4 is objected to as being dependent upon claim 3. Further, the claims 1-5 are rejected under 35 U.S.C. 101, and would only be allowable if the 35 U.S.C. 101 rejections for these claims have been overcome. Claims 3 and 4 are allowable over prior art only.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to WENWEI ZENG whose telephone number is (571)272-7111. The examiner can normally be reached Monday-Friday, 8am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Usmaan Saeed can be reached at (571) 272-4046. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/WenWei Zeng/Examiner, Art Unit 2146
/USMAAN SAEED/Supervisory Patent Examiner, Art Unit 2146
Prosecution Insights