CTNF 18/071,895 CTNF 95653 DETAILED ACTION Notice of Pre-AIA or AIA Status 07-03-aia AIA 15-10-aia The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA. Remarks [0050] defines “a computer readable storage medium”, as that term is used in the present disclosure, is not to be construed as storage in the form of transitory signals per se. Claim Objections 07-29-01 AIA Claim s 2, 6-8, 10, 14, and 16 are objected to because of the following informalities: Claim 2 line 2-3; claim 6 line 2-3; claim 10 line 2-3; claim 14 line 3; claim 16 line 2 ; claim 20 line 3 “the first sequence W L ” should be “the first weight sequence W L ”. Claim 2 line 4; claim 6 line 5; claim 10 line 4; claim 14 line 5; claim 16 line 4; claim 20 line 5; “the number of ‘1’s” should be “a number of ‘1’s” because there is lack of antecedent basis for such limitation. Claim 2 line 4; claim 6 line 5; claim 10 line 4; claim 14 line 5; claim 16 line 4; claim 20 line 5; “the two sequences” should be “two sequences” because there is lack of antecedently for such limitation. Claim 2 line 5; claim 6 line 5; claim 10 line 5; claim 14 line 6; claim 16 line 5; claim 20 line 6 “the said logical AND operations” should be “the first and second logical AND operations” as antecedently recited. Claim 7 line 2 and claim 8 line 2 “the method” should be “the computer implemented method” as antecedently recited . Appropriate correction is required. Claim Rejections - 35 USC § 112(b) The following is a quotation of 35 U.S.C. 112(b): (b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention. 07-34-01 Claims 8, 11-14, and 17-20 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention. Claim 8 line 3 recites “the weights”. It is unclear whether the weights are referring to the set of defined node weights or different weights. For examination purposes, Examiner interprets such limitation as “weights”. Claims 11-14 line 2 recite “the one or more processors”. There is lack of antecedent basis for such limitation. For examination purposes, Examiner interprets as the one or more computer systems as antecedently recited. Claims 17-20 line 2 recite “the one or more processors”. There is lack of antecedent basis for such limitation. For examination purposes, Examiner interprets as “the one or more computer processors” as antecedently recited. Claim Rejections - 35 USC § 101 07-04-01 AIA 07-04 35 U.S.C. 101 reads as follows: Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title. Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. Claim 1 recites a method Under Prong One of Step 2A of the USPTO current eligibility guidance (MPEP 2106), the claim recites limitations cover mathematical calculations, relationship, and/or formula, such as determining a set of network activations according to network input data (see at least figure 2 step 220 [0042] describes the step of determining using known processing methods including shifting, scaling, and quantization, which are mathematical operations); for each activation x of the set of network activations and each node weight w of the set of defined node weights: generating an activation sequence X ( x n x , x n x - 1 , ..., x 2 , x 1 , x 0 ) which is a unary representation of x/nx, a first weight sequence W L = (W, W, ..., W) by repeating W for nx/nw times, and a second weight sequence W R = (V, V, ..., V) by repeating V for nx/lnw times, where the sequence W = ( W n w , W n w - 1 , ..., W 2 , W 1 , W 0 ) is a unary representation of w/nw, the sequence V= ( W 0 , W 1 , W 2 , ..., W n w - 1 , W n w ) is the reverse of W, and nx is a multiple of nw so that W L and W R are of length nx (see at least figure 2 step 230, [0023] describes examples of generating unary representation of X and W, which are mathematical operations) ; and computing Ms(X, W L ) + Ms(X, W R ) as an approximation for 2(x/nx)(w/nw), wherein Ms(.,.) denotes stochastic multiplication; and providing a neural network output based, at least in part, on a set of computed approximations for 2(x/nx)(w/nw) values (see at least figure 2 step 240-260, and [0023] describes stochastic multiplication includes performing AND operation, [0045] describes the step of providing output of a neural network based on the computed values) . Therefore, the claim includes limitations that fall within the “Mathematical Concepts” grouping of abstract ideas. Accordingly, the claim recites an abstract idea. Under Prong Two of Step 2A , this judicial exception is not integrated into a practical application. The claim additionally recites a computer implemented method comprising receiving a trained neural network having a set of defined node weights and providing a neural network output. However, the additional elements are recited at a high level of generality, i.e., as a computer performing a computer function of receiving, processing data, and providing output, wherein the steps of receiving data (e.g., trained neural network) and providing output amount to mere data gathering, which are considered as insignificant extra/post solution activities. Moreover, the limitations reciting neural network is mere generally linking the use of the judicial exception into a technological environment or field of use, such as neural network. Such additional elements fail to provide a meaningful limitation on the judicial exception, and amount to no more than mere instructions to apply the exception using computer element. Thus, the claim is directed to an abstract idea. Under Step 2B , as discussed with respect to Prong Two of Step 2A, the additional elements in the claim amount no more than mere instructions to apply the exception using a component. The same conclusion is reached in step 2B, i.e., mere instructions to apply an exception on a computer element cannot integrate a judicial exception into a practical application at step 2A or provide an inventive concept that is furnished by an element or combination of elements that is recited in the claim in addition to (beyond) the judicial exception. The steps of receiving a trained neural network and providing a neural network output are considered to be insignificant extra/post solution activities in step 2A, and are determined to be well-understood, routine, conventional activity in the field. Court decisions cited in MPEP 2106.05(d)(II) section (i), indicate that mere receiving or transmitting data over a network, is well-understood, routing, conventional function when it is claimed in a merely generic manner. Thus, the additional element fails to ensure the claim as a whole amount to significantly more than the judicial exception itself. Accordingly, the claim is not patent-eligible under 35 U.S.C. 101. Claim 2 further recites wherein the stochastic multiplication comprises a first logical AND operation on the activation sequence X and the first sequence WL, and a second logical AND operation on the activation sequence X and the second weight sequence WR, followed by counting the number of '1's in the two sequences resulting from the said logical AND operations. Such limitations cover mathematical calculations, relationship, and/or formula (performing AND operations and counting number of 1). The claim does not recite additional element that would integrate the judicial exception into a practical application under step 2A prong two or ensure the claim as a whole amount to significantly more than the judicial exception itself under step 2B. Accordingly, the claim is not patent-eligible under 35 U.S.C. 101. Claim 3 further recites wherein an activation sequence X is generated using a circuit comprising an up-counter or down-counter circuit element and a comparator circuit element. The step of generating an activation sequence X covers mathematical calculations, relationship, and/or formula (see at least figure 2 step 230, [0023] describes examples of generating unary representation of X and W, which are mathematical operations), and the additional elements, such as the circuit comprising an up-counter or down-counter circuit element and a comparator circuit elements, are recited at a high level of generality, e.g., computer components performing computer function and amount to no more than mere instructions to apply the judicial exception using computer components. Thus, the claim does not recite additional element that would integrate the judicial exception into a practical application under step 2A prong two or ensure the claim as a whole amount to significantly more than the judicial exception itself under step 2B. Accordingly, the claim is not patent-eligible under 35 U.S.C. 101. Claim 4 further recites wherein a first weight sequence WL is generated using a circuit comprising an up-counter circuit element and a comparator circuit element. The step of generating a first weight sequence WL covers mathematical calculations, relationship, and/or formula (see at least figure 2 step 230, [0023] describes examples of generating unary representation of X and W, which are mathematical operations), and the additional elements, such as the circuit comprising an up-counter or down-counter circuit element and a comparator circuit elements, are recited at a high level of generality, e.g., computer components performing computer function and amount to no more than mere instructions to apply the judicial exception using computer components. Thus, the claim does not recite additional element that would integrate the judicial exception into a practical application under step 2A prong two or ensure the claim as a whole amount to significantly more than the judicial exception itself under step 2B. Accordingly, the claim is not patent-eligible under 35 U.S.C. 101. Claim 5 further recites wherein a second weight sequence WR is generated using a circuit comprising a down-counter circuit element and a comparator circuit element. The step of generating a second weight sequence WR covers mathematical calculations, relationship, and/or formula (see at least figure 2 step 230, [0023] describes examples of generating unary representation of X and W, which are mathematical operations), and the additional elements, such as the circuit comprising an up-counter or down-counter circuit element and a comparator circuit elements, are recited at a high level of generality, e.g., computer components performing computer function and amount to no more than mere instructions to apply the judicial exception using computer components. Thus, the claim does not recite additional element that would integrate the judicial exception into a practical application under step 2A prong two or ensure the claim as a whole amount to significantly more than the judicial exception itself under step 2B. Accordingly, the claim is not patent-eligible under 35 U.S.C. 101. Claim 6 further recites wherein the stochastic multiplication comprises a first logical AND operation on the activation sequence X and the first sequence WL, and a second logical AND operation on the activation sequence X and the second weight sequence WR, followed by counting the number of '1's in the two sequences resulting from the said logical AND operations. Such limitations cover mathematical calculations, relationship, and/or formula (performing AND operation on sequence of bits as described in [0023] and counting number of 1 in the result sequences). Moreover, the claim recites additional elements, such as a first AND gate circuit element, a second AND gate circuit element, and a parallel counter circuit element, that are at a high level of generality e.g., computer components performing computer function and amount to no more than mere instructions to apply the judicial exception using computer components. Thus, the claim does not recite additional element that would integrate the judicial exception into a practical application under step 2A prong two or ensure the claim as a whole amount to significantly more than the judicial exception itself under step 2B as the additional elements are merely recited as a result of the mathematical operations, such as having an AND gate to perform AND operation and counter to perform counting. Accordingly, the claim is not patent-eligible under 35 U.S.C. 101. Claim 7 further recites wherein the neural network output provided by the method is an inference output of the neural network. Such limitation merely describes the type of output of neural network, such as an inference output. Thus, such limitation mere generally linking the use of the judicial exception into a technological environment or field of use, such as neural network, under step 2A prong two. Therefore, the claim does not recite additional element that would integrate the judicial exception into a practical application under step 2A prong two or ensure the claim as a whole amount to significantly more than the judicial exception itself under step 2B. Accordingly, the claim is not patent-eligible under 35 U.S.C. 101. Claim 8 further recites wherein the neural network output provided by the method is a forward propagation path output of the neural network during retraining to fine-tune the weights prior to inference. Such limitation merely describes the type of output of neural network, such as a forward propagation path output. Thus, such limitation mere generally linking the use of the judicial exception into a technological environment or field of use, such as neural network, under step 2A prong two. Therefore, the claim does not recite additional element that would integrate the judicial exception into a practical application under step 2A prong two or ensure the claim as a whole amount to significantly more than the judicial exception itself under step 2B. Accordingly, the claim is not patent-eligible under 35 U.S.C. 101. Claims 9-14 recite product claims having similar limitations as claims 1-6. Thus, they are rejected for the same reasons. Furthermore, claim 9 recites a computer program product, the computer program product comprising one or more computer readable storage media and collectively stored program instructions on the one or more computer readable storage media, the stored program instructions which, when executed, cause one or more computer systems to perform operations. Under step 2A prong two, such additional elements are recited at a high level of generality, e.g., computer components performing computer functions of storing and executing instructions. Thus, the claim does not recite additional element that would integrate the judicial exception into a practical application under step 2A prong two or ensure the claim as a whole amount to significantly more than the judicial exception itself under step 2B. Accordingly, the claims are not patent-eligible under 35 U.S.C. 101. Claims 15-20 recite system claims having similar limitations as method claims 1-6. Thus, they are rejected for the same reasons. Furthermore, claim 15 recites a computer system for providing neural network inference outputs, the computer system comprising: one or more computer processors; one or more computer readable storage devices; and stored program instructions on the one or more computer readable storage devices for execution by the one or more computer processors, the stored program instructions which, when executed, cause the one or more computer processors to perform operations. Under step 2A prong two, such additional elements are recited at a high level of generality, e.g., computer components performing computer functions of storing and executing instructions. Thus, the claim does not recite additional element that would integrate the judicial exception into a practical application under step 2A prong two or ensure the claim as a whole amount to significantly more than the judicial exception itself under step 2B. Accordingly, the claims are not patent-eligible under 35 U.S.C. 101. Allowable Subject Matter Claims 1-20 would be allowable if rewritten or amended to overcome the claim objections, rejections under 35 U.S.C. 112(b) and 101 as appropriate, set forth in this Office action. 13-03-01 AIA The following is a statement of reasons for the indication of allowable subject matter: Regarding claims 1, 9, and 15 , the prior art of records does not teach or suggest a combination of limitations, including for each activation x of the set of network activations and each node weight w of the set of defined node weights: generating an activation sequence X ( x n x , x n x - 1 , ..., x 2 , x 1 , x 0 ) which is a unary representation of x/nx, a first weight sequence W L = (W, W, ..., W) by repeating W for nx/nw times, and a second weight sequence W R = (V, V, ..., V) by repeating V for nx/lnw times, where the sequence W = ( W n w , W n w - 1 , ..., W 2 , W 1 , W 0 ) is a unary representation of w/nw, the sequence V= ( W 0 , W 1 , W 2 , ..., W n w - 1 , W n w ) is the reverse of W, and nx is a multiple of nw so that W L and W R are of length nx; and computing Ms(X, W L ) + Ms(X, W R ) as an approximation for 2(x/nx)(w/nw), wherein Ms(.,.) denotes stochastic multiplication; and providing a neural network output based, at least in part, on a set of computed approximations for 2(x/nx)(w/nw) values. Mohajer – US 20190121839 teaches a device includes shuffling circuitry configured to receive an input unary bit stream and generate a shuffled bit stream by selecting n-tuple combinations of bits of the input unary bit stream. The device also includes stochastic logic circuitry having a plurality of stochastic computational units configured to perform operations on the shuffled bit stream in parallel to produce an output unary bit stream, each of the stochastic computational units operating on a different one of the n-tuple combinations of the bits as illustrated in figure 1, where [0036] describes stochastic computing can be performed using AND gate to perform multiplication between two uncorrelated bit streams. However, Mohajer does not teach or suggest the concept of generate two sequences of weight, one forward (WL) and the other reverse (WR) and compute Ms (X,WL) + Ms (X,WR) as required in the independent claims. Riedel – US 20170359082 teaches a stochastic computation using deterministic bit streams, wherein a device includes an integrated circuit comprising a computational unit configured to process at least two input bit streams that each include a sequential set of data bits or two or more sets of data bits in parallel that is deterministically encoded to represent numerical values. Figure 3B illustrates multiplication by repeating a first input bit stream, where 100 is repeated to perform AND operation using AND gate. However, Riedel does not teach or suggest the concept of generate two sequences of weight, one forward (WL) and the other reverse (WR) and compute Ms (X,WL) + Ms (X,WR) as required in the independent claims. Jenson – NPL A Deterministic Approach to Stochastic Computation (IDS filed on 11/30/2022) teaches that a deterministic approach to stochastic computation on data represented by random bit streams, which allows complex arithmetic to be performed with very simple logic, but suffers from high latency and poor precision. Thus, Jenson teaches that if properly structured, the same arithmetical constructs can operate on deterministic bit stream using relatively prime stream length as illustrated by example 6 performing multiplication using a single AND gate. However, Jenson does not teach or suggest the concept of generate two sequences of weight, one forward (WL) and the other reverse (WR) and compute Ms (X,WL) + Ms (X,WR) as required in the independent claims. Najafi – US 20200143234 teaches a sorting network that operates on unary representation as illustrated in figure 5, wherein figure 7 illustrates a unary stream generator configured to generate a unary stream using counter and comparator. However, Najafi does not teach or suggest for each activation x of the set of network activations and each node weight w of the set of defined node weights: generating an activation sequence X ( x n x , x n x - 1 , ..., x 2 , x 1 , x 0 ) which is a unary representation of x/nx, a first weight sequence W L = (W, W, ..., W) by repeating W for nx/nw times, and a second weight sequence W R = (V, V, ..., V) by repeating V for nx/lnw times, where the sequence W = ( W n w , W n w - 1 , ..., W 2 , W 1 , W 0 ) is a unary representation of w/nw, the sequence V= ( W 0 , W 1 , W 2 , ..., W n w - 1 , W n w ) is the reverse of W, and nx is a multiple of nw so that W L and W R are of length nx; and computing Ms(X, W L ) + Ms(X, W R ) as an approximation for 2(x/nx)(w/nw), wherein Ms(.,.) denotes stochastic multiplication; and providing a neural network output based, at least in part, on a set of computed approximations for 2(x/nx)(w/nw) values. Najafi – US 20190289345 teaches stochastic multiplication of deterministic bitstream using a relatively prime method, where the first bitstream is repeated four times and the second bitstream is repeated 3 times as illustrated in figure 4B. However, Najafi does not teach or suggest for each activation x of the set of network activations and each node weight w of the set of defined node weights: generating an activation sequence X ( x n x , x n x - 1 , ..., x 2 , x 1 , x 0 ) which is a unary representation of x/nx, a first weight sequence W L = (W, W, ..., W) by repeating W for nx/nw times, and a second weight sequence W R = (V, V, ..., V) by repeating V for nx/lnw times, where the sequence W = ( W n w , W n w - 1 , ..., W 2 , W 1 , W 0 ) is a unary representation of w/nw, the sequence V= ( W 0 , W 1 , W 2 , ..., W n w - 1 , W n w ) is the reverse of W, and nx is a multiple of nw so that W L and W R are of length nx; and computing Ms(X, W L ) + Ms(X, W R ) as an approximation for 2(x/nx)(w/nw), wherein Ms(.,.) denotes stochastic multiplication; and providing a neural network output based, at least in part, on a set of computed approximations for 2(x/nx)(w/nw) values. Therefore, the prior art of records does not teach or suggest the combination of limitations as required in the claims . Conclusion Any inquiry concerning this communication or earlier communications from the examiner should be directed to HUY DUONG whose telephone number is (571)272-2764. The examiner can normally be reached Mon-Friday 7:30-5:30. Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Andrew Caldwell can be reached at (571) 272-3702. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300. Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. /HUY DUONG/Examiner, Art Unit 2182 (571)272-2764 Application/Control Number: 18/071,895 Page 2 Art Unit: 2182 Application/Control Number: 18/071,895 Page 3 Art Unit: 2182 Application/Control Number: 18/071,895 Page 5 Art Unit: 2182 Application/Control Number: 18/071,895 Page 6 Art Unit: 2182 Application/Control Number: 18/071,895 Page 7 Art Unit: 2182 Application/Control Number: 18/071,895 Page 8 Art Unit: 2182 Application/Control Number: 18/071,895 Page 9 Art Unit: 2182 Application/Control Number: 18/071,895 Page 10 Art Unit: 2182 Application/Control Number: 18/071,895 Page 11 Art Unit: 2182 Application/Control Number: 18/071,895 Page 12 Art Unit: 2182 Application/Control Number: 18/071,895 Page 13 Art Unit: 2182
Prosecution Insights