Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This application has been examined. Claims 1-20 are pending.
The Group and/or Art Unit location of your application in the PTO has changed. To aid in correlating any papers for this application, all further correspondence regarding this application should be directed to Group Art Unit 2175.
Specification
The title of the invention is not descriptive. A new title is required that is clearly indicative of the invention to which the claims are directed.
Claim Rejections - 35 USC § 112
The following is a quotation of the second paragraph of 35 U.S.C. 112:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 9-10, 12-13 are rejected under 35 U.S.C. 112(b) as failing to particularly point out and distinctly claim the subject matter regarded as the invention:
Claims 9, 10, 12, and 13 are rejected under 35 U.S.C. § 112(b) as indefinite.
Claim 2 defines the index i used throughout its dependent claims as follows: "i is an integer greater than 1, and i is less than or equal to n" - i.e., 2 ≤ i ≤ n, so that i may equal n, the last group in the systolic array. Claims 9, 10, 12, and 13 depend from claim 2 (through claims 4, 6, 7, 8, and/or 11), and each additionally recites structure belonging to an "(i+1)th group of feature operation units," e.g., claim 9’s "exponent comparison subcomponent i+1... an exponent comparison subcomponent in an (i+1)th group of feature operation units in the n groups of feature operation units," and claim 12’s "an output end of ... a first operation subunit i+1 (MEU[i+1]); the first operation subunit i+1 is a first operation subunit in an (i+1)th group of feature operation units in the n groups of feature operation units." Because claim 2 (as incorporated into claims 9, 10, 12, and 13) expressly permits i = n, these claims read on an embodiment in which no "(i+1)th group" exists among the n groups recited in claim 1 (there is no "(n+1)th group"). It is therefore unclear what structure, if any, is required to satisfy the "(i+1)th group"/"exponent comparison subcomponent i+1"/"first operation subunit i+1 (MEU[i+1])" limitations when i = n, and the metes and bounds of claims 9, 10, 12, and 13 cannot be determined with reasonable certainty. See MPEP § 2173.05(b); Nautilus, Inc. v. Biosig Instruments, Inc., 572 U.S. 898 (2014). To the extent these claims are construed to cover the i = n embodiment, they may also lack adequate written description under 35 U.S.C. § 112(a), because the Specification describes the exponent-comparison-subcomponent chain and the "first operation subunit i+1" generally (see Instant Spec. ¶¶ 0040-0041, 0047-0048) without describing what circuitry (if any) is present, or how the recited signal is otherwise generated or handled, at the boundary case where the ith group is the last group in the array. Applicant may wish to amend claim 2 (and any independent basis relied on by claims 9-13) to restrict i such that i+1 ≤ n (i.e., 2 ≤ i ≤ n-1), or to expressly address the terminal-group case.
Claims 18 and 19 are rejected under 35 U.S.C. § 112(b) as indefinite for the corresponding reason.
Claim 17 defines i as "an integer greater than or equal to 1, and i is less than or equal to n" - i.e., 1 ≤ i ≤ n, so that i may equal 1. Claim 18, which depends from claim 17, additionally requires "a preceding group of feature operation units adjacent to the ith group of feature operation units," denoted "an (i-1)th group of feature operation units," which the (i-1)th group is "configured to perform the feature operation on an (i-1)th piece of feature subdata." Because claim 17 permits i = 1, claim 18 reads on an embodiment in which no "(i-1)th group" exists (there is no "0th group" and no "0th piece of feature subdata"). As with claims 9, 10, 12, and 13 above, it is unclear what satisfies these limitations when i = 1, rendering claim 18 (and claim 19, which depends from claim 18) indefinite under 35 U.S.C. § 112(b), and potentially unsupported under 35 U.S.C. § 112(a) for the i = 1 embodiment. This appears to be an oversight, since the corresponding apparatus claim 2 correctly restricts i to be greater than 1 (thereby avoiding the analogous problem for the (i-1)th-group limitation) - Applicant may wish to conform claim 17’s index restriction to that used in claim 2.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. § 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art t which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-5, 15-18, 20 are rejected under AIA 35 U.S.C. § 103 as being unpatentable over Elmer (US No. 11,842,169).
In order to expedite and avoid piecemeal prosecution, the following rejection is made to the extent that the claims are understood, by considering those elements which are understood and interpreting their function in a manner which is consistent with the recited goals of the claims, and then applying the best available art.
The examiner relies on the entire teachings of Elmer references; the applicant should carefully consider the entire teachings of the above-mentioned references to better understand the examiner’s position.
In regard to claim 1, Elmer teaches a data processing apparatus, provided with a systolic array (the systolic array 100 of figure 1; col. 4:47-50) configured to perform a feature operation on feature data (the streamed input data set processed by the array’s multiply-delayed-accumate PEs) obtained from feature extraction on service data of a target service (Elmer’s array is disclosed as forming part of "a neural network processor (602), col. 23:20-23" and it is well understood in the art - consistent with Elmer’s own background discussion that systolic arrays are used “to accelerate the workload in neural networks,” col. 2:16-17 - that the values streamed into a neural-network systolic array are feature values derived from feature extraction performed on the underlying service/application data; this preamble recitation is, in any event, non-structural, intended-use language that does not further limit an otherwise fully-disclosed systolic array), the feature data comprising n pieces of feature subdata arranged in sequence (the sequential stream of input data elements clocked into the array one per systolic interval), and n being an integer greater than or equal to 1
the systolic array comprising a feature operation module, the feature operation module comprising n groups of feature operation units (the PEs, e.g., 112a, 112b, ...,112n, arranged along a row/column of the array, each corresponding to one piece of the sequentially streamed feature subdata), the n groups of feature operation units being configured to perform the feature operation on the n pieces of feature subdata in the feature data, and the n groups of feature operation units being connected according to association operation logic between the n pieces of feature subdata (Elmer discloses that “[f]or example, a PE 112a (of the column 112) may generate a first scaled input (from the first input data set) during a first systolic interval, perform an accumulation during the second systolic interval, and transmit a partial sum to a PE 112b via the column output bus 106 during the second systolic interval” (Elmer col. 5:51-56), i.e., the operation performed on the (i-1)th subdata (by PE 112a) precedes, and its result is applied to, the operation performed on the ith subdata (by PE 112b) - exactly the recited “association operation logic.).
a group of feature operation units (each PE, e.g., PE 200 of figure 2) compris[es] a first operation subunit (multiplier 208) and a second operation subunit (adder 210), and the first operation subunit and the second operation subunit being connected according to feature operation logic of the corresponding feature subdata (“[t]he multiplier 208 can generate a multiplication result (a product) 250 for storing in the delay register 251[; t]he adder 210 can generate an addition result 238 (a sum) based on a stored multiplication result 253 from the delay register 251,” col. 10:66-11:3, i.e., “[e]ach processing element in the systolic array can have a pipeline configured to perform a multiplication during a first systolic interval and to perform an accumulation during a second systolic interval,” Abstract); and
the n groups of feature operation units performing the feature operation on the n pieces of feature subdata in a preset sequence corresponding to the association operation logic, and in the preset sequence, in any two adjacent groups of feature operation units, a time at which a first group of feature operation units starts the feature operation is at least one preset clock cycle earlier than a time at which a second group of feature operation units starts the feature operation (Elmer’s Figs. 9A-9F depict the array’s PEs beginning their respective multiply-and-accumulate operations at successively delayed systolic intervals (interval 0, 1, 2, 3, 4, 5, ...), i.e., a downstream PE cannot begin operating on a given systolic wavefront until at least one clock-cycle-length systolic interval after the immediately preceding PE has begun, this staggered timing being sequenced by Elmer’s clocked “computation controller (606)” (“The computation controller 606 may be configured to provide controls to various components of the neural network processor 602 to perform neural network computations,” col. 24:13-16) and per-PE “operation decoder (256)” (“[t]he operation decoder 256 may be configured to decode the opcode 230 to determine an operation to be executed by the PE 200,” col. 12:20-21). To whatever extent Elmer is argued not to use Applicant’s precise claim terminology (e.g., “association operation logic,” “preset clock cycle”). It would nevertheless have been obvious to a person of ordinary skill in the art (POSA), applying Elmer’s own express systolic-interval-based synchronous pipelining scheme, to operate Elmer’s PE array with adjacent PEs/groups beginning their respective multiply-accumulate operations offset by at least one clock cycle, this being nothing more than the way Elmer’s own disclosed architecture is expressly stated to function in order to “increase clock speeds, reduce circuit area, and/or reduce dynamic power consumption” (Abstract).).
In regard to claim 2, for the reasons given above as to claim 1m, claim 2 merely assigns index labels "(i-1)th" and "ith" to "any two pieces of feature subdata arranged adjacently" and to "any two adjacent groups of feature operation units," and recites that the association operation logic "compris[es]: a feature operation sequence of the (i-1)th piece of feature subdata precedes a feature operation sequence of the ith piece of feature subdata, and a feature operation result of the (i-1)th piece of feature subdata is applied to a feature operation process of the ith piece of feature subdata," with the (i-1)th group "connected to" the ith group according to that logic (As shown above, Elmer’s PE 112a (the “(i-1)th group”) computes its multiply-accumulate result during an earlier systolic interval and forwards that result via the column output bus (106) to the input of the adjacent, downstream PE 112b (the “ith group”), where it is used (“applied to”) PE 112b’s own accumulation (Elmer: col. 5:51-56) - i.e., the “(i-1)th group ... is connected to the ith group... according to the association operation logic,” reading directly on claim 2).
In regard to claim 3, claim 3 adds "a beater," configured "to control the feature operation process between the n groups of feature operation units through beat processing," where "a time interval between two rounds of consecutive beat processing performed by the beater includes at least one preset clock cycle," the (i-1)th group being "controlled to start the feature operation... when the beater performs a round of beat processing at a time Ti-1," and the ith group being controlled to start "when the beater performs a next round of beat processing at a time Ti adjacent to the beat processing at the time Ti-1." Elmer discloses exactly this kind of clocked, round-by-round sequencing hardware: a “computation controller (606)” that “provide[s] controls to various components of the neural network processor (602) to perform neural network computations” (Elmer col. 24:13-16), together with a per-PE “operation decoder (256)” that “decode[s] the opcode (230) to determine an operation to be executed by the PE (200)” (col. 12:20-21) once per systolic interval, thereby issuing a new control/enable signal (“a round of beat processing”) to advance each successive PE to begin its multiply-accumulate operation one clock-cycle-based systolic interval after its upstream neighbor. Applicant’s own Specification confirms that a “beater” is nothing more than such interval-sequencing clock control logic (Instant Spec. ¶¶ 0033-0034: “the beater may control a feature operation process between the n groups of feature operation units through beat processing,” where “[t]he clock cycle... is defined as a reciprocal of a clock frequency” (¶ 0022)). Because Elmer’s computation-controller/operation-decoder combination performs the identical function - issuing successive, clock-cycle-spaced control pulses that trigger each PE/group to begin its operation - It would have been obvious to a person of ordinary skill in the art before the effective filling date of the claimed invention to a person having ordinary skill in the art to recognize (or, to whatever extent needed, trivially relabel) that clocked sequencing logic as the claimed “beater” performing “beat processing,” this being purely a difference in terminology rather than in underlying circuit function.
In regard to claim 4, claim 4 recites that "the feature operation logic of the ith piece of feature subdata comprises: performing first operation processing on the ith piece of feature subdata, and applying a feature operation result of the first operation processing to a process of second operation processing," with "an input end of a first operation subunit i" configured "to receive the ith piece of feature subdata, and to perform the first operation processing," an "output end of the first operation subunit i... connected to an input end of a second operation subunit i," and "an input end of the second operation subunit i... configured to receive a first operation result of the first operation subunit i, and to perform the second operation processing on the first operation result, to obtain a feature operation result of the ith group of feature operation units." This is precisely Elmer’s disclosed PE architecture: the multiplier (208) (“first operation subunit i”) receives the streamed input feature value (“the ith piece of feature subdata”) and performs a multiplication (“first operation processing”); its multiplication result (250) is captured in delay register (251) and supplied to adder (210) (“second operation subunit i”), which performs the accumulation (“second operation processing”) using that stored multiplication result to “obtain a feature operation result of the ith group of feature operation units” (the PE’s accumulated output). Elmer confirms this connectivity: “[t]he multiplier 208 can generate a multiplication result (a product) 250 for storing in the delay register 251[; t]he adder 210 can generate an addition result 238 (a sum) based on a stored multiplication result 253 from the delay register 251” (Elmer col. 10:66-col. 11:3).
In regard to claim 5, Claim 5 requires that "the (i-1)th group of feature operation units is connected to the ith group of feature operation units" such that "the second operation subunit i-1 in the (i-1)th group... is connected to the second operation subunit i in the ith group." Elmer discloses exactly this adder-to-adder chaining: the accumulated partial sum computed by adder (210) of PE 112a (the “(i-1)th group,” specifically its “second operation subunit i-1”) is transmitted, via the “column output bus (106),” directly to the corresponding accumulation-input of the adjacent, downstream PE 112b (the “ith group,” specifically its “second operation subunit i”) (Elmer col. 5:51-56), reading on the claimed connection topology.
In regard to claim 15, claim 15 extends the staggered-start-time limitation of claim 1 from "groups" within one feature operation module to "m feature operation modules" within the systolic array, each "respectively configured to perform a feature operation on the feature data to obtain a feature operation result of the feature data under a corresponding feature operation module," with adjacent modules staggered by at least one preset clock cycle. Elmer’s systolic array is expressly disclosed as a two-dimensional array: “[f]or example, the systolic array 100 may include four PEs in each row, and four PEs in each column” (Elmer col. 4:47-49). Each column (or row) of Elmer’s PEs constitutes an independent multiply-accumulate chain that processes the same streamed input data set against a different, column-specific set of locally-supplied values, yielding a distinct output for that column - i.e., each column is “a feature operation module... configured to perform a feature operation on the feature data to obtain a feature operation result... under a corresponding feature operation module,” exactly as claimed. Because Elmer’s systolic wavefront propagates cell-to-cell, one systolic interval (clock cycle) at a time, in both the row and column directions of its 2-D PE array (this being, as reflected in the very name “systolic array,” the fundamental skewed-clocking characteristic of the architecture - (Note: see also Ng reference, the foundational Kung systolic-array patent, confirming that staggered/skewed clocking between adjacent processing cells in both array dimensions is the defining feature of any systolic array), adjacent columns (modules) necessarily begin processing the same input data staggered by at least one clock cycle, for the same reasons discussed as to claim 1.
In regard to claim 16, for the reasons given above as to claims 1 and 2, respectively. Claim 16 recites a method of “receiving feature data” and “invoking the n groups of feature operation units to perform a feature operation on the n pieces of feature subdata in a preset sequence,” which is simply the method necessarily performed by, and inherent in the operation of, the apparatus of claim 1 as disclosed by Elmer (i.e., Elmer’s systolic array necessarily “receives” the streamed input data set and “invokes” its PEs/groups to process it in the staggered, association-operation-logic-based sequence already discussed, Elmer col. 4:47-5:56). Claim 17 correspondingly recites the method-claim analog of claim 2’s (i-1)th/ith indexing and invoking limitations, met by Elmer for the same reasons given as to claim 2.
In regard to claim 18, for the reasons given above as to claim 4. Claim 18 recites the method-claim analog of claim 4’s first-operation-subunit/second-operation-subunit processing steps (“invoking the first operation subunit i to perform first operation processing... to obtain a first operation result; and invoking the second operation subunit i to perform second operation processing on the first operation result and the feature operation result of the (i-1)th group... to obtain the feature operation result of the ith group”), which is met by Elmer’s multiplier-then-adder PE pipeline for the same reasons discussed as to claim 4 (Elmer col. 10:66-11:3).
In regard to claim 20, claim 20 recites “[a] non-transitory computer-readable storage medium containing a computer program that, when being executed, causes an Artificial Intelligence (AI) processor to perform” the method of claim 16 (receiving feature data and invoking the n groups of feature operation units in the recited preset sequence). Elmer’s neural network processor (602) is disclosed as instruction/opcode-driven: each PE’s “operation decoder (256)... decode[s] the opcode (230) to determine an operation to be executed by the PE (200)” (Elmer col. 12:20-21) - i.e., the array-level “computation controller (606)... provide[s] controls... to perform neural network computations” (col. 24, ll. 13-16) - i.e., Elmer’s systolic array is orchestrated by stored instructions/opcodes (a “computer program”) that, when executed/decoded by Elmer’s AI/neural-network processor, cause it to invoke the PE array’s groups to perform the feature operation in the disclosed sequence. It is, in addition, well understood in the art - and admitted as much by Applicant’s own Specification’s generic references to “a computer-readable storage medium” storing “a computer program” executed by an “AI processor” (Instant Spec. ¶¶ 0002, 0007, 0071-0072) - that storing the control instructions for an AI accelerator’s dataflow on a non-transitory computer-readable medium is a conventional, well-known implementation choice. Claim 20 is therefore obvious over Elmer (optionally in further view of Applicant’s own admission that such storage-medium implementations were conventional) for the same reasons discussed as to claim 16.
6. Claim 6 is rejected under 35 U.S.C. § 103 as obvious over Elmer in view of the AAPA.
Claim 6 requires that "the first operation processing comprises weighting operation processing, and the second operation processing comprises merging operation processing," with "a group of the n groups of feature operation units corresponds to a weight," the weight corresponding to group i (the "ith weight") being used by "the first operation subunit i" to "perform[] the weighting operation processing on the ith piece of feature subdata... to obtain the first operation result." Elmer discloses that its multiplier (208) multiplies the streamed feature value against a value associated with that PE to “generate a first scaled input” (Elmer col. 5:51-52), i.e., a multiply-by-a-locally-supplied-value (“weighting”) operation, and its adder performs the accumulation (“merging”) of that scaled result into the running partial sum - but Elmer does not use the specific term "weight" for the multiplicand supplied to each PE. Applicant’s own Specification, however, admits that it was already well known in systolic arrays generally that "data flowing in the horizontal pulsation is usually feature data, and data flowing in the longitudinal pulsation is usually weight data" (Instant Spec. ¶ 0021), i.e., that each PE column/PE is supplied with a distinct, PE-specific "weight" value against which the streaming feature data is multiplied. It would therefore have been obvious to a POSA to understand (or configure) Elmer’s per-PE multiplicand as the well-known, art-recognized "weight" value used in weight-stationary systolic MAC arrays generally, with a reasonable expectation of success, since Elmer’s multiplier already performs the identical "scale the streamed input by a locally-available value" function that a weighting operation requires.
7. Claims 7-10, 19 are rejected under 35 U.S.C. § 103 as obvious over Elmer in view of AAPA and further in view of Oklobdzija (US No. 11,366,638).
The examiner relies on the entire teachings of Elmer and Oklobdzija references; the applicant should carefully consider the entire teachings of the above-mentioned references to better understand the examiner’s position.
In regard to claim 7, Elmer does not describe, in detail, how its generic multiplier (208) and adder (210) are internally organized to process floating-point operands (Elmer states only that its PE datapath "can include circuitry configured to multiply, store, and/or accumulate integer, floating point, reduced precision floating point, double, or other data formats," without further floating-point-specific architectural detail). Oklobdzija supplies this missing teaching: it discloses that “[t]he operands entering the multiplier are ‘normalized’ floating point numbers,” where the first bit is 1 (Oklobdzija col. 6:27-28), decomposed into a feature exponent/weight exponent and a feature mantissa (significand)/weight mantissa - “the ith weight is decomposed into a weight exponent and a weight mantissa” is squarely met by Oklobdzija’s BF16/FP32 exponent-and-significand format (col. 4:5-12, Fig. 1) used by its “first operation subunit i” analog, the FP-CS-MAC unit’s multiplier stage, which “comprises an exponent operation component” - Oklobdzija’s exponent-addition/exponent-comparison circuitry (col. 14:53-67, Fig. 7B) - “and a mantissa operation component” - Oklobdzija’s significand multiplier/shifter datapath (col. 6:37-41; col. 11:48-51, Figs. 4a-4b, 6, 7C). Oklobdzija further discloses the recited connectivity: “an input end of the exponent operation component is configured to receive the feature exponent and the weight exponent” (“[t]he Exponent part performs addition of operand exponents,” col. 6:24-25, i.e., the exponent adder that computes “emult” from the multiplier’s two exponents), with the exponent-operation result routed “to the mantissa operation component and the second operation subunit i-1” (Oklobdzija’s Exponent Comparison circuit 652 outputs its comparison/alignment result both to the significand-shift datapath and, via the accumulate stage, to the running accumulator, col. 15:6-10, 33-37), and “an input end of the mantissa operation component is configured to receive the feature mantissa, the weight mantissa, and the exponent operation result of the exponent operation component” (the significand multiplier and shifter receive the significands together with the shift/comparison result from circuit 652, col. 15:53-57). Finally, claim 7 requires that “a time at which the exponent operation component starts the exponent operation is at least one clock cycle earlier than a time at which the mantissa operation component starts the mantissa operation.” Oklobdzija expressly discloses this “early exponent compare” pipelining: the exponent addition/comparison is completed in an earlier pipeline stage specifically so that “(early exponent compare) enables use of a pipeline having an accumulator stage with a shorter critical timing path and operable at higher clock speeds” (col. 6:19-21) - i.e., Oklobdzija’s exponent path is, by design, computed at least one pipeline stage (clock cycle) ahead of the corresponding mantissa/significand-path processing that depends on it. It would have been obvious to a person of ordinary skill in the art before the effective filling date of the claimed invention to a person having ordinary skill in the art to incorporate Oklobdzija’s known floating-point exponent/mantissa decomposition and early-exponent-compare pipelining into Elmer’s generic, format-agnostic multiply-delayed-accumulate PE (which Elmer itself states is intended to support “floating point” data) in order to enable floating-point operation of Elmer’s systolic array while obtaining Oklobdzija’s own stated benefit of a shorter critical path and higher clock speed - this is simply “the combination of familiar elements according to known methods... yield[ing] predictable results.” KSR Int’l Co. v. Teleflex Inc., 550 U.S. 398, 416 (2007).
In regard to claim 8, for the reasons given as to claims 3 and 7 above. Claim 8 maps the exponent-then-mantissa-then-merge-then-output sequence of claim 7 onto four successive beats of the beater (times Ti, Ti+1, Ti+2, and Ti+3, each separated by at least one preset clock cycle). Oklobdzija’s multi-stage pipeline discloses exactly this kind of successive-stage timing: the exponent operation component (652) “starts the exponent operation” and, one or more pipeline stages later, “obtains the exponent operation result” that is then used by the mantissa/significand path, which itself later feeds the carry-save “merging” (accumulate) stage, whose result is in turn obtained and forwarded onward - Oklobdzija’s own described six-stage pipeline (“[o]ne implementation of a three mode Floating point Carry-Save MAC (FP-CS-MAC) unit comprises 6 pipeline stages,” col. 3:46-47: a first main unit (the BF16 Multiplier) implemented in two pipeline stages; “[t]he third pipeline stage is a Carry-Save Accumulate stage”; “[t]he next two stages convert the result in carry-sum format back into regular normalized sign-magnitude format”; and “[t]he last pipeline stage performs normalization and rounding to produce results,” col. 3:51-61) exhibits precisely this kind of clock-cycle-by-clock-cycle staged advancement of a computation through an exponent stage, then a mantissa stage, then a merge stage, then an output stage. Combined with Elmer’s clocked “computation controller”/“operation decoder” sequencing (mapped to the claimed “beater” performing successive rounds of “beat processing,” as discussed for claim 3), it would have been obvious to a person of ordinary skill in the art before the effective filling date of the claimed invention to a person having ordinary skill in the art to synchronize Oklobdzija’s floating-point pipeline stages to Elmer’s beat-processing clock so that each of the four sub-steps recited in claim 8 begins on a successive beat (Ti, Ti+1, Ti+2, Ti+3), for the same reasons (enabling floating-point operation at Elmer’s systolic clock rate) discussed for claim 7.
In regard to claim 9, claim 9 recites that "the exponent operation component comprises an exponent addition subcomponent and an exponent comparison subcomponent i," with the exponent addition subcomponent receiving the feature exponent and weight exponent and performing "merging processing" (i.e., addition, for the multiplication of feature-by-weight) to output a "merged exponent" to exponent comparison subcomponent i; exponent comparison subcomponent i additionally receiving, from "an exponent comparison subcomponent i-1... in an (i-1)th group of feature operation units," a "local exponent... of the exponent comparison subcomponent i-1"; comparing the merged exponent to that local exponent and outputting "an exponent having a larger value" onward to an "exponent comparison subcomponent i+1... in an (i+1)th group" as the new local exponent (i.e., propagating a running/local maximum exponent down the chain); and additionally determining, from the difference between the merged exponent and the local exponent of subcomponent i-1, "an alignment shift amount... of the exponent comparison subcomponent i," output to "the second operation subunit i-1 and the mantissa operation component." This is precisely Oklobdzija’s Exponent Comparison circuit (652): Oklobdzija discloses an exponent adder that “merge[s] the feature exponent and the weight exponent” to compute “emult” (the “merged exponent”), which is then compared, in circuit 652, against “eaccu” - the accumulator’s currently held, running exponent value (the claimed “local exponent of the exponent comparison subcomponent i-1”) - to generate condition bits (“[t]he Comparison circuit 652... generates the following condition bits that are stored in the 16-bit condition register 650: emult: Multiplier Exponent; eaccu: Accumulator Exponent; z_diff-emult and eaccu are same; mgrt-emult is greater than eaccu; agrt-eaccu is greater than emult,” col. 14, ll. 53-67; “[t]he condition register 650 stores the results of a comparison of the Eacc from summation S(i-1) and the E_mult register 520 stores the term A(i)*B(i)... during the generation of the Eacc for summation S(i),” col. 15, ll. 6-10) that both (i) select and forward the larger of the two exponents onward as the new running/local exponent for use by the next accumulate step (“[t]his output is the Eacc signal, also known as the New-Exponent, which is an input to Exponent Accumulator (Eacc) register 654,” col. 15:33-37 - the claimed output “to the exponent comparison subcomponent i+1”), and (ii) set the significand alignment-shift amount (the claimed “alignment shift amount... output... to the second operation subunit i-1 and the mantissa operation component”) supplied to the shifters (SHR_8/16/24, SHL_8) that condition the significand/mantissa data before or during the carry-save accumulation (“the select signal for the shift registers SHR8/16/24/SHL8 609 and 610,” col. 15:53-57). Elmer does not disclose this running-exponent-comparison chain (its adder (210) is generic and format-agnostic); Oklobdzija supplies it. It would have been obvious to a person of ordinary skill in the art before the effective filling date of the claimed invention to a person having ordinary skill in the art to combine Oklobdzija’s known running-maximum-exponent comparison/alignment technique with Elmer’s chained, PE-to-PE partial-sum-forwarding architecture (already discussed for claims 1-2) for the same reason discussed for claim 7 - to enable Elmer’s systolic multiply-accumulate array to correctly and efficiently accumulate a stream of floating-point partial products, which by definition requires some mechanism for tracking a running exponent Elmernd computing the corresponding alignment shift at each accumulation step, exactly the function Oklobdzija’s circuit 652 is designed to perform.
In regard to claim 10, for the reasons discussed above as to claim 9. Claim 10 recites that "the mantissa operation component comprises a mantissa multiplication subcomponent and a mantissa shift subcomponent," the multiplication subcomponent receiving the feature mantissa and weight mantissa and outputting a "mantissa multiplication result" to the shift subcomponent, which also receives the alignment-shift amount/exponent-comparison output described in claim 9, and which performs "right-shift processing on the mantissa multiplication result according to the alignment shift amount... if the merged exponent is less than the local exponent of the exponent comparison subcomponent i-1," or else outputs the (unshifted) "mantissa multiplication result... as the first operation result... if the merged exponent is greater than or equal to the local exponent." This is precisely Oklobdzija’s significand datapath: the “8x8-bit integer multiplier” (mantissa multiplication subcomponent, col. 6:37-41) computes the significand product and forwards it to the shifter circuitry (mantissa shift subcomponent) comprising SHR_8/16/24 (right-shift, used when the multiplier’s own exponent (“emult”) is less than the running accumulator exponent (“eaccu”), i.e., condition “agrt”) and a pass-through/SHL_8 path (used when “emult” is greater than or equal to “eaccu,” i.e., conditions “mgrt”/“z_diff”) (col. 14:62-67; col. 15:53-64), exactly the conditional right-shift-or-pass-through logic recited in claim 10. The same combination and motivation discussed for claims 7 and 9 applies.
In regard to claim 19, for the reasons given above as to claims 6 and 7, claim 19 recites the method-claim analog of claim 7’s weighting-operation/exponent-mantissa-decomposition limitations (“invoking the exponent operation component to perform an exponent operation on the feature exponent and the weight exponent to obtain an exponent operation result... and invoking the mantissa operation component to perform a mantissa operation on the feature mantissa, the weight mantissa, and the exponent operation result... to obtain a first operation result of the first operation subunit i”), which is met by the Elmer/Oklobdzija combination discussed above for the same reasons and with the same motivation to combine (Oklobdzija: col. 6:19-28; col. 14:53-67).
8. Claims 11-12, 14 are rejected under 35 U.S.C. § 103 as obvious over Elmer in view of Oklobdzija.
Claim 11 recites that "the second operation subunit i comprises a merging component and a shift component," the merging component receiving "the first operation result of the first operation subunit i" (the significand/mantissa product, possibly aligned) and "a feature operation result of the (i-1)th group of feature operation units output by the second operation subunit i-1" (the running accumulator sum), and performing "merging processing" on the two to obtain "an initial operation result of the ith group of feature operation units," which is then passed to the shift component (together with the two merging inputs) for normalization, ultimately producing "the feature operation result of the ith group of feature operation units," output "to the second operation subunit i+1." This maps directly onto (a) Oklobdzija’s carry-save accumulation stage - “[t]he outputs of shifter circuits 609 & 610 route to the carry-save adder (CSA) 614” (col. 11, ll. 50-51), a carry-save adder (“the merging component”) that combines the (aligned) significand product from the multiplier path with the previous accumulator value S(i-1) to produce a new, non-normalized carry-save partial sum (“the initial operation result”; “[t]he significand circuit includes a significand shifter, responsive to exponent comparison signals... to align the multiplier output significand and the feedback sum and carry values for addition,” col. 12:6-9) - followed by (b) Oklobdzija’s carry-save-to-normal conversion and LZA-based normalization circuitry (“the shift component,” discussed further as to claims 12-13 below) that normalizes that partial sum into the final accumulated result, combined with (c) Elmer’s PE-to-PE forwarding of the accumulated running sum from one PE (group i) to the next, adjacent PE (group i+1) via the column output bus, as already discussed for claims 1, 2, and 5 (Elmer col. 5:51-56). The same motivation to combine discussed for claim 7 (enabling Elmer’s systolic array to perform floating-point accumulation using Oklobdzija’s known merge-then-normalize accumulate architecture) applies here.
In regard to claim 12, claim 12 recites that "the shift component comprises a leading zero anticipator (LZA) subcomponent, a shift control subcomponent, and a shift processing subcomponent," with the LZA subcomponent receiving the first operation result of first operation subunit i and the (i-1)th group’s feature operation result (i.e., the two operands feeding the merge/carry-save-add of claim 11) and performing "leading zero anticipation" on the initial (merged) operation result "to obtain a normalized shift amount"; the shift control subcomponent, connected to the LZA output and to an output of "a first operation subunit i+1 (MEU[i+1])" that supplies "the alignment shift amount of the exponent comparison subcomponent i+1," determining a "target shift direction" and "target shift amount" by combining the normalized shift amount with that alignment shift amount; and the shift processing subcomponent then shifting the initial operation result according to the target shift direction/amount to produce the final "feature operation result of the ith group of feature operation units," output to second operation subunit i+1. This maps directly onto Oklobdzija’s LZA/normalization circuitry: “[l]eading Zero/Leading One Anticipator (second LZA/LOA circuit) 710 will calculate two numbers: number of Leading Zeroes 711 (in case the significand 716 is positive) and number of leading Ones 712 (in case the significand 716 is negative)[; d]epending on the significand sign bit 719, the correct position will be selected by the multiplexer, LZA POS Selection circuit 714 and will be stored into the register 730” (Oklobdzija: col. 17, ll. 46-52; see also col. 17, ll. 18-25: “[a] second circuit LZA/LOA 710 receives input bus 702 and bus 704[; t]he second circuit LZA/LOA 710 outputs two buses POS_P[5:0]... and POS_N[5:0]... to a third LZA POS Selection circuit 714[; t]he output of the LZA POS Selection circuit 714 is the POS[5:0] 715 which maps to the register 730 as the 6-bit position, designating the amount of shift left required to normalize significand”), which resolves these (in conjunction with alignment information already computed for, and carried forward from, the next accumulate stage’s own exponent-comparison circuit - i.e., Oklobdzija’s pipelined “early exponent compare” architecture, discussed for claim 7, in which the (i+1)th stage’s exponent-comparison/alignment result is computed early enough to be available for use by the ith stage’s normalization logic) into a single position value “Pos[5:0]” on line 723 which controls the “SHL Left shifter circuit 735” (“the shift processing subcomponent”) and is also “an input to the Exponent adder circuit 740” (col. 18:7-11), which “adjusts (increments) the exponent for the number of positions the significand is shifted to the left as indicated by the POS[5:0], and provide[s] an output on line 736 to register 748” (col. 18:11-17) (i.e., forwards updated alignment/exponent context onward, corresponding to the claimed connection to “the second operation subunit i+1”). Elmer does not disclose any LZA or floating-point normalization circuitry; Oklobdzija supplies it, and it would have been obvious to a person of ordinary skill in the art before the effective filling date of the claimed invention to a person having ordinary skill in the art to combine it with Elmer for the reasons discussed as to claim 7 (enabling floating-point operation of Elmer’s systolic accumulate chain), it being well understood that some LZA-based (or equivalent) normalization stage is a practical necessity of any floating-point adder/accumulator, as further evidenced by the substantial body of prior art directed to LZA circuits generally (see Section IX, below).
In regard to claim 14, claim 14 recites that "the feature operation module further comprises a precision control unit," whose input is configured to receive only "a feature operation result of an nth group of feature operation units" (i.e., only the last group’s output, not every group’s output) and which performs "precision control processing" on that result "to obtain a feature operation result of the feature data under the feature operation module." Oklobdzija discloses exactly this last-stage-only rounding architecture: its FP-CS-MAC unit maintains the running accumulation internally in redundant carry-save form, without rounding, across its internal pipeline stages, and performs precision-control (rounding) processing only once, at the final stage - “[a]ccording to some aspects, the final stage is pipeline-6 which performs rounding of the result to a standard Floating point Sign/Magnitude number” ('638 Patent col. 19:4-6; see also col. 3:60-61: “[t]he last pipeline stage performs normalization and rounding to produce results”) - i.e., only on the final accumulated result rather than after each individual multiply-accumulate step. Elmer does not disclose any rounding/precision-control architecture (its adder (210) is generic); Oklobdzija supplies it. It would have been obvious to a person of ordinary skill in the art before the effective filling date of the claimed invention to a person having ordinary skill in the art to combine Oklobdzija’s final-stage-only rounding technique with Elmer’s chained systolic PE array (in which the “nth group” corresponds to the last PE in the accumulate chain) because doing so was already an art-recognized way to reduce the number of rounding operations performed during an accumulation and thereby reduce both critical-path delay and hardware cost relative to rounding after every stage - precisely the tradeoff and benefit Applicant’s own Specification acknowledges was understood in the art, since the Specification’s own description of the “existing systolic array” of Fig. 2 (AAPA) admits that a “standard floating point multiplication unit (20) and a floating point addition unit (21)” each having “a precision control assembly (22)... configured to perform a rounding operation” after each row (i.e., rounding at every stage) was well known, and that reducing the number of such roundings (as in the claimed invention, and as in Oklobdzija) was a recognized design objective. The combination therefore involves nothing more than applying a known technique (single, final-stage rounding, as taught by Oklobdzija and as motivated by the very problem AAPA identifies) to a known device (Elmer’s systolic PE chain) to yield the predictable result of reduced rounding overhead.
9. Claim 13 is rejected under 35 U.S.C. § 103 as obvious over Elmer in view of Oklobdzija, and further in view of Ng (US No. 5,204,825).
The examiner relies on the entire teachings of Elmer, Oklobdzija and Ng references; the applicant should carefully consider the entire teachings of the above-mentioned references to better understand the examiner’s position.
Claim 13 recites that "the shift processing subcomponent comprises a left-shift device, a right-shift device, and a selection device," the left-shift and right-shift devices each independently receiving the target shift amount and the initial operation result and producing, respectively, a "left-shift result" and a "right-shift result," and the selection device choosing between them, according to the "target shift direction," to output "the feature operation result of the ith group of feature operation units" to second operation subunit i+1. Oklobdzija discloses a left-shifter (“SHL Left shifter circuit 735,” col. 18:7-11) used for LZA-directed normalization (discussed for claim 12) and separately discloses right-shift circuitry (SHR_8/16/24, col. 15:53-64) used elsewhere in the same FP-CS-MAC unit for exponent-difference alignment, together with condition-bit-driven selection logic that already chooses among multiple shift/pass-through datapaths depending on a comparison result. To the extent Oklobdzija’s normalization stage is not shown to use two fully parallel (left and right) shift datapaths merged by a single final selection device (rather than, e.g., a single shifter driven by a signed shift-amount value), that specific circuit-level implementation choice is expressly confirmed to have been conventional, well-known design practice by Ng, whose own background discussion of prior floating-point-adder normalization hardware describes “FIG. 1 [as] a block diagram of a prior art subtractor circuit requiring a normalization corrector” (Ng, col. 2:21-22) - i.e., shifting a result by a predicted amount and then applying “a normalization corrector” (an “adjust-by-one normalization corrector 50... required to determine whether shifted result 45 is correctly normalized, and shift it one bit if it is not,” col. 1:49-53) - i.e., a second, alternative shift/selection path used to correct the output when the predicted shift is off by one position - as the conventional pre-1991 approach that Ng’s own invention was designed to improve upon (“[t]he need for normalization corrector 50 undesirably slows down subtraction operations,” col. 1:54-55). This confirms that implementing a bidirectional (left-or-right) normalization shift using two parallel directional shift paths and a final output-selecting multiplexer, controlled by a shift-direction signal, was one of a small number of well-known, finite, and predictable circuit-design alternatives for realizing that function; it therefore would have been obvious to a person of ordinary skill in the art before the effective filling date of the claimed invention to a person having ordinary skill in the art, in implementing the Elmer/Oklobdzija combination discussed above, to realize the LZA-directed normalization shifter using the well-known left-shift-device/right-shift-device/selection-device topology (particularly given that both a right-shift datapath and a left-shift datapath, together with condition-bit-based selection logic, are already present and taught, for related purposes, within Oklobdzija’s own FP-CS-MAC unit), this being no more than “the use of a known technique to improve similar devices... in the same way.” KSR, 550 U.S. at 417.
Examiner's note:
Examiner has cited particular columns and line numbers in the references applied to the claims above for the convenience of the Applicant. Although the specified citations are representative of the teachings of the art and are applied to specific limitations within the individual claim, other passages and figures may apply as well. It is respectfully requested from the Applicant in preparing responses, to fully consider the references in entirety as potentially teaching all or part of the claimed invention, as well as the context of the passages as taught by the prior art or disclosed by the Examiner.
Conclusion
10. All claims are rejected.
11. The prior arts made of record and not relied upon are considered pertinent to applicant's disclosure.
U.S. Patent No. 4,727,503, teaches ("Systolic array"), foundational systolic-array patent describing skewed/staggered clocking between adjacent processing cells generally.
U.S. No. 9,697,463 discloses a weight-stationary systolic array (the Google TPU matrix-multiply unit) for neural-network feature-map/convolution computation, relevant to claim 1’s "feature data obtained from feature extraction" and claim 6’s weight-per-PE limitation.
U.S. Pub. No. US 2022/0350861, teaches additional systolic-array/accelerator architecture reference.
U.S. Patent No. 11,816,446 B2 / EP 4 066 100, discloses a systolic array PE supporting multiple floating-point/integer data types, relevant to claims 6-7.
U.S. Pub. No. US 2023/001514, discloses an enhanced fused multiplier-adder PE architecture for systolic arrays (carry-save partial products), relevant to claim 4-5 and 11.
U.S. Pub. No. US 2023/0206052, discloses reduced-precision PE arrays with a compensator/accumulator architecture; relevant background for claim 14.
U.S. Patent No. 4,926,369, teaches early foundational LZA patent, relevant to claims 12-13.
U.S. Patent No. 5,993,051, teaches combined leading one and leading zero anticipator - relevant to claims 12-13.
U.S. Patent No. 6,499,044, teaches leading zero/one anticipator for floating poin - relevant to claims 12-13.
U.S. Patent No. 6,594,679, teaches leading-zero anticipator having an independent sign bit determination module - relevant to claims 12-13.
U.S. Patent No. 10,061,561, discloses early-shift/masking techniques for floating-point addition rounding; considered but not relied upon because its masking architecture differs from the claimed left-shift/right-shift/selection-device structure.
U.S. Patent No. 8,489,663/US 2013/0282779, teaches additional LZA/normalization background reference.
U.S. Patent No. 11,842,486, teaches additional systolic-array/host-processor integration background.
12. Any inquiry concerning this communication or earlier communications from the examiner should be directed to examiner Raymond Phan, whose telephone number is (571) 272-3630. The examiner can normally be reached on Monday-Friday from 6:30AM- 3:00PM. The Group Fax No. (571) 273-8300.
Communications via Internet e-mail regarding this application, other than those under 35 U.S.C. 132 or which otherwise require a signature, may be used by the applicant and should be addressed to [raymond.phan@uspto.gov].
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Andrew Jung can be reached at (571) 270-3779. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
All Internet e-mail communications will be made of record in the application file. PTO employees do not engage in Internet communications where there exists a possibility that sensitive information could be identified or exchanged unless the record includes a properly signed express waiver of the confidentiality requirements of 35 U.S.C. 122. This is more clearly set forth in the Interim Internet Usage Policy published in the Official Gazette of the Patent and Trademark on February 25, 1997 at 1195 OG 89.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see hop://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free).
Any inquiry of a general nature or relating to the status of this application should be directed to the TC 2100 central telephone number is (571) 272-2100.
/RAYMOND N PHAN/
Primary Examiner, Art Unit 2175