Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Preliminary Amendment
The preliminary amendment filed 02/03/2023 does not comply with the requirements of 37 CFR 1.121(c)(2). Eight claims in the preliminary amendment are labeled as Currently Amended, but there is no underlining or strikethrough to show the changes to the claims. A marked up version of the original filing that was used for examination, including strikethrough where language is removed and margin edits where text is added is included with this Office action as an appendix. Any amendment submitted must be based on the attached amendment with regards to the identifiers per 37 CFR 1.121(c). Any future response that does not comply with the requirements of 37 CFR 1.121 may be held noncompliant.
Drawings
Figure 1 should be designated by a legend such as --Prior Art-- because only that which is old is illustrated. See MPEP § 608.02(g).
Figure 2 is objected to under 37 CFR 1.84(o) for lacking suitable descriptive legends. Examiner suggests using the labels as described in paragraph [0033] of the specification.
Figures 4 and 5 are objected to under 37 CFR 1.84(p) for not having legible text in the drawings. The mathematical notation in the drawings is not legible.
Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. The figure or figure number of an amended drawing should not be labeled as “amended.” If a drawing figure is to be canceled, the appropriate figure must be removed from the replacement sheet, and where necessary, the remaining figures must be renumbered and appropriate changes made to the brief description of the several views of the drawings for consistency. Additional replacement sheets may be necessary to show the renumbering of the remaining figures. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
Claim Objections
Claim 9 is objected to because of the following informalities: “read from the secondary main memory” should read “read from the secondary . Appropriate correction is required.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitations use a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitations are: the first and second operation modules in claims 1 and 6, which are being interpreted as being implemented by hardware or software per paragraph [0062] of the specification.
Because these claim limitations are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, they are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(d):
(d) REFERENCE IN DEPENDENT FORMS.—Subject to subsection (e), a claim in dependent form shall contain a reference to a claim previously set forth and then specify a further limitation of the subject matter claimed. A claim in dependent form shall be construed to incorporate by reference all the limitations of the claim to which it refers.
The following is a quotation of pre-AIA 35 U.S.C. 112, fourth paragraph:
Subject to the following paragraph [i.e., the fifth paragraph of pre-AIA 35 U.S.C. 112], a claim in dependent form shall contain a reference to a claim previously set forth and then specify a further limitation of the subject matter claimed. A claim in dependent form shall be construed to incorporate by reference all the limitations of the claim to which it refers.
Claims 5 and 10 are rejected under 35 U.S.C. 112(d) or pre-AIA 35 U.S.C. 112, 4th paragraph, as being of improper dependent form for failing to further limit the subject matter of the claim upon which it depends, or for failing to include all the limitations of the claim upon which it depends. Claim 5's recitation of the LSTM parameter matrix comprising the first and second submatrices is already recited in claim 1, and therefore does not further limit the subject matter of the claim upon which it depends. Claim 10 is a method claim that corresponds to apparatus claim 5 and has the same issue with regards to its dependency on claim 6. Applicant may cancel the claims, amend the claims to place the claims in proper dependent form, rewrite the claims in independent form, or present a sufficient showing that the dependent claims complies with the statutory requirements.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-10 are rejected under 35 U.S.C. 103 as being unpatentable over colah's blog, “Understanding LSTM Networks”, hereinafter Olah; in view of Smith et al., “Anatomy of High-Performance Many-Threaded Matrix Multiplication”, hereinafter Smith; and further in view of Hennessy et al., Computer Architecture: A Quantitative Approach, hereinafter Hennessy.
Regarding claim 1, Olah discloses for each of the K frames, the second operation module is operable to: cause the processor to compute a second intermediate result vector corresponding to each frame according to a second submatrix of the LSTM parameter matrix, the first intermediate result vector and an LSTM output vector of a previous frame; and update an LSTM gated vector and an LSTM state vector, and compute an LSTM output vector of a current frame according to the first intermediate result vector and the second intermediate result vector (p. 3, “repeating module in an LSTM”), but does not disclose either the reading of parameters into cache and doing a multiply-accumulate operation or the memory hierarchy.
Smith discloses an algorithm which is operable to read input vectors of K frames from a current layer into the processor cache and read one row after another from a first submatrix of a LSTM parameter matrix into the processor cache, to enable the processor to perform an multiply-accumulate operation between the input vectors of the K frames and one row after another of the first submatrix, until traversing all rows of the first submatrix to obtain a first intermediate result vector corresponding to each of the K frames (Figure 1), wherein K is greater than 1 and K is selected such that sizes of the input vectors of the K frames and of one row of the first submatrix of the LSTM parameter matrix are smaller than a size of the processor cache (Section II, ¶ 3).
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to modify the LSTM as disclosed by Olah to use the cache aware matrix multiplication algorithm as disclosed by Smith because Smith’s algorithm maximizing cache allows faster access to data when performing matrix multiplication compared to repeatedly reading data from memory or storage (Hennessy, Figure 2.1(b)).
Smith does not disclose the memory hierarchy of a computer. Hennessy discloses a system comprising a processor, a processor cache, a main memory, a secondary memory, a first operation module and a second operation module, wherein an access speed of the processor cache is higher than that of the main memory, and an access speed of the main memory is higher than that of the secondary memory (Figure 2.1(b)).
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to use the memory hierarchy as disclosed by Hennessy to the system as disclosed by Olah and Smith because “a memory hierarchy […] takes advantage of locality and trade-offs in the cost-performance of memory technologies.” (p. 72).
Regarding claim 2, Smith discloses the device according to claim 1, wherein the second operation module is operable to read the first intermediate result vector of the current frame and the LSTM output vector of the previous frame into the processor cache, and enable the processor to access the second submatrix in one of the main memory and the secondary memory (Figure 1), and Olah discloses thereby the processor computes the second intermediate result vector for each frame according to the second submatrix of the LSTM parameter matrix, the first intermediate result vector and the LSTM output vector of the previous frame (Page 9, Ct = ft * Ct-1 + (1 – ft) * Ct).
Regarding claim 3, Smith discloses the device according to claim 1, wherein the first submatrix of the LSTM parameter matrix of the current layer is stored in the main memory (Figure 1).
Regarding claim 4, while Smith does not disclose storing the layer in the secondary memory, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to use a secondary memory instead of a primary memory as disclosed by Hennessy to re-use the parameters on a system reboot, because storage is not cleared the same way RAM is when a system boots.
Regarding claim 5, as discussed in the 35 U.S.C. 112(d) rejection above, it does not limit the scope as claim 1 and is rejected for the same reasons as claim 1.
Regarding claims 6-10, they are method claims that correspond to apparatus claims 1-5, respectively, and are rejected for the same reasons.
Discussion of Pertinent Art
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Zhang et al. (US 11,526,728) discloses splitting neural network data across cache layers to maximize throughput and the scheduling of performing LSTM operations into phases dependent on past outputs and independent of past outputs. Nurvitadhi et al. (US 2018/0189638) discloses techniques for streamlining LSTM operation by mapping the algorithm to a flow graph to best optimize the hardware, but does not use a cache-aware technique for multiply-accumulate.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Matthew Strapp whose telephone number is (571)272-9343. The examiner can normally be reached Monday-Friday 8:00 AM-4:00 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Andrew Caldwell can be reached at (571)272-3702. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/M.S./
Matthew StrappExaminer, Art Unit 2182 (571)272-9343
/ANDREW CALDWELL/Supervisory Patent Examiner, Art Unit 2182
