DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
The action is responsive to Applicant’s amendment filed on 5/6/2026.
Claims 1, 4-8, 11-28 are pending.
Claims 2-3, 9-10 are cancelled.
Response to Arguments
Applicant’s arguments with respect to the rejections previously made and the amended claims filed on 5/6/2026 have been fully considered. In view of the claim amendments, the rejections are being updated accordingly.
35 USC 101 Rejections
Applicant’s arguments have been fully considered.
In response to the arguments, it is submitted that amended claims further empathized that the claimed process is directed to mathematic relationships in the Mathematical Concept group of the abstract ideas.
As stated by the Applicant in the remarks, amended claim 1 recites "calculating, by the attention computation module, an accurate attention score for only the candidates to perform a self-attention operation." The specification explains that the attention calculation module 113 may calculate an accurate attention score only for the selected candidate keys… normalizing, by the output module, the attention scores and storing normalized values in a matrix memory….The specification further explains that when the attention score value is approximated as in Equation 4, the operation in which d multiply-and-accumulates have been originally required is changed to a combination of a simple bitwise xor, a lookup table reference (cosine value), and a multiplication. As-Filed Specification, paragraph [0042] …”, the claim may be intended to
specific technical transformation that reduces computational complexity, which constitutes a concrete technical improvement rather than an abstract mathematical relationship.
However, as indicated by at least the underlined elements, both the claim and the specification in Applicant’s remarks are directed to an abstract idea having different mathematic relationships between different data elements without significantly more.
E.g. calculating accurate attention score for candidate data elements using a mathematical equation, and usage of mathematical value (e.g. cosine value).
Yet, neither the claim nor the specification indicates how these mathematical relationships would be directed to a specific technical transformation that reduces computational complexity, which constitutes a concrete technical improvement. The outcome claimed calculations is merely a data value of an attention score that is not functionally involve in any of technical transformation or improvement, which indicate that claimed element(s) in any of the claimed step would not have any impact on any technical transformation or improvement.
Thus, it is submitted that the amended claims fail to overcome the 35 USC 101 rejections, see the updated claim analysis below for detail.
35 USC 103 Rejections
Applicant’s arguments have been fully considered.
In response to the arguments, it is submitted that in view of the claim amendment, the rejections as set forth in the previous office action are hereby withdrawn.
Claim Objections
Claim 1 is objected to because of the following informalities: improper amendment marking for previously presented limitation of “converting the Hamming distance between the query hash and the key hash into an angle and performing a bias removal on the angle”. Appropriate correction is required.
Claim 18 is objected to because of the following informalities: fail to spell out acronym “XOR”. Appropriate correction is required.
Claim 21 is objected to because of the following informalities: fail to spell out acronym “IDs”. Appropriate correction is required.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1, 4-8, 11-28 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Independent claims 1 and 8 each recites a process of reducing memory access operation by generating hash values of query and keys, calculating a similarity estimate, selecting a subset of keys as candidates by comparing the similarity estimate with a threshold, calculating scores for the candidates, and normalizing the attention scores. The calculating the similarity estimate includes (i)calculating a Hamming distance, (ii)converting the Hamming distance into an angle with bias removal, (iii)calculating a cosine function value, and (iv) multiplying the cosine by a normalized key value to obtain an inner product.
The claimed process is similar to a method of mathematic relationships, which is in one of the groupings of abstract ideas (i.e. Mathematical Concepts) according to Prong One in Step 2A of the 2019 Patent Subject Matter Eligibility Guidance since the claimed steps are directed a series of mathematic calculations with different types of data relationships.
E.g., the claimed similarity estimate calculation involves multiple mathematical calculations with respect to different type of data elements, including a Hamming distance calculation and a cosine function calculation as claimed.
Also, the performance of a self-attention operation involves mathematic computation as indicated in the specification of instant application (e.g., para [0017-0019]).
The additional steps of “determining access on keys” and “storing normalized values” are directed to recite insignificant extra-solution activities at Step 2A Prong Two, and also would be well-understood, routine, and conventional at Step 2B. This is nothing more than gathering and receiving information on which the process of mathematic relationships is performed.
The claimed steps with mathematic relationships do not render the claims to a practical application because the claimed process does not show how performing a series of mathematic relationships to reduce memory access operations and multiply-accumulate operations would direct the claimed process to a particular useful application.
Merely reciting amended limitations of “reducing memory access operations and multiply-accumulate operations compared to performing self-attention operation on all of a plurality of keys by” performing a series of mathematical relationships recited in claim does not show how the memory access operations and multiply-accumulate operations are being reduced rendering the claimed process to a particular useful application. Since none of the claimed mathematical relationships performances involves in any memory access operations or multiply-accumulate operations relative to the performing self-attention operation on all of a plurality of keys, nor the claim recited any steps on the memory access operations and multiply-accumulate operations are being impacted.
The claims do not show how the mathematical relationships performances would add the necessary degree of specificity to the independent claims to integrate the alleged abstract idea into a practical application and to satisfy of the Step 2A of the patent eligibility analysis since there is no to practical application being generated from the mathematical relationship.
The additional steps of “determining access on keys” and “storing normalized values” are directed to recite insignificant extra-solution activities at Step 2A Prong Two, and also would be well-understood, routine, and conventional at Step 2B. This is nothing more than gathering and receiving information on which the process of mathematic relationships is performed, which would not be integrated into to a practical application or necessary improvement that would impose meaningful limit on the judicial exception.
The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. Additional elements (e.g. queries, keys, candidates) are directed to types of information, which do not impose a meaningful limit on the judicial exception, such that the claims are more than a drafting effort design to monopolize exception, because the claimed steps could be performed in a same manner to achieve the same outcome with other types of information other than the ones being used in the claims.
Hence, the claims do not include additional elements or the combination of the elements are sufficient to amount to significantly more than the judicial exception and fail to integrate the judicial exception into practical application according to Prong Two in Step 2A of the 2019 Patent Subject Matter Eligibility Guidance because the claimed elements or their combination do not impose any meaningful limits on practicing the abstract idea.
Further, in view of Step 2B of the 2019 Patent Subject Matter Eligibility Guidance, it is determined that the computing elements (such a memory, processor, hash computation module, candidate selection module, an attention computation module, and an output module, matrix ma) in the claim amount to no more than usage of a generic computing system having a generic computing components operating an generic manner of parallel processing, which fails to provide an inventive concept or significantly more than abstract idea because the elements do not necessary improve the functional of a computing system or an improvement to a technical field since network computing is well known.
Dependent claims 4-6, 11-12, and 16-28 recite additional mathematic relationships with a series of mathematic calculations, including perform an inner product operation between matrixes; adjust threshold(s) based on a degree of approximation; filter key; calculate a norm value; perform sign random projection, binary hash vector, angular distance, bitwise XOR operation; multiply an orthogonal matrix; multiply the threshold; correct angle by a biased angle; generate the hash values by k unit vectors randomly determined in a d-dimensional vector space, wherein the hash function for a vector x is defined as h(x) = (hvi(x), hv2(x), ..., hvk(x)), where hv(x) = sign(v - x).
These mathematic calculations further emphasizes that the claimed process is directed to the abstract idea of mathematic relationships. In addition, additional elements (e.g. threshold, structure, lookup table, current query) recited in the dependent claims are directed to types of information materials. The information materials which do not impose a meaningful limit on the judicial exception, such that the claim is more than a drafting effort design to monopolize exception, because the claimed steps could be performed in a same manner to achieve the same outcome with other types of information other than the ones being used in the claims as stated above.
Additionally, the computing elements (such as computer program product, instructions) in the claim 4-7 and 11-28 amount to no more than usage of a generic computing system having a generic computing components, which fails to provide an inventive concept or significantly more than abstract idea because the elements do not necessary improve the functional of a computing system or an improvement to a technical field since network computing is well known.
Thus, for at least the reasoning above, the pending claims are not patent eligible.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 1, 4-8, 11-28 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
The term “accurate attention” in independent claims 1 and 8 is a relative term which renders the claim indefinite. The term “accurate attention” is not defined by the claim, the specification does not provide a standard for ascertaining the requisite degree, and one of ordinary skill in the art would not be reasonably apprised of the scope of the invention.
In addition to incorporate the deficiency of claim 1 stated above, claim 4 also recites “the at least one processor calculates an attention score by performing an inner product operation” that contradicts with limitation of “a hardware module comprising a hash computation module, a candidate selection module, an attention computation module, and an output module operating… calculating, by the attention computation module, an accurate attention score for only the candidates to perform a self-attention operation; and normalizing, by the output module, the attention scores… and at least one processor…” recited in claim 1. The contradiction is rendering claim 4 being further indefinite.
In addition to incorporate the deficiency of claim 1 stated above, claim 6 also recites “wherein the at least one processor includes a hash computation module that generates hash values of queries and keys, a candidate selection module that filters keys based on the one or more thresholds, an attention computation module that generates attention scores for the candidates, and an output module, and the memory comprises a hash memory and a matrix memory…” that contradicts with limitation of “a hardware module comprising a hash computation module, a candidate selection module, an attention computation module, and an output module operating… calculating, by the attention computation module, an accurate attention score for only the candidates to perform a self-attention operation; and normalizing, by the output module, the attention scores… and at least one processor …” …” recited in claim 1. The contradiction is rendering claim 6 being further indefinite.
In addition to incorporate the deficiency of claim 8 stated above, claim 13 also recites “wherein at least one processor of an electronic device operating the calculating the similarity estimate, the selecting the subset of keys of the plurality of keys as the candidates, and performing the self-attention operation includes a hash computation module that generates hash values of queries and keys, a candidate selection module that filters keys based on the one or more thresholds, an attention computation module that generates attention scores for the candidates, and an output module, and a memory of the electronic device…” that contradicts with limitation of “the method performed by a hardware module comprising a hash computation module, a candidate selection module, an attention computation module, and an output module…the method comprising… calculating, by the candidate selection module, a similarity estimate …selecting, by the candidate selection module, a subset of keys…” recited in claim 8. The contradiction is rendering claim 13 being further indefinite.
Each of the dependent claims not specifically addressed is being rejected for incorporate the deficiency of the claim it depends on.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Michelle Owyang whose telephone number is (571)270-1254. The examiner can normally be reached Monday-Friday, 8am-6pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Charles Rones can be reached at (571)272-4085. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MICHELLE N OWYANG/ Primary Examiner, Art Unit 2168
Prosecution Insights