DETAILED ACTION
Notice of AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 4/21/2026 has been entered.
Response to Amendment
Applicant’s amendment and remarks dated 4/21/2026 have been considered. Claims 2-3 are cancelled. Claims 11-13 and 16 were previously cancelled. Claims 1, 4-10, 14-15, and 17-18 are pending.
35 U.S.C. 112(a) Rejections. The rejections to claims 1-10, 14-15, and 17-18 under 35 U.S.C. 112(a) are withdrawn in view of Applicant’s amendments to the independent claims.
Response to Arguments
On page 10 of Applicant’s 4/21/2026 Amendment and remarks, with respect to the rejections under 35 U.S.C. 101, with respect to Step 2A, Prong 2, Applicant argues that as amended, the independent claims “provide a specific technical solution that improves the functioning of a computer system itself (Step 2A, Prong 2), and include an unconventional combination of elements that amounts to significantly more than any alleged abstract idea (Step 2B)”. In particular, Applicant argues:
PNG
media_image1.png
334
548
media_image1.png
Greyscale
The examiner respectfully disagrees. The entirety of claim 1 recites mental processes as explained in the detailed rejection. There is no “technical solution that improves the functioning of a computer system itself” because no actual technology pertaining to a computer system is recited in the claims.
The examiner respectfully disagrees with Applicant’s arguments that “The amended claims solve this by introducing a specific mathematical and structural architecture.” No structural architecture limitations are present in the claims. “Partitioning the input into first and second sequences, corresponding them to first and second learnable parameter matrices, and executing a parameterized non-linear feature mapping function combined with a fast Fourier transform (FFT)” are each mental processes as explained in the detailed rejections, and as admitted by Applicant, could also be considered to be mathematical relationships and calculations, which are another type of judicial exception.
The examiner acknowledges that para. 0060 of the instant specification states that “embodiments of the disclosure provide an efficient Transformer architecture that is referred to as Fourier sparse attention for Transformer (FSAT) for performing fast context-awareness.” However, this alleged architecture is not recited in the claims themselves. Moreover, the specification and drawings do not appear to disclose this alleged architecture either.
On page 10 of Applicant’s 4/21/2026 Amendment and remarks, with respect to the rejections under 35 U.S.C. 101, with respect to Step 2B, Applicant argues:
PNG
media_image2.png
254
536
media_image2.png
Greyscale
The examiner respectfully disagrees. The analyses under 35 U.S.C. 101, for Step 2B, and the analyses under 35 U.S.C. 103 for obviousness are entirely separate, and Applicant has provided no citations to the MPEP or other legal authority mandating that a finding of non-obviousness under 35 U.S.C. 103 automatically overcomes a rejection under 35 U.S.C. 101 for lack of eligible subject matter.
As explained by MPEP 2106.05(d), “Another consideration when determining whether a claim recites significantly more than a judicial exception is whether the additional element(s) are well-understood, routine, conventional activities previously known to the industry.” However, as explained herein, the entirety of the claims are mental processes, and therefore there are no “additional elements” to consider under the “well-understood, routine, conventional activities” consideration.
Moreover, as explained by MPEP 2106.05(d), the “well-understood, routine, conventional activity” consideration is not a standalone test, but rather a consideration for eligibility. The examiner is not aware of any evidence, either for or against, a finding that the claims recite “well-understood, routine, conventional activity” and therefore this consideration does not weigh in favor, or against, a finding of subject matter eligibility.
On pages 11-12 of Applicant’s 4/21/2026 Amendment and remarks, with respect to the rejections under 35 U.S.C. 103, Applicant argues that all such rejections should be withdrawn in view of the amendments to the independent claims to incorporate the allowable subject matter of now-cancelled claim 3.
The examiner agrees. All previous rejections under 35 U.S.C. 103 are hereby withdrawn.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1, 4-10, 14-15, and 17-18 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Regarding Step 1 of the Alice/Mayo framework, Claims 1 and 4-10 are directed to a method (a process), Claims 14 and 17-18 are directed to an electronic device (a machine), and Claim 15 is directed to one or more non-transitory computer readable storage media (an article of manufacture), which each fall within one of the four statutory categories of inventions.
Regarding Claim 1
Step 2A, prong 1 (Is the claim directed to a law of nature, a natural phenomenon or an abstract idea).
Claim 1 recites the following mental processes, that in each case under the broadest reasonable interpretation, covers performance of the limitation in the mind (including an observation, evaluation, judgment, opinion) or with the aid of pencil and paper, or mathematical calculations.
An information processing method, the method comprising (under the broadest reasonable interpretation, this limitation can be performed mentally by a human, for example, a human can mentally process information)
performing a feature crossing process on at least two target vectors in an input sequence of target information to obtain an output sequence of the target information; and (under the broadest reasonable interpretation, this limitation can be performed mentally by a human, for example, a human can mentally (or using pencil and paper), perform a vector cross product on 2 vectors to obtain an output sequence)
performing a feature perception process on the output sequence of the target information to obtain a target sequence of the target information, (under the broadest reasonable interpretation, this limitation can be performed mentally by a human, for example, a human can mentally process the “output sequence of the target information” (the output of the vector cross product) to obtain a target sequence (e.g., performing “processing” such as rounding numbers, performing normalization, etc.))
wherein the target sequence represents semantic information of each target object in the target information correlated to other target objects in the target information (under the broadest reasonable interpretation, this limitation can be performed mentally by a human, for example, a human can mentally review the resulting target sequence and understand that such target sequence represents sematic information for different target objects (e.g., different words) in the target information)
wherein the performing of the feature crossing process on the at least two target vectors in the input sequence of target information to obtain the output sequence of the target information comprises: (see explanation above)
determining corresponding crossed hidden states based on a feature function for the at least two target vectors in the input sequence of the target information, and (under the broadest reasonable interpretation, this limitation can be performed mentally by a human, for example, a human can mentally determine states that are hidden based on a feature function, e.g., a function that identifies that the 2nd entry in a vector is a hidden state)
determining the crossed hidden states based on a fast Fourier transform to obtain the output sequence of the target information, and (under the broadest reasonable interpretation, this limitation can be performed mentally by a human, for example, a human can mentally perform a FFT, and from the resulting output sequence, determine hidden states in the output sequence, such as designating the 2nd entry in a vector as a hidden state)
wherein the feature function comprises a parameterized non-linear feature mapping function. (under the broadest reasonable interpretation, this limitation can be performed mentally by a human, for example, a human can mentally apply a non-linear feature mapping function using parameters, such as a feature function designating only the 2nd entry in a vector as a hidden state)
wherein the determining of the corresponding crossed hidden states based on the feature function for the at least two target vectors in the input sequence of the target information comprises: (see explanation above)
determining a first sequence and a second sequence based on the input sequence of the target information, and (under the broadest reasonable interpretation, this limitation can be performed mentally by a human, for example, a human can mentally decompose an input sequence into first and second sequences)
determining the corresponding crossed hidden states based on the feature function for a first target vector in the first sequence and a second target vector in the second sequence, and (under the broadest reasonable interpretation, this limitation can be performed mentally by a human, for example, a human can mentally apply the feature function described with respect to claim 2 to target vectors with respect to the first and second sequences)
wherein the first target vector is different from the second target vector; (under the broadest reasonable interpretation, this limitation can be performed mentally by a human, for example, a human can mentally confirm that the first and second target vectors differ)
wherein in the feature function, the first target vector corresponds to a first learnable parameter matrix, and the second target vector corresponds to a second learnable parameter matrix. (under the broadest reasonable interpretation, this limitation can be performed mentally by a human, for example, a human can mentally note that the first and second target vectors correspond to learnable parameter matrices)
Regarding Step 2A, Prong 2, the claim does not include any additional elements that integrate the judicial exception into a practical application and regarding Step 2B, there are no additional elements recited that amount to significantly more than the judicial exception.
Regarding Claim 4
Step 2A, Prong 1
performing a first feature function corresponding to the first target vector and the first learnable parameter matrix based on the fast Fourier transform for real input to obtain a first feature information; (under the broadest reasonable interpretation, this limitation can be performed mentally by a human, for example, a human can mentally perform a feature function that corresponds to the first target vector and the first learnable parameter matrix based on the FFT only on real numbers)
performing a second feature function corresponding to the second target vector and the second learnable parameter matrix based on the fast Fourier transform for real input to obtain a second feature information; and (under the broadest reasonable interpretation, this limitation can be performed mentally by a human, for example, a human can mentally perform a feature function that corresponds to the second target vector and the second learnable parameter matrix based on the FFT only on real numbers)
performing a convolution transform of the first feature information and the second feature information based on an inverse fast Fourier transform for real input to obtain the output sequence of the target information. (under the broadest reasonable interpretation, this limitation can be performed mentally by a human, for example, a human can mentally perform a convolution transform on the first and second feature information based on an inverse FFT on real numbers)
Regarding Step 2A, Prong 2, the claim does not include any additional elements that integrate the judicial exception into a practical application and regarding Step 2B, there are no additional elements recited that amount to significantly more than the judicial exception.
Regarding Claim 5
Step 2A, Prong 1
deleting a same element values in a cross matrix corresponding to the output sequence of the target information, the element values being hidden states after crossing of the at least two target vectors; and (under the broadest reasonable interpretation, this limitation can be performed mentally by a human, for example, a human can mentally delete elements in a cross matrix, where such elements are hidden states)
performing the feature perception process on the cross matrix with the same element values deleted. (under the broadest reasonable interpretation, this limitation can be performed mentally by a human, for example, a human can mentally perceive features from the cross matrix after element values are deleted, such as determining that the sum of each row or column vector is a feature)
Regarding Step 2A, Prong 2, the claim does not include any additional elements that integrate the judicial exception into a practical application and regarding Step 2B, there are no additional elements recited that amount to significantly more than the judicial exception.
Regarding Claim 6
Step 2A, Prong 1
performing the feature perception process on hidden states of a target vector pair having correlation in the output sequence of the target information, the target vector pair being the target vectors for which the feature crossing have been performed. (under the broadest reasonable interpretation, this limitation can be performed mentally by a human, for example, a human can mentally perceive features on hidden states of a vector pair, such as perceiving elements in the vector pair that match as features)
Regarding Step 2A, Prong 2, the claim does not include any additional elements that integrate the judicial exception into a practical application and regarding Step 2B, there are no additional elements recited that amount to significantly more than the judicial exception.
Regarding Claim 7
Step 2A, Prong 1
performing the feature perception process on dominant elements in a cross matrix corresponding to the output sequence of the target information, and wherein the dominant elements include non-zero elements. (under the broadest reasonable interpretation, this limitation can be performed mentally by a human, for example, a human can mentally perceive features only based on non-zero elements in a cross matrix corresponding to the output sequence of the target information)
Regarding Step 2A, Prong 2, the claim does not include any additional elements that integrate the judicial exception into a practical application and regarding Step 2B, there are no additional elements recited that amount to significantly more than the judicial exception.
Regarding Claim 8
Step 2A, Prong 1
determining column indexes of the dominant elements in the cross matrix corresponding to the output sequence of the target information; (under the broadest reasonable interpretation, this limitation can be performed mentally by a human, for example, a human can mentally determine such column indexes of the non-zero elements in a cross matrix)
determining a confidence of a sparse matrix based on the column indexes, and (under the broadest reasonable interpretation, this limitation can be performed mentally by a human, for example, a human can mentally determine such a confidence, such as by using equation 8 in para. 0132 of the instance disclosure, which is also a mathematical calculation)
obtaining a sparse attention matrix based on the confidence of the sparse matrix and determination of an attention probability matrix; and (under the broadest reasonable interpretation, this limitation can be performed mentally by a human, for example, a human can mentally obtain such a sparse attention matrix, such as by multiplying the sparse matrix by the attention probability matrix and a scalar confidence value)
determining the target sequence of the target information based on the sparse attention matrix. (under the broadest reasonable interpretation, this limitation can be performed mentally by a human, for example, a human can mentally use the sparse attention matrix to determine a target sequence of the target information, such as reading-out the different rows to determine the target sequence)
Regarding Step 2A, Prong 2, the claim does not include any additional elements that integrate the judicial exception into a practical application and regarding Step 2B, there are no additional elements recited that amount to significantly more than the judicial exception.
Regarding Claim 9
Step 2A, Prong 1
wherein the performing of the feature perception process on the output sequence of the target information to obtain the target sequence of the target information is performed (under the broadest reasonable interpretation, this limitation can be performed mentally by a human, for example, a human can mentally perceive features from an output sequence to obtain a target sequence of the target information)
a gradient truncation is performed on a back-transferred positive gradient, and wherein the back-transferred gradient is determined by a loss value of the attention model and a mean value of the column indexes (under the broadest reasonable interpretation, this limitation can be performed mentally by a human, for example, a human can mentally perform the gradient truncation and then back-transfer the gradient; the examiner further notes that calculating a gradient is a mathematical calculation)
Step 2A, Prong 2
Regarding the “based on a pre-constructed attention model” limitation, such limitation is recited at a high-level of generality and amounts to no more than adding the words “apply it” (or an equivalent) with the judicial exception. In particular, the claim only recites the additional element of a pre-constructed attention model. This additional element is recited at a high-level of generality and amounts to no more than mere instructions to apply the exception using a generic computer component (a generic computing model). Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea (See MPEP 2106.05(f)).
Regarding the “wherein, when the attention model is trained,” limitation, such limitation is recited at a high-level of generality and amounts to no more than adding the words “apply it” (or an equivalent) with the judicial exception. In particular, the claim only recites the additional element of a pre-constructed attention model. This additional element is recited at a high-level of generality and amounts to no more than mere instructions to apply the exception using a generic computer component (a generic computing model). Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea (See MPEP 2106.05(f)).
Step 2B
Regarding the “based on a pre-constructed attention model” limitation, such limitation is recited at a high-level of generality and amounts to no more than adding the words “apply it” (or an equivalent) with the judicial exception, because the limitation merely provides instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. Accordingly, this additional element does not add significantly more than the judicial exception. (See MPEP 2106.05(f)).
Regarding the “wherein, when the attention model is trained” limitation, such limitation is recited at a high-level of generality and amounts to no more than adding the words “apply it” (or an equivalent) with the judicial exception, because the limitation merely provides instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. Accordingly, this additional element does not add significantly more than the judicial exception. (See MPEP 2106.05(f)).
Regarding Claim 10
Step 2A, Prong 1
determining an element vector based on the dominant elements selected from the sparse attention matrix and a corresponding value vector; and (under the broadest reasonable interpretation, this limitation can be performed mentally by a human, for example, a human can mentally determine such an element vector based on the dominant elements, such as by extracting the dominant elements into a new vector corresponding to the recited “element vector”)
accumulating the element vector into positions of the target sequence corresponding to the column indexes of the selected dominant elements to determine the target sequence of the target information. (under the broadest reasonable interpretation, this limitation can be performed mentally by a human, for example, a human can mentally accumulate the element vector into positions of the target sequence corresponding to the column indexes of the selected dominant elements to determine the target sequence of the target information)
Regarding Step 2A, Prong 2, the claim does not include any additional elements that integrate the judicial exception into a practical application and regarding Step 2B, there are no additional elements recited that amount to significantly more than the judicial exception.
Regarding Claim 14
Step 2A, Prong 1
Claim 14 recites an electronic device that corresponds to the method of claim 1, and therefore the analysis under Step 2A, Prong 1 with respect to claim 1 also applies to this claim 14. While claim 14 recites additional generic computing components (“processors”, “memory”, “instructions”), such additional generic computing components do not change the analysis under Step 2A, Prong 1.
Step 2A, Prong 2
Claim 14 recites an electronic device that corresponds to the method of claim 1. While claim 14 recites additional generic computing components (“processors”, “memory”, “instructions”), such additional generic computing components do not change the analysis under Step 2A, Prong 2.
Step 2B
Claim 14 recites an electronic device that corresponds to the method of claim 1. While claim 14 recites additional generic computing components (“processors”, “memory”, “instructions”), such additional generic computing components do not change the analysis under Step 2B.
Regarding Claim 15
Step 2A, Prong 1
Claim 15 recites one or more non-transitory computer readable storage media for storing instructions that when executed by a computer perform the method of claim 1. While claim 15 recites additional generic computing components (“non-transitory computer readable storage media”, “computer instructions”, “processor”), such additional generic computing components do not change the analysis under Step 2A, Prong 1.
Step 2A, Prong 2
Claim 15 recites one or more non-transitory computer readable storage media for storing instructions that when executed by a computer perform the method of claim 1. While claim 15 recites additional generic computing components (“non-transitory computer readable storage media”, “computer instructions”, “processor”), such additional generic computing components do not change the analysis under Step 2A, Prong 2.
Step 2B
Claim 15 recites one or more non-transitory computer readable storage media for storing instructions that when executed by a computer perform the method of claim 1. While claim 15 recites additional generic computing components (“non-transitory computer readable storage media”, “computer instructions”, “processor”), such additional generic computing components do not change the analysis under Step 2B.
Claim 17 depends from claim 14 and claims an electronic device that corresponds to the method of claim 5, and is therefore rejected for the same reasons explained above with respect to claims 5 and 14.
Claim 18 depends from claim 14 and claims an electronic device that corresponds to the method of claim 6, and is therefore rejected for the same reasons explained above with respect to claims 6 and 14.
Allowable Subject Matter
Claims 1, 4-10, 14-15, and 17-18 would be allowed over the prior art, provided that the rejections under 35 U.S.C. 101 are overcome.
The following is a statement of reasons for the indication of allowable subject matter:
Independent claims 1, 14, and 15 would be considered allowable over the prior art, if the rejections under 35 U.S.C. 101 overcome, because none of the references of record either alone or in combination fairly disclose or suggest the combination of limitations specified in independent claims 1, 14, and 15, including at least:
wherein the determining of the corresponding crossed hidden states based on the feature function for the at least two target vectors in the input sequence of the target information comprises:
determining a first sequence and a second sequence based on the input sequence of the target information, and
determining the corresponding crossed hidden states based on the feature function for a first target vector in the first sequence and a second target vector in the second sequence, and
wherein the first target vector is different from the second target vector; and
wherein in the feature function, the first target vector corresponds to a first learnable parameter matrix, and the second target vector corresponds to a second learnable parameter matrix.
The closest prior art of record discloses:
Lee-Thorp, James, et al. "Sparse mixers: Combining moe and mixing to build a more efficient bert." arXiv preprint arXiv:2205.12399 (May 9, 2021), hereinafter referenced as LEE-THORP, discloses the Fnet model that uses a Fast Fourier Transform in a mixing sublayer to operate on text token sequences. (p. 5, sections 3.2-3.3).
US 20210264449 A1, hereinafter referenced as TSAI, teaches using an XGBoost technique that reduces the level of information confusion achieved. (para. 0029).
US 20060253517 A1, hereinafter referenced as ARGENTAR, discloses finding patterns in text with respect to corresponding 2-tuples and n-tuple pairs. (paras. 0031, 0118, 0140).
Xu, Chengfeng, et al. "Recurrent convolutional neural network for sequential recommendation." The world wide web conference. 2019, hereinafter referenced as XU, teaches applying a non-linear activation function to hidden states by a neural network. (p. 3399, section 1)
However, the examiner has found that the distinct feature of the Applicant's claimed invention over the prior art is the explicit claiming of the aforementioned limitations in combination with all the other limitations as specified in claims 1, 14, and 15. In particular, one of ordinary skill would not have been motivated to implement the recited “determining the corresponding crossed hidden states based on the feature function for the at least two target vectors in the input sequence of the target information” function in the precise manner recited (using both first and second sequences, each having a respective target vector corresponding to learnable parameter matrices) without the hindsight aid of Applicant’s disclosure. Therefore, because the limitations of claims 1, 14, and 15 are not anticipated nor made obvious by the prior art of record, such claims would be allowable over the prior art if the rejections under 35 U.S.C. 101 are overcome.
Dependent Claims 14-10 and 17-18 would be allowed over the prior art because they depend from an allowable independent base claim, provided that the rejections under 35 U.S.C. 101 are overcome.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MICHAEL C LEE whose telephone number is (571)272-4933. The examiner can normally be reached M-F 12:00 pm - 8:00 pm ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Omar Fernandez Rivas can be reached at 571-272-2589. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MICHAEL C. LEE/Examiner, Art Unit 2128
Prosecution Insights