DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Remarks
The Office Action has been made issued in response to amendment filed October 14, 2025. Claims 1-4 and 7-18 are pending of which 9-18 are withdrawn. Applicant’s arguments have been carefully and respectfully considered in light of the instant amendment, and are not persuasive. Accordingly, this action has been made FINAL.
Claim Rejections – 35 USC section § 101
In an effort to overcome the 101 rejection, Applicant has amended independent claim 1 to recites “a processor configured to: transform received image data into a sequence of embeddings and a memory, communicatively coupled to the processor, the memory configured to store [[a]] the sequence of embeddings in an encoder layer of a vision transformer, the memory comprising: L encoder layers; and H attention heads in each encoder layer...". Applicant then concluded that now the claims are “tangible, physical elements in the form of a processor communicatively coupled to a physical memory, wherein the physical memory is configured in a specific way to store the sequence of embeddings in an encoder layer of a vision transformer, and wherein the physical memory is organized into L encoder layers and H attention heads” (see pages 5-6 of the Remarks).
In response, while this amendment has reduced the potentially 101 issues, they still do not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. As discussed in the previous and included in this Office Action, the Examiner lay out the reasons the claims as a whole is directed to the abstract idea of mathematics written in prose. The Examiner also showed that the additional features were not sufficient to overcome the 101 rejections because not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea.
Allowable Subject Matter
Applicant fails to include all the limitations of the objected claims. Therefore, the allowable subject matter in objected claims 5-6 are withdrawn.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-8 are rejected under 35 U.S.C. 101
Regarding independent claim 1 and dependent claims 2-4 and 7-8
Step 1 Analysis: Claim 1 is directed to a device, which falls within one of the four statutory categories.
Step 2A Prong 1 Analysis: Claim 1 recites, in part, “H attention heads in each layer in which h' of the attention heads comprise an attention mask added before a Softmax operation, and h of the attention heads comprise unmasked attention heads, wherein the attention mask added before the Softmax operation extracts local information and the unmasked attention heads are based on global dependencies, and in which H=h'+h”, as drafted, are elements that, under broadest reasonable interpretation, covers “mathematical concepts” and “explicit mathematics” grouping of abstract ideas.
The limitation of “H attention heads in each encoder layer in which h' of the attention heads comprise an attention mask added before a Softmax operation, and h of the attention heads comprise unmasked attention heads, wherein the attention mask added before the Softmax operation extracts local information and the unmasked attention heads are based on global dependencies, and in which H=h'+h” is a mathematical equation written in prose and explicit mathematics.
The limitation of “wherein the attention mask comprises at least one of a hard mask or a soft mask, the hard mask selecting one or more closest neighbors of a patch of the received image data and the soft mask multiplying a set of weights of one or more closest neighbors of the patch by a magnification factor” is a mathematical equation written in prose and explicit mathematics.
Accordingly, the claim recites an abstract idea.
Step 2A Prong 2 Analysis: The judicial exception is not integrated into a practical application. Particularly, the claim recites the following additional limitations:
The additional elements “a memory, communicatively coupled to the processor, the memory configured to store the sequence of embeddings in an encoder layer of a vision transformer, the memory”
The limitations “a memory, communicatively coupled to the processor” and “the memory configured to store the sequence of embeddings in an encoder layer of a vision transformer, the memory” are recited at a high level of generality, i.e., as a generic processor and memory performing a generic computer function of processing data (memory for the transformer and processor for executing the steps for training the transformer). This generic processor and memory limitations are no more than mere instructions to apply the exception using generic computer components.
Accordingly, these additional elements do not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. Please see MPEP §2106.04.(a)(2).III.C.
The claim element of “L encoder layers” impart additional elements, the additional elements merely constitute pre-solution activity involving the transformation of input data into a representation that captures context and dependencies. Such extra-solution activity does not integrate the abstract idea into a practical application. Please see MPEP §2106.05(g). Accordingly, these additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea. The claim as a whole is directed to an abstract idea. Please see MPEP §2106.04.(a)(2).III.C.
In view of the of the foregoing, the additional step does not integrate the abstract idea into a practical application.
For all of the foregoing reasons, claim 1 does not comply with the requirements of 35 USC 101.
Accordingly, the dependent claims 2-8 do not provide elements that overcome the deficiencies of the independent claim 1.
Claim 2 recites in part “wherein at least one attention mask multiplies a Query vector and a Key vector to form element-wise products“ do not overcome the rejection of the parent claim 1 as stated above because the additional elements does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea.
Claim 3 recites in part “wherein at least one attention mask comprises a 3x3 attention mask “do not overcome the rejection of the parent claim 1 as stated above because the additional elements does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea.
Claim 4 recites in part “wherein at least one attention mask comprises a 5x5 attention mask “do not overcome the rejection of the parent claim 1 as stated above because the additional elements does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea.
Claim 7 recites in part “wherein a learnable bias a is added to at least one attention mask“ do not overcome the rejection of the parent claim 1 as stated above because the additional elements does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea.
Claim 8 recites in part “wherein the learnable bias a is added to diagonal elements of the at least one attention map“ do not overcome the rejection of the parent claim 1 as stated above because the additional elements does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea.
Examiner Notes
The claims as a whole is directed to a data structure stored or residing in a memory, i.e. printed matter. The transformer disclosed does is not being used to process an image or video or data of any kind. Thus, the Examiner only need to find prior art which discloses a transformer without the particulars disclosed by Applicant to meet the limitations of the claims. The Examiner has determined as required the printed matter analysis, that the claim subject matter is directed toward printed matter. Further, since there is no new and nonobvious functional relationship between the printed matter and the substrate, the examiner have not given printed matter any patentable weight. See In re Lowry, 32 F.3d 1579, 1583-84, 32 USPQ2d 1031, 1035 (Fed. Cir. 1994); In re Ngai, 367 F.3d 1336, 70 USPQ2d 1862 (Fed. Cir. 2004); In re Gulack, 703 F.2d 1381, 1385, 217 USPQ 401, 403-04 (Fed. Cir. 1983) (see MPEP 2111.05). However, for the purposes of compact prosecution, the Examiner has mapped the claims to the references as shown below.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-2 and 7 are rejected under 35 U.S.C. 103 as being unpatentable over
Dosovitski et al (NPL titled: AN IMAGE IS WORTH 16X16 WORDS: TRANSFORMERS FOR IMAGE RECOGNITION AT SCALE) in view of JIAO et al (Pub No.: 20220067533).
As to independent claim 1, Dosovitski discloses a device (vision transformer – see abstract) comprising: a processor (section 2, [p][002]) configured to:
transform received image data into a sequence of embeddings (token embeddings – see section 3.1); and a memory (see section 3.2), communicatively coupled to the processor, the memory configured to store the sequence of embeddings (token embeddings – see section 3.1) in an encoder layer (transformer encoder – see Fig 1) of a vision transformer (vision transformer – see abstract), the memory comprising:
L layers encoder (see section 3.1, [p][002]); and H attention heads in each layer in which h' of the attention heads comprise an attention mask added before a Softmax operation (relative attention as a bias term and add it to the logits of the main attention (content-based attention) before applying the softmax – see section D.4, [p][001]), the unmasked attention heads are based on global dependencies (globally average-pooling – see section D.3, [p][002]); however, Dosovitski does not expressly disclose h of the attention heads comprise unmasked attention heads, and in which H=h^'+h and wherein the attention mask added before the Softmax operation extracts local information and wherein the attention mask comprises at least one of a hard mask or a soft mask, the hard mask selecting one or more closest neighbors of a patch of the received image data and the soft mask multiplying a set of weights of one or more closest neighbors of the patch by a magnification factor3
JIAO discloses a transformer based neural network including h of the attention heads comprise unmasked attention heads (another attention network 110 that does not make use of a mask data structure – see [p][0042]), and in which H=h^'+h (combination mechanism, 914 – see Fig 9) and wherein the attention mask added before the Softmax operation extracts local information (effectively determining the influence between pairs of data items within local neighborhoods of data items – see [p][0054]) wherein the attention mask comprises at least one of a hard mask (see [p][0074]) or a soft mask, the hard mask selecting one or more closest neighbors of a patch of the received image data (modified attention data structure 612 accounts for influence of neighboring data items in the sequence of data items – see [p][0074]) and the soft mask multiplying a set of weights of one or more closest neighbors of the patch by a magnification factor .
Dosovitski and JIAO are combinable because they are from the same field of endeavor of utilizing a vision transformer for processing an image. It would have been obvious to one of ordinary skill in the art before the effective filling date of the invention to have incorporated the a transformer based neural network of JIAO into the vision transformer of Dosovitski in order increased accuracy without markedly increasing the number of machine-trained parameter values used by the transformer-based neural network (see [p][0008]). Such a modification is the result of combining prior art elements according to known methods, they would have performed as expected, and the results would have been predictable.
As to claim 2, Dosovitski teaches the vision transformer, wherein at least one attention mask multiplies a Query vector and a Key vector to form element-wise products (see section Appendix A).
As to claim 7, Dosovitski teaches the vision transformer, wherein a learnable bias a is added to at least one attention mask (relative attention as a bias term and add it to the logits of the main attention - see section D.4, [p][001]).
Claims 3-4 are rejected under 35 U.S.C. 103 as being unpatentable over
Dosovitski et al (NPL titled: AN IMAGE IS WORTH 16X16 WORDS: TRANSFORMERS FOR IMAGE RECOGNITION AT SCALE) in view of JIAO et al (Pub No.: 20220067533) as applied to claim 1 further in view of Nakai et al (NPL titled: Att-DARTS: Differentiable Neural Architecture Search for Attention).
As to claims 3-4, Dosovitski teaches that the kernel size 3 x 3 (see section D.6); however, the combination of Dosovitski and JIAO does not expressly disclose the vision transformer, wherein at least one attention mask comprises a 3×3 attention mask and wherein at least one attention mask comprises a 5x5 attention mask.
Nakai discloses differential neural architecture including wherein at least one attention mask comprises a 3×3 attention mask (attention module is inserted after each operation. sep conv 3x3 and sep conv 5x5 denote 3×3 and 5×5 separable convolutions - see Fig 4 description) and wherein at least one attention mask comprises a 5x5 attention mask (attention module is inserted after each operation. sep conv 3x3 and sep conv 5x5 denote 3×3 and 5×5 separable convolutions – see Fig 4 description).
Dosovitski, Nakai and JIAO are combinable because they are from the same field of endeavor of utilizing deep learning machines for processing an image. It would have been obvious to one of ordinary skill in the art before the effective filling date of the invention to have incorporated the differential neural architecture of Nakai into the vision transformer of Dosovitski as modified by JIAO to use attention modules to improve the performances of deep learning networks by discarding information of no interest (see abstract). Such a modification is the result of combining prior art elements according to known methods, they would have performed as expected, and the results would have been predictable.
Allowable Subject Matter
Claim 8 would be allowable if rewritten to overcome the rejection(s) under 35 U.S.C. 101, set forth in this Office action and to include all of the limitations of the base claim and any intervening claims.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Inquiries
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ANDRAE S ALLISON whose telephone number is (571)270-1052. The examiner can normally be reached on Monday-Friday 9am-5pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Chineyere Wills-Burns, can be reached on (571) 272-9752. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ANDRAE S ALLISON/Primary Examiner, Art Unit 2663 June 9, 2025