The present application, filed on or after 16 March 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
This office action is in response to Applicant’s submission filed on 20 December 2023. THIS ACTION IS NON-FINAL.
Status of Claims
Claims 16-29 are pending.
Claims 1-15 are cancelled.
Claims 22 includes limitations interpreted under 35 U.S.C. 112(f), because it uses a generic placeholder coupled with functional language without reciting sufficient structure to achieve the function.
Claims 22-27 are rejected under 35 U.S.C. 112(a) as failing to comply with the written description requirement.
Claims 22-27 are rejected under 35 U.S.C. 112(b) as indefinite.
Claim 16-29 are rejected under 35 U.S.C. 101 for being directed to a judicial exception (i.e., a law of nature, a natural phenomenon, or an abstract idea) without significantly more.
There is no art rejection for claims 16-29.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
In claim 22, claim limitations "1x1 convolution module", "attention and aggregation module", “shift and summation module”, “addition module”, have been interpreted under 35 U.S.C. 112(f), because they use a generic placeholder coupled with functional language without reciting sufficient structure to achieve the function. The specification does not provide descriptions of the structure of these elements.
If applicant wishes to provide further explanation or dispute the examiner’s interpretation of the corresponding structure, applicant must identify the corresponding structure with reference to the specification by page and line number, and to the drawing, if any, by reference characters in response to this Office action.
If applicant does not intend to have the claim(s) limitations treated under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112 , sixth paragraph, applicant may amend the claim(s) so that it/they will clearly not invoke 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, or present a sufficient showing that the claim recites/recite sufficient structure, material, or acts for performing the claimed function to preclude application of 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
For more information, see MPEP § 2173 et seq. and Supplementary Examination Guidelines for Determining Compliance With 35 U.S.C. 112 and for Treatment of Related Issues in Patent Applications, 76 FR 7162, 7167 (Feb. 9, 2011).
Claim Rejections - 35 USC § 112
112(b) Rejection
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
A claim is indefinite if, when read in light of the specification, it fails to inform, with reasonable certainty, those skilled in the art about the scope of the invention. Nautilus, Inc. v. Biosig Instruments, Inc., 110 USPQ.2d 1688, U.S. Supreme Court (2014).
Claims 22-27 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor, or for pre-AIA the applicant regards as the invention.
Regarding claim 22, "1x1 convolution module", "attention and aggregation module", “shift and summation module”, “addition module”, the specification does not provide sufficient structural / procedural description to determine the scope of these limitations, the claims are therefore indefinite.
Regarding claims 23-27, which depend on above rejected claim 22, are rejected for the same reason.
35 U.S.C. 112(a) Rejections
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
Claims 22-27 are rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for pre-AIA the inventor(s), at the time the application was filed, had possession of the claimed invention.
Regarding claims 22, "1x1 convolution module", "attention and aggregation module", “shift and summation module”, “addition module”, the specification does not provide sufficient structural / procedural description to determine the scope of these limitations”, the specification does not provide sufficient structural / procedural description for these limitations. The claims are therefore rejected for failing to comply with the written description requirement.
Regarding claims 23-27, which depend on above rejected claim 22, are rejected for the same reason.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Judicial Exception
Claims 16-29 of the claimed invention are directed to a judicial exception, an abstract idea, without significantly more.
Regarding claims 16-29,
(Independent Claims) With regards to claim 16 / 22,28 / 29,
Step 1: The claim recites a process / machine / article of manufacturing, which falls into one of the statutory categories.
Step 2A – Prong 1: the claim, in part, recites: “projecting input visual data into a plurality of intermediate feature maps by performing a plurality of 1x1 convolution operations; generating an attention weighted map by performing attention and aggregation operations on the plurality of intermediate feature maps; generating a convolved feature map by performing shift and summation operations on the plurality of intermediate feature maps; and adding the attention weighted map and the convolved feature map based on at least one scalar” (mental process and/or math concept), as drafted, is a process that, under its broadest reasonable interpretation, covers mathematical concepts but for the recitation of generic computer components. That is, the steps of “projecting … by performing a plurality of 1x1 convolution operations”, “generating …. by performing attention and aggregation operations”, “generating …. by performing shift and summation operations”, “adding the attention weighted map and the convolved feature map”, based on their broadest reasonable interpretation, describe mathematical relationships and algorithms. Mathematical relationship and algorithms have been found by the courts to be abstract ideas, e.g., see MPEP 2106.04(a)(2) A. Mathematical Relationships, iv. organizing information and manipulating information through mathematical correlations, Digitech Image Techs., LLC v. Electronics for Imaging, Inc., 758 F.3d 1344, 1350, 111 USPQ2d 1717, 1721 (Fed. Cir. 2014). The patentee in Digitech claimed methods of generating first and second data by taking existing information, manipulating the data using mathematical functions, and organizing this information into a new form. The court explained that such claims were directed to an abstract idea because they described a process of organizing information through mathematical correlations, like Flook's method of calculating using a mathematical formula. 758 F.3d at 1350, 111 USPQ2d at 1721. If a claim limitation, under its broadest reasonable interpretation, covers mathematical relationships, then it falls within the “Mathematical Concepts” grouping of abstract ideas.
Accordingly, the claim recites an abstract idea.
Step 2A – Prong 2: This judicial exception is not integrated into a practical application. In particular, the claim recites the additional elements of “an apparatus for computer vision processing …”, “… convolution module”, “attention and aggregation module”, “shift and summation module”, “addition module”, “… memory and processor coupled to the memory…”, “non-transitory computer readable medium on which is stored computer code for computer vision processing …’, which is mere instructions to implement an abstract idea using generic computing device, or merely uses a computer as a tool to perform an abstract idea (see MPEP 2106.05(f)). Accordingly. the additional elements individually or in combination do not integrate the judicial exception into a practical application. The claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above, the claim recites the additional elements of “an apparatus for computer vision processing …”, “… convolution module”, “attention and aggregation module”, “shift and summation module”, “addition module”, “… memory and processor coupled to the memory…”, “non-transitory computer readable medium on which is stored computer code for computer vision processing …, which is mere instructions to implement an abstract idea using generic computing device, or merely uses a computer as a tool to perform an abstract idea (see MPEP 2106.05(f)). Accordingly, the additional elements individually or in combination do not amount to significantly more than the judicial exception. The claim is not patent eligible.
(Dependent claims)
Claims 17-21 /23-27 are dependent on claim 16 / 22 and include all the limitations of claim 16 / 22. Therefore, claims 17-21 /23-27 recite the same abstract ideas.
With regards to claim 17 / 23 the claim recites further limitation of “wherein the input visual data include: (i) image data obtained from at least one of a optical sensor, a radar sensor, an ultrasonic sensor, and a nuclear magnetic resonance sensor, or (ii) a feature map obtained from a previous layer of a deep network based on the image data” (math concept and/or mental process), which is further details of the data processed by the math operation, a math concept. Except citing generic computer elements to implement the abstract idea, there is no additional element showing integration into a practical application or adding something signifi With regards to claim 17 / 23 the claim recites further limitation of “wherein the input visual data include: (i) image data obtained from at least one of a optical sensor, a radar sensor, an ultrasonic sensor, and a nuclear magnetic resonance sensor, or (ii) a feature map obtained from a previous layer of a deep network based on the image data” (math concept and/or mental process), which is further details of the data processed by the math operation, a math concept. Except citing generic computer elements to implement the abstract idea, there is no additional element showing integration into a practical application or adding something significantly more to the abstract idea. The claim is not patent eligible.
With regards to claim 18 / 24 the claim recites further limitation of “wherein the plurality of 1x1 convolution operations includes three 1x1 convolution operation paths, and an intermediate feature map output from each path is reshaped into a number Nh of intermediate feature maps, Nh is a number of heads of a self-attention operation” (math concept and/or mental process), which is further steps of math operation, a math concept. Except citing generic computer elements to implement the abstract idea, there is no additional element showing integration into a practical application or adding something significantly more to the abstract idea. The claim is not patent eligible.
With regards to claim 19 / 25 the claim recites further limitation of “wherein the generating of the attention weighted map includes: generating a number Nh of groups of intermediate feature maps based on the plurality of intermediate feature maps, each group including three intermediate feature maps respectively serving as query, key, and value for self-attention operation, wherein Nh is a number of heads of the self-attention operation; generating Nh attention weighted maps by performing attention and aggregation operations respectively on each group of intermediate feature maps; and concatenating the Nh attention weighted maps” (math concept and/or mental process), which is further steps of math operation, a math concept. Except citing generic computer elements to implement the abstract idea, there is no additional element showing integration into a practical application or adding something significantly more to the abstract idea. The claim is not patent eligible.
With regards to claim 20 / 26 the claim recites further limitation of “wherein the generating of the convolved feature map includes: generating a number Nc of groups of intermediate feature maps based on the plurality of intermediate feature maps, each group including a number k2 of intermediate feature maps, wherein k is a size of a convolution kernel for a k x k convolution operation, and Nc is an integer greater than one; generating Nc convolved feature maps by performing shift and summation operations respectively on each group of intermediate feature maps; and concatenating the Nc convolved feature maps” (math concept and/or mental process), which is further steps of math operation, a math concept. Except citing generic computer elements to implement the abstract idea, there is no additional element showing integration into a practical application or adding something significantly more to the abstract idea. The claim is not patent eligible.
With regards to claim 21 / 27 the claim recites further limitation of “wherein the adding of the attention weighted map and the convolved feature map includes: adjusting a channel size of at least one of the attention weighted map and the convolved feature map to make the attention weighted map and the convolved feature map have the same channel size” (math concept and/or mental process), which is further steps of math operation, a math concept. Except citing generic computer elements to implement the abstract idea, there is no additional element showing integration into a practical application or adding something significantly more to the abstract idea. The claim is not patent eligible.
Allowable Subject Matter
Claims 16-29 include allowable subject matter since when reading the claims in light of the specification, as per, MPEP §2111.01 or Toro Co. v. White Consolidated Industries Inc., 199F.3d 1295, 1301, 53 USPQ2d 1065, 1069, 1069 (Fed.Cir. 1999), none of the references of record alone or in combination disclose or suggest the combination of limitations specified in claims 16-29.
In interpreting the claims, in light of the specification filed on 20 December 2023, the Examiner finds the claimed invention to be patentably distinct from the prior arts of record.
Regarding the amended independent claims, the primary reason for the allowance is the inclusion of the specific process / structure of vision data processing pipeline with projection using 1x1 convolution, weighted attention feature mapping, feature convolution using shift & summation, and addition of attention map with convolved map.
None of the cited prior art references, singly or in combination, fully teaches all limitations of independent claims 16, 22, 28 and 29.
Regarding the dependent claims, which include all the limitations of the independent claims, are also allowed.
The followings are references close to the invention claimed:
Yin et al., US-PGPUB NO.20220284232A1 [hereafter Yin] teaches visual data processing with feature map. However Yin does not teach the specific process / structure of vision data processing pipeline with projection using 1x1 convolution, weighted attention feature mapping, feature convolution using shift & summation, and addition of attention map with convolved map.
Ayush et al., US-PATENT NO.11663463B2 [hereafter Ayush] teaches attention processing for visual data. However Ayush does not teach the specific process / structure of vision data processing pipeline with projection using 1x1 convolution, weighted attention feature mapping, feature convolution using shift & summation, and addition of attention map with convolved map.
Lopez et al., US-Patent NO.12406023B1 [hereafter Lopez] teaches convolutional neural network for visual data processing. However Lopez does not teach the specific process / structure of vision data processing pipeline with projection using 1x1 convolution, weighted attention feature mapping, feature convolution using shift & summation, and addition of attention map with convolved map.
Bello et al., “Attention augmented convolutional networks”, ICCV 2019 [hereafter Bello] teaches attention processing with convolutional neural networks. However Bello does not teach the specific process / structure of vision data processing pipeline with projection using 1x1 convolution, weighted attention feature mapping, feature convolution using shift & summation, and addition of attention map with convolved map.
Wang et al., “ECA-Net: Efficient channel attention for deep convolutional neural networks”, CVPR 2020 [hereafter Wang] teaches attention processing with deep convolutional neural networks. However Wang does not teach the specific process / structure of vision data processing pipeline with projection using 1x1 convolution, weighted attention feature mapping, feature convolution using shift & summation, and addition of attention map with convolved map.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to TSU-CHANG LEE whose telephone number is 571-272-3567. The fax number is 571-273-3567.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Omar Fernandez Rivas, can be reached 571-272-2589.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/TSU-CHANG LEE/
Primary Examiner, Art Unit 2128
Prosecution Insights