DETAILED ACTION
Notice of Pre-AIA or AIA Status.
1. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
2. Claims 1-17 filed on 11/22/2024 are pending and being examined. Claim 1 is independent form.
Priority
3. This application is a CON of PCT/CN2022/094521 filed on 05/23/2022
Claim Rejections - 35 USC § 101
4. 35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
5. Claims 1-9 and 14-15 are rejected under 35 U.S.C. 101 because the claimed inventions are directed to non-statutory subject matter (an abstract idea without significantly more).
5-1. Regarding independent claim 1, the claim recites a method for feature extraction using a neural network, the method comprising:
[1] extracting a first set of features from an input set of features by processing said input set of features through at least two residual blocks of the neural network connected successively to each other;
[2] extracting a second set of features from the input set of features, said extracting the second set of features comprising the steps of: [2-a] extracting a third set of features and a fourth set of features from the input set of features by a non-local attention processing, [2-b] implementing a group convolution by a multi-head attention mechanism to the input set of features, the third set of features and the fourth set of features to obtain the second set of features.
Step 1:
With regard to step (1), claim 1, is directed to a method for feature extraction using a neural network. The claim 1 therefore is one of statutory categories of invention, i.e., a process.
Step 2A-1:
With regard to 2A-1, The elements recited in claim 1, as drafted, under their broadest reasonable interpretation, encompass a process(es) which falls within mathematical concepts. For example, “extracting a first set of features from an input set of features by processing said input set of features through at least two residual blocks of the neural network connected successively to each other” in step [1] and “extracting a second set of features from the input set of features, said extracting the second set of features comprising the steps of [2-1] and [2-b] in step [2] in the context of this claim are mathematical calculations and fall within the “mathematical concepts” grouping of abstract ideas. In other words, even though “a neural network (NN)” is recited by the claim, the NN mere generally encompasses mathematical calculations (i.e., abstract idea) without limiting how the NN functions to a practical application. Claim 1 therefore recites an abstract idea. If a claim limitation is directed to organizing human activity, can be practically performed in human mind, or falls within mathematical concepts, then the claim recites an abstract idea. See MPEP 2106.04(a)(2).
Step 2A-2:
The 2019 PEG defines the phrase "integration into a practical application" to require an additional element or a combination of additional elements in the claim to apply, rely on, or use the judicial exception. In the instant case, as mentioned in Step 2A-1, there is no additional element rather than a high generic neural network. Therefore, the claim as a whole does not integrate the judicial exception into a practical application.
Step 2B:
As explained above, the method for feature extraction using a neural network encompasses mathematical calculations without any additional. The claim therefore is ineligible.
5-2. Regarding dependent claims 2-9, and 14, they are dependent from claim 1 and viewed individually, these additional elements are under its broadest reasonable interpretation, either covers performance of the limitation in the mind, performing a mathematical algorithm or extra solution activity for data gathering and do not provide meaningful limitations to transform the abstract idea into a patent eligible application of the abstract idea such that the claims amount to significantly more than the abstract idea itself. And, when the claims are viewed as a whole, they do not improve a technology by allowing the technology to perform a function that it previously was not capable of performing; and they do not provide any limitations beyond generally linking the use of the abstract idea to a broad technological environment (i.e., computer-based analysis of generic data). Hence, the claimed invention does not constitute significantly more than the abstract idea, so the claims are rejected under 35 USC § 101 as being directed to non-statutory subject matter.
5-3. Regarding independent claim 15, the claim recites a non-transitory storage medium which is analogous to apparatus claim 1, grounds of rejection analogous to those applied to claim 1 are applicable to claims 15. Furthermore, the claim is a method that does not recite any additional elements, and according to step 2A-2 does not integrate the abstract idea into a practical application because it does not recite any additional elements that impose any meaningful limits on practicing the abstract idea. The claim recites an abstract idea.
Because the claim fails under (2A), the claim is further evaluated under (2B). The claim herein does not include any additional elements that are sufficient to amount to significantly more than the judicial exception. The claims are not patent eligible.
Claim Rejections - 35 USC § 103
6. In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
7. The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
8. Claims 1-17 are rejected under 35 U.S.C. 103 as being unpatentable over Liu et al (“Non-local Attention Optimized Deep Image Compression”, 2019, hereinafter “Liu”) in view of Vaswani et al (“Attention Is All You Need”, 2017, hereinafter “Vaswani”).
Regarding claim 1, Liu discloses a method for feature extraction using a neural network (the Non-Local Attention Optimized Deep Image Compression (NLAIC) framework; see abstract and fig.1), the method comprising:
extracting a first set of features from an input set of features by processing said input set of features through at least two residual blocks of the neural network connected successively to each other (to generate output features based on input features, wherein the main branch of the non-local attention module (NLAM) of an autoencoder constructing the NLAIC comprises 3 residual blocks connected successively to each other, see the lower branch of fig.2 (b) and Sec. 3.1);
extracting a second set of features from the input set of features (to generate output features based on input features, besides the main branch, the attention mask branch of the non-local attention module (NLAM) extracts non-local features including global features from the input features, see the upper branch of fig.2 (b) and Sec. 3.1).
Liu does not disclose “said extracting the second set of features comprising the steps of: extracting a third set of features and a fourth set of features from the input set of features by a non-local attention processing, implementing a group convolution by a multi-head attention mechanism to the input set of features, the third set of features and the fourth set of features to obtain the second set of features” as recited by claim 1. However, in the same field of endeavor, Vaswani teaches the “multi-head attention” approach to “draw global dependencies between input and output” instead of “a single attention function” and concatenate all the h head attentions as an output attention mask. See the right of fig.2 and Section 3.2.2. It would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention was made to incorporate the teachings of Vaswani into the teachings of Liu and modify the teachings of Vaswani and set h=2 in the attention mask branch of Liu. Suggestion or motivation for doing so would have been “to draw global dependencies between input and output” as taught by Vaswani, see Section 1, the last paragraph. Therefore, the claim is unpatentable over Liu in view of Vaswani.
Regarding claim 2, the combination of Liu and Vaswani discloses the method of claim 1, wherein: the input set of features is a multidimensional vector representing an image data (Liu, see input feature vector X in Eq(1)).
Regarding claim 3, the combination of Liu and Vaswani discloses the method of claim 2, wherein the first set of features is a set of local features of the image data and the second set of features is a set of global features of the image data (Liu, the NLAM includes the local main and global non-local branches shown by fig.2).
Regarding claim 4, the combination of Liu and Vaswani discloses the method according to claim 1, comprising further the steps of: obtaining a fifth set of features by processing the second set of features by at least two convolutional layers of the neural network (Liu, see the right dash frame in the main branch of fig.2. Vaswani, see the cascading processing “Concat” in the right of fig.2 and Sec. 3.2.2).
Regarding claim 5, the combination of Liu and Vaswani discloses the method according to claim 4, further comprising the steps of: performing cascading processing on the first set of features and the fifth set of features (Vaswani, see the cascading processing “Concat” in the right of fig.2 and Sec. 3.2.2), and fusing the first set of features and fifth set of features after cascading processing to thereby obtain a mixed set of features (Liu, see the fusing processing “
⊗
” in fig.2).
Regarding claim 6, the combination of Liu and Vaswani discloses the method according to claim 5, further comprising: performing convolution processing on the mixed set of features by at least one convolutional layer of the neural network to obtain a sixth set of features, and outputting the sixth set of features (Liu, see the fusing processing “
⊗
” in fig.2).
Regarding claim 7, the combination of Liu and Vaswani discloses the method according to claim 1, wherein implementing the group convolution comprises the steps of: multiplying the third set of features and the fourth set of features (Liu, see the fusing processing “
⊗
” in fig.2), adding the input set of features to the multiplication of the third set of features and the fourth set of features to obtain a resultant set of features (Liu, see the fusing processing “⊕” in fig.2), and concatenating the resultant set of features by the number of heads of the multi-head attention mechanism to obtain the second set of features (Vaswani, see the cascading processing “Concat” in the right of fig.2 and Sec. 3.2.2).
Regarding claim 8, the combination of Liu and Vaswani discloses the method according to claim 1, wherein extracting the third set of features comprises: processing the input set of features through at least three residual blocks of the neural network successively connected to each other (Liu, see the lower/main branch of fig.2 (b) and Sec. 3.1).
Regarding claim 9, the combination of Liu and Vaswani discloses the method according to claim 1, wherein extracting the fourth set of features comprises: processing the input set of features by at least three residual blocks of the neural network successively connected to each other; processing the output of the last of the at least three residual blocks by a convolutional layer of the neural network (Liu, see the 3 residual blocks after “NLM” in the mask branch of fig.2 (b)); and performing activation processing by an activation layer of the neural network to the output of the at least one convolutional layer (Liu, see the “sigmoid” block in the mask branch of fig.2 (b)).
Regarding claim 10, the combination of Liu and Vaswani discloses the method for learned image compression using a neural network (Liu, see “Main Encoder” in fig.1 and Table 1, wherein a NLAM module of the Main Encoder generates the output feature vector Y by transforming the input feature vector X into more compact latent features), the method comprising performing the steps of:
extracting a set of features (x) from an input image data to be compressed (Liu, see the input feature vector X shown in fig. 2, wherein the input feature vector X is extracted from a corresponding input image as shown by fig.1); and extracting a set of features (y) indicating a latent representation of the input image data to be compressed from the set of features (x) extracted from the input image data to be compressed by performing the steps (Liu, the NLAM module generates the output feature vector Y by “transforming the input feature vector into more compact latent features”; see fig.2 and sec. 3.2 and sec. 3.3) of:
performing at least four steps of downsampling convolution processing by at least four convolutional layers of the neural network arranged in a stream like manner on the extracted set of features (x) from the input image data (Liu, see the four convolution layers in the “Main Encoder” in Table 1, wherein the Main Encoder receives the input image X as shown by fig.1), and
performing the steps of the feature extraction method of claim 1 at least once after two steps of downsampling convolution processing by two convolutional layers from the at least four convolutional layer by using as the input set of features a set of features based on the output of the second convolutional layer from the two convolutional layers to thereby extract the set of features (y) indicating the latent representation of the input image data (Liu, see the “Main Encoder” column in Table 1 from row 1 (i.e., “Con: 5x5x192 s2”) to row 4 (i.e., “NLAM” which is a non-local feature attention module for extracting compact latent features Y)); and the method further comprising the step of: outputting the extracted set of features (y) indicating a latent representation of the input image data (Liu, see Sec. 3.4, paragraph 1: “our novel NLAM scheme to transform the input pixels into more compact latent features”).
Regarding claim 11, the combination of Liu and Vaswani discloses the method of claim 10, further comprising: inserting a step of processing by at least one residual block of the neural network at least between two steps of downsampling processing by two convolutional layers of the at least four downsampling convolutional layers arranged in a stream like manner (Liu, see the second row and 6-th row in “Main Encoder” column in Table 1).
Regarding claim 12, the combination of Liu and Vaswani discloses the method for learned image decompression using a neural network (Liu, see the “Main Decoder” in fig.1 and Table 1, wherein the Main Decoder is to reconstruct the compressed input feature vector
Y
^
to a decompression output image
X
^
), the method comprising:
extracting a set of features from an input set of features to be decompressed; and extracting a set of features indicating a reconstructed image of the input image data from the extracted set of features (wherein a NLAM module of the Main Decoder reconstructs the output feature vector
X
^
by decoding the input feature vector
Y
^
; Liu, see the “Main Decoder” in fig.1 and Table 1,) by performing the steps of:
performing the steps of the feature extraction method of claim 1 using as the input set of features the extracted set of features from the input set of features to be decompressed (Liu, see the 1st row “NLAM”, wherein the NLAM module of the Main Decoder generates the output feature vector
X
^
by decoding the input feature vector
Y
^
), performing upsampling convolution processing by at least four convolutional layers of the neural network arranged in a stream like manner on the extracted set of features from the input set of features to be decompressed (Liu, see the up-sampling with stride 2 as shown in the 2nd, 4th, 6th, and 8th rows in the “Main Decoder” in Table 1), and performing the feature extraction method of claim 1 at least once after two steps of upsampling convolution processing by two convolutional layers from the at least four convolutional layer by using as the input set of features a set of features based on the output of the second convolutional layer from the two convolutional layers to thereby extract a set of features indicating a reconstructed image of the input image data (Liu, see the 5th row “NLAM” in the “Main Decoder” in Table 1); and the method further comprising: outputting the extracted set of features indicating a reconstructed image of the input image data (Liu, see the “output image” in fig.1).
Regarding claim 13, the combination of Liu and Vaswani discloses the method of claim 12, wherein the method further comprises: inserting a step of processing by at least one residual block of the neural network at least once between two steps of upsampling processing by two convolutional layers of the at least four upsampling convolutional layers arranged in a stream like manner (Liu, see the “ResBlock” in the 3rd row and 7th row in the “Main Decoder” in Table 1).
Regarding claim 14, the combination of Liu and Vaswani discloses the method according to claim 1, wherein performing the step of processing by a residual block comprises: performing convolution by at least two convolutional layers of the neural network (Liu, see the two “Conv” in each of the “ResBlock” modules in fig.2 (b)); performing activation by at least one activation layer of the neural network after each of the at least two convolutional layers of the neural network; implementing a skip connection providing an alternative path from an input of the residual block to an output of the residual block; and obtaining the output of the residual block by fusing the skip connection and the last activation layer of the residual block (Liu teaches this element because, the “Output feature” in Liu is masked by the mask M, 0<M<1, defined by Eq(5), see the right below Eq(5), Liu states “This attention mask M, having its element 0 < Mk < 1; Mk
∈
R, is element-wise multiplied with feature maps from the main branch to perform adaptive processing.” In other words, as M->0, a skip connection processing will be implemented.).
Regarding claim 15, claim 15 is an inherent variation of claim 1, thus it is interpreted and rejected for the reasons set forth in the rejection of claim 1.
Regarding claim 16, claim 16 is an inherent variation of claim 10, thus it is interpreted and rejected for the reasons set forth in the rejection of claim 10.
Regarding claim 17, claim 17 is an inherent variation of claim 12, thus it is interpreted and rejected for the reasons set forth in the rejection of claim 12.
Conclusion
9. The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Ding et al, US 20220101492.
10. Any inquiry concerning this communication or earlier communications from the examiner should be directed to RUIPING LI whose telephone number is (571)270-3376. The examiner can normally be reached 8:30am--5:30pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, HENOK SHIFERAW can be reached on (571)272-4637. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit https://patentcenter.uspto.gov; https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center, and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/RUIPING LI/Primary Examiner, Ph.D., Art Unit 2676
Prosecution Insights