DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1-2 and 5-6 are rejected under 35 U.S.C. 102(a)(1) as being unpatentable by Liang (US 20190042895 A1).
Regarding Claim 1, representative of Claims 5 and 6, Liang teaches an image classification apparatus comprising:
a deep-layer feature vector extraction unit that extracts a low-resolution deep-layer feature vector of an input image ([0042]: As shown in FIG. 4, the deep convolutional neural network includes several convolution groups and several fully connected layers… When extracting the convolution feature, outputs (pool4 and pool5 as shown in FIG. 4) of pooling layers of the last two convolution groups of the five convolution groups and an output (fc2 as shown in FIG. 4) of the last fully connected layer are extracted, and PCA transform is performed on the outputs layer by layer to reserve a principal component. Examiner interpreting convolution groups 1-5 contributing to pool5 as shown in Fig. 4 as the deep layer extraction);
a shallow-layer feature vector extraction unit that extracts a high-resolution shallow-layer feature vector of the input image ([0042]: As shown in FIG. 4, the deep convolutional neural network includes several convolution groups and several fully connected layers… When extracting the convolution feature, outputs (pool4 and pool5 as shown in FIG. 4) of pooling layers of the last two convolution groups of the five convolution groups and an output (fc2 as shown in FIG. 4) of the last fully connected layer are extracted, and PCA transform is performed on the outputs layer by layer to reserve a principal component. Examiner interpreting convolution groups 1-4 contributing to pool4 as shown in Fig. 4 as the shallow layer extraction);
a concatenation unit that concatenates the deep-layer feature vector and the shallow-layer feature vector and outputs a concatenated feature vector ([0045]: a pre-trained deep convolutional neural network, the output (pool4) of the pooling layer of the fourth convolution group may be extracted, and all values of the output are strung into a first vector, [0046]: PCA transform is performed on the first vector, and a first number of principal components are reserved to obtain a first insertion vector. [0048] In step 205, an output of a pooling layer of a fifth convolution group is extracted, all values of the output are strung into a second vector, and the first insertion vector is inserted into a header of the second vector);
a similarity calculation unit that retains a weight matrix of respective classes and calculates similarities from the concatenated feature vector and the weight matrix of respective classes ([0056] It should be understood that, for the three images in the to-be-authenticated ternary image group, each of the images corresponds to one feature vector, and the ternary image group corresponds to three feature vectors, [0057] In step 209, a cosine similarity degree between each pair of feature vectors of the three feature vectors corresponding to the to-be-authenticated ternary image group is calculated as three matching scores, [0016]: fusion vector is inputted into a pre-trained SVM classifier to authenticate and determine whether the images in the to-be-authenticated multivariate image group are consistent with one another); and
a classification determination unit that determines a classification of the input image based on the similarities ([0016]: fusion vector is inputted into a pre-trained SVM classifier to authenticate and determine whether the images in the to-be-authenticated multivariate image group are consistent with one another).
Regarding Claim 2, Liang teaches the image classification apparatus according to claim 1. In addition, Liang teaches wherein the shallow-layer feature vector extraction unit shares at least one convolutional layer with the deep-layer feature vector extraction unit ([0042]: As shown in FIG. 4, the deep convolutional neural network includes several convolution groups and several fully connected layers. Each of convolution groups includes several convolution sub-layers and one pooling layer. Examiner notes Fig. 4 depicts each convolution group leading into the next convolution group).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 3 is rejected under 35 U.S.C. 103 as being unpatentable over Liang (US 20190042895 A1) in view of Roh (US 20180165551 A1).
Regarding Claim 3, Liang teaches the image classification apparatus according to claim 2. Liang does not explicitly teach the remaining limitations of Claim 3. Roh teaches wherein the larger the number of training images at the time of learning, the smaller the number of convolutional layers that the shallow-layer feature vector extraction unit shares with the deep-layer feature vector extraction unit ([0034]: Including multiple levels of abstraction in the convolution map may improve accuracy (i.e., semantics) and/or localization. For example, low-level features from an earlier part of the convolution network 206 may include structures or boundaries of objects, while high-level features from a later part of the convolution network 206 may be capable of generating robust and abstracted object categories. [0030] In block 404, the computing device 100 inputs the image data into the multi-layer convolution network 206 to generate a convolution map. In some embodiments, in block 406, the computing device 100 may represent the convolution map data at multiple levels of abstraction of the multi-layer convolution network 206. For example, the computing device 100 may store or otherwise provide access to the convolution map data generated by each convolution layer 208. In some embodiments, the computing device 100 may provide access to each convolution layer 208, or to a sampling or otherwise less than all of the convolution layers 208. Examiner notes one of ordinary skill would be able to incorporate a feature vector from an earlier convolution layer when more images are incorporated in training. Doing so would provide a diversity of features for a classification task).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present invention to have modified Liang to include the teachings of Roh by including selective convolutional layers to provide multi-level features for classification. Doing so would improve classification accuracy by incorporating a diversity of features.
Claim(s) 4 is rejected under 35 U.S.C. 103 as being unpatentable over Liang (US 20190042895 A1) in view of Mei (S. Mei, Y. Chen, J. Ji, J. Hou and Q. Du, "Fusing different levels of deep features by deep stacked neural network for hyperspectral images," 2017 IEEE International Geoscience and Remote Sensing Symposium (IGARSS), Fort Worth, TX, USA, 2017, pp. 759-762, doi: 10.1109/IGARSS.2017.8127063).
Regarding Claim 4, Liang teaches the image classification apparatus according to claim 2. Although Liang teaches that convolution outputs of a number greater than two may be used for concatenation, Liang does not explicitly teach the remaining limitations of Claim 4. Mei teaches wherein as the number of training images at the time of learning increases, the shallow-layer feature vector extraction unit extracts a plurality of shallow-layer feature vectors from feature maps output by a larger number of convolutional layers, and the concatenation unit that concatenates the deep-layer feature vector and the plurality of shallow-layer feature vectors and outputs a concatenated feature vector ([Section 2.2, paragraph 1]: The structure of our proposed DSNN to extract different levels of deep learning features is shown in Fig. 2. In the proposed DSNN, basic unit of CNN is concatenated to each other sequentially to extract different levels of deep features. All levels of deep features are then fed to a high-level feature extraction and classification unit, [Section 2.2, paragraph 2]: It is observed that the proposed DSNN with different number of stacked CNNs can extract different levels of deep features. Generally, the more stacked CNNs, the deeper the features. For example, the DSNN with three stacked CNNs will extract deeper features than that extracted with two stacked CNNs…Different levels of deep features can be fused by concatenating them to form a fused deep features F=[F1;F2;…;Fk]. As a result, the performance of classification can be improved).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present invention to have modified the teachings of Liang to include the teachings of Mei by including a plurality of features at a plurality of levels by including more convolutions contributing to the fused feature. Doing so would provide more features at various levels which would improve classification accuracy.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JANICE VAZ whose telephone number is (703)756-4685. The examiner can normally be reached Monday-Friday 9:00-5:00pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Matthew Bella can be reached at (571) 272-7778. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JANICE E. VAZ/Examiner, Art Unit 2667
/MATTHEW C BELLA/Supervisory Patent Examiner, Art Unit 2667
Prosecution Insights