DETAILED ACTIONS
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Acknowledgment is made of applicant’s claim this application is in benefit of foreign priority from Korean Patent Application No. KR10-2022-0069865 filed on June 9, 2022.
Status of Claims
Claims 1 and 4-5 are pending. Claims 2-3 and 6 are canceled.
Response to Amendment
The amendment filed 11/24/2025 has been entered. Claims 1 and 4-5 remain pending in the application. Claims 2-3 and 6 are canceled.
Response to Arguments
Applicant’s arguments with respect to claim 1 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1 and 4-5 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Guo et al., "Deep Collaborative Attention Network for Hyperspectral Image Classification by Combining 2-D CNNand 3-D CNN" (published August 2020), hereinafter referred to as Guo.
Claim 1
Guo discloses a material classification apparatus based on a multi-spectral NIR band (Guo, Section III, “All experiments are conducted with python language and tensorflow framework, and results are demonstrated on a PC equipped with an Intel Core i5 with 2.8 GHz, memory 8G, and Nvidia GeForce GTX 1060 3G graphics card.”), comprising:
an input unit (Guo, Fig. 1, input image, Section II.A, In 3-D CNN, the input data are convolved with 3-D kernels before going through activation function to produce the feature maps.”) configured to acquire a multi-band NIR image of a target (Guo, Section III.A, “The first dataset is the Indian Pines (IP) image acquired by the Airborne Visible/Infrared Imaging Spectrometer (AVIRIS). This image, with size of 145 × 145 pixels, contains 220 bands covering the wavelength range of 0.4-2.5 μm, and the spectral and spatial resolutions are 10 nm and 17 m, separately.”, the images are within the NIR spectral range because near-infrared (NIR) spectral range typically spans from 0.75 µm to 1.4 µm (750–1400 nm));
an attention module (Guo, Section 1, “we first use 2-D CNN and 3-D CNN to extract spectral-spatial features, respectively, and then combine these two kinds of features with a “NonLocalBlock”. This block is termed as a typical spatial attention mechanism to make salient features be emphasized. Then, we proposed “Conv_Block” which is similar to the light weight dense block to extract correlation information contained in the feature maps.”) configured to generate a spatio-spectral correlation map (Guo, Section I, “The 3-D CNN and 2-D CNN layers are collaborated for the proposed model in such a way that we will achieve abundant spectral as well as spatial feature maps. By combining of these feature maps, the new feature maps contain rich spectral-spatial correlation information”) that encodes cross-correlations between band indices (i, j) across spatial coordinates based on spatial information of the multi-band NIR image (Section II, “the input of the original HSI can be denoted as X ∈ RH×W×D, the output Y ∈ RH×W×C denotes the class probability of each pixel, where H, W, D, and C are indicated as height, width, number of bands, and number of classes, separately. In the CACNN, due to high spectral resolution and hundreds of channels along the spectral dimension, we use PCA algorithm to remove the spectral redundancy in raw HSI data (X). The PCA reduces spectral bands from D to B while maintaining the same spatial dimensions (i.e., width) W and height H). The low-dimensional I ∈ RH×W×B is a new data after PCA. In order to exploit better spectral-spatial features, we design a substructure with 2-D CNN and 3-DCNN. In this substructure, spatial feature information are extracted by 2-DCNN,and meanwhile spectral-spatial contexts are exploited by 3-D CNN.”, Section II.A, “Before feeding the deep network, we create neighboring patches P ∈ RS×S×B by choosing an S ×S neighborhood of the central pixel from I, centered at the spatial location (α, β) and including B bands.”, B bands in the different band indices); and
a classification network model configured to analyze the spatio-spectral correlation map and output a material classification label for the target (Guo, Section II.D, “To gain high-level discriminative features, we design deep multilayer feature fusion. After NonLocaLBlock and Conv_Block ,feature maps contain abundant high-level spectral spatial correlation information. As shown in Table I, to fuse multilayer feature maps (concat_4), we reduce spatial size of feature maps by Max-Pooling and Conv2D and make them with same spatial size. Moreover, the obtained feature maps are transformed to new feature maps (1 × 1 × 296) with conv2D so as to better assist classification. The concat5 denotes fusion of deep spatial features, spectral-spatial features and obtained discriminative features, which is fed into a 1 × 1 convolutional layer to achieve expected results. The output vector is ˆy =[ˆ y1, ˆy2,..., ˆ yC]. And the truth one-hot label y = [y1,y2,...,yC] is the number of land-cover categories.”, the correlation feature map is used to assist classification by using the last 1x1 convolutional layer),
wherein the input unit Guo, Fig. 1, input image, Section II.A, In 3-D CNN, the input data are convolved with 3-D kernels before going through activation function to produce the feature maps.”) obtains the multi-band NIR imaqe of the target (Guo, Section III.A, “The first dataset is the Indian Pines (IP) image acquired by the Airborne Visible/Infrared Imaging Spectrometer (AVIRIS). This image, with size of 145 × 145 pixels, contains 220 bands covering the wavelength range of 0.4-2.5 μm, and the spectral and spatial resolutions are 10 nm and 17 m, separately.”, the images are within the NIR spectral range because near-infrared (NIR) spectral range typically spans from 0.75 µm to 1.4 µm (750–1400 nm)) by optically dividinq a near-infrared wavelength band into n sub-bands, where n is equal to or qreater than 2 (Guo, Section II, “The PCA reduces spectral bands from D to B while maintaining the same spatial dimensions (i.e., width W and height H). The low-dimensional I ∈ RH×W×B is a new data after PCA.”, Section III.A, “In our experiments, there are 200 bands retained by removing 20 water absorption bands.”), and,
wherein the attention module (Guo, Section 1, “we first use 2-D CNN and 3-D CNN to extract spectral-spatial features, respectively, and then combine these two kinds of features with a “NonLocalBlock”. This block is termed as a typical spatial attention mechanism to make salient features be emphasized. Then, we proposed “Conv_Block” which is similar to the light weight dense block to extract correlation information contained in the feature maps.”) comprises a 3D convolution-based model (Guo, Fig. 1, 3D convolution) whose temporal dimension is set as a multi-spectral axis (Guo, Section II, “The PCA reduces spectral bands from D to B while maintaining the same spatial dimensions (i.e., width W and height H). The low-dimensional I ∈ RH×W×B is a new data after PCA.”, the last axis in the dimension is replaces with spectral band number) to qenerate the spatio-spectral correlation map (Guo, Section I, “The 3-D CNN and 2-D CNN layers are collaborated for the proposed model in such a way that we will achieve abundant spectral as well as spatial feature maps. By combining of these feature maps, the new feature maps contain rich spectral-spatial correlation information”), which operates without temporal sequence inputs and without temporal attention units (Guo, Fig. 1, Guo dos not consider temporal attention units in the system shown in Fig. 1).
Claim 4
Mou discloses the material classification apparatus of claim 1 Guo, Section III, “All experiments are conducted with python language and tensorflow framework, and results are demonstrated on a PC equipped with an Intel Core i5 with 2.8 GHz, memory 8G, and Nvidia GeForce GTX 1060 3G graphics card.”), wherein the attention module further receives a visible light image of the target and uses the received visible light image to generate the spatial-spectral correlation map (Guo, Fig. 1, spectral bands input, Section III.A, “The first dataset is the Indian Pines (IP) image acquired by the Airborne Visible/Infrared Imaging Spectrometer (AVIRIS). This image, with size of 145 × 145 pixels, contains 220 bands covering the wavelength range of 0.4-2.5 μm, and the spectral and spatial resolutions are 10 nm and 17 m, separately.”, visible spectrum typically have wavelengths ranging from approximately 0.4-0.7 μm, so the images that was used in the reference is within the visible wavelength range).
Claim 5
Guo discloses a material classification method based on a multi-spectral NIR band (Guo, Fig. 1), comprising:
acquiring a multi-band NIR image of a target (Mou, Fig. 1, spectral bands input, Section III.A, “It includes 145 × 145 pixels with a 20 m/pixel spatial resolution and 200 spectral bands covering from 400 to 2500 nm after removing 20 water absorption channels”, near-infrared band begins just beyond the visible spectrum, typically with wavelengths ranging from approximately 750 nm to 2500 nm, so the images that was used in the reference is within the NIR wavelength range), by optically dividing a near-infrared wavelength band into n sub-bands, where n is equal to or greater than 2 (Guo, Section II, “The PCA reduces spectral bands from D to B while maintaining the same spatial dimensions (i.e., width W and height H). The low-dimensional I ∈ RH×W×B is a new data after PCA.”, Section III.A, “In our experiments, there are 200 bands retained by removing 20 water absorption bands.”);
generating a spatio-spectral correlation map considering spatial information on the multi-band NIR image that encodes cross-correlations between band indices (i, i) across spatial coordinates ((Section II, “the input of the original HSI can be denoted as X ∈ RH×W×D, the output Y ∈ RH×W×C denotes the class probability of each pixel, where H, W, D, and C are indicated as height, width, number of bands, and number of classes, separately. In the CACNN, due to high spectral resolution and hundreds of channels along the spectral dimension, we use PCA algorithm to remove the spectral redundancy in raw HSI data (X). The PCA reduces spectral bands from D to B while maintaining the same spatial dimensions (i.e., width) W and height H). The low-dimensional I ∈ RH×W×B is a new data after PCA. In order to exploit better spectral-spatial features, we design a substructure with 2-D CNN and 3-DCNN. In this substructure, spatial feature information are extracted by 2-DCNN,and meanwhile spectral-spatial contexts are exploited by 3-D CNN.”, Section II.A, “Before feeding the deep network, we create neighboring patches P ∈ RS×S×B by choosing an S ×S neighborhood of the central pixel from I, centered at the spatial location (α, β) and including B bands.”, B bands in the different band indices) by applyinq the multi-band NIR imaqe to a trained 3D convolution-based attention module (Guo, Fig. 1, 3D convolution) whose temporal dimension is set as a multi-spectral axis (Guo, Section II, “The PCA reduces spectral bands from D to B while maintaining the same spatial dimensions (i.e., width W and height H). The low-dimensional I ∈ RH×W×B is a new data after PCA.”, the last axis in the dimension is replaces with spectral band number) and operates without temporal sequence inputs and without temporal attention units (Guo, Fig. 1, Guo dos not consider temporal attention units in the system shown in Fig. 1); and
outputting a material classification label for the target by applying the spatio-spectral correlation map to the trained classification model (Guo, Section II.D, “To gain high-level discriminative features, we design deep multilayer feature fusion. After NonLocaLBlock and Conv_Block ,feature maps contain abundant high-level spectral spatial correlation information. As shown in Table I, to fuse multilayer feature maps (concat_4), we reduce spatial size of feature maps by Max-Pooling and Conv2D and make them with same spatial size. Moreover, the obtained feature maps are transformed to new feature maps (1 × 1 × 296) with conv2D so as to better assist classification. The concat5 denotes fusion of deep spatial features, spectral-spatial features and obtained discriminative features, which is fed into a 1 × 1 convolutional layer to achieve expected results. The output vector is ˆy =[ˆ y1, ˆy2,..., ˆ yC]. And the truth one-hot label y = [y1,y2,...,yC] is the number of land-cover categories.”, the correlation feature map is used to assist classification by using the last 1x1 convolutional layer).
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DENISE G ALFONSO whose telephone number is (571)272-1360. The examiner can normally be reached Monday - Friday 7:30 - 5:30.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Amandeep Saini can be reached at (571)272-3382. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DENISE G ALFONSO/Examiner, Art Unit 2662 /AMANDEEP SAINI/Supervisory Patent Examiner, Art Unit 2662