DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are: an acquisition module, a determining module, a feature fusion module, and a processing module in claim 10.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1, 3, 6, 7, 10-12, 16 and 20 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Cao et al ("Attention fusion for one-stage multispectral pedestrian detection." Sensors 21.12 (2021): 4184, https://www.mdpi.com/1424-8220/21/12/4184, pages 1-17, retrieved from the Internet on 8/28/2026).
Regarding claim 1, Cao discloses an image processing method, comprising:
acquiring a first image feature obtained by extracting a first feature from a first image (Ci in equation 1; section 3.3, where Ci and Ti are defined as the color and thermal features at the i-th level) and a second image feature obtained by extracting a second feature from a second image (Ti in equation 1; section 3.3, where Ci and Ti are defined as the color and thermal features at the i-th level), wherein an image type of the first image is different from an image type of the second image (fig. 4; section 3.3, the color and thermal modalities);
determining feature selection weights respectively corresponding to the first image feature and the second image feature based on the first image feature and the second image feature (equations 1-3; section 3.3, Then, a new compact feature F2 is created to adaptively learn the fusion weights of the color and thermal features);
performing feature fusion on the first image feature and the second image feature based on the feature selection weights respectively corresponding to the first image feature and the second image feature to obtain fused feature data (equation 5; section 3.3, With this method, the features at all of the levels are adaptively aggregated at each scale. The output of the MCFF can be used as the input of a new fusion branch for feature extraction); and
performing object detection processing based on the fused feature data to obtain an object detection result (section 3; section 4.3.2, In this paper, the MCFF adaptively selects the features from the color stream and thermal stream depending on the illumination conditions to detect pedestrians).
Regarding claim 3, Cao discloses an image processing method wherein the step of determining the feature selection weights respectively corresponding to the first image feature and the second image feature based on the first image feature and the second image feature comprises:
performing first feature fusion processing on the first image feature and the second image feature to obtain an initial fused feature (fig. 4; equation 1; section 3.3);
determining a feature importance vector for the first image feature and the second image feature based on the initial fused feature (equations 2-3; section 3.3); and
determining the feature selection weights respectively corresponding to the first image feature and the second image feature by using the feature importance vector (equation 4; section 3.3).
Regarding claim 6, Cao discloses an image processing method wherein the step of performing the feature fusion on the first image feature and the second image feature based on the feature selection weights respectively corresponding to the first image feature and the second image feature to obtain the fused feature data comprises:
performing feature selection on the first image feature based on a feature selection weight corresponding to the first image feature to obtain first selected feature data of the first image feature (equations 4-5); and
performing the feature selection on the second image feature by using a feature selection weight corresponding to the second image feature to obtain second selected feature data of the second image feature (equation 5); and
performing second feature fusion processing on the first selected feature data and the second selected feature data to obtain the fused feature data (fig. 4; equation 5; section 3.3).
Regarding claim 7, Cao discloses an image processing method wherein the image processing method is applied to a pre-trained network (section 4.2); and wherein the pre-trained network comprises a feature encoder (section 3.1, feature extraction), a feature reweighting device (fig. 4; section 3.3), and an object detector (detection head in fig. 2; section 3.1); wherein
the feature encoder is configured to acquire the first image feature obtained by extracting the first feature from the first image (Ci in equation 1; section 3.3, where Ci and Ti are defined as the color and thermal features at the i-th level) and the second image feature obtained by extracting the second feature from the second image (Ti in equation 1; section 3.3, where Ci and Ti are defined as the color and thermal features at the i-th level), wherein the image type of the first image is different from the image type of the second image (fig. 4; section 3.3, the color and thermal modalities);
the feature reweighting device is configured to determine the feature selection weights respectively corresponding to the first image feature and the second image feature based on the first image feature and the second image feature (equations 1-3; section 3.3, Then, a new compact feature F2 is created to adaptively learn the fusion weights of the color and thermal features); and perform the feature fusion on the first image feature and the second image feature based on the feature selection weights respectively corresponding to the first image feature and the second image feature to obtain the fused feature data (equation 5); and
the object detector is configured to perform the object detection processing based on the fused feature data to obtain the object detection result (section 3; section 4.3.2, In this paper, the MCFF adaptively selects the features from the color stream and thermal stream depending on the illumination conditions to detect pedestrians).
Regarding claims 10-12, the claims recite similar subject matter as claim 1 and are rejected for the same reasons as stated above.
Regarding claim 16, the claim recites similar subject matter as claim 6 and is rejected for the same reasons as stated above.
Regarding claim 20, the claim recites similar subject matter as claim 3 and is rejected for the same reasons as stated above.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 2, 13, 15 and 19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Cao et al ("Attention fusion for one-stage multispectral pedestrian detection." Sensors 21.12 (2021): 4184, https://www.mdpi.com/1424-8220/21/12/4184, pages 1-17, retrieved from the Internet on 8/28/2026) in view of Nie (US20210383166).
Regarding claim 2, Cao teaches an image processing method wherein the step of acquiring the first image feature obtained by extracting the first feature from the first image and the second image feature obtained by extracting the second feature from the second image comprises:
acquiring the first image and the second image (section 3.3, color and thermal images).
Cao fails to teach for any one of images, performing convolution processing on the image, and performing squeeze-and-excitation processing on a result obtained after convolution processing to obtain an image feature corresponding to the image.
However Nie teaches performing convolution processing (para. [0063]) on an image (para. [0067]), and performing squeeze-and-excitation processing on a result obtained after convolution processing (640 in fig. 6) to obtain an image feature corresponding to the image (para. [0087], Further, the SE block 640 may include another fully connected layer fc2 followed by a sigmoid, the output of which has the dimensions 1×1×C (where, generally, C′=C/16) and is used to rescale the initial features of the concatenated feature map 625, for example, via channel-wise multiplication).
Therefore taking the combined teachings of Cao and Nie as a whole, it would have been obvious to one of ordinary skill in the art at the time the invention was filed to incorporate the steps of Nie into the method of Cao. The motivation to combine Nie and Cao would be to reduce or, alternatively, avoid noise information (para. [0040] of Nie).
Regarding claim 13, the modified method of Cao teaches an image processing method wherein the step of determining the feature selection weights respectively corresponding to the first image feature and the second image feature based on the first image feature and the second image feature comprises:
performing first feature fusion processing on the first image feature and the second image feature to obtain an initial fused feature (fig. 4; equation 1; section 3.3 of Cao);
determining a feature importance vector for the first image feature and the second image feature based on the initial fused feature (equations 2-3; section 3.3 of Cao); and
determining the feature selection weights respectively corresponding to the first image feature and the second image feature by using the feature importance vector (equation 4; section 3.3 of Cao).
Regarding claim 15, the modified method of Cao teaches an image processing method wherein the step of performing the feature fusion on the first image feature and the second image feature based on the feature selection weights respectively corresponding to the first image feature and the second image feature to obtain the fused feature data comprises:
performing feature selection on the first image feature based on a feature selection weight corresponding to the first image feature to obtain first selected feature data of the first image feature (equations 4-5 of Cao); and
performing the feature selection on the second image feature by using a feature selection weight corresponding to the second image feature to obtain second selected feature data of the second image feature (equation 5 of Cao); and
performing second feature fusion processing on the first selected feature data and the second selected feature data to obtain the fused feature data (fig.4; equation 5; section 3.3 of Cao).
Regarding claim 19, the modified method of Cao teaches an image processing method wherein the image processing method is applied to a pre-trained network (section 4.2 of Cao); and wherein the pre-trained network comprises a feature encoder (section 3.1 of Cao, feature extraction), a feature reweighting device (fig. 4; section 3.3 of Cao), and an object detector (detection head in fig. 2; section 3.1 of Cao); wherein
the feature encoder is configured to acquire the first image feature obtained by extracting the first feature from the first image (Ci in equation 1; section 3.3 of Cao, where Ci and Ti are defined as the color and thermal features at the i-th level) and the second image feature obtained by extracting the second feature from the second image (Ti in equation 1; section 3.3 of Cao, where Ci and Ti are defined as the color and thermal features at the i-th level), wherein the image type of the first image is different from the image type of the second image (fig. 4; section 3.3 of Cao, the color and thermal modalities);
the feature reweighting device is configured to determine the feature selection weights respectively corresponding to the first image feature and the second image feature based on the first image feature and the second image feature (equations 1-3; section 3.3 of Cao, Then, a new compact feature F2 is created to adaptively learn the fusion weights of the color and thermal features); and perform the feature fusion on the first image feature and the second image feature based on the feature selection weights respectively corresponding to the first image feature and the second image feature to obtain the fused feature data (equation 5 of Cao); and
the object detector is configured to perform the object detection processing based on the fused feature data to obtain the object detection result (section 3; section 4.3.2 of Cao, In this paper, the MCFF adaptively selects the features from the color stream and thermal stream depending on the illumination conditions to detect pedestrians).
Claim(s) 4 and 17 is/are rejected under 35 U.S.C. 103 as being unpatentable over Cao et al ("Attention fusion for one-stage multispectral pedestrian detection." Sensors 21.12 (2021): 4184, https://www.mdpi.com/1424-8220/21/12/4184, pages 1-17, retrieved from the Internet on 8/28/2026) in view of Li et al ("Selective kernel networks." 2019 IEEE/CVF conference on computer vision and pattern recognition (CVPR). IEEE, 2019, pages 510-519, retrieved from the Internet on 8/28/2026).
Regarding claim 4, Cao teaches an image processing method wherein the step of determining the feature importance vector for the first image feature and the second image feature based on the initial fused feature comprises:
performing global pooling processing on the initial fused feature to obtain an intermediate fused feature (equation 2; section 3.3).
Cao fails to teach performing full connection processing on the intermediate fused feature, and performing normalization processing on the intermediate fused feature after the full connection processing to obtain the feature importance vector.
However Li teaches performing full connection processing on an intermediate fused feature (equation 3; section 3.1), and performing normalization processing on the intermediate fused feature after the full connection processing (section 3.1, Batch Normalization) to obtain a feature importance vector (equation 5; section 3.1, compact feature descriptor).
Therefore taking the combined teachings of Cao and Li as a whole, it would have been obvious to one of ordinary skill in the art at the time the invention was filed to incorporate the steps of Li into the method of Cao. The motivation to combine Li and Cao would be to utilize parameters more efficiently (section 4.1 of Li) and improve the efficiency and effectiveness of object recognition (section 5 of Li).
Regarding claim 17, the modified method of Cao teaches an image processing method wherein the step of performing the feature fusion on the first image feature and the second image feature based on the feature selection weights respectively corresponding to the first image feature and the second image feature to obtain the fused feature data comprises:
performing feature selection on the first image feature based on a feature selection weight corresponding to the first image feature to obtain first selected feature data of the first image feature (equations 4-5 of Cao); and
performing the feature selection on the second image feature by using a feature selection weight corresponding to the second image feature to obtain second selected feature data of the second image feature (equation 5 of Cao); and
performing second feature fusion processing on the first selected feature data and the second selected feature data to obtain the fused feature data (fig.4; equation 5; section 3.3 of Cao).
Claim(s) 5 and 18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Cao et al ("Attention fusion for one-stage multispectral pedestrian detection." Sensors 21.12 (2021): 4184, https://www.mdpi.com/1424-8220/21/12/4184, pages 1-17, retrieved from the Internet on 8/28/2026) in view of Lu et al (US20200057805).
Regarding claim 5, Cao fails to teach an image processing wherein the step of determining the feature selection weights respectively corresponding to the first image feature and the second image feature by using the feature importance vector comprises:
determining the feature selection weights respectively corresponding to the first image feature and the second image feature based on the feature importance vector, the first image feature and the second image feature.
However Lu teaches determining feature selection weights (para. [0066], αt) respectively corresponding to a first image feature and a second image feature (para. [0064], v1,…vk) based on a feature importance vector (para. [0063]-[0064], ht), the first image feature and the second image feature (para. [0064], v1,…vk).
Therefore taking the combined teachings of Cao and Lu as a whole, it would have been obvious to one of ordinary skill in the art at the time the invention was filed to incorporate the steps of Lu into the method of Cao. The motivation to combine Lu and Cao would be to significantly improve the performance of image captioning (para. [0059] of Lu).
Regarding claim 18, the modified method of Cao teaches an image processing method wherein the step of performing the feature fusion on the first image feature and the second image feature based on the feature selection weights respectively corresponding to the first image feature and the second image feature to obtain the fused feature data comprises:
performing feature selection on the first image feature based on a feature selection weight corresponding to the first image feature to obtain first selected feature data of the first image feature (equations 4-5 of Cao); and
performing the feature selection on the second image feature by using a feature selection weight corresponding to the second image feature to obtain second selected feature data of the second image feature (equation 5 of Cao); and
performing second feature fusion processing on the first selected feature data and the second selected feature data to obtain the fused feature data (fig.4; equation 5; section 3.3 of Cao).
Claim(s) 8 is/are rejected under 35 U.S.C. 103 as being unpatentable over Cao et al ("Attention fusion for one-stage multispectral pedestrian detection." Sensors 21.12 (2021): 4184, https://www.mdpi.com/1424-8220/21/12/4184, pages 1-17, retrieved from the Internet on 8/28/2026) in view of Kecskemethy et al (US20190197366).
Regarding claim 8, Cao fails to teach an image processing method wherein the feature encoder and the feature reweighting device are obtained by training in following manner:
acquiring a first sample image and a second sample image, wherein an image type of the first sample image is different from an image type of the second sample image;
extracting a first feature from the first sample image by using a feature encoder to be trained to obtain a first sample feature; and extracting a second feature from the second sample image by using the feature encoder to be trained to obtain a second sample feature;
determining sample feature selection weights respectively corresponding to the first sample feature and the second sample feature by using a feature reweighting device to be trained, and performing the feature fusion on the first sample feature and the second sample feature based on the sample feature selection weights respectively corresponding to the first sample feature and the second sample feature to obtain sample fused feature data;
performing first decoding processing and second decoding processing on the sample fused feature data by using a feature decoder to obtain a first decoded image corresponding to the first sample image and a second decoded image corresponding to the second sample image; and
determining a feature contrastive loss based on the first sample image, the second sample image, the first decoded image, and the second decoded image; and updating the feature encoder to be trained and the feature reweighting device to be trained based on the feature contrastive loss to obtain the feature encoder and the feature reweighting device.
However Kecskemethy teaches acquiring a first sample image and a second sample image, wherein an image type of the first sample image is different from an image type of the second sample image (para. [0042]);
extracting a first feature from the first sample image by using a feature encoder to be trained to obtain a first sample feature (20 in fig. 1; para. [0044]); and extracting a second feature from the second sample image by using the feature encoder to be trained to obtain a second sample feature (21 in fig. 1; para. [0044]);
determining sample feature selection weights respectively corresponding to the first sample feature and the second sample feature by using a feature reweighting device to be trained (para. [0044], para. [0051]), and performing the feature fusion on the first sample feature and the second sample feature based on the sample feature selection weights respectively corresponding to the first sample feature and the second sample feature to obtain sample fused feature data (25 in figs. 1-2; para. [0054]);
performing first decoding processing and second decoding processing on the sample fused feature data by using a feature decoder (30 and 31 in figs. 1-2; para. [0054]) to obtain a first decoded image corresponding to the first sample image and a second decoded image corresponding to the second sample image (para. [0054], [0056]); and
determining a feature contrastive loss based on the first sample image, the second sample image, the first decoded image, and the second decoded image (para. [0055], mean squared error); and updating the feature encoder to be trained and the feature reweighting device to be trained based on the feature contrastive loss to obtain the feature encoder and the feature reweighting device (para. [0055]).
Therefore taking the combined teachings of Cao and Kecskemethy as a whole, it would have been obvious to one of ordinary skill in the art at the time the invention was filed to incorporate the steps of Kecskemethy into the method of Cao. The motivation to combine Kecskemethy and Cao would be to improve the accuracy of identifying regions of interest (para. [0010] of Kecskemethy).
Claim(s) 14 is/are rejected under 35 U.S.C. 103 as being unpatentable over Cao et al ("Attention fusion for one-stage multispectral pedestrian detection." Sensors 21.12 (2021): 4184, https://www.mdpi.com/1424-8220/21/12/4184, pages 1-17, retrieved from the Internet on 8/28/2026) and Li et al ("Selective kernel networks." 2019 IEEE/CVF conference on computer vision and pattern recognition (CVPR). IEEE, 2019, pages 510-519, retrieved from the Internet on 8/28/2026) in view of view of Lu et al (US20200057805).
Regarding claim 14, the modified method of Cao fails to teach an image processing method wherein the step of determining the feature selection weights respectively corresponding to the first image feature and the second image feature by using the feature importance vector comprises:
determining the feature selection weights respectively corresponding to the first image feature and the second image feature based on the feature importance vector, the first image feature and the second image feature.
However Lu teaches determining feature selection weights (para. [0066], αt) respectively corresponding to a first image feature and a second image feature (para. [0064], v1,…vk) based on a feature importance vector (para. [0063]-[0064], ht), the first image feature and the second image feature (para. [0064], v1,…vk).
Therefore taking the combined teachings of modified Cao and Lu as a whole, it would have been obvious to one of ordinary skill in the art at the time the invention was filed to incorporate the steps of Lu into the method of modified Cao. The motivation to combine Lu and modified Cao would be to significantly improve the performance of image captioning (para. [0059] of Lu).
Allowable Subject Matter
Claim 9 is objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Related Art
Gowda et al (US20230237785) – see figs. 3-5
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to LEON VIET Q NGUYEN whose telephone number is (571)270-1185. The examiner can normally be reached Mon-Fri 11AM-7PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Gregory Morse can be reached at 571-272-3838. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/LEON VIET Q NGUYEN/Primary Examiner, Art Unit 2663
Prosecution Insights