DETAILED ACTION
Response to Arguments
Applicant’s amendments in the Amendment filed April 21, 2026 (herein “Amendment”) have re-written the pending claims and in turn have overcome the previous claim objections against claims 1–10, and claim rejections under 35 U.S.C. 112(b) against claims 1–10. Therefore the objections against claims 1–10, and rejections under 35 U.S.C. 112(b) against previous claims 1–10 are withdrawn.
Applicant's arguments filed in the Amendment regarding the rejection of claims 1–6 under non-statutory obviousness type double patenting have been fully considered but they are not persuasive. The claims in co-pending application no. 18/398,205 have also been amended and are currently not patentably distinguishable from the presently amended claims in the present application. Accordingly, the double patenting rejection is maintained, although updated to include the amended limitations, and provide updated secondary references where appropriate.
Applicant is reminded to comply with Amendment requirements set forth in MPEP §714, specifically providing that (with emphasis added in bold):
A canceled claim can be reinstated only by a subsequent amendment presenting the claim as a new claim with a new claim number. The original numbering of the claims must be preserved throughout the prosecution. When claims are canceled, the remaining claims must not be renumbered. For example, when applicant cancels all of the claims in the original specification and adds a new set of claims, the claim listing must include all of the canceled claims with the status identifier (canceled) (the canceled claims may be aggregated into one statement). The new claims must be numbered consecutively beginning with the number next following the highest numbered claim previously presented (whether entered or not) in compliance with 37 CFR 1.126.
Accordingly, Applicant’s portion of their amendment presenting the amended claims as a renumbered clean set is not entered. However, the amended claim set with the change markings which is compliant is considered the present claim set, and the amendments as presented in the amended claim set with change markings are entered. Reference to the claim numbers in this Action are those of the amended claim set with change markings.
Claim Objections
Claim 1 and therefore pending claims 2, 4–6, and 9–10 which depend therefrom are objected to because of the following informalities: in the last few lines of claim 1, there is recited “the original image” twice, however, this should be pluralized to be aligned with the antecedent basis “original images.” Appropriate correction is required.
Claim 10 is further objected to for depending upon claim 5, when the further features recited in claim 10 of “the optimization algorithm” feature antecedence based in claim 9 where “an optimization algorithm” is first introduced, and not in claim 5. Therefore, claim 10 must be amended to change its dependency to claim 9, not claim 5.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 1 and therefore pending claims 2, 4–6, and 9–10 which depend therefrom are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
First, claim 1 recites “the three sets of dimension-reduced channels,” although only the “another copy” and the “last copy” of the channels are claimed to “go through” (which is informal and imprecise language in and of itself that would be better recited as “processed by”) a dimension reduction. Without explicitly stating that the “one copy” (first copy) is also processed by a dimension reduction, an indefiniteness arises as to whether and where the first/one copy channel processing branch reciting only a “second inverted residual structure” requires a dimension reduction.
Second, claim 1 recites in the inputting limitation “the backbone-stair … comprises suitable structures” (emphasis added). The term “suitable” in claim 1 is a relative term which renders the claim indefinite. The term “suitable” is not defined by the claim, the specification does not provide a standard for ascertaining the requisite degree, and one of ordinary skill in the art would not be reasonably apprised of the scope of the invention. Therefore it is unclear and indefinite what would be included in the broadest reasonable interpretation for a “suitable” structure for a backbone (feature extraction) processing.
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The filing of a terminal disclaimer by itself is not a complete reply to a nonstatutory double patenting (NSDP) rejection. A complete reply requires that the terminal disclaimer be accompanied by a reply requesting reconsideration of the prior Office action. Even where the NSDP rejection is provisional the reply must be complete. See MPEP § 804, subsection I.B.1. For a reply to a non-final Office action, see 37 CFR 1.111(a). For a reply to final Office action, see 37 CFR 1.113(c). A request for reconsideration while not provided for in 37 CFR 1.113(c) may be filed after final for consideration. See MPEP §§ 706.07(e) and 714.13.
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer.
Claims 1–2, 4–6 and 9–10 are provisionally rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1, 3, and 5–8 of copending Application No. 18/398,205 (herein “‘205 application”) in view of Luqman et al., “Damage Detection and Localization in Masonry Structure Using Faster Region Convolutional Networks,” International Journal of GEOMATE, July 2019, Vol.17, Issue 59, pp.98-105Geotec., Const. Mat. & Env., DOI: https://doi.org/10.21660/2019.59.8272 ISSN: 2186-2982 (Print), 2186-2990 (Online), Japan (herein “Luqman”). This is a provisional nonstatutory double patenting rejection as the co-pending application, while being in a current status of the Notice of Allowance having been issued, has not yet issued as a patent.
Regarding claim 1 of the present application, the correspondence to claims 1 and 8 (which depends therefrom) of the ‘205 application is as follows, with deficiencies noted in square brackets [], and examiner added explanation added in curly brackets {}:
Limitation from claims 1 of the present application
Limitation from claims 1 and 8 of the ‘205 application
A method for fast detecting road cracks in images using a depth-stage dependent and hyperparameter-adaptive lightweight CNN-based model, comprising the following steps:
A three-stage modular convolutional neural network (CNN) for rapid classification of concrete cracks in images {would include roads made of concrete and having cracks}, comprising: (Claim 8 – A computer-implemented method for classifying concrete cracks in digital images, the method comprising: providing the three-stage modularized CNN according to any one of the claims 1–7)
collecting original images of road surface [and establishing a dataset from the original images, the dataset including a training set and a validation set];
an input layer configured to receive an input image: (Claim 8 - receiving, at the input layer of said CNN system, digital images)
inputting the images from the dataset into a backbone-stair to obtain feature maps, wherein the backbone-stair is depth-stage dependent and comprises suitable structures in different depths: a convolutional layer, a stair1 structure, a convolution block attention module (CBAM), a stair2 structure, another CBAM, and a stair3 structure,
a shallow-layer feature extraction module, designated as Stair1, operatively connected to the input layer … a first convolutional block attention module (CBAM) operatively connected to the shallow-layer feature extraction module (Stair1) … a mid-layer feature extraction module, designated as Stair2, … a second CBAM operatively connected to the mid-layer feature extraction module (Stair2); a deep-layer feature extraction module, designated as Stair3, operatively connected to the second CBAM …
(Claim 8 - processing the received digital images through the sequentially connected stages of said CNN, including performing feature extraction by the shallow-, mid-, and deep layer feature extraction modules (Stair1, Stair2, Stair3))
wherein basic components in the stair1 structure and the stair2 structure have variations according to an adjustment of hyperparameters;
Stair1 … having an expansion factor configurable as an integer during model initialization … {an expansion factor is a hyperparameter} wherein Stair2 comprises a plurality of basic block_2 structural blocks each having a stride configurable to 1 or 2 during model initialization {a stride is a hyperparameter}
wherein the stair1 structure comprises two variations for processing feature maps output from the convolutional layer: when an expansion factor is 1, the feature maps go through an inverted residual structure with convolutions; when the expansion factor is not 1, the feature maps go through a convolutional operation;
Stair1 comprises two basic block_1 structural blocks, each basic block_1 comprising an inverted residual block with convolutions (Convs) and having an expansion factor configurable as an integer during model initialization, wherein when the expansion factor is not 1, the basic block 1 consists of a first 3x3 Conv layer and a second 1 x 1 Conv layer; and when the expansion factor is 1, the basic block I consists of a single 3x3 Conv layer:
wherein the stair2 structure comprises two variations for processing feature maps output from the first CBAM: when a kernel stride is 1, channels of the feature maps are split into two equal parts using a split operation, one part goes through a first inverted residual structure with depth-wise separable convolutions while the other part remains unchanged, after that, the two equal parts of channels are concatenated and then subjected to a first shuffle operation;
a mid-layer feature extraction module, designated as Stair2, operatively connected to the first CBAM, wherein Stair2 comprises a plurality of basic block_2 structural blocks each having a stride configurable to 1 or 2 during model initialization, wherein: when configured for a stride of 1, the basic block_2 comprises a dual-branch structure having: (i) a partitioner unit configured to partition an input feature map of basic block 2 (stride = 1) into a first portion and a second portion; (ii) a first processing branch (stride = 1), coupled to the partitioner unit, configured to process the first portion, the first processing branch comprising a first 1x1 Conv layer, a 3x3 depthwise separable convolutional (DConv) layer, and a second 1 x 1 Conv layer connected in series {a series of three convolutional layers, one that constricts, a middle one that widens, and a third that constricts is known in the art as being an inverted residual structure}; (iii) a second processing branch (stride = 1), coupled to the partitioner unit, configured to pass through the second portion without processing (other part remains unchanged); (iv) a concatenator unit (stride = 1), coupled to outputs of the first and second processing branches (stride = 1), configured to concatenate an output of the first processing branch and an output of the second processing branch; and (v) a first channel shuffle unit, coupled to an output of the concatenator unit (stride = 1), configured to apply a channel shuffle operation to the concatenated output;
when the kernel stride is 2, channels of the feature maps are replicated into three copies, one copy goes through a second inverted residual structure including dimension reduction at the end, another copy goes through a depth-wise separable convolution followed by a first dimension reduction, a last copy goes through a max pooling operation followed by a second dimension reduction, and finally, the three sets of dimension-reduced channels are concatenated and then subjected to a second shuffle operation;
when configured for a stride of 2, the basic block 2 comprises a triple-branch structure having: (i) a replicator unit configured to replicate an input feature map of basic block 2 (stride = 2) into a first copy, a second copy, and a third copy; (ii) a first processing branch (stride = 2), coupled to the replicator unit, configured to process the first copy, the first processing branch comprising a first 1x 1 Conv layer, a 5x5 DConv layer, and a second 1x1 Conv layer connected in series {a series of three convolutional layers, one that constricts, a middle one that widens, and a third that constricts (dimension reduction) is known in the art as being an inverted residual structure that because of the DConv (depth convolution) would be depthwise}; (iii) a second processing branch (stride = 2), coupled to the replicator unit, configured to process the second copy, the second processing branch comprising a 5x5 DConv layer and a 1 x1 Conv layer connected in series {a series of three convolutional layers, one that constricts, a middle one that widens, and a third that constricts (dimension reduction) is known in the art as being an inverted residual structure that because of the DConv (depth convolution) would be depthwise}; (iv) a third processing branch (stride = 2), coupled to the replicator unit, configured to process the third copy, the third processing branch comprising a 3x3 max pooling layer and a 1 x 1 Conv layer {dimension reduction} connected in series; a concatenator unit, coupled to outputs of the first, second, and third processing branches (stride = 2), configured to concatenate the outputs of the first, second, and third processing paths (stride = 2); and (vi) a second channel shuffle unit, coupled to an output of the concatenator unit (stride = 2), configured to apply a channel shuffle operation to the concatenated output
wherein the stair3 structure comprises a residual structure for processing feature maps output from the second CBAM, the residual structure consisting of depth separable convolution and efficient channel attention(ECA); and
a deep-layer feature extraction module, designated as Stair3, operatively connected to the second CBAM, wherein Stair3 comprises an inverted residual structure with an efficient channel attention (ECA) module, the inverted residual structure in Stair 3 sequentially comprising: a first 1x1 Conv layer, a 5x5 DConvlayer, the ECA module, and a second 1 x 1 Conv layer;
While claims 1 and 8 of the ‘205 application teach the specific stair3 structure as given above, they do not teach the further processing upon the feature maps output from a structure.
Accordingly, Claims 1 and 8 of the ‘205 application do not teach or recite, but Luqman teaches establishing a dataset from the original images, the dataset including a training set and a validation set (Luqman page 103, database creation of original images taken by a drone of cracks in masonry, the images used for training and testing);
inputting feature maps obtained from the structure to a region proposal network (RPN) to generate proposals (Luqman page 101, feature maps output from a Fast R-CNN structure input to an RPN to generate boxes around portions believed to feature cracks in masonry), projecting the proposals onto the feature maps outputted by the structure to obtain corresponding feature matrices (Luqman pages 101–102, a conv layer followed by the ReLU activation function is slid on each pixel to obtain feature maps as shown in figure 5 as a matrix), passing the feature matrices through a region of interest (ROI) head to output predicted bounding boxes of road cracks in the feature maps (Luqman pages 101–102, sliding window features are mapped into a vector and given to a softmax layer and regressors, where section 3.2.3 on page 102 details the post RPN object proposal processing as being Fast R-CNN processing including ROI pooling processing of the obtained region proposals and outputting location and size of anchors/rectangular bounding boxes), and mapping the predicted bounding boxes back to the original image using post-processing to obtain positions and types of road cracks in the original image (Luqman page 102, fig. 8, results including labeled boxes on the input image showing the location of cracks in bricks, and where bricks can be used as road material, thus road cracks).
Therefore, taking the teachings and recitations of claims 1 and 8 of the ‘205 application and Luqman together as a whole, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have modified the image processing claimed in claims 1 and 8 of the ‘205 application with the image database, RPN and ROI pooling/processing disclosed by Luqman at least because doing so would provide detecting damage in structure with promising computational speed. See Luqman Abstract.
Regarding claim 2 of the present application, claim 2 corresponds with claims 3 and 8 (where 8 depends from claim 3) of the ‘205 application.
Regarding claim 4 of the present application, claim 4 corresponds with claim 3 and 8 of the ‘205 application.
Regarding claim 5 of the present application, claim 5 corresponds with the limitation of claim 1of the ‘205 application reciting “wherein Stair3 comprises an inverted residual structure with an efficient channel attention (ECA) module … the ECA module, and a second 1x1 Conv layer; and a fully connected layer operatively connected to the deep-layer feature extraction module (Stair3), configured to output a crack classification result.”
Regarding claim 6 of the present application, claim 5 corresponds with claims 6 and 8 of the ‘205 application.
Regarding claim 9 of the present application, claim 1 and 8 of the ‘205 application do not explicitly teach where Luqman teaches in a training phase, calculating a loss function that includes a classification loss and a regression loss for the RPN and the ROI head (Luqman page 102, network training including training the RPN again after training the Fast R-CNN which includes regressor output (regression loss) and a softmax output (classification loss)); and updating network parameters using an optimization algorithm to minimize the loss function until a network model converges (Luqman page 103, section 4.1, training converged on a 96.5% mean average precision, with the loss stabilization graph (showing the minimization of the loss function till convergence) in fig. 9 on page 104).
Therefore, taking the teachings and recitations of claims 1 and 8 of the ‘205 application and Luqman together as a whole, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have modified the image processing claimed in claims 1 and 8 of the ‘205 application with the training and convergence disclosed by Luqman at least because doing so would provide detecting damage in structure with promising computational speed. See Luqman Abstract.
Regarding claim 10 of the present application, claim 10 corresponds with claims 7 and 8 of the ‘205 application.
Allowable Subject Matter
Claims 1–2, 4–6 and 9–10 would be allowable if rewritten or amended to overcome the rejection(s) under 35 U.S.C. 112(b) and the obviousness-type double patenting rejections set forth in this Office action.
The following is an examiner’s statement of reasons for finding allowable subject matter. Regarding claim 1, and therefore claims depending therefrom, the cited art of record does not, in any combination obvious to one having ordinary skill in the art before the effective filing date of the claimed invention, teach or suggest the specific processing performed by the hyperparameter-adaptive lightweight CNN-based model claimed and all supporting limitations thereof. The closest cited art of record includes Guo and Luqman.
Both Guo and Luqman are directed towards detecting damage including cracks in pavement material, used for roads. Guo teaches the front end processing of feature extraction with a backbone using the MobileNetV3 network that includes depthwise separable convolution, inverse residual structures, and channel attention mechanisms. Gao also teaches channel attention processing (fig. 6) using spatial information in three branches including concatenating and pooling. Gao further also teaches detection and location of cracks using bounding boxes.
Luqman also teaches feature extraction of input images with cracks in pavement (masonry) material, but not to the level of detail given by Gao. Luqman however does detail the backend processing to include a region proposal network and region of interest processing including ROI pooling.
However neither Gao nor Luqman teach the specific sequence of processing claimed, including the CBAM and stair structures, and where the stair structures have variations according to hyperparameters such as an expansion factor and a kernel stride value.
Therefore, none of the cited prior art, whether considered alone or in a combination obvious to a person having ordinary skill in the art teaches or suggests the limitations of claim 1, and therefore claims 1, and claims 2, 4–6 and 9–10 which depend therefrom are allowable over the prior art.
Any comments considered necessary by applicant must be submitted no later than the payment of the issue fee and, to avoid processing delays, should preferably accompany the issue fee. Such submissions should be clearly labeled “Comments on Statement of Reasons for Allowance.”
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure: Yang et al., US Patent Application Publication No. US 2020/0250462 A1, directed towards processing feature maps of images to locate features, using a convolution block attention module, and in consideration of setting parameters of a network based on training losses.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MICHELLE M KOETH whose telephone number is (571)272-5908. The examiner can normally be reached Monday-Thursday, 09:00-17:00, Friday 09:00-13:00, EDT/EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Vincent Rudolph can be reached at 571-272-8243. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
MICHELLE M. KOETH
Primary Examiner
Art Unit 2671
/MICHELLE M KOETH/Primary Examiner, Art Unit 2671
Prosecution Insights