DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Objections
Claims 7, 14, and 19 objected to because of the following informalities: all of these claims recite “the regression map a four two-dimensional regression map,” which is grammatically incorrect, and should recite a verb between these two nouns to complete the phrase. For example, a suggestion would be “the regression map includes four two-dimensional regression maps,” as would be supported by ¶40 of the originally filed Specification. Appropriate correction is required.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The text of those sections of Title 35, U.S. Code not included in this action can be found in a prior Office action.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1–2, 6–9, 13–15 and 18–20 are rejected under 35 U.S.C. 103 as being unpatentable over Zhou et al., “Bottom-up Object Detection by Grouping Extreme and Center Points,” arXiv:1901.08043v3 [cs.CV] 25 Apr 2019, https://doi.org/10.48550/arXiv.1901.08043 (herein “Zhou”) in view of Lan et al., “SaccadeNet: A Fast and Accurate Object Detector,” 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (herein “Lan”).
Regarding claims 1, 8 and 15, with substantive differences between the claims noted in curly brackets {}, deficiencies of Zhou noted in square brackets [], and with claim 1 as exemplary, Zhou teaches {an apparatus for object detection, comprising – claim 1 / a method for object detection, the method comprising – claim 8 / a non-transitory computer readable storage medium comprising instructions that, when executed, cause a processor to at least – claim 15}: {at least one memory; instructions in the apparatus; and processor circuitry to execute the instructions to: - claim 1} (Zhou page 6, section 5.2, implementation of disclosed bottom-up object detection (method) by grouping, performed on a GPU which would be understood by a person having ordinary skill in the art (herein “PHOSITA”) to include memory (non-transitory computer readable storage medium) and execute instructions on the processor of the GPU)
receive an input image (Zhou fig. 3, page 3, network takes an image as input);
identify a first grouping reference box for a first object representation in the input image, the first grouping reference box based on feature extraction performed with a feature extractor network (Zhou fig. 3, page 4, sections 4, and 4.1, HourglassNet backbone network used to detect keypoints (feature extraction) per class, with resulting heatmaps that are used to group detections in a geometric manner, thus a reference for grouping (grouping reference), and where the detected keypoints pertain to different sides of an object (first object representation in the image) where page 3, Keypoint detection section, further details that the output of the hourglass network is heatmaps, and where fig. 4 illustrates that bounding boxes are determined from the keypoints);
extract a first coordinate and a second coordinate for a [corner] location from a heatmap, the heatmap used to determine the first grouping reference box (Zhou page 4, section 4.1, a center coordinate comprised of c =
l
x
+
t
x
2
,
t
y
+
b
y
2
(two coordinates, thus a first and second coordinate) determining the center of the bounding box (first grouping reference box) is determined from a heatmap);
generate a second grouping reference box for the first object representation based on the [corner] location (Zhou page 5, section 4.2, a center object has two box choices as a result of the center grouping algorithm on page 4, Algorithm 1, where a smaller box (second grouping reference box) and a much larger box are both determined from the center point (the location)); and
when the [corner] location of the first grouping reference box surpasses a corner location threshold of the second grouping reference box, update the first grouping reference box with the second grouping reference box (Zhou page 5, section 4.2, if boxes are contained within a larger bounding box, thus, the bounding box having a corner defining a box encompassing (surpassing) a box with a corner within the larger box, the larger box then is determined to be a ghost box and removed, and the smaller box (second grouping reference box) remains (update)).
Zhou does not, but Lan teaches calculating corner coordinate locations for a bounding box from the center point (Lan page 10397, left column, first and second full paragraphs, part of section 3.3, Aggregation-Attn refines the object boundary boxes by calculating the corresponding top-left, top-right, bottom-left, bottom-right corners from a known centering position).
Therefore, taking the teachings of Zhou and Lan together as a whole, it would have been obvious to a PHOSITA before the effective filing date of the claimed invention to have modified the bounding box determination and ghost box suppression teachings of Zhou with the calculation of corner coordinates of the boxes as disclosed in Lan at least because doing so would provide more accurate boundary estimates (see Lan page 10397, upper right column).
Regarding claims 2, 9 and 20, Zhou teaches wherein the feature extractor network is a convolutional encoder-decoder network for keypoint-based detection (Zhou page 3, keypoint detection section, the keypoint estimation using a fully convolutional encoder-decoder network).
Regarding claims 6, 13 and 18, with claim 6 as exemplary, Zhou teaches wherein the processor circuitry is to train a reference box model to determine a width and a height of the second grouping reference box (Zhou fig. 3, and page 4, section 4, disclosed network is trained to produce offset maps, used for grouping per algorithm 1, where page 5, section 4.2 teaches Algorithm 1 determines a smaller box (second grouping reference box) that has a width and height, from the center point based on the offset maps, thus the training based offset maps are used “to determine” the second grouping reference box, and thus its height and width).
Regarding claims 7, 14 and 19, with claim 7 as exemplary, Zhou teaches wherein the processor circuitry is to generate a regression map for the second grouping reference box, the regression map a four two-dimensional regression map identified using smooth L1 training of the reference box model (Zhou fig. 3, page 4, left column, CornerNet section, network trained to produce four 2-channel (two-dimensional) category-agnostic offset maps, the offset maps trained with (thus in inferencing, they are identified using) Smooth L1 loss applied at ground truth peak locations, where for sub-pixel accuracy of the extreme points, the CornerNet network (also used by the disclosed ExtremeNet – see last paragraph on page 4), regresses to category-agnostic keypoint offset for each corner, thus the offset maps are regression maps).
Claims 3–5, 10–12 and 16–17 are rejected under 35 U.S.C. 103 as being unpatentable over Zhou in view of Lan, as set forth above regarding independent claims 1, 8 and 15, and further in view of He et al., “Softer-NMS: Rethinking Bounding Box Regression for Accurate Object Detection,” arXiv:1809.08545v1 [cs.CV] 23 Sep 2018 (herein “He”).
Regarding claims 3, 10 and 16, with claim 3 being exemplary, while Zhou does teach using a soft non-maxima suppression for the ghost box suppression, thus at least suggesting a soft-grouping algorithm and a non-maximum suppression (NMS) algorithm, Zhou does not explicitly teach the limitations of claim 3. However, He teaches wherein, when the corner location includes a first corner location and a second corner location, the processor circuitry is to group the first corner location and the second corner location using a soft-grouping (SG) algorithm and a non-maximum suppression (NMS) algorithm (He page 4, section 3.2, Algorithm 1, merging (grouping) the bounding boxes using a softer-NMS algorithm which determines corner coordinates for each box M (thus a first corner location and a second corner location) using a non-maximum suppression method).
Therefore, taking the teachings of Zhou as modified above by Lan, and He together as a whole, it would have been obvious to a PHOSITA before the effective filing date of the claimed invention to have modified the corner location grouping teachings of Zhou with softer-NMS algorithm as disclosed in He at least because doing so would improve localization accuracy for object detection (see He Abstract).
Regarding claims 4, 11, and 17, with claim 4 as exemplary, Zhou as modified by Lan does not explicitly teach where He teaches wherein the processor circuitry is to determine a distance metric corresponding to the first grouping reference box and the second grouping reference box, the distance metric shared between the SG algorithm and the NMS algorithm (He page 4, section 3.2, as shown in algorithm 1, for softer-NMS (softer grouping with NMS), an IoU metric is calculated for both the detection scores S and the M boxes and B matrix of initial boxes, where page 2, object detection loss section discusses the IoU as being an Intersection over Union loss function (distance metric) for bounding box prediction). The same motivation set forth above for combining the teachings of He with Zhou and Lan for claims 3 and 11 equally applies for claims 4, 11 and 17.
Regarding claims 5 and 12, with claim 5 as exemplary, Zhou as modified by Lan does not explicitly teach where He teaches wherein the distance metric is an Intersection over Union (IoU) distance metric determined as part of the NMS algorithm (He page 4, section 3.2, as shown in algorithm 1, for softer-NMS, the IoU distance is measured between box M and matrix B of initial detection boxes, where page 2, object detection loss section discusses the IoU as being an Intersection over Union loss function for bounding box prediction). The same motivation set forth above for combining the teachings of He with Zhou and Lan for claims 3 and 11 equally applies for claims 5 and 12.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
Dong et al., “CentripetalNet: Pursuing High-quality Keypoint Pairs for Object Detection,” 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), DOI 10.1109/CVPR42600.2020.01053, directed towards object detection using keypoint pairs and bounding boxes.
Newell et al., “Stacked Hourglass Networks for Human
Pose Estimation,” B. Leibe et al. (Eds.): ECCV 2016, Part VIII, LNCS 9912, pp. 483–499, 2016. Springer International Publishing AG 2016. DOI: 10.1007/978-3-319-46484-8 29, directed towards convolutional network architecture including feature extraction generating keypoints and heatmaps for human posed detection.
The following is not currently considered prior art, but is being made of record as pertinent to applicant’s disclosure: Wei et al., “RETHINKING THE GROUPING PROCESS IN CORNER-BASED DETECTORS,” 2022 IEEE International Conference on Multimedia and Expo (ICME), Doi: 10.1109/ICME52920.2022.9859905, directed towards a soft grouping based corner detector for object detection in images.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MICHELLE M KOETH whose telephone number is (571)272-5908. The examiner can normally be reached Monday-Thursday, 09:00-17:00, Friday 09:00-13:00, EDT/EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Vincent Rudolph can be reached at 571-272-8243. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
MICHELLE M. KOETH
Primary Examiner
Art Unit 2671
/MICHELLE M KOETH/Primary Examiner, Art Unit 2671
Prosecution Insights