Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1, 6, 8 is/are rejected under 35 U.S.C. 103 as being unpatentable over Hoehne (US 20200279128 A1) in view of Otake (US 12423939 B2) and further in view of Zheng Ge (YOLOX: Exceeding YOLO Series in 2021 (prior art disclosed by instant application 18736715)).
Regarding claims 1, 6 and 8, Hoehne discloses a memory storing instructions ([0010] An object detection system (ODS) 102 may increase system throughput by reducing the processing power, memory, and number of computing cycles that would otherwise be required to identify the objects of an image 104.); and
a processor configured to execute the instructions to implement ([0010] An object detection system (ODS) 102 may increase system throughput by reducing the processing power, memory, and number of computing cycles that would otherwise be required to identify the objects of an image 104.):
detection operation for deriving bounding boxes of a detection target object and determine scores indicating reliability of the bounding boxes, based on a given image ([0028] In an embodiment, NMS may compare a first bounding box relative to an object 110A to a second, at least partially overlapping bounding box on the same object 110A. Then, for example, based on the relative confidence scores 114, an NMS processor (not shown) may discard the bounding box 108 associated with the lesser confidence scores.),
NMS (Non-Maximum Suppression) units, each of which is provided for each of the two or more detection units ([0028] In an embodiment, NMS may compare a first bounding box relative to an object 110A to a second, at least partially overlapping bounding box on the same object 110A. Then, for example, based on the relative confidence scores 114, an NMS processor (not shown) may discard the bounding box 108 associated with the lesser confidence scores.);
wherein each NMS unit eliminates bounding box overlap in overlapping bounding boxes output from a corresponding detection unit of the two or more detection units ([0028] In an embodiment, NMS may compare a first bounding box relative to an object 110A to a second, at least partially overlapping bounding box on the same object 110A. Then, for example, based on the relative confidence scores 114, an NMS processor (not shown) may discard the bounding box 108 associated with the lesser confidence scores.);
an overlap removal unit that removes either a first bounding box or a second bounding box when the first bounding box and the second bounding box overlap ([0028] In an embodiment, NMS may compare a first bounding box relative to an object 110A to a second, at least partially overlapping bounding box on the same object 110A. Then, for example, based on the relative confidence scores 114, an NMS processor (not shown) may discard the bounding box 108 associated with the lesser confidence scores.),
wherein the first bounding box is a bounding box obtained from a first partial image which is one of two adjacent partial images and overlapping with boundary of the two adjacent partial images ([0028] In an embodiment, NMS may compare a first bounding box relative to an object 110A to a second, at least partially overlapping bounding box on the same object 110A.),
and the second bounding box is a bounding box obtained from a second partial image which is the other of the two adjacent partial images and overlapping with the boundary ([0028] In an embodiment, NMS may compare a first bounding box relative to an object 110A to a second, at least partially overlapping bounding box on the same object 110A.)
wherein the overlap removal unit removes the first bounding box when center of the first bounding box protrudes from the first partial image and removes the second bounding box when center of the second bounding box protrudes from the second partial image (abstract: Pixels that do not correspond to a center of at least one of the bounding boxes are iteratively removed from the plurality of pixels until a subset of pixels each of which correspond to a center of at least one of the bounding boxes remains. Based on the subset, a final bounding box associated with each object of the image is determined based on an overlapping of the bounding boxes of the subset of pixels and the corresponding confidence scores.
[0027] In an embodiment, an image processing system may use an algorithm or combine or process the hundreds or thousands (or more) of varying bounding boxes 108 to identify a set of final bounding boxes 120 for image 104. The final boxes 120 may include one box per object 110. In an embodiment, the final boxes 120 may be calculated or determined from the predicated boxes 108 based on their overlap. For example, non-maximum suppression (NMS) may be used to compare two overlapping predicted bounding boxes 108 and eliminate the less accurate one (e.g., based on their relative confidence scores 114).
[0028] In an embodiment, NMS may compare a first bounding box relative to an object 110A to a second, at least partially overlapping bounding box on the same object 110A. Then, for example, based on the relative confidence scores 114, an NMS processor (not shown) may discard the bounding box 108 associated with the lesser confidence scores. NMS may repeat this process over and over again for the hundreds or thousands of bounding box combinations received from a neural network 106.).
Hoehne does not explicitly disclose a division unit that divides an input image to obtain partial images corresponding to partial regions indicated by region setting values, when a pair of the region setting values that indicate the partial regions in the input image and designation information designating the detection unit to be input destination for partial images corresponding to the partial regions is given;
a partial image input unit that inputs the individual partial images to designated detection unit; and
wherein the partial image input unit does not input a partial image for which no detection unit is designated as the input destination to any of the detection units.
In a similar field of endeavor of candidate region matching, Otake teaches a division unit that divides an input image to obtain partial images corresponding to partial regions indicated by region setting values (fig. 6e: claim 12, lines 3-5: FIG. 6E shows the adjusted confidence scores for the detection results. In this example, the confidence score for the detection result B is reduced to 0.68.), wherein the dividing the input image is performed when a pair of the region setting values that indicate the partial regions in the input image and designation information designating a detection unit of the two or more detection units to be input destination for partial images corresponding to the partial regions is given (fig 6d: col 9, lines 40-55: The designated region b (612) partially includes a person in a left end part, and thus a part of the face is detected incompletely in the detection results B and C as shown in FIG. 6D. On the other hand, the same person's face is completely detected in the designated region a (611), and thus a process is performed to properly unify these detection results. In the following processes, the detection results in all designated regions are treated simultaneously. In this example, three detection results A to C are acquired. In the example shown in FIG. 6D, rectangular candidate areas 613 to 615 corresponding to the detection results A to C are displayed in the display unit 104 such that they are superimposed on the input image 610.);
a partial image input unit that inputs the individual partial images to designated detection unit; and
wherein the partial image input unit does not input a partial image for which no detection unit is designated as the input destination to any of the two or more detection units (col 5, lines 20-25: For each overlap-detected group output by the overlap determination unit 203, the result correction unit 206 deletes detection results other than the detection result corresponding to the representative area determined by the representative area determination unit 204.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to combine the known system of Overlapping bounding Box Reductions, as disclosed by Hoehne, with the known methods of generating partial images from whole image, as taught by Otake, in order to yield the predictable results of filtering overlapping bounding boxes to select the highest-confidence prediction per object. Hoehne and Otake do not explicitly disclose two or more detection units wherein accuracy of the detection operation of each detection unit is different.
In a similar field of endeavor of candidate region matching, Ge teaches two or more detection units ("Figure 3:Training curves for detectors with YOLOv3 head or decoupled head. We evaluate the AP on COCO val every 10 epochs. It is obvious that the decoupled head converges much faster than the YOLOv3 head and achieves better result finally."), wherein accuracy of the detection operation of each detection unit is different ("Figure 3:Training curves for detectors with YOLOv3 head or decoupled head. We evaluate the AP on COCO val every 10 epochs. It is obvious that the decoupled head converges much faster than the YOLOv3 head and achieves better result finally.").
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to combine the known system of Overlapping bounding Box Reductions using partial images, as disclosed by Hoehne and Otake, with the known methods of using multiple detection heads, as taught by Ge, in order to yield the predictable results of using a non-maximum suppression that can select the best box from the localization head while using the confidence score from the classification head, yielding higher overall precision.
Response to Arguments
Applicant's arguments filed 08/07/2026 have been fully considered but they are not persuasive.
Regarding the claim that prior art Hoehne does not disclose claim 5 (which is now cancelled and amended into the independent claims), the examiner most respectfully disagrees.
Hoehne discloses:
(abstract: Pixels that do not correspond to a center of at least one of the bounding boxes are iteratively removed from the plurality of pixels until a subset of pixels each of which correspond to a center of at least one of the bounding boxes remains. Based on the subset, a final bounding box associated with each object of the image is determined based on an overlapping of the bounding boxes of the subset of pixels and the corresponding confidence scores.
[0027] In an embodiment, an image processing system may use an algorithm or combine or process the hundreds or thousands (or more) of varying bounding boxes 108 to identify a set of final bounding boxes 120 for image 104. The final boxes 120 may include one box per object 110. In an embodiment, the final boxes 120 may be calculated or determined from the predicated boxes 108 based on their overlap. For example, non-maximum suppression (NMS) may be used to compare two overlapping predicted bounding boxes 108 and eliminate the less accurate one (e.g., based on their relative confidence scores 114).
[0028] In an embodiment, NMS may compare a first bounding box relative to an object 110A to a second, at least partially overlapping bounding box on the same object 110A. Then, for example, based on the relative confidence scores 114, an NMS processor (not shown) may discard the bounding box 108 associated with the lesser confidence scores. NMS may repeat this process over and over again for the hundreds or thousands of bounding box combinations received from a neural network 106.).
The instant application states:
[0105] The overlap removal means may correct the score of the bounding box by multiplying the score of the bounding box which is obtained by the detection means 93 with lower accuracy of detection operation among the first bounding box and the second bounding box by a predetermined coefficient, compare the corrected score with the score of the other bounding box, and remove the bounding box corresponding to lower score.
[0106] The overlap removal means may remove the first bounding box when center of the first bounding box protrudes from the first partial image and remove the second bounding box when center of the second bounding box protrudes from the second partial image.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to associate the removal of a partially overlapping bounding box that is not centered on the object of interest, as disclosed by Hoehne, to have a similar effect of removing a bounding box that has a lower confidence score for an object, due to the object not being centered around the object, as taught by applicant’s specification.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
US 20220300739 A1 with respect to claim 1: [0038] The above example conventional five-step NMS post-processing method requires a huge number of sigmoid and/or exponential calculations to get the values needed for selection due to large amount of Deep learning model outputted results. Examples of this are shown in FIG. 2 (YOLO V3) and FIG. 3 (YOLO V5). The YOLO V3 post-process, which outputs 3 layers of feature maps, is shown in FIG. 2. Taking the 1.sup.st layer output of a 416 input_shape as an example: Outs[0].shape=(13, 13, 3, 85) referring to 13 width×13 column×3 anchor×85 channels (1 confidence+80 class score+4 box position) as shown. The similar post-process for YOLO V5 is shown in FIG. 3, and also has 3 layers features maps as outs, outs[0].shape=(1, 3, 80, 80, 85).
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to AHMED A NASHER whose telephone number is (571)272-1885. The examiner can normally be reached Mon - Fri 0800 - 1700.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Emily Terrell can be reached at (571) 270-3717. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/AHMED A NASHER/Examiner, Art Unit 2675
/EMILY C TERRELL/Supervisory Patent Examiner, Art Unit 2666
Prosecution Insights