DETAILED ACTION
Notice of Pre-AIA or AIA Status
1. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
2. Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55.
Oath/Declaration
3. The receipt of Oath/Declaration is acknowledged.
Information Disclosure Statement
4. The information disclosure statement (IDS) submitted on 12/08/2023 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Drawings
5. The drawing(s) filed on 12/08/2023 are accepted by the Examiner.
Status of Claims
6. Claims 1-14 are pending in this application.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
7. The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
8. This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier.
Such claim limitation(s) is/are:
“an object detector module” in claims 1, 6-9, 12 and 14;
“an object detection filtering module” in claims 1, 10, 12 and 14; and
“a threshold determining module” in claim 12.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
(a) Claim 1: ‘an object detector module configured to detect objects in an image’ corresponds to Fig. 3, ‘object detector module 304’. “object detector module 304 is configured to output one or more object detections 310 of objects classified as the first object class, wherein each object detection being associated with an object location (such as a center point, bounding box, etc.,) and a confidence score. As schematically shown in FIG. 3 in reference 310, each object location may comprise a bounding box of the detected object in the input image 302, 303.” [0039].
(b) Claim 1: ‘an object detection filtering module configured to operate on the output of the object detector module to remove any object detection associated with a confidence score below a first threshold' corresponds to Fig. 3 ‘object detection filtering module 306’. “object detection filtering module 306 is configured to remove any object detection of the one or more object detections in the output 310 being associated with a confidence score below a first threshold (the confidence threshold). The object detection filtering module 306 may then output the remaining object detections 312 for further processing according to the application in which the object detection system is implemented.” [0040].
(c) Claim 12: ‘a threshold determining module configured to: run the object detector module on a plurality of images depicting a scene and store outputs from the object detector module' corresponds to Fig. 3 ‘threshold determining module 308’. “threshold determining module 308 that is configured to run the object detector module on a plurality of images 302 depicting a scene and store outputs 310 from the object detector module 304, for example in storage 309.” [0041].
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 102
9. In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
10. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
11. The text of those sections of Title 35, U.S. Code not included in this action can be found in a prior Office action.
12. Claims 1-14 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Yamasaki (US 2022/0262031).
Regarding Claim 1:
Yamasaki discloses a method for setting a threshold in an object detection system (Yamasaki discloses an information processing method for adjusting a threshold for likelihoods in an object detection/estimation system, explicitly for improving detection accuracy by region-specific threshold control. See Abstract; ¶¶[001-0004]; Fig. 4 flowchart), the object detection system comprising:
an object detector module configured to detect objects in an image and to output one or more object detections of objects classified as a first object class, each object detection being associated with an object location and a confidence score (Yamasaki discloses an estimation unit that detects particular objects (e.g., people or heads) in an input image and outputs likelihoods indicating certainty of presence, each likelihood being associated with a position in the image. Likelihoods explicitly range from 0 to 1 and represent confidence scores. See ¶¶[0021], [0029], [0047-0049]; Figs. 3 and 4. Note that in this example the object is classified as a person), and
an object detection filtering module configured to operate on the output of the object detector module to remove any object detection associated with a confidence score below a first threshold (Yamasaki discloses excluding likelihoods below a threshold from counting targets by setting them to zero or otherwise removing them from further processing. This filtering operation is explicitly performed after threshold determination. See ¶¶[0029], [0033-0035], [0055]; Fig. 4 (steps S407-S409)), the method comprising:
a) running the object detector module on a plurality of images depicting a scene and storing outputs from the object detector module (Yamasaki discloses executing object detection on plural frames (moving images) and storing likelihood information and positions in a recording unit for later threshold adjustment. See ¶¶[0024], [0031], [0046], [0062]; Fig. 2 (recording unit 205));
b) for each region of a plurality of regions in the scene: based on the stored outputs, determining a representative confidence score of object detections being associated with an object location within said region (Yamasaki discloses dividing the image into a plurality of partial regions and, for each region, analyzing likelihood information within that region, including counts, distributions, and accumulated likelihoods, to characterize detection reliability locally. See ¶¶[0027-0033], [0036-0044]; Figs. 3, 5A-5C, 6 and 7);
[Examiner note: While Yamasaki does not use the phrase “representative confidence score”, the accumulated likelihoods, histograms, and region-level likelihood characterizations constitute a representative statistical measure of confidence for detections within each region.]
c) setting the first threshold for object detections associated with an object location within a first region of the plurality of regions based on the representative confidence score determined for the first region, wherein the first threshold is set to have a lower value if a higher representative confidence score has been determined for the first region compared to if a lower representative confidence score has been determined for the first region (Yamasaki explicitly teaches that in regions where many high likelihoods are present (indicating reliable detections), the threshold is lowered or eliminated, and in regions with fewer high likelihoods, the threshold is raised. This establishes the claimed inverse relationship. See ¶¶[0032-0035], [0063-0066]; Figs. 7-8),
characterized in that:
setting the first threshold comprises determining a difference between the representative confidence score and a predefined threshold confidence score, wherein upon the difference being above 0, the first threshold is set to a value lower than the predefined threshold confidence score, wherein upon the difference being below 0, the first threshold is set to a value higher than the predefined threshold confidence score (Yamasaki discloses using predetermined baseline threshold values (e.g., 0.02, 0.03) and adjusting thresholds upward or downward depending on whether region-level likelihood characteristics exceed or fall below those baseline values. See ¶¶[0029], [0032], [0065-0069]),
wherein the predefined threshold confidence score is a baseline confidence score of the object detection filtering module (Yamasaki explicitly discloses baseline thresholds used by the system to determine which likelihoods are excluded, prior to and after adjustment. See ¶¶[0029], [0033], [0065]).
Regarding Claim 2:
Yamasaki further discloses the method according to claim 1, wherein determining a representative confidence score comprises disregarding any object detections associated with a confidence score below a second threshold when determining the representative confidence score (Yamasaki discloses excluding likelihoods below a predetermined value from further consideration when analyzing region-level likelihood information and when calculating detection counts. Likelihoods below the threshold are explicitly removed or set to zero prior to further processing. See ¶¶[0029], [0033], [0053-0055]; Fig. 4 (steps S406-S409).
Regarding Claim 3:
Yamasaki further discloses the method according to claim 1, wherein a deviation between the first threshold and the predefined threshold confidence score increases with increasing size of the difference (Yamasaki discloses graduated threshold adjustment, wherein the degree to which the threshold is lowered depends on how many high likelihoods are present in the region (e.g., different ranges of likelihood counts correspond to progressively lower thresholds). This teaches an increasing deviation from a baseline threshold as region-level confidence increases. See ¶¶[0067-0073]; Figs. 7-8.).
Regarding Claim 4:
Yamasaki further discloses the method according to claim 3, wherein the value of the first threshold is capped at a minimum value and a maximum value (Yamasaki discloses threshold values selected from predefined discrete ranges, including cases where the threshold is not reduced below a minimum (e.g., zero or elimination) and not increased beyond defined upper values. The use of bounded threshold values inherently caps threshold adjustment. See ¶¶[0065-0069]; Fig. 7 (table 700)).
Regarding Claim 5:
Yamasaki further discloses the method according to claim 1, wherein, for each region of the plurality of regions, the representative confidence score is an average confidence score of object detections being associated with an object location within said region (Yamasaki discloses computing region-level confidence characteristics using accumulated likelihoods, histograms, and statistical aggregation of likelihood values within each region. These operations constitute averaging or equivalent statistical summarization of confidence scores across detections in a region. See ¶¶[0031], [0036], [0068]; Fig. 6).
Regarding Claim 6:
Yamasaki further discloses the method according to claim 1, wherein the object detector module is configured to associate each of the one or more object detections with a respective timestamp, wherein, for each region of the plurality of regions, the representative confidence score is a weighted average confidence score of object detections being associated with an object location within said region, wherein a weight is based on one or more of: a value of the confidence score, and the associated timestamp (Yamasaki discloses weighting the influence of likelihoods based on their magnitude (higher likelihoods are treated as more reliable) and updating threshold determinations over time for moving images, such that more recent detection results influence threshold setting. This teaches weighting confidence values and temporal relevance when determining region-level confidence. See ¶¶[0024], [0046], [0062], [0072]).
Regarding Claim 7:
Yamasaki further discloses the method according to claim 1, wherein the object detector module is configured to associate each of the one or more object detections with a respective timestamp, wherein the method further comprises: d) for each region of the plurality of regions in the scene: determining a latest timestamp for the region, by determining a latest timestamp in the stored output of an object detection being associated with an object location within said region; and e) adjusting the first threshold set in step c) for object detections associated with an object location within the first region of the plurality of regions based on the latest timestamp determined for the first region, wherein the first threshold is adjusted to have a higher value if an earlier latest time stamp has been determined for the first region, compared to if a later latest time stamp has been determined for the first region (Yamasaki discloses repeatedly performing detection and threshold adjustment for moving images and updating partial regions and thresholds as estimation results change between frames. Regions that have not recently produced reliable detections are treated differently from regions with recent high-confidence detections, thereby adjusting thresholds based on recency. See ¶¶[0046], [0062], 0072]).
Regarding Claim 8:
Yamasaki further discloses the method according to claim 1, further comprising the step of: continuously receiving images at the object detector module; wherein the steps (a) - (c) are performed at a plurality of points in time, wherein for each point in time, the object detector module is run on the latest received N>1 images Yamasaki explicitly discloses processing moving images, acquiring multiple frames, and repeatedly executing threshold adjustment processing as new frames are received. This corresponds to performing the method on the latest plurality of images over time. See ¶¶[0024], [0046], [0062], [0072]).
Regarding Claim 9:
Yamasaki further discloses the method of claim 7, further comprising the step of: continuously receiving images at the object detector module; wherein the steps (a) - (e) are performed at a plurality of points in time, wherein for each point in time, the object detector module is run on the latest received N>1 images (As discussed for claim 8, Yamasaki discloses repeated execution of detection, likelihood acquisition, region-based threshold adjustment, and updating across successive frames of a moving image. The temporal repetition applies equally to all steps, including threshold adjustment based on detection timing. See ¶¶[0046], [0062], [0072]).
Regarding Claim 10:
Yamasaki further discloses the method of claim 1, wherein step c) is performed for M>1 regions of the plurality of regions in the scene, the method further comprising the step of: receiving, at the object detection filtering module, a first object detection classified as the first object type, the object detection defined by a bounding box and being associated with and a confidence score, wherein said bounding box overlaps the M regions, determining the first threshold to be used for said first object detection based on the first threshold set for each region of the M regions (Yamasaki discloses handling likelihoods distributed across multiple partial regions and creating grouped or clustered regions for threshold adjustment, including cases where detections span multiple regions and region-level threshold logic is combined to determine filtering behavior. See¶¶[0057-0061]; Figs. 5A-5C).
Regarding Claim 11:
Yamasaki further discloses the method of claim 10, wherein the step of determining the first threshold to be used for said first object detection comprises one of: calculating an average value of the first thresholds set for each of the M regions and using the average value as the first threshold for the first object detection; calculating a maximum value of the first thresholds set for each of the M regions and using the maximum value as the first threshold for the first object detection; calculating a minimum value of the first thresholds set for each of the M regions and using the minimum value as the first threshold for the first object detection (Yamasaki discloses determining threshold values for regions based on region groupings and applying region-level threshold logic to detections associated with those regions. Selecting a threshold from among region thresholds (including conservative or permissive choices) is consistent with Yamasaki’s region aggregation teachings. See ¶¶[0057-0061], [0067-0069]).
Regarding Claim 12:
An object detection system (Yamasaki discloses an information processing apparatus/system for object detection and threshold adjustment. See Abstract; ¶¶[0003-0004], [0016-0018]; Figs. 1 and 2) comprising:
an object detector module configured to detect objects in an image and to output one or more object detections of objects classified as a first object class, each object detection being associated with an object location and a confidence score (Yamasaki discloses an estimation unit (204) configured to detect particular objects (e.g., people, heads) in an input image and to output likelihoods indicating certainty of presence, each likelihood being associated with position information in the image. Likelihoods are numerical confidence scores. See ¶¶[0021], [0029], [0047-0049]; Figs. 2-4);
an object detection filtering module configured to operate on the output of the object detector module to remove any object detection associated with a confidence score below a first threshold (Yamasaki discloses an adjustment unit (206) that compares likelihoods to a threshold and excludes likelihoods below the threshold from further processing (e.g., by setting them to zero), thereby removing low-confidence detections. See ¶¶[0029], [0033-0035], [0055]; Fig. 4 (steps S407-S409)); and
a threshold determining module configured to:
run the object detector module on a plurality of images depicting a scene and store outputs from the object detector module (Yamasaki discloses that the estimation unit executes object detection on plural images/frames and that likelihoods and position information are stored in a recording unit (205) for later use in threshold adjustment. See ¶¶[0024], [0031], [0046], [0062]; Fig. 2);
for each region of a plurality of regions in the scene:
based on the stored outputs, determine a representative confidence score of object detections being associated with an object location within said region (Yamasaki discloses dividing an image into a plurality of partial regions and, for each region, analyzing stored likelihood information (counts, accumulations, distributions) to characterize detection reliability locally. These region-level likelihood characterizations constitute representative confidence measure for each region. See ¶¶[0027-0033], [0036-0044]; Figs. 3, 5A-5C, 6 and 7);
set the first threshold for object detections associated with an object location within a first region of the plurality of regions based on the representative confidence score determined for the first region, wherein the first threshold is set to have a lower value if a higher representative confidence score has been determined for the first region compared to if a lower representative confidence score has been determined for the first region (Yamasaki explicitly teaches adjusting thresholds per region such that regions with many high-confidence likelihoods (indicating reliable detections) have lower thresholds, while regions with fewer high-confidence likelihoods have higher thresholds. This establishes the claimed inverse relationship between representative confidence and threshold value. See ¶¶[0032-0035], [0063-0069]; Figs. 7-8),
characterized in that:
the threshold determining module is configured to set the first threshold by determining a difference between the representative confidence score and a predefined threshold confidence score (Yamasaki discloses using predetermined baseline threshold values and adjusting region-specific thresholds upward or downward depending on whether region-level likelihood characteristics exceed or fall below those baseline values. The adjustment is explicitly relative to a predefined threshold used by the filtering logic. See ¶¶[0029, [0032], [0065-0069]),
wherein upon the difference being above 0, the first threshold is set to a value lower than the predefined threshold confidence score, wherein upon the difference being below 0, the first threshold is set to a value higher than the predefined threshold confidence score, wherein the predefined threshold confidence score is a baseline confidence score of the object detection filtering module (Yamasaki discloses baseline thresholds used by the system to determine which likelihoods are excluded prior to adjustment and which serve as reference values for subsequent region-specific threshold setting. See ¶¶[0029], [0033], [0065]).
Regarding Claim 13:
Yamasaki further discloses the object detection system of claim 12, being implemented in a monitoring camera (Yamasaki explicitly discloses that the object detection and threshold adjustment system is implemented in, or operates in conjunction with, a monitoring camera. The disclosure repeatedly identifies the imaging apparatus as a monitoring camera and describes the system as being used for analyzing images captured by such cameras. See ¶¶[0016], [0018], [0021], [0079]; Fig. 1 (imaging apparatus 110 as a monitoring camera)).
Regarding Claim 14: (drawn to a computer-readable storage medium)
The proposed rejection of system claim 12, over Yamasaki is similarly cited to reject the steps of the computer-readable storage medium of claim 14, because these steps occur in the operation of the system as discussed above. Thus, the arguments similar to that presented above for claim 12 are equally applicable to claim 14.
It is noted that Yamasaki discloses a computer-readable storage medium at least at ¶[0079-0081].
Conclusion
13. The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Koivisto et al. (US 2019/0258878) discloses wherein detected object data representative of locations of detected objects in a field of view may be determined. One or more clusters of the detected objects may be generated based at least in part on the locations and features of the cluster may be determined for use as inputs to a machine learning model(s). A confidence score, computed by the machine learning model(s) based at least in part on the inputs, may be received, where the confidence score may be representative of a probability that the cluster corresponds to an object depicted at least partially in the field of view. Further examples provide approaches for determining ground truth data for training object detectors, such as for determining coverage values for ground truth objects using associated shapes, and for determining soft coverage values for ground truth objects.
Chen et al. (US 2021/0150218) relates to a method of acquiring detection zone in image and method of determining zone usage, particularly to a method of acquiring detection zone and method of determining zone usage based on multiple moving traces of multiple objects in an image.
14. Any inquiry concerning this communication or earlier communications from the examiner should be directed to NEIL R MCLEAN whose telephone number is (571)270-1679. The examiner can normally be reached Monday-Thursday, 6AM - 4PM, PST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Akwasi M Sarpong can be reached at 571.270.3438. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/NEIL R MCLEAN/Primary Examiner, Art Unit 2681