DETAILED ACTION
Claim Objections
Claims 1, 6 and 11 are objected to because of the following informalities: “PHD” is claimed. Acronym/abbreviations must be defined. Appropriate correction is required.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-11 is/are rejected under 35 U.S.C. 103 as being unpatentable over Zhang et al (NPL: Single-Image Crowd Counting via Multi-Column Convolutional Neural Network) in view of Vo et al (NPL: The Gaussian Mixture Probability Hypothesis Density Filter), in further view of Redmon et al (NPL: You Only Look Once: Unified, Real-Time Object Detection)
Regarding claim 1, Zhang discloses an information processing apparatus comprising:
at least one memory configured to store instructions (abstract Multi-column Convolutional Neural Network (MCNN) architecture to map the image to its crowd density map; wherein a MCNN would require a computer or embedded system with computational hardware and memory to run); and
at least one processor (abstract Multi-column Convolutional Neural Network (MCNN) architecture to map the image to its crowd density map; wherein a MCNN would require a computer or embedded system with computational hardware to run) configured to execute the instructions to perform:
training a neural network by use of one or more combinations of prepared learning image data and an ideal PHD for each of mutually different types of target objects (pg. 590-591 CNN needs to be trained to estimate the crowd density map from an input image; convert an image with labeled people heads to a map of crowd density; see further Fig. 2 Original images and corresponding crowd density maps obtained by convolving geometry-adaptive Gaussian kernels (e.g. the density map is the ideal distribution of the object locations, akin to PHD));
acquiring image data (pg. 594 Fig. 5 e.g. start with acquiring a test image);
generating likelihood data for each of a plurality of partial regions included in the image data by inputting the acquired image data to the trained neural network (pg. 591 2.1 Density map based on crowd counting: One is a network whose input is the image and the output is the estimated head count. The other one is to output a density map of the crowd (say how many people per square meter), and then obtain the head count by integration);
computing a distribution of a likelihood of existence of the target objects with respect to a position and a size by computing a total sum of the likelihood data (pg. 590 Input of the MCNN is the image, and its output is a crowd density map whose integral gives the overall crowd count), and
extracting, from the computed distribution, one or more partial distributions each of which relates to one target object (pg. 594 Table 1: Comparation of Shanghaitech dataset with existing datasets: Num is the number of images; Max is the maximual crowd count; Min is the minimal crowd count; Ave is the average crowd count; Total is total number of labeled people); and
Zhang fails to specifically teach where Vo teaches an ideal PHD for each of mutually different types of target objects (pg. 4094 C. The Probability Hypothesis Density (PHD) Filter: In other words, the integral v of over any region X gives the expected number of elements of that are in S. Hence, the total mass gives the expected number of elements of X), and fails to teach where Redmon teaches outputting, for each of the one or more partial distribution, a position and a size of the one target object relating to the partial distribution, based on a statistic of the partial distribution (pg. 1 Introduction: We reframe object detection as a single regression problem, straight from image pixels to bounding box coordinates and class probabilities. Using our system, you only look once (YOLO) at an image to predict what objects are present and where they are; pg. 2 Fig. 2: The Model. Our system models detection as a regression problem. It divides the image into an S × S grid and for each grid cell predicts B bounding boxes, confidence for those boxes, and C class probabilities; pg. 2 2. Unified Detection: Each bounding box consists of 5 predictions: x, y, w, h, and confidence. The (x, y) coordinates represent the center of the box relative to the bounds of the grid cell. The width and height are predicted relative to the whole image. Finally the confidence prediction represents the IOU between the predicted box and any ground truth box).
Therefore, it would have been obvious to one with ordinary skill in the art before the effective filing date of the invention to have implemented the teaching of an ideal PHD for each of mutually different types of target objects from Vo, and the teaching of outputting, for each of the one or more partial distribution, a position and a size of the one target object relating to the partial distribution, based on a statistic of the partial distribution from Redmon into the information processing apparatus as disclosed by Zhang. The motivation for doing this is to improve detection accuracy and improve estimating the time-varying number of targets and their states.
Regarding claim 2, the combination of Zhang, Vo and Redmon disclose the information processing apparatus according to claim 1, wherein the likelihood data is represented by a distribution conforming to a predetermined model (Zhang pg. 591 2.1 Density map based on crowd counting: One is a network whose input is the image and the output is the estimated head count. The other one is to output a density map of the crowd (say how many people per square meter), and then obtain the head count by integration), and for the each partial region, the trained neural network outputs a likelihood that a target object exists in the partial region and a parameter value of the predetermined model (Zhang pg. 591 2.2. convert an image with labeled people heads to a map of crowd density. Density map via geometry-adaptive kernels: If there is a head at pixel xi, we represent it as a delta function δ(x − xi); To convert this to a continuous density function, we may convolve this function with a Gaussian kernel[17] Gσ so that the density is F(x) = H(x) ∗ Gσ(x)).
Regarding claim 3, the combination of Zhang, Vo and Redmon disclose the information processing apparatus according to claim 1, wherein the at least one processor is configured to execute the instructions to perform:
computing a number of target objects included in the image data, based on an integral value of the distribution represented by the total sum of the likelihood data (Zhang pg. 590 Introduction: Input of the MCNN is the image, and its output is a crowd density map whose integral gives the overall crowd count), and
extracting as many as the number of the partial distributions from the distribution represented by the total sum of the likelihood data (Redmon pg. 1 Fig. 1 & Introduction: first generate potential bounding boxes in an image and then run a classifier on these proposed boxes. After classification, post-processing is used to refine the bounding boxes, eliminate duplicate detections, and rescore the boxes based on other objects in the scene; A single convolutional network simultaneously predicts multiple bounding boxes and class probabilities for those boxes; pg. 2 2. Unified Detection: Our network uses features from the entire image to predict each bounding box. It also predicts all bounding boxes across all classes for an image simultaneously. This means our network reasons globally about the full image and all the objects in the image.). The motivation to combine is discussed in claim 1 above.
Regarding claim 4, the combination of Zhang, Vo and Redmon disclose the information processing apparatus according to claim 1, wherein the at least one processor is configured to execute the instructions to perform: extracting the partial distributions (Vo pg. 4092 Introduction; It is shown that when the initial prior intensity is a Gaussian mixture, the posterior intensity at any subsequent time step is also a Gaussian mixture. Moreover, closed-form recursions for the weights, means, and covariances of the constituent Gaussian components are derived; pg. 4094 C. The Probability Hypothesis Density (PHD) Filter: the integral of over any region gives the expected number of elements of that are in) an integral value of each of which is 1 from the distribution represented by the total sum of the likelihood data (Zhang pg. 590 Introduction: Input of the MCNN is the image, and its output is a crowd density map whose integral gives the overall crowd count; pg. 591 If there is a head at pixel xi, we represent it as a delta function δ(x − xi). Hence an image with N heads labeled can be represented as a function:
PNG
media_image1.png
90
240
media_image1.png
Greyscale
; To convert this to a continuous density function, we may
convolve this function with a Gaussian kernel[17] Gσ so that the density is F(x) = H(x) ∗ Gσ(x); wherein Gσ(x) is a gaussian kernal normalized to 1). The motivation to combine is discussed in claim 1 above.
Regarding claim 5, the combination of Zhang, Vo and Redmon disclose the information processing apparatus according to claim 1, wherein the at least one processor is configured to execute the instructions to perform:
generating the likelihood data for each of mutually different types of the target objects (Redmon pg. 2 2. Unified Detection: Each grid cell predicts B bounding boxes and confidence scores for those boxes. These confidence scores reflect how confident the model is that the box contains an object and also how accurate it thinks the box is that it predicts.; Each grid cell also predicts C conditional class probabilities, Pr(ClassijObject). These probabilities are conditioned on the grid cell containing an object);
computing, for each of mutually different types of the target objects, a distribution of a likelihood of existence of the target objects (Redmon abstract: we frame object detection as a regression problem to spatially separated bounding boxes and associated class probabilities) and extracting the partial distribution from the distribution (Vo pg 4103 Conclusion: the posterior intensity at any time step is also a Gaussian mixture; derived closed-form recursions for the weights, means, and covariances of the constituent Gaussian components of the posterior intensity); and
outputting a position and a size of a target object relating to the each partial distribution along with a type of the target objects relating to the partial distribution (Redmon Fig. 2 Our system models detection as a regression problem. It divides the image into an S X S grid and for each grid cell predicts B bounding boxes, confidence for those boxes, and C class probabilities). The motivation to combine is discussed in claim 1 above.
Regarding claim(s) 6-10 (drawn to a method):
The rejection/proposed combination of Zhang, Vo and Redmon, explained in the rejection of apparatus claim(s) 1-5, anticipates/renders obvious the steps of the method of claim(s) 6-10 because these steps occur in the operation of the proposed combination as discussed above. Thus, the arguments similar to that presented above for claim(s) 1-5 is/are equally applicable to claim(s) 6-10.
Regarding claim(s) 11 (drawn to a CRM):
The rejection/proposed combination of Zhang, Vo and Redmon, explained in the rejection of apparatus claim(s) 1, anticipates/renders obvious the steps of the computer readable medium of claim(s) 11 because these steps occur in the operation of the proposed combination as discussed above. Thus, the arguments similar to that presented above for claim(s) 1 is/are equally applicable to claim(s) 11.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to KEVIN KY whose telephone number is (571)272-7648. The examiner can normally be reached Monday-Friday 9-5PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Vincent Rudolph can be reached at 571-272-8243. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/KEVIN KY/Primary Examiner, Art Unit 2671