DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 13-14 are rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter. The claim(s) does/do not fall within at least one of the four categories of patent eligible subject matter because the claim is directed to towards a “computer program “ and a "computer program product" that includes "code instructions", which broadly encompasses a computer program per se. Such computer programs, per se, are not, in and of themselves, methods or machines, nor are they physical products of manufacture or compositions of matter. Therefore, such programs do not fall into any of the categories of eligible subject matter defined in 35 U.S.C. § 101 and are not, by themselves, eligible for patent protection. Such programs can be eligible for patent protection if claimed as embodied on or in a computer readable storage device or medium, but only if the claim clearly and unambiguously excludes transitory, propagating signals from the full scope of the claimed subject matter, as such signals are also not eligible under 35 U.S.C. § 101. It is suggested that amending the claim language to define the computer program product as having the code instructions embodied on a "non-transitory computer-readable medium" would satisfy these requirements and would limit the claimed invention to eligible subject matter.
Allowable Subject Matter
Claims 1-12 are allowed.
The following is an examiner’s statement of reasons for allowance:
Independent claims 1 and 11 are is directed to a method and the corresponding device, respectively, for object attribute classification in an image. The claims recites obtaining, from an ANN entity, a plurality of object proposals associated with feature map layers of different spatial resolutions. Each object proposal indicates an object class confidence score, an estimated object location, and attribute class confidence scores for a first attribute. The claims further recites identifying a first set including a main object proposal and one or more other object proposals, ranking the feature map layers in terms of ability to extract information useful for classification of the first attribute, and determining the attribute class using the attribute class confidence scores of the members of the first set while taking into account both the feature map layer ranking and object location overlaps with the main object proposal. The claims further requires a particular scoring procedure. For each candidate attribute class, the method determines terms equal or proportional to a product of: (i) an object location overlap size between a member of the first set and the main object proposal; (ii) an overall ranking score for the feature map layer associated with that member; and (iii) the attribute class confidence score indicated by that member. The overall attribute class score is then determined from a sum of those terms, and the attribute class having the highest overall attribute class score is selected.
The closest prior art of record includes Liu (US11087130B2), Shaw (US20210042588A1), and Solovyev et al., (Weighted Boxes Fusion: Ensembling Boxes from Different Object Detection Models, 2021).
Liu (US11087130B2) teaches simultaneous object localization and attribute classification using a multitask deep neural network. In particular, Liu teaches an ANN based object detection and attribute prediction environment including multiscale feature maps, bounding box regression, object class confidence scores, and attribute label confidence scores. Accordingly, Liu teaches the general use of a neural network to localize objects, classify objects, and predict object attributes using multiscale feature information. However, Liu does not teach or suggest the claimed post processing in which a first set including a main object proposal and one or more other object proposals is used to determine an attribute class by taking into account both object location overlap with the main object proposal and an attribute specific ranking of feature map layers. Liu does not teach ranking feature map layers according to their ability to extract information useful for classification of a particular attribute, nor does it teach determining an overall attribute class score from terms proportional to object location overlap size, feature map layer ranking score, and attribute class confidence score.
Shaw (US20210042588A1) teaches region proposal based object recognition and graph based non maximum suppression. Shaw teaches processing region proposals, class labels, class confidence scores, and overlapping proposal regions in order to retain or select object detections. Thus, Shaw shows that overlap based grouping or suppression of region proposals using confidence information was known in the object detection field. However, Shaw is directed to object detection selection and region proposal suppression, not to determining an object attribute class using attribute class confidence scores from a main object proposal and one or more other object proposals. Shaw does not teach ranking feature map layers by usefulness for classifying a particular attribute and does not teach weighting attribute class confidence scores according to both proposal overlap and feature map layer ranking.
Solovyev teaches combining object detection predictions by fusing bounding boxes and associated confidence scores. This reference shows that weighted fusion of object detections and confidence scores was known for improving object detection results. However, Solovyev is directed to fusion of bounding box detections, not to determining an attribute class for an object. The reference does not teach using attribute class confidence scores from multiple object proposals to determine an attribute class, does not teach ranking feature map layers according to their usefulness for a particular attribute, and does not teach the claimed product based attribute scoring term involving object location overlap size, feature map layer ranking score, and attribute class confidence score. Accordingly, while the prior art of record teaches ANN based object localization and attribute prediction, overlap based proposal processing, non-maximum suppression, and weighted fusion of object detections, the prior art does not teach or suggest the specific claimed combination. In particular, the prior art does not teach or suggest determining an object attribute class by: (i) identifying a first set including a main object proposal and one or more other object proposals; (ii) ranking feature map layers according to their ability to extract information useful for classification of the first attribute; and (iii) for each candidate attribute class, determining an overall attribute class score based on a sum of terms equal or proportional to a product of object location overlap size with the main object proposal, an overall ranking score for the feature map layer associated with the proposal, and the proposal’s attribute class confidence score.
Any comments considered necessary by applicant must be submitted no later than the payment of the issue fee and, to avoid processing delays, should preferably accompany the issue fee. Such submissions should be clearly labeled “Comments on Statement of Reasons for Allowance.”
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to WASSIM MAHROUKA whose telephone number is (571)272-2945. The examiner can normally be reached Monday-Thursday 8:00-5:00 EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Stephen Koziol can be reached at (408) 918-7630. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/WASSIM MAHROUKA/Primary Examiner, Art Unit 2665
Prosecution Insights