Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claim 17 is rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter. The claim(s) does/do not fall within at least one of the four categories of patent eligible subject matter because the limitation “computer readable storage medium”, given its broadest reasonable interpretation, includes both transitory and non-transitory forms of signal transmission, in which transitory forms of signal transmission are not directed towards any of the statutory categories (see MPEP 2106.03). The Examiner recommends the applicant amend the claim language such that it explicitly states “A non-transitory computer readable medium having instructions…”.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-3, 9-11, and 17 are rejected as being unpatentable over Hua et al. (US 2016/0217349; hereinafter “Hua”) in view of Guo et al. (CN107590489; hereinafter “Guo”).
Regarding Claim 1, Hua teaches a method for object classification, comprising: executing a multi-class object classification model (OCM) on an input image depicting an object, wherein the multi-class OCM is configured to output, for each respective class of a plurality of classes, a respective confidence score indicative of a likelihood of the object being of a member the respective class ([0097], Hua teaches applying a classifier on multimedia data input, where the classifier will output a similarity value (i.e., a confidence value). The Examiner notes [0047], wherein the multimedia data input is an image, and [0019], [0046-0048], wherein the classifier is a multi-class classifier used for object recognition/categorization.); detecting a plurality of confidence scores outputted by the multi-class OCM that are within a threshold range, wherein classes associated with the plurality of confidence scores are candidate classes of the object (Fig. 7, [0101-0103], Hua teaches selecting a predetermined number of labels above a pre-determined threshold based on the highest ranked similarity.);
Hua does not explicitly disclose executing, on the input image, a single-class OCM for each of the candidate classes, wherein the single-class OCM is configured to determine whether a given object is a member of a specific candidate class of the candidate classes; and outputting a final class for the object in the input image based on a result of each single-class OCM (Fig. 8, [0104-0109], The Examiner notes that Hua teaches a process wherein the highest ranked labels (based on similarity) are processed and “re-ranked”, however this process does not involve the claimed process of applying a “single-class OCM”.).
PNG
media_image1.png
265
557
media_image1.png
Greyscale
Guo teaches executing, on the input image, a single-class OCM for each of the candidate classes, wherein the single-class OCM is configured to determine whether a given object is a member of a specific candidate class of the candidate classes; and outputting a final class for the object in the input image based on a result of each single-class OCM (Page 3, Guo teaches refining the preliminary detection results received from a multi-classifier, wherein a binary classifier (i.e., single-class OCM) for teach target type (i.e., class of object) is applied to determine a final accurate detection result.).
Hua and Guo are considered to be analogous to the claimed invention as they are in the same field of applying multi-class classifiers to images, and furthermore both Hua and Guo perform some degree of post-processing on the results output from a multi-class classifier. Therefore, it would have been obvious to one having ordinary skill in the art before the effective filling date of the claimed invention to modify the invention of Hua such that Hua’s post-processing of highest-ranking labels was replaced by Guo’s method, such that the highest-ranking labels are reclassified again using the binary classifiers taught by Guo, in order to ultimately determine a final class for an object in an image. The motivation for this combination being the ability to use label-specific models to refine the results produced by the multi-class classifier.
Claims 9 and 17 are the apparatus and computer readable medium (CRM) claims corresponding to claim 1, and are similarly rejected (see Fig. 1, Hua).
Regarding Claim 2, Hua in view of Guo teaches the method of claim 1, wherein the plurality of confidence scores are each greater than a threshold confidence score ([0103], Hua teaches selecting labels which are above a threshold. The Examiner notes that Hua’s threshold is analogous to the claimed “threshold confidence score” since the threshold defines which set of labels and their associated similarities are selected. The Examiner further notes [0050] wherein the threshold is based on statistics which are further based on the similarity values.).
Claim 10 is the apparatus claim corresponding to claim 2, and is similarly rejected (see Fig. 1, Hua).
Regarding Claim 3, Hua in view of Guo teaches the method of claim 1, wherein the threshold range is between a highest confidence score of the plurality of confidence scores and a lower confidence score that is a fixed amount below the highest confidence score ([0103], Hua teaches selecting a predetermined number of labels corresponding to similarity values above a predetermined threshold, wherein the range of the similarity values given by the number of labels is analogous to the claimed “threshold range”. The Examiner notes that Hua performs a ranking of the labels prior to determining a recognition result, and in a situation where multiple labels are returned (i.e., a predetermined number such as 5 labels), the 5th label present in the ranked list represents the claimed “lower confidence score” as it is a fixed amount (i.e., 5) below the highest confidence score, which is given by the top-ranking label.).
Claim 11 is the apparatus claim corresponding to claim 3, and is similarly rejected (see Fig. 1, Hua).
Claims 4 and 12 are rejected as being unpatentable over Hua in view of Guo in view of Steelberg et al. (US 2020/0285910; hereinafter “Steelberg”).
Regarding Claim 4, Hua in view of Guo teaches the method of claim 1.
Hua in view of Guo does not teach wherein the input image comprises a plurality of objects, further comprising: generating, by the multi-class OCM, a boundary around each classified object of the plurality of objects in the input image.
Steelberg teaches wherein the input image comprises a plurality of objects, further comprising: generating, by the multi-class OCM, a boundary around each classified object of the plurality of objects in the input image (Fig. 2, [0040], Steelberg teaches applying a classification engine to an image, wherein each object in an image is identified with a bounding box. The Examiner notes that Steelberg’s classification engine can identify different classes of objects from the same image.).
Hua, Guo, and Steelberg are considered to be analogous to the claimed invention as they are in the same field of applying multi-class classifiers to images. Therefore, it would have been obvious to one having ordinary skill in the art before the effective filling date of the claimed invention to modify the Hua in view of Guo such that it incorporates Steelberg’s method of identifying and utilizing a bounding box around each classified object in an image. The motivation for this combination being the ability to identify the locations of the detected objects in the image.
Claim 12 is the apparatus claim corresponding to claim 3, and is similarly rejected (see Fig. 1, Hua).
Claim 5 and 13 are rejected as being unpatentable over Hua in view of Guo in view of Steelberg in view of Elan et al. (US 2024/0193968; hereinafter “Elan”).
Regarding Claim 5, Hua in view of Guo in view of Steelberg teaches the method of claim 4.
Hua in view of Guo in view of Steelberg does not explicitly disclose extracting an image of the object from the input image based on a generated boundary around the object, wherein each single-class OCM is executed on the image extracted.
Elan teaches extracting an image of the object from the input image based on a generated boundary around the object, wherein each single-class OCM is executed on the image extracted ([0132], Elan teaches processing an image with a classifier which outputs bounding box information regarding an object of interest, from which the object of interest is extracted and provided to other subsequent neural networks for further processing.).
Hua, Guo, Steelberg, and Elan are considered to be analogous to the claimed invention as they are in the same field of applying classifiers to images. Therefore, it would have been obvious to one having ordinary skill in the art before the effective filling date of the claimed invention to modify the Hua in view of Guo in view of Steelberg such that the identified bounded objects taught by Hua in view of Guo in view of Steelberg are extracted using Elan’s methods, and furthermore processed using the specific binary classifier taught by Guo. The motivation for this combination being the ability to reduce the computational load by specifying a subset of an image for further processing.
Claim 13 is the apparatus claim corresponding to claim 5, and is similarly rejected (see Fig. 1, Hua).
Allowable Subject Matter
Claims 6-8 and 14-16 objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PROMOTTO TAJRIAN ISLAM whose telephone number is (703)756-5584. The examiner can normally be reached Monday - Friday 8:30 am - 5:00 pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Chan Park can be reached at (571) 272-7409. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/PROMOTTO TAJRIAN ISLAM/Examiner, Art Unit 2669
/CHAN S PARK/Supervisory Patent Examiner, Art Unit 2669