DETAILED ACTION
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA . Claims 1-20 are all the claims pending in the application. Claims 1-2, 13-15 and 20 are rejected. Claims 3-12 and 16-19 are objected to.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1, 13-14 and 20 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by “Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks” by Ren et al. (hereinafter “Ren”).
As to independent claim 1, Ren discloses a model training method (Abstract, Section 3, and Fig. 2 disclose that Ren is directed to a trained deep learning framework called Faster Region-Convolutional Neural Network (Faster R-CNN) comprising a Region Proposal Network (RPN) and Fast R-CNN), comprising: obtaining a first reference bounding box from a first candidate bounding box set (Section 3 discloses that Fast R-CNN inputs a bounding box i from a plurality of k anchor box proposals predicted by the RPN in response to an input image), and obtaining a real bounding box corresponding to the first reference bounding box and a real category of the real bounding box (Section 3 discloses “ground-truth label pi*” and “ground-truth box” ti* for each bounding box i); obtaining a target detection model by inputting the first reference bounding box, the real bounding box and the real category into a model to be trained for an iterative model training until the iterative model training meets a training termination criteria (Section 3 discloses that the trained Faster R-CNN is obtained by training according to a loss function (1) that involves “ground-truth label pi*” and “ground-truth box” ti*, wherein the Fast R-CNN portion inputs the bounding box i predicted by the RPN, wherein Section 4.2 discloses that the training is performed for a fixed number (240 k) “iterations”, such a fixed number indicating a stopping criteria for the iterative training); wherein, the model to be trained comprises a generation sub-model and a determination sub-model; each model training of the iterative model training comprises: performing a prediction based on the first reference bounding box, and obtaining a first prediction bounding box and a first prediction category by the generation sub-model (Section 3 discloses that the anchor bounding box i proposed by the RPN is input to the Fast R-CNN which predicts a label pi and a bounding box ti); generating, by the determination sub-model, a determination result set based on the real bounding box, the first prediction bounding box, the real category, and the first prediction category (Section 3 and equation (1) discloses calculating a loss function based on the set of i bounding boxes and corresponding ground-truth labels pi*, ground-truth boxes ti*, predicted labels pi and predicted bounding boxes ti); wherein the determination result set comprises a first determination result and a second determination result; determining a total loss value of the model to be trained based on the first determination result and the second determination result (Section 3 and equation (1) discloses that the loss function comprises a “classification loss Lcls” based on the ground-truth labels pi* and predicted labels pi, and a “regression loss” Lreg based on ground-truth boxes ti* and predicted bounding boxes ti); and updating model parameters of the generation sub-model and model parameters of the determination sub-model based on the total loss value (Sections 3-4 disclose training the Faster R-CNN model by updating “network weights” according to the loss function).
As to claim 13, Ren further discloses obtaining a second reference bounding box by extracting a target area of an image to be detected; obtaining a second prediction bounding box and a second prediction category of the second reference bounding box by inputting the second reference bounding box into the target detection model; and generating a target detection result of an image to be detected based on the second prediction bounding box and the second prediction category (Sections 3-4 and Fig. 1 discloses that the Faster R-CNN model, once trained, is deployed at “test-time” in which the model inputs an image, the RPN predicts proposal (reference) bounding boxes which are input to the Fast R-CNN which predicts a bounding box and classification label and outputs an image with the bounding box and classification label thereon as a target detection result; see Figs. 5-6 of such test set target detection results).
Independent claim 14 recites a computer device, comprising: a processor; and a storage device storing computer-executable instructions, which when executed by the processor, cause the processor (Section 4.2 of Ren discloses that the model is trained and tested using an “8-GPU implementation”, wherein such graphics processing units require software necessarily stored on a storage device; Abstract and Section 3 of Ren references such “Python code” which “has been made publicly available”) to perform the steps of the method recited in independent claim 1. Accordingly, claim 14 is rejected for reasons analogous to those discussed above in conjunction with claim 1.
Independent claim 20 recites a non-transitory storage medium having instructions stored thereon, when the instructions are executed by a processor of a computer device, the processor is configured to perform a model training method, wherein the method comprises (Section 4.2 of Ren discloses that the model is trained and tested using an “8-GPU implementation”, wherein such graphics processing units require software necessarily stored on a storage medium; Abstract and Section 3 of Ren references such “Python code” which “has been made publicly available”) the steps of the method recited in independent claim 1. Accordingly, claim 20 is rejected for reasons analogous to those discussed above in conjunction with claim 1.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 2 and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Ren in view of “Region-Based Convolutional Networks for Accurate Object Detection and Segmentation” by Girshick et al. (hereinafter “Girshick”).
As to claim 2, Ren further discloses that the first determination result represents a bounding box similarity of the first prediction bounding box and the real bounding box, and the second determination result represents a category similarity between the first prediction category and the real category (Section 3 and equation (1) discloses that the loss function comprises a “classification loss Lcls” based on the log loss (i.e., lack of similarity) between ground-truth labels pi* and predicted labels pi, and a “regression loss” Lreg based on the difference (i.e., lack of similarity) between ground-truth boxes ti* and predicted bounding boxes ti).
Ren discloses multiple examples of images including multiple objects for which the bounding box and classification label are predicted (See at least Figs. 3, 5, and 6). Accordingly, Ren likely inherently requires that the ground truth bounding boxes are compared with the predicted bounding boxes for objects of the same category or class. Nonetheless, Ren does not expressly disclose this feature. That is, Ren does not expressly disclose that the bounding box similarity is performed when a constraint is met, the constraint is that a category of a target object in the first prediction bounding box is predicted as a target category of the real category by the generation sub-model.
Girshick, like Ren, is directed to a “Region-based Convolutional Network” or R-CNN which inputs an image, extracts region proposals, and performs bounding box regression and classification on the proposals (Abstract and Fig. 1). In fact, Girshick is listed as a co-author on the Ren paper, and the Fast R-CNN component of Ren is derived from the R-CNN model disclosed in Girshick (See cited reference [2] in Ren: “R. Girshick, “Fast R-CNN,”’). Girshick discloses that R-CNN learns a “set of class-specific bounding-box regressors” such that the bounding box is predicted “after scoring each selective search proposal with a class-specific detection SVM” (Section 7.3). That is, R-CNN will only “transform an input proposal P into a predicted ground-truth box G^” when a constraint is met, the constraint is that a category of a target object in the first prediction bounding box is predicted as a target category of the real category by the generation sub-model (Section 7.3 discloses that R-CNN only applies a class-specific bounding box regressor after a proposal has been scored and classified which ensures that the regression loss for a particular class of objects is only ever calculated using proposals matched to ground-truth boxes of that particular class).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Ren to enforce a constraint that bounding box regression is conditional upon the predicted class of the object in the bounding box matching the ground-truth class, as taught by Girshick, to arrive at the claimed invention discussed above. Such a modification is the result of combining prior art elements according to known methods to yield predictable results. It is predictable that the proposed modification would have reduced computation considerably since the model would not waste time calculating loss between predicted bounding boxes with objects that don’t match the ground truth class.
Claim 15 recites features nearly identical to those recited in claim 2. Accordingly, claim 15 is rejected for reasons analogous to those discussed above in conjunction with claim 2.
Allowable Subject Matter
Claims 3-12 and 16-19 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Pertinent Art
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Yuan (“Multi-branch Bounding Box Regression for Object Detection”; cited in IDS filed 1/10/25) is directed to a bounding box regression and classification using a multi-branch R-CNN framework. Yuan discloses a backbone convolutional layer that regresses a first bounding box B0 which the framework uses to predict two bounding boxes B1 and D1 and a classification C1. A category cross-modal loss is minimized using classification C1 and its ground truth category, and a distance loss is calculated between the two predicted bounding boxes B1 and D1, wherein the model is trained on those two losses in multiple stages. These teachings are relevant to the broadest reasonable interpretation of the independent claims.
Shen (“Generative Adversarial Learning Towards Fast Weakly Supervised Detection”) is directed to a deep object detection framework using generative adversarial learning. Shen discloses a Surrogator network which inputs an image with region proposals and outputs a classification and a bounding box, a Generator which inputs images and performs bounding box regression thereon, and a Discriminator which distinguishes the “real” bounding boxes generated by the Surrogator from the “fake” bounding boxes generated by the Generator.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SEAN M CONNER whose telephone number is (571)272-1486. The examiner can normally be reached 10 AM - 6 PM Monday through Friday, and some Saturday afternoons.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Greg Morse can be reached at (571) 272-3838. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SEAN M CONNER/Primary Examiner, Art Unit 2663
Prosecution Insights