Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Examiner notes the entry of the following papers:
Amended claims filed 2/25/2026.
Applicant’s arguments/remarks filed in amendment 2/25/2026.
Claims 1, 7-8, and 14-15 are amended. Claims 5-6, 12-13, and 19-20 are cancelled. Claims 1-4, 7-11, and 14-18 are pending.
Response to Arguments
Applicant presents arguments. Each is addressed.
Applicant argues “…the combination of Whatmough, Liang, and Chen fails to disclose at least the limitations…” of the amended claims. (Remarks, page 7, paragraph 3, line 1.) The argument is moot in view of new grounds of rejection necessitated by amendment.
Applicant argues “…Chen fails to disclose that the total loss function includes ‘a distance intersection over union (IoU) loss function’…” of the amended claims. (Remarks, page 8, paragraph 3, line 1.) The argument is moot in view of new grounds of rejection necessitated by amendment.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-4, 7-11, and 14-18 are rejected under 35 U.S.C. § 103 as being unpatentable over Whatmough, et al (FixyNN: Efficient Hardware for Mobile Computer Vision via Transfer Learning, herein Whatmough), and Kong, et al (YOLO-G: A Lightweight Network Model for Improving the Performance of Military Targets Detection, herein Kong).
Regarding claim 1,
Whatmough teaches a method of performing multiple machine learning tasks through a shared framework, executable by a processor (Whatmough, Figure 1, and, page 1, abstract, line 2 “This paper proposes FixyNN, which consists of a fixed-weight feature extractor that generates ubiquitous CNN features, and a conventional programmable CNN accelerator which processes a dataset-specific CNN. Image classification models for FixyNN are trained end-to-end via transfer learning, with the common feature extractor representing the transferred part, and
the programmable part being learnt on the target dataset.” And, page 6, column 1, paragraph 6 “The programmable accelerator is based on published results for the NVIDIA Deep Learning Accelerator (NVDLA) (Nvidia).”
PNG
media_image1.png
470
452
media_image1.png
Greyscale
In other words, “FixyNN” which consists of a fixed-weight feature extractor that generates ubiquitous CNN features, and a conventional programmable CNN accelerator which processes a dataset-specific CNN is a method of performing multiple machine learning tasks through a shared framework, and NVIDIA deep learning Accelerator (NVDLA) is a processor.), comprising:
receiving data corresponding to a plurality of predetermined machine learning tasks (Whatmough, Figure 1. In other words, input is receiving data and, plurality of Task Specific CNNs is plurality of predetermined machine learning tasks.);
performing, on the received data, one or more steps of the machine learning tasks associated with the received data [by a shared backbone] of a machine learning model (Whatmough, Figure 1. And, page 1, abstract, line 2 “This paper proposes FixyNN, which consists of a fixed-weight feature extractor that generates ubiquitous CNN features, and a conventional programmable CNN accelerator which processes a dataset-specific CNN.” In other words, generate ubiquitous CNN features (by executing the shared layers of the FFE) from the received data is performing one or more steps of the machine learning tasks associated with the received data of a machine learning model.), wherein the shared backbone is
[trained based on minimizing a composite loss function associated with each machine learning task of the plurality of machine learning tasks, the composite loss function including a distance intersection over union (IoU) loss function for comparing a detected box outlining a feature to a bounding box, a focal loss function for bounding box confidence, and a cross entropy function for classification loss]; and
completing, by a plurality of sub-networks associated with each of the plurality of predetermined machine learning tasks, the predetermined plurality of machine learning tasks on the received data (Whatmough, Figure 1. In other words, Task Specific CNNs is plurality of sub-networks, and processing with Task Specific CNNs is completing, by a plurality of sub-networks associated with each of the plurality of machine learning tasks, the plurality of machine learning tasks. ) .
Thus far, Whatmough does not explicitly teach a shared backbone network.
Kong teaches a shared backbone (Kong, Figure 2, and, page 55549, column 2, paragraph 1, line 10 “To solve the above problems, we improved the structure of Darknet53 network and replaced the feature extraction network with GhostNet”
PNG
media_image2.png
386
904
media_image2.png
Greyscale
In other words, Darknet53 network is shared backbone.)
Kong teaches trained based on minimizing a composite loss function associated with each machine learning task of the plurality of machine learning tasks, the composite loss function including a distance intersection over union (IoU) loss function for comparing a detected box outlining a feature to a bounding box, a focal loss function for bounding box confidence, and a cross entropy function for classification loss (Kong, page 55553, column 1, paragraph 1, line 1 “ In the regression prediction of bounding box by IOU, if there is no overlap between the prediction box and the ground truth box, the gradient of loss function will be zero. To solve this problem, we introduced DIOU loss function for the regression prediction of bounding box, which can effectively solve the problems of inaccurate regression and slow convergence.” And, page 55553, column 2, paragraph 2, line 3 “In order to reduce the influence of negative examples on model optimization, we introduced Focal loss function to reduce the weight of negative examples, so that it can focus on more difficult and complicated classified samples.” And, page 55553, paragraph 3, line 1 “The classification loss can be calculated by the cross entropy loss function. The input image is divided into S X S grid cells, and each grid corresponds to three target prediction boxes.” And, page 55555, column 2, paragraph 4, line 8 “In the comparative experiment of several detection algorithms, we trained and tested YOLO-G and several advanced detection algorithms on self-built dataset, and the running environment of each algorithm is consistent.” In other words, DIoU loss function is distance intersection over union (IoU) loss function, bounding box is bounding box, focal loss function is focal loss function, classification loss can be calculated by cross entropy function is cross entropy function for classification loss, combining DIoU loss with focal loss and cross entropy loss is a composite loss function, and trained and tested is trained using based on minimizing the composite loss function.)
Both Whatmough and Kong are directed to a method for performing multiple machine learning tasks, among other things. Whatmough teaches a method of performing multiple machine learning tasks through a shared framework, executable by a processor, comprising receiving data corresponding to a plurality of predetermined machine learning tasks, performing, on the received data, one or more steps of the machine learning tasks associated with the received data by a machine learning model, and completing, by a plurality of sub-networks associated with each of the plurality of predetermined machine learning tasks, the predetermined plurality of machine learning tasks on the received data; but does not explicitly teach a shared backbone wherein the shared backbone is trained based on minimizing a composite loss function associated with each machine learning task of the plurality of machine learning tasks, the composite loss function including a distance intersection over union (IoU) loss function for comparing a detected box outlining a feature to a bounding box, a focal loss function for bounding box confidence, and a cross entropy function for classification loss. Kong teaches a shared backbone wherein the shared backbone is trained based on minimizing a composite loss function associated with each machine learning task of the plurality of machine learning tasks, the composite loss function including a distance intersection over union (IoU) loss function for comparing a detected box outlining a feature to a bounding box, a focal loss function for bounding box confidence, and a cross entropy function for classification loss.
In view of the teaching of Whatmough, it would be obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of Kong into Whatmough. This would result in a method of performing multiple machine learning tasks through a shared framework, executable by a processor, comprising receiving data corresponding to a plurality of predetermined machine learning tasks, performing, on the received data, one or more steps of the machine learning tasks associated with the received data by a shared backbone of a machine learning model, wherein the shared backbone is trained based on minimizing a composite loss function associated with each machine learning task of the plurality of machine learning tasks, the composite loss function including a distance intersection over union (IoU) loss function for comparing a detected box outlining a feature to a bounding box, a focal loss function for bounding box confidence, and a cross entropy function for classification loss, and completing, by a plurality of sub-networks associated with each of the plurality of predetermined machine learning tasks, the predetermined plurality of machine learning tasks on the received data.
One of ordinary skill in the art would be motivated to do this to speed up execution. (Kong, abstract, line 1 “ Military target detection technology is the foundation and key to perceive and analyze the battlefield situation, and it is also the premise of target tracking technology. Aiming at the task of military target detection, the detection performance of traditional detection algorithms is poor in complex environment. We realized automatic detection of military targets in complex environment through deep learning. In this research, we improved the components of YOLOv3 and proposed a novel military target detection algorithm (YOLO-G).”)
Regarding claim 2,
The combination of Whatmough and Kong teaches the method of claim 1, wherein
the plurality of predetermined machine learning tasks comprises object classification, scene classification, and situation recognition (Kong, Figure 13, and page 55560, column 2, paragraph 1, line 14 “However, our method can effectively detect all kinds of targets with high confidence. Although the fifth scene is very simple, it contains the situation of occlusion. From the detection results, it can be seen that other detection models can correctly detect the two kinds of targets in the figure, except SSD which identifies Soldier_RPG as Soldier_R. Faster R-CNN has a better detection effect on the occluded target, second only to our YOLO-G model,
but superior to other one-stage detectors.”
PNG
media_image3.png
662
1164
media_image3.png
Greyscale
In other words, soldiers are objects, scene is scene, and situation is situation recognition.)
Regarding claim 3,
The combination of Whatmough and Kong teaches the method of claim 1, wherein each of the one or more sub-networks is configured to
perform a specific individual machine learning task (Whatmough, Figure 1. In other words, the Task Specific CNN backends shows tasks 1 through N, where each specific CNN backend is addressing a specific individual task.).
Regarding claim 4,
The combination of Whatmough and Kong teaches the method of claim 1, wherein the shared backbone is configured to
perform one or more common initial steps of the plurality of predetermined machine learning tasks (Whatmough, Figure 1, and, page 2, column 1, paragraph 2, line 3 “The first part of the network implements a set of layers that are common for all CV tasks, essentially producing a set of universal low-level CNN features that are shared for multiple different
tasks or datasets. The second part of the network provides a task-specific CNN back-end.” In other words, the fixed weight feature extractor (FFE) executes a set of layers that are common for all CV tasks which is perform one or more initial steps of the plurality of machine learning tasks. Examiner notes that shared backbone is previously mapped to Kong.) .
Regarding claim 7,
The combination of Whatmough and Kong teaches the method of claim 5, wherein
each sub-network from among the plurality of sub-networks is trained for the individual machine learning task associated with the sub-network separately from the shared backbone (Whatmough, Figure 1, and page 2, column 2, paragraph 1 “Section 4 describes how a fixed feature-extractor can be used with transfer learning principles to train networks for a variety of CV datasets of varying sizes.” In other words, task specific CNN is subnetwork, and fixed feature extractor (FFE) can be used with transfer learning principles to train networks (the task specific CNNs) is each sub-network is trained for the individual machine learning task, separately from the shared backbone. Examiner notes that since the FFE is used to train each of the sub-networks, it necessarily must be trained first, which is separately from the sub-networks. Examiner further notes, that shared backbone is previously mapped to Liang. )
Claims 8-11, and 14 are computer system claims corresponding to method claims 1-4, and 7, respectively. Otherwise, they are the same. The combination of Whatmough and Kong teaches a computer system (Kong, page 55555, column 1, paragraph 2, line 1 “All the experiments in this article were carried out on the workstation with Ubuntu16.04 operating system. The graphics processor is GeForce RTX 2080ti x 2, and the memory is 16GB.” In other words, workstation is computer system.) Therefore, claims 8-11 are rejected for the same reasons as claims 1-4, respectively.
Claims 15-18 are non-transitory, computer readable medium claims corresponding to method claims 1-4, respectively. Otherwise, they are the same. The combination of Whatmough and Kong teaches a non-transitory computer readable medium. (Kong, page 55555, column 2, paragraph 2, line 1. See above mapping. In other words, memory is 16GB is non-transitory, computer readable medium.) Therefore, claims 15-18 are rejected for the same reasons as claims 1-4, respectively.)
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to BART RYLANDER whose telephone number is (571)272-8359. The examiner can normally be reached Monday - Thursday 8:00 to 5:30.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Miranda Huang can be reached at 571-270-7092. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/B.I.R./Examiner, Art Unit 2124
/MIRANDA M HUANG/Supervisory Patent Examiner, Art Unit 2124
Prosecution Insights