DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of Claims
Claims 1-22 are pending. Claims 1, 5-6, 8, 13, 15, 17 and 20 have been amended. Claims 21-22 are new. Claims 1-21 are rejected. Claim 22 is objected to as being but would be allowable if rewritten in independent form including all of the limitations of the base claim.
Response to Arguments
This final office action is in response to the amendment filed February 26, 2026. Claims 1-22 are pending in this application and have been considered below.
Applicant’s arguments with respect to claims 1-22 have been considered but are moot in view of new ground(s) of rejection set forth below, necessitated by Applicant's amendments.
Applicant argues Nguyen's inter-class network is itself the classifier, not a separate classifier differing from the NN. This is acknowledged. The new rejection relies on Neurala (US 2024/0289625) as primary, which explicitly teaches Module B as an incremental classifier (ART, RBM, or SVM) that is separate from and differs from the base DNN (Module A).
Applicant argues there was no proper motivation to combine the references. The new combination of Neurala, Savvides, and Nguyen are all within the field of computer vision and neural network-based object detection/recognition, providing ample motivation to combine under KSR.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1-21 is/are rejected under 35 U.S.C. 103 as being unpatentable over Neurala (US 2024/0289625 A1, hereinafter "Neurala") in view of Savvides et al. (US 2018/0068198 A1 – hereinafter “Savvides”) in view of Nguyen et al. (US 2020/0042864 A1 – hereinafter “Nguyen”).
Claims 1, 8, and 15.
Neurala, Savvides, and Nguyen disclose a non-transitory computer readable medium storing instructions that, when executed by at least one processor, cause the at least one processor to perform operations (Neurala ¶49) for transfer learning (Neurala ¶¶111-112 discloses “greedy layer-wise pre-training allows DNNs to perform unsupervised learning, by training each layer in turn, from the bottom-up … This process is called pre-training as it tends to precede supervised learning later ("fine-tuning").”), comprising:
(a) obtaining new object images and new object bounding shape information indicative of new object bounding shapes that are indicative of dimensions of a new object, wherein each new object image includes a new object that is associated with a new object bounding shape of the new object bounding shapes (Neurala ¶49 teaches obtaining the images, “An input source 100, such as a digital camera … acquires information/data from the environment.” ¶¶54-55 teaches the new object aspect, ¶54 teaches “continuously processes data ( e.g., tensor 210) from the DNN 200 as the input source 100 provides data relating to the environment” for on-the-fly learning after deployment, and ¶55 states, “In the fast learning mode, when a novel set of features is presented as input from Module A.” The novel set of features are the new object images being fed in. Savvides, claim 1, teaches "receiving the image . .. projecting the bounding box back to each of the at least two feature maps to identify a region of interest," where bounding boxes define the dimensions of the detected object.);
(b) feeding the new object images to a neural network (NN) that is trained to detect objects that differ from the new object (Neurala, ¶48: "Module A includes a pretrained DNN, and Module Bis based on a fast-learning Adaptive Resonance Theory (ART) paradigm, where the DNN feeds to the ART the output of one of the latter feature layers.” ¶50: "the input data is fed to a pre-trained Deep Neural Network (DNN) 200 in Module A. The DNN 200 includes a stack 202 of convolutional layers 204 ... The DNN 200 can be factory pre-trained before deployment." Module A is pre-trained on known object classes prior to deployment; when a new object is encountered post-deployment, the new object images are fed through this already-trained NN (¶51 : "A desired raw convolutional output of high level feature extraction layer 204 is accessed to serve as input to Module B 104").);
(c) generating, per each layer out of a group of candidate layers of the NN and for each new object image of the new object images, (i) a features map regarding a new object bounding shape of the new object image, and (ii) a features map regarding an external region of the new object image, the external region is located outside the new object bounding shape of the new object image (Savvides, ¶42: “the region proposal is projected into feature maps from multiple convolution layers. ROI-pooling is performed in each layer." ¶52: “additional ROI-pooling operations are performed for each region proposal in convolution feature maps to represent the body context features.” The context ROI corresponds to the external region outside the bounding shape. ¶68: "the object detection system projects the bounding box back to each of the at least two feature maps of differing scale to identify an ROI in each of the at least two feature maps. At step 350, the object-detection system also identifies a corresponding context region for each bounding box." The ROI is the bounding shape region feature map; the context region is the external region feature map; both are generated per convolutional layer);
(d) building, per each layer out of a group of candidate layers of the NN, an object classifier configured to distinguish between a bounding shape region related to the new object and an external region related to the new object wherein an object classifier of a corresponding layer is built based on (i) features maps, generated by the corresponding layer, regarding the bounding shape region, and (ii) features maps, generated by the corresponding layer, regarding the external region (Neurala, ¶48: "Module B is based on a fast-learning Adaptive Resonance Theory (ART) paradigm, where the DNN feeds to the ART the output of one of the latter feature layers ... Other configurations are possible, where multiple DNN layers can provide inputs to one or more Modules B." ¶52: "There can be one-to-one or one-to-many correspondence between each late stage convolutional layer 204 in Module A 102 and a respective fast learning neural network classifier in Module B 104." Module B classifiers (ART, RBM, SVM per ¶39) are structurally separate from the DNN of Module A and thus "differ from the NN." The combined Neurala/Savvides system builds these per-layer Module B classifiers using the ROI features (bounding shape region) and context region features (external region) extracted at each candidate layer per Savvides ¶¶ 42, 68);
(e) selecting, out of the group of candidates layers, a selected layer, wherein the selecting is based on a comparison between one or more classifier parameters of object classifiers associated with the candidates layers (Nguyen, ¶21: "classification results from each neural network engine are analyzed and the best output is identified … a performance score can be assigned to each of the classification results." if ¶63: "a performance score can be assigned to each of the layers based on the classification result. Layers having a performance score below a given threshold (e.g., 50% accuracy) can be eliminated." ¶36: the layer selection process repeats until "the performance score is above an accuracy threshold." The per-layer performance scores are compared to select the best-performing layer.); and
(f) associating the selected layer with a detection of the new object (Neurala, ¶¶ 48, 90: the user can triggering fast learning mode for new objects so the machine can learn the new object and modify its behavior and adapt to the new object. ¶52: the one-to-one correspondence between each convolutional layer and its respective Module B classifier means that layer is the dedicated detection path for the associated object path. Once Module B learns the new object at a given layer, that layer is the associated detection layer for that object. Nguyen, ¶36: confirms the selection upon satisfactory performance, the selected layer configuration is associated with the classification task, and "process 400 is repeated” with another object in the input file, confirming each object is associated with its selected layer.).
Neurala does not explicitly teach generating feature maps for a bounding shape region
versus an external region as recited in (c)-(d). Savvides teaches this in the same field.
It would have been obvious to modify Neurala's feature extraction using Savvides' multiscale ROI-pooling. While Neurala provides a general high-level feature map, Savvides teaches that projecting bounding boxes back to multiple layers of differing scales isolates the object from background noise. A person of ordinary skill would combine these to ensure the ‘fast learning’ weights in Neurala are updated using only the most relevant, scale-invariant pixels, increasing classification accuracy for new objects in cluttered environments.
Neurala and Savvides do not explicitly teach selecting a layer based on classifier
parameter comparison as recited in (e). Nguyen teaches this in the same field.
Applying Nguyen’s performance scoring to the per-layer classifiers of Neurala/Savvides system would have been obvious. Since different convolutional layers have different levels of feature discriminability for specific objects., Nguyen’s method allows the system to automatically identify which layer produces the most reliable result. This selection process optimizes the system by associating a single, high-performing detection branch with the object, ensuring the edge device does not waste memory or power on underperforming layers.
Claims 2, 9, and 16.
Neurala, Savvides and Nguyen teach the non-transitory computer readable medium according to claim 8, wherein the associating includes adding a new branch to the NN, wherein the branch starts from the selected layer (Neurala, ¶52: "There can be one-to-one or one-to-many correspondence between each late stage convolutional layer 204 in Module A 102 and a respective fast learning neural network classifier in Module B 104." Module B adds a new classification branch connecting from the selected layer of Module A, reading on adding a new branch starting from the selected layer.).
Claims 3, 10, and 17.
Neurala, Savvides and Nguyen teach non-transitory computer readable medium according to claim 8, that stores instructions for training the new branch to detect the new object and generate a bounding shape around the new object (Neurala, ¶55: Module B uses fast, one-shot learning where "the corresponding weight vector is updated to generalize and cover the new input" or "a new category node is introduced in category layer F2 216." Combined with Savvides' bounding box generation (¶68: the system "projects the bounding box back ... to identify an ROI"), the trained branch detects the new object and outputs bounding shapes.).
Claims 4, 11, 18.
Neurala, Savvides and Nguyen teach non-transitory computer readable medium according to claim 8, wherein the one or more classifier parameters comprise a sensitivity index (Nguyen ¶35: "At 415, a performance score is generated by comparing the classification results with the ground truth data ... " ¶63: "Layers having a performing score below a given threshold (e.g., 50% accuracy) can be eliminated.").
Claim 5.
Neurala, Savvides and Nguyen teach wherein the object classifier is generated using one or a regression model or using a support vector machine (Neurala, ¶41 : Module B can be a Support Vector Machines (SVM)).
Claims 12 and 19.
Neurala, Savvides and Nguyen teach a non-transitory computer readable medium according to claim 8, wherein the NN is a convolutional NN (Neurala, ¶65).
Claims 6 and 13.
Neurala, Savvides and Nguyen teach the non-transitory computer readable medium according to claim 8, that stores instructions for repeating steps a - e for another new object to provide another selected layer associated with a detection of the other new object, wherein the other new object differs from the new object and from the objects that were used to train the NN (Neurala teaches continual/lifelong learning (Title/Abstract) where the process repeats for each new object class, with each class potentially associated with a different candidate layer.).
Claims 7 and 14.
Neurala, Savvides and Nguyen teaches the non-transitory computer readable medium according to claim 13, wherein the other selected layer is selected regardless of the selecting of the selected layer associated with the new object (Neurala's per-object layer selection is independent: each new object class undergoes its own evaluation of candidate layers. Combined with Nguyen's per-layer performance scoring, each object's best layer is determined independently).
Claim 21.
Neurala, Savvides and Nguyen teaches method according to claim 1, wherein per each layer out of the group of candidate layers of the NN, the object classifier is generated using a regression model or using support vector machines (Neurala teaches SVM classifiers as Module B implementations as shown above).
Allowable Subject Matter
Claim 22 is objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Ross Varndell whose telephone number is (571)270-1922. The examiner can normally be reached M-F, 9-5 EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, O’Neal Mistry can be reached at (313)446-4912. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see https://ppair-my.uspto.gov/pair/PrivatePair. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Ross Varndell/Primary Examiner, Art Unit 2674
Prosecution Insights