Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Amendments
This action is in response to amendments filed June 23rd, 2026, in which Claims 1-3, 6-11, 14, 17-19, and 22 are amended. Claims 4, 5, 12, 13, 20, and 21 are cancelled. The amendments have been entered, and Claims 1-3, 6-11, 14-19, and 22-24 are currently pending.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-3, 6-11, 14-19, and 22-24 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claims 1, 9, and 17 recite to fine-tune at least one subset of the trained machine learning model from a frozen part of the trained machine learning model, which is indefinite because it appears to be saying to alter a frozen part of the machine learning model (fine-tune one subset from a frozen part), in which case the part is no longer frozen. It is unclear whether this is the intended interpretation, because it appears contradictory. For the purpose of examination, fine-tuning any previously pre-trained (where the end of pre-training denotes frozen, since the pre-training is complete) will be considered to fall within the scope of the claim limitation.
Dependent claims are rejected for inheriting the indefiniteness of a parent claim.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1-3, 7, 8; 9-11, 15, 16; 17-19, 23, and 24 are rejected under 35 U.S.C. 102 (a)(1) as being anticipated by Rajpura et al., “Transfer Learning by Finetuning Pretrained CNNs Entirely with Synthetic Images.” Lin et al., “Microsoft COCO: Common Objects in Context,” is relied upon to demonstrate an inherency in Rajpura.
Regarding Claim 1, Rajpura teaches a machine learning based system for optimizing learning time of a machine learning model (Rajpura, title, “Finetuning Pretrained CNNs” denotes optimizing learning time by finetuning a pretrained model rather than starting from scratch), the machine learning model comprising: one or more hardware processors; and a memory unit computed to the one or more hardware processors, wherein the memory unit comprises a set of program instructions in form of a plurality of subsystems, configured to be executed by the one or more hardware processors (Rajpura, pg. 5, 2nd paragraph, “All the experiments were performed on a workstation with Intel Core i7-5960X processor” denotes that they perform their method on a computer, in which program instructions to perform the functions are inherent), wherein the plurality of subsystems comprises: a data obtaining subsystem configured to obtain a plurality of data associated with a first one or more images, wherein the first one or more images is obtained from a publicly available image database comprising a plurality of images for training a machine learning model (Abstract, “A CNN pretrained on the COCO dataset” where images in COCO are data in a public image database); a data training subsystem configured to train the machine learning model on a second plurality of data associated with second one or more images corresponding to first one or more products (Abstract, “A CNN pretrained on the COCO dataset” where the pretraining is training on all the images in the COCO dataset, including second one or more images corresponding to first one or more products, that is, any products in the set of COCO categories. See Lin, pg. 3, Fig. 2(a), the toothbrush and Wii controller to demonstrate that COCO includes second one or more images corresponding to first one or more products) wherein the second one or more images in a database comprises one or more images corresponding to the first one or more products, irrespective of whether the second one or more images comprises of one or more products on which the trained machine learning model is to be performed (Rajpura is intended to recognize, pg. 3, Section 2, “an RGB image captured inside a refrigerator” and not necessarily a Wii controller or a toothbrush), a data extracting subsystem configured to extract a third plurality of data associated with third one or more images corresponding to second one or more products from the database, wherein the third one or more images corresponding to the second one or more products are pre-stored in the database (Rajpura, pg. 3, Section 2, “synthetic rendered images from available 3D models” also see pg. 2, Fig. 1, “2d Rendered Annotated Images for training” are images of products in the refrigerator); an image analyzing subsystem configured to learn to recognize the third one or more images corresponding to the second one or more products by using the machine learning model trained on the second one or more images corresponding to the first one or more products (Rajpura, Abstract, “fine-tuning pretrained CNNs entirely on synthetic images is an effective strategy to achieve transfer learning” denotes that the classifier network is configured to learn to recognize the third one or more images corresponding to the second one or more products, e.g. the synthetic images of products in the refrigerator, where third images were previously identified as the synthetic images and second products were previously identified as products in a refrigerator) wherein a number of the first one or more products is higher than a number of the second one or more products (COCO has Lin, Abstract, “3.5 million labeled instances in 328k images” and pg. 7, Fig. 5, 91 categories with 10,000+ instances per category while Rajpura only uses a maximum of 6000 images total, thus fewer second products than first products); a fine-tuning subsystem configured to fine-tune at least one subset of the trained machine learning model from a frozen part of the trained machine learning model to recognize the third one or more analyzed images corresponding to the second one or more products using a transfer learning method (Rajpura, Abstract, “fine-tuning pretrained CNNs entirely on synthetic images is an effective strategy to achieve transfer learning … A CNN pretrained on the COCO dataset and fine-tuned with our 4000 synthetic images … fine-tuning with selected layers” where a pretrained model has completed training, and was static/frozen); and the image analyzing subsystem configured to analyze fourth one or more images corresponding to the second one or more products by applying trained weights of the fine-tuned at least one subset of the trained machine learning model on the fourth one or more images corresponding to the second one or more products (Rajpura, pg. 2, Fig. 1, “Real world images” are analyzed, also see pg. 6, Fig. 2, “Detection Results” on real world images of products in a refrigerator) wherein the fine-tuned at least one subset of the trained machine learning model is trained on the third one or more images corresponding to the second one or more products (Rajpura, Abstract, “fine-tuning pretrained CNNs entirely on synthetic images is an effective strategy to achieve transfer learning” where classifier network was trained on the third one or more images corresponding to the second one or more products, e.g. the synthetic images of products in the refrigerator, where third images were previously identified as the synthetic images and second products were previously identified as products in a refrigerator) wherein the fine-tuned at least one subset of the trained machine learning model is required during learning to recognize the second one or more products (Rajpura, Abstract, “fine-tuning with selected layers” is part of the fine-tuned CNN) wherein the fourth one or more images comprises one or more real world test images corresponding to the second one or more products, and wherein the fine-tuned at least one subset of the trained machine learning model is performed for analyzing the one or more real world test images corresponding to the second one or more products (Rajpura, pg. 2, Fig. 1, “Real world images” are analyzed, also see pg. 6, Fig. 2, “Detection Results”).
Regarding Claim 2, the Rajpura/Shermin combination of Claim 1 teaches the machine learning based system of Claim 1 (and thus the rejection of Claim 1 is incorporated). Rajpura further teaches (partially inherently though Lin) to receive the second plurality of data associated with the second or more images corresponding to the first one or more products (Abstract, “the COCO dataset” must have been obtained to pretrain the model on it); provide a first plurality of labels related to the second one or more images corresponding to the first one or more products to the machine learning model, wherein the first plurality of labels comprises at least one of: objects comprised in the second one or more images (Lin, Abstract, “a total of 2.5 million labeled instances in 329k images” pg. 3, Fig. 2); and train the machine learning model by correlating the second one or more images corresponding to the first one or more products, with the first plurality of labels related to the second one or more images, wherein the machine learning model is a supervised machine learning model (Rajpura, pg. 4, “For neural network training we use … Faster-RCNN (with ResNet-101 as feature mapping network) and SSD using TensorFlow and weights pretrained on COCO dataset” are supervised machine learning model training methods).
Regarding Claim 3, the Rajpura/Shermin combination of Claim 1 teaches the machine learning based system of Claim 1 (and thus the rejection of Claim 1 is incorporated). Rajpura further teaches to: obtain the third one or more images corresponding to the second one or more products from the database (Rajpura, pg. 2, Fig, 1, “2D Rendered Annotated Images for training”), provide a second plurality of labels related to the third one or more images corresponding to the first one or more products, to the machine learning model, wherein the first plurality of labels comprises at least one of : objects comprised in the second one or more images (Rajpura, pg. 2, Fig, 1, “2D Rendered Annotated Images for training”).
Regarding Claim 7, the Rajpura/Shermin combination of Claim 1 teaches the machine learning based system of Claim 1 (and thus the rejection of Claim 1 is incorporated). Rajpura has already been shown to teach wherein the trained machine learning model is a convolutional neural network (CNN) model (Rajpura, title, “Finetuning Pretrained CNNs”).
Regarding Claim 8, the Rajpura/Shermin combination of Claim 1 teaches the machine learning based system of Claim 1 (and thus the rejection of Claim 1 is incorporated). The rejection has already been shown to teach wherein the first one or more products and the second one or more products are different products (the COCO dataset comprising e.g. toothbrush and the fine-tuning dataset comprising products in a refrigerator).
Claims 9-12, 15, and 16 recite the machine learning based method performed by the system of Clams 1-3, 7, and 8, respectively, and is thus rejected for reasons set forth in the rejections of Claims 1-3, 7, and 8, respectively. Similarly, Claims 17-19, 23, and 24 recite a non-transitory computer-readable storing medium having instructions to perform that method and are also rejected for reasons set forth in the rejections of Claims 1-3, 7, and 8, respectively.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 6, 14, and 22 are rejected under 35 U.S.C. 103 as being unpatentable over Rajpura et al., “Transfer Learning by Finetuning Pretrained CNNs Entirely with Synthetic Images,” in view of Shermin et al., “Enhanced Transfer Learning with ImageNet Trained Classification Layer.”
Regarding Claim 6, the Rajpura/Shermin combination of Claim 1 teaches the machine learning based system of Claim 1 (and thus the rejection of Claim 1 is incorporated). Rajpura does not teach, but Shermin teaches providing probabilistic values to the fourth one or more analyzed images corresponding to the second one or more products, between 0 and 1 (Shermin, pg. 4, 1st paragraph, “the new classification module, which has a FC classification layer C with a Softmax layer” where “softmax” denotes an output with the probability of each class). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use the Rajpura/Shermin fine-tuning method for classification, with a softmax output as Shermin does, rather than localization, as Rajpura does. The motivation to do so is that all sorts of classification tasks on images are desirable (Lin, pg. 1, Fig. 1, Image classification, Object localization, and segmentation are all tasks that image recognition systems perform).
Claim 14 recites the machine learning based method performed by the system of Claims 6, and is thus rejected for reasons set forth in the rejections of Claim 6. Similarly, Claim 22 recites a non-transitory computer-readable storing medium having instructions to perform that method and is also rejected for reasons set forth in the rejections of Claim 6.
Response to Arguments
Applicant’s arguments filed June 23rd, 2026 have been fully considered, but are not fully persuasive.
Applicant’s amendments have caused the withdrawal of the Claim Objections and 35 U.S.C. 112(b) rejections of the previous office action. However, the amendments have also required a new 35 U.S.C. 112(b) rejection to be made.
Applicant’s arguments regarding the prior art rejections of the previous office action have been fully considered, but are unpersuasive.
Applicant first asserts that “Rajpura does not teach or suggest the presently claimed staged subsystem architecture, in which: a machine learning model is trained on second or more images corresponding to first one or more products.” However, the pre-trained model of Rajpura is indeed trained on the COCO dataset, e.g. images corresponding to general products (e.g. toothbrush and Wii controller).
Applicant next asserts that “Rajpura does not teach or suggest the presently claimed staged subsystem architecture, in which: … third one or more images corresponding to second one or more products are extracted from a database.” However, Rajpura synthetically generates and then uses (e.g. extracts from storage) images of items in a refrigerator, i.e. corresponding to second one or more products.
Applicant next asserts that “Rajpura does not teach or suggest the presently claimed staged subsystem architecture, in which: … the image analyzing subsystem learns to recognize the third one or more images using the previously trained machine learning model; the fine-tuning subsystem fine-tunes at least one subset of the trained machine learning model from a frozen part of the trained machine learning model.” However, Rajpura uses the synthetic images in training, learning to recognize them and fine-tuning the (previously frozen) pre-trained model to perform the recognition.
Applicant next asserts that “Rajpura does not teach or suggest the presently claimed staged subsystem architecture, in which: … the trained weights of the fine-tuned subset are subsequently applied for analyzing fourth one or more images corresponding to real-world test images.” This step is clearly illustrated in Rajpura, Fig. 1, where the fine-tuned model (e.g. using the weights that were trained in fine-tuning) is used to recognize real images.
Applicant next asserts that “Rajpura does not disclose or suggest: the claimed staged interaction between second, third, and fourth image datasets.” However, each of these datasets are clearly identified in the rejection: Second one or more images are images of the COCO dataset corresponding to general products, upon which the pre-trained classifier is trained (e.g. toothbrush and Wii controller). Third one or more images are the synthetic images of specific products (in this case, items in a refrigerator). Fourth one or more images are real images of items in a refrigeration on which inference is performed. See Rajpura, Fig. 1 for an illustration of these datasets.
Applicant next asserts that “Rajpura does not disclose or suggest: … the claimed subsystem orchestration involving distinct data obtaining, data training, data extracting, image analyzing, and fine-tuning subsystems.” However, the rejection indicates that all of these steps are performed, and in Rajpura, all of these steps are performed by software. Any software code or routine that performs the recited function is the recited subsystem for that function.
Applicant next asserts that “Rajpura does not disclose or suggest: … the claimed sequence in which trained weights learned using third images are subsequently applied for analyzing fourth real-world test images.” This step is clearly illustrated in Rajpura, Fig. 1.
Applicant’s arguments regarding Shermin are moot, as Shermin was only used in the previous rejection to teach a second round of fine-tuning, which the amendments to the claims have removed.
Applicant’s arguments regarding the other independent and dependent claims rely upon the same features argued with respect to Claim 1 and are likewise unpersuasive.
Conclusion
Applicant’s amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to BRIAN M SMITH whose telephone number is (469)295-9104. The examiner can normally be reached Monday - Friday, 8:00am - 4pm Pacific.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kakali Chaki can be reached at (571) 272-3719. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/BRIAN M SMITH/ Primary Examiner, Art Unit 2122
Prosecution Insights