DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Acknowledgement is made of Applicant’s claim of priority from U.S. Provisional Application No. 63/408,712, filed September 21, 2022.
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on November 6, 2025 has been entered.
Response to Arguments
Applicant’s arguments, filed November 6, 2025, with respect to the claim objections have been fully considered and are persuasive. The objection to the claims has been withdrawn.
Applicant’s arguments, filed November 6, 2025, with respect to the 35 USC 103 have been fully considered but are moot because of the new grounds of rejection presented in the sections below. Applicant argues that the claim amendments overcome the prior art of record, however, in analogous fields of endeavor, Brownlee and Dighe teach the newly added limitations. Thus, the 35 USC 103 rejection of the claims is upheld.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1, 6, 8-9 and 11 are rejected under 35 U.S.C. 103 as being unpatentable over Hiroaki Kingetsu (US 2022/0207307 A1) in view of Jason Brownlee (“A Gentle Introduction to Mixture of Experts Ensembles”, November 7, 2021) further in view of Dapogny et al. (US 2024/0161483 A1, with Foreign priority to Application No. EP 4057184, filed March 11, 2021, US PGPub used herein as a translation and for mapping purposes) and Dighe et al. (US 2022/0093095 A1)
Regarding claim 1, Kingetsu teaches a method for reusing and retraining image recognition models, comprising:
receiving first image data (Kingetsu, Para. [0066], training data is given and a correct answer label of “dog” is given to the training data. Para. [0090], the training data corresponds to data on email spam, electricity demand prediction, stock price prediction, data on poker hands, image data, or the like);
determining, based on the first image data, a reference label of the first image data using a trained teacher model inferencing the first image data, wherein the reference label represents a ground-truth inference of the first image data (Kingetsu, Para. [0067], the computing system trains the parameters of the teacher model such that the output result of the teacher model obtained at the time of inputting the training data approaches the correct answer label (i.e., ground-truth inference) of “dog”),
determining a first feature label of the first image data using the first image recognition model (Kingetsu, Para. [0067], the computing system trains the parameters of the Student Model such that the output result obtained at the time of inputting the training data approaches the output result of the Teacher Model);
responsive to the validating, adaptively installing the first image recognition model for reuse (Kingetsu, Para. [0119], the detection unit may notify the training unit of information indicating that accuracy degradation has been detected and retrain the machine learning model);
receiving second image data (Kingetsu, Para. [0119], the training unit retrains the machine learning model by using a training data set that is newly designated (i.e., second image data)); and
processing the second image data for inferencing (Kingetsu, Para. [0119], the training unit retrains the machine learning model by using a training data set that is newly designated (i.e., second image data)).
Although Kingetsu teaches a plurality of inspector models created based on the knowledge distillation of the machine learning model (Kingetsu, Para. [0166]), Kingetsu does not explicitly teach “selecting, based on the first image data using a selection model, a first image recognition model from a plurality of trained image recognition models, wherein the selection model comprises a gating network, wherein the gating network selects the first image recognition model based on level of confidence of the first image recognition model predicting a label of the first image data without further training the plurality of trained image recognition models based on the first image data”. However, in an analogous field of endeavor, Brownlee teaches a model is used to interpret the predictions made by each expert and to aid in deciding which expert to trust for a given input. This is called the gating model, or the gating network, given that it is traditionally a neural network model. The gating network takes as input the input pattern that was provided to the expert models and outputs the contribution that each expert should have in making a prediction for the input. The gating network might have a softmax output that gives a probability-like confidence score (i.e., level of confidence of the first image recognition model predicting a label of the first image data) for each expert. Finally, the mixture of expert models must make a prediction, and this is achieved using a pooling or aggregation mechanism. This might be as simple as selecting the expert with the largest output or confidence provided by the gating network. Alternatively, a weighted sum prediction could be made that explicitly combines the predictions made by each expert and the confidence estimated by the gating network (i.e., select the model based on level of confidence of prediction for input without further training the models) (Brownlee, “Gating Model” and “Pooling Method”).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the method of Kingetsu with the teachings of Brownlee by including selecting the first image recognition model using a gating network that outputs a probability-like confidence score for each model in making a prediction for the input and selects the model with the largest confidence. One having ordinary skill in the art would have been motivated to combine these references because doing so would allow for an automated approach to predictive modeling that allows for broad problem-solving, as recognized by Brownlee.
Although Kingetsu in view of Brownlee teaches a teacher model determining a reference label (Kingetsu, Para. [0067]) and retraining the machine learning model data based on accuracy degradation detection (Kingetsu, Para. [0119]), they do not explicitly teach “wherein the teacher model generates the reference label by inferencing” and “validating an accuracy of the first image recognition model in determining the first feature label of the first image data based on a comparison between the reference label and the first feature label”. However, in an analogous field of endeavor, Dapogny teaches a method of inference of one or more predictive models to obtain one or more predictions by applying one of the one or more predictive models to the provided image (Dapogny, Para. [0041]). Dapogny further teaches computing a cost (or loss) function based on the prediction of the student model and the prediction of the teacher model, and updating the parameters of the student model based on the computed cost function and by backpropagation (Dapogny, Para. [0064]).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the method of Kingetsu in view of Brownlee with the teachings of Dapogny by including the teacher model generating the reference label based on inference and validating an accuracy (i.e., cost/loss function) based on comparison between the reference label and the first feature label. One having ordinary skill in the art would have been motivated to combine these references because doing so would allow for a low-cost network that is trained by retraining a student network using knowledge distillation from the teacher network, as recognized by Dapogny.
Although Kingetsu in view of Brownlee further in view of Dapogny teaches a teacher model determining a reference label (Kingetsu, Para. [0067]), they do not explicitly teach ”the trained teacher model performs inferencing of the first image data more accurately than the first image recognition model by consuming more memory resources than the first image recognition model”. However, in an analogous field of endeavor, Dighe teaches that student-teacher training is a training technique where a (typically) more accurate and computationally expensive teacher model trains a less computationally expensive student model to mimic the teacher model’s outputs and/or determinations (Dighe, Para. [0277]).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the method of Kingetsu in view of Brownlee further in view of Dapogny with the teachings of Dighe by including that the teacher model is more accurate by consuming more computation resources than the student model (i.e., first image recognition model). One having ordinary skill in the art would have been motivated to combine these references because doing so would allow for knowledge distillation from a teacher model to students that use fewer resources, as recognized by Dighe. Thus, the claimed invention would have been obvious to one having ordinary skill in the art before the effective filing date.
Regarding claim 6, Kingetsu in view of Brownlee further in view of Dapogny and Dighe teaches the method of claim 1, and further teaches wherein the teacher model determines the reference label based on the first image data, wherein the reference label is more accurate in inferencing the first image data than the first feature label in inferencing the first image data using the first image recognition model (Kingetsu, Para. [0067], the computing system trains the parameters of the teacher model such that the output result of the teacher model approaches the correct answer label of “dog”. Furthermore, the computing system trains the parameters of the student model such that the output result of the student model approaches the output result of the teacher model. An output of the teacher model is referred to as a “soft target”. A correct answer label of the training data is referred to as a “hard target”).
Regarding claim 8, Kingetsu in view of Brownlee further in view of Dapogny and Dighe teaches the method of claim 1, further comprising:
receiving, based on a predefined rule associated with a timing of capturing a frame of video data, the first image data (Dapogny, Para. [0069], the method of machine-learning may further comprise determining, based on the prior condition, whether the one or more images are to be provided to the first and second model. In other words, the prior condition may be a(n) (automatic) determining criteria whether a provided image from the first stream is to be used in the on-the-fly adaptation. Para. [0068], the prior condition may be computed based on one or more of: image metadata, time stamp, information extracted from the image's content. In examples, the information extracted from the image may comprise the time of the day, weather, luminosity, backlight, movement of the camera, and/or hazing).
The proposed combination as well as the motivation for combining the Kingetsu, Brownlee, Dapogny and Dighe references presented in the rejection of Claim 1, apply to Claim 8 and are incorporated herein by reference. Thus, the method recited in Claim 8 is met by Kingetsu in view of Brownlee further in view of Dapogny and Dighe.
Regarding claim 9, Kingetsu in view of Brownlee further in view of Dapogny and Dighe teaches the method of claim 1, further comprising:
receiving, based on a change of scenery captured in a frame of video data, the first image data (Dapogny, Para. [0069], the method of machine-learning may further comprise determining, based on the prior condition, whether the one or more images are to be provided to the first and second model. In other words, the prior condition may be a(n) (automatic) determining criteria whether a provided image from the first stream is to be used in the on-the-fly adaptation. Para. [0068], the prior condition may be computed based on one or more of: image metadata, time stamp, information extracted from the image's content. In examples, the information extracted from the image may comprise the time of the day, weather, luminosity, backlight, movement of the camera, and/or hazing).
The proposed combination as well as the motivation for combining the Kingetsu, Brownlee, Dapogny and Dighe references presented in the rejection of Claim 1, apply to Claim 9 and are incorporated herein by reference. Thus, the method recited in Claim 9 is met by Kingetsu in view of Brownlee further in view of Dapogny and Dighe.
Regarding claim 11, Kingetsu in view of Brownlee further in view of Dapogny and Dighe teaches the method of claim 1, further comprising:
iteratively retraining the first image recognition model using the comparison between the reference label and the first feature label according to a predetermined level of accuracy (Kingetsu, Para. [0119], the detection unit may notify the training unit of information indicating that accuracy degradation has been detected and retrain the machine learning model data by using a training dataset that is newly designated); and
adding the iteratively retrained first image recognition model to the plurality of trained image recognition models (Kingetsu, Para. [0126], the training unit retrains the machine learning model by using a new training data set and proceeds to Step S102. Para. [0123], In Step S102, the creating unit generates the inspector models from the distillation data table).
Claim 2 is rejected under 35 U.S.C. 103 as being unpatentable over Hiroaki Kingetsu (US 2022/0207307 A1) in view of Brownlee (“A Gentle Introduction to Mixture of Experts Ensembles”, November 7, 2021) further in view of Dapogny et al. (US 2024/0161483 A1, with Foreign priority to Application No. EP 4057184, filed March 11, 2021, US PGPub used herein as a translation and for mapping purposes) and Dighe et al. (US 2022/0093095 A1), as applied to claims 1, 6, 8-9 and 11 above, and further in view of Talagala et al. (US 2019/0108417 A1) and Jacobs et al. (US 2020/0302784 A1).
Regarding claim 2, Kingetsu in view of Brownlee further in view of Dapogny and Dighe teaches the method of claim 1, as described above.
Although Kingetsu in view of Brownlee further in view of Dapogny and Dighe teaches selecting a best model using a gating network (Brownlee, “A Gentle Introduction to Mixture of Experts Ensembles”), they do not explicitly teach “selecting, based on the first image data using the selection model, a second image recognition model from the plurality of trained image recognition models”, “determining a third feature label associated with the first image data using the selected second image recognition model”, “determining variances of the first feature label and the third feature label from the reference label” and “selecting, based on the variances of the first feature label and the third feature label from the reference label, the first image recognition model”. However, in an analogous field of endeavor, Talagala teaches a model selection module that determines which of the machine learning models is the best fit for the objective that is being analyzed. The best-fitting machine learning model may be the machine learning model that produced results most similar to the actual results for the training data (i.e., reference label) (e.g., the most accurate machine learning model) (Talagala, Para. [0066]).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the method of Kingetsu in view of Brownlee further in view of Dapogny and Dighe with the teachings of Talagala by including selecting the image recognition model based on the variance (i.e., similarity) between the results of the machine learning models (i.e., first feature label and third feature label) and the actual results for the training data (i.e., reference label). One having ordinary skill in the art would have been motivated to combine these references because doing so would allow for a machine learning system that delivers accurate and relevant results, as recognized by Talagala.
Although Kingetsu in view of Brownlee further in view of Dapogny, Dighe and Talagala teaches the machine learning models are pushed to the inference pipelines that comprise the logical pipeline grouping for the objective, each of which is executing on live data coming from an edge device, e.g., input data (Talagala, Para. [0062]), they do not explicitly teach “receiving the first image data from an edge device associated with a 5G telecommunication network”. However, in an analogous field of endeavor, Jacobs teaches a 5G network (or another network protocol) can connect the edge device with other devices and objects (Jacobs, Para. [0031]).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the method of Kingetsu in view of Brownlee further in view of Dapogny, Dighe and Talagala with the teachings of Jacobs by including the edge device being connected to a 5G telecommunication network. One having ordinary skill in the art would have been motivated to combine these references because doing so would allow for reducing a latency of data communications and to conserve bandwidth, as recognized by Jacobs. Thus, the claimed invention would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention.
Claims 3 and 7 are rejected under 35 U.S.C. 103 as being unpatentable over Hiroaki Kingetsu (US 2022/0207307 A1) in view of Brownlee (“A Gentle Introduction to Mixture of Experts Ensembles”, November 7, 2021) further in view of Dapogny et al. (US 2024/0161483 A1, with Foreign priority to Application No. EP 4057184, filed March 11, 2021, US PGPub used herein as a translation and for mapping purposes) and Dighe et al. (US 2022/0093095 A1), as applied to claims 1, 6, 8-9 and 11 above, and further in view of Talagala et al. (US 2019/0108417 A1).
Regarding claim 3, Kingetsu in view of Brownlee further in view of Dapogny and Dighe teaches the method of claim 1, as described above.
Although Kingetsu in view of Brownlee further in view of Dapogny and Dighe teaches selecting a best model using a gating network (Brownlee, “A Gentle Introduction to Mixture of Experts Ensembles”), they do not explicitly teach “responsive to the selecting the first image recognition model, installing the first image recognition model for inferring captured image data” and “determining, based on inferencing using the first image recognition model, a second feature label associated with the second image data”. However, in an analogous field of endeavor, Talagala teaches that after the machine learning model is selected, it is pushed to the policy pipeline for validation, verification, or the like, which then pushes it back to the inference pipelines (Talagala, Para. [0069]). Talagala further teaches the inference pipelines use the generated machine learning model and the corresponding analytics engine to generate machine learning results/predictions on input data that is associated with the objective. For example, if a user wants to know whether an email is spam, the training pipeline may generate a machine learning model using a training data set that includes emails that are known to be both spam and not spam. After the machine learning model is generated, the policy pipeline pushes the machine learning model to the inference pipelines, where it is used to predict whether one or more emails (i.e., first and second data), e.g., provided as input data, are spam (Talagala, Para. [0061]).
The proposed combination as well as the motivation for combining the Kingetsu, Brownlee, Dapogny, Dighe and Talagala references presented in the rejection of Claim 2, apply to Claim 3 and are incorporated herein by reference. Thus, the method recited in Claim 3 is met by Kingetsu in view of Brownlee further in view of Dapogny, Dighe and Talagala.
Regarding claim 7, Kingetsu in view of Brownlee further in view of Dapogny teaches the method of claim 1 as described above.
Although Kingetsu in view of Brownlee further in view of Dapogny and Dighe teaches selecting a best model using a gating network (Brownlee, “A Gentle Introduction to Mixture of Experts Ensembles”), they do not explicitly teach “wherein the selection model selects one or more ranked image recognition models according to a fit between an image recognition model and the first image data”. However, in an analogous field of endeavor, Talagala teaches a model selection module that determines which of the machine learning models is the best fit for the objective that is being analyzed. The best-fitting machine learning model may be the machine learning model that produced results most similar to the actual results for the training data (i.e., reference label) (e.g., the most accurate machine learning model) (Talagala, Para. [0066]).
The proposed combination as well as the motivation for combining the Kingetsu, Brownlee, Dapogny, Dighe and Talagala references presented in the rejection of Claim 2, apply to Claim 7 and are incorporated herein by reference. Thus, the method recited in Claim 7 is met by Kingetsu in view of Brownlee further in view of Dapogny, Dighe and Talagala.
Claim 4 is rejected under 35 U.S.C. 103 as being unpatentable over Hiroaki Kingetsu (US 2022/0207307 A1) in view of Brownlee (“A Gentle Introduction to Mixture of Experts Ensembles”, November 7, 2021) further in view of Dapogny et al. (US 2024/0161483 A1, with Foreign priority to Application No. EP 4057184, filed March 11, 2021, US PGPub used herein as a translation and for mapping purposes) and Dighe et al. (US 2022/0093095 A1), as applied to claims 1, 6, 8-9 and 11 above, and further in view of Takimoto et al. (US 2022/0129675 A1).
Regarding claim 4, Kingetsu in view of Brownlee further in view of Dapogny and Dighe teaches the method of claim 1, as described above.
Although Kingetsu in view of Brownlee further in view of Dapogny and Dighe teaches selecting a best model using a gating network (Brownlee, “A Gentle Introduction to Mixture of Experts Ensembles”), they do not explicitly teach “selecting, based on the first image data using the selection model, a set of image recognition models from the plurality of trained image recognition models”, “ranking, based on probability values associated with a likelihood of respective image recognition models accurately recognizing the first image data, the set of image recognition models”, and “selecting, based on the ranked set of image recognition models, the first image recognition model”. However, in an analogous field of endeavor, Takimoto teaches the CPU obtains a score for the result of object detection processing for each of the P captured images in correspondence with each of the M candidate learning models. The CPU then performs ranking (ranking creation) of the M candidate learning models based on the scores, and selects N candidate learning models from the M candidate learning models (i.e., N=1) (Takimoto, Para. [0137]).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the method of Kingetsu in view of Brownlee further in view of Dapogny and Dighe with the teachings of Takimoto by including selecting a set of candidate learning models and ranking the models based on a score of the result, and selecting the first image recognition model based on the rank. One having ordinary skill in the art would have been motivated to combine these references because doing so would allow for ensuring high performance of a learning model for a new input with a low operation cost, as recognized by Takimoto. Thus, the claimed invention would have been obvious to one having ordinary skill in the art before the effective filing date.
Claim 5 is rejected under 35 U.S.C. 103 as being unpatentable over Hiroaki Kingetsu (US 2022/0207307 A1) in view of Brownlee (“A Gentle Introduction to Mixture of Experts Ensembles”, November 7, 2021) further in view of Dapogny et al. (US 2024/0161483 A1, with Foreign priority to Application No. EP 4057184, filed March 11, 2021, US PGPub used herein as a translation and for mapping purposes) and Dighe et al. (US 2022/0093095 A1), as applied to claims 1, 6, 8-9 and 11 above, and further in view of Jacobs et al. (US 2020/0302784 A1).
Regarding claim 5, Kingetsu in view of Brownlee further in view of Dapogny and Dighe teaches the method of claim 1, as described above.
Although Kingetsu in view of Brownlee further in view of Dapogny and Dighe teaches a stream of images from a live video camera (Dapogny, Para. [0064]), they do not explicitly teach “receiving, by an edge server associated with the 5G telecommunication network, the first image data from an edge device via a wireless network of the 5G telecommunication network, wherein the edge device includes a camera for capturing the first image data”. However, in an analogous field of endeavor, Jacobs teaches a 5G network (or another network protocol) can connect the edge device with the augmented reality device, the centrally located components of the movement analytics platform, and/or other objects (Jacobs, Para. [0031]) and teaches augmented reality device includes a camera that can capture video data, image data, and/or the like (Jacobs, Para. [0046]).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the method of Kingetsu in view of Brownlee further in view of Dapogny and Dighe with the teachings of Jacobs by including an edge device connected to a 5G network and wherein the edge device includes a camera for capturing the first image data. One having ordinary skill in the art would have been motivated to combine these references, because doing so would allow for reducing a latency of data communications and to conserve bandwidth, as recognized by Jacobs. Thus, the claimed invention would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention.
Claim 10 is rejected under 35 U.S.C. 103 as being unpatentable over Hiroaki Kingetsu (US 2022/0207307 A1) in view of Brownlee (“A Gentle Introduction to Mixture of Experts Ensembles”, November 7, 2021) further in view of Dapogny et al. (US 2024/0161483 A1, with Foreign priority to Application No. EP 4057184, filed March 11, 2021, US PGPub used herein as a translation and for mapping purposes) and Dighe et al. (US 2022/0093095 A1), as applied to claims 1, 6, 8-9 and 11 above, and further in view of Atsushi Nogami (US 2022/0269996 A1) and Balasubramanian et al. (US 2022/0172100 A1).
Regarding claim 10, Kingetsu in view of Brownlee further in view of Dapogny and Dighe teaches the method of claim 1, as described above.
Although Kingetsu in view of Brownlee further in view of Dapogny and Dighe teaches selecting a best model using a gating network (Brownlee, “A Gentle Introduction to Mixture of Experts Ensembles”), they do not explicitly teach “counting a number of occasions of selecting the first image recognition model”. However, in an analogous field of endeavor, Nogami teaches recording the count at which a model is selected, and that this count of selection is recorded for each model, and the records are added up such that the model evaluation value rises as the selection count increases (Nogami, Para. [0135]).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the method of Kingetsu in view of Brownlee further in view of Dapogny and Dighe with the teachings of Nogami by including recording a count of the number of occasions of selecting the first image recognition model. One having ordinary skill in the art before the effective filing date would have been motivated to combine these references because doing so would allow for evaluating the selection of a model, as recognized by Nogami.
Although Kingetsu in view of Brownlee further in view of Dapogny, Dighe and Nogami teaches selecting a best model using a gating network (Brownlee, “A Gentle Introduction to Mixture of Experts Ensembles”), they do not explicitly teach “removing, based on the number of occasions of selecting the first image recognition model, the first image recognition model from the plurality of trained image recognition models”. However, in an analogous field of endeavor, Balasubramanian teaches that when the model metrics indicate that the machine learning model is performing poorly (for example, has an error rate above a threshold value, has a statistical distribution that is not an expected or desired distribution (for example, not a binomial distribution, a Poisson distribution, a geometric distribution, a normal distribution, Gaussian distribution, etc.), has an execution latency above a threshold value, has a confidence level below a threshold value)) and/or is performing progressively worse (for example, the quality metric continues to worsen over time), the model training system can instruct the virtual machine instance to delete the ML training container and/or to delete any model data stored in the training model data store (Balasubramanian, Para. [0206]).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the method of Kingetsu in view of Brownlee further in view of Dapogny, Dighe and Nogami with the teachings of Balasubramanian by deleting the first image training model from the data store when the number of occasions of model selection (as taught by Nogami) is above a threshold value. One having ordinary skill in the art would have been motivated to combine these references, because doing so would allow for removing a machine learning model that is performing incorrectly based on model metrics, as recognized by Balasubramanian. Thus, the claimed invention would have been obvious to one having ordinary skill in the art before the effective filing date.
Claim 12 is rejected under 35 U.S.C. 103 as being unpatentable over Hiroaki Kingetsu (US 2022/0207307 A1) in view of Brownlee (“A Gentle Introduction to Mixture of Experts Ensembles”, November 7, 2021) further in view of Dapogny et al. (US 2024/0161483 A1, with Foreign priority to Application No. EP 4057184, filed March 11, 2021, US PGPub used herein as a translation and for mapping purposes) and Dighe et al. (US 2022/0093095 A1), as applied to claims 1, 6, 8-9 and 11 above, and further in view of Zheng et al. (US 2018/0348781 A1).
Regarding claim 12, Kingetsu in view of Brownlee further in view of Dapogny and Dighe teaches the method of claim 11, as described above.
Although Kingetsu in view of Brownlee further in view of Dapogny and Dighe teaches retraining the machine learning model when accuracy degradation is detected (Kingetsu, Para. [0119]), they do not explicitly teach “selecting a plurality of candidate models for retraining from the plurality of trained image recognition models” and “iteratively processing the plurality of trained image recognition models until a change of a level of accuracy in labeling is less than a predetermined threshold”. However, in an analogous field of endeavor, Zheng teaches determining whether the model to be updated involves the global models. If the global models are to be updated, appropriate labeled training data are used to re-train. The re-trained global models are then tested, using benchmark testing data selected for testing the global models. If the testing result is satisfactory, the global models are updated. If the testing result is not satisfactory, the processing goes back to re-train (e.g., iteratively) the global models (Zheng, Para. [0156]).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the method of Kingetsu in view of Brownlee further in view of Dapogny and Dighe with the teachings of Zheng by including determining a set of candidate models to retrain iteratively until the testing result (i.e., change in level of accuracy) is satisfactory. One having ordinary skill in the art would have been motivated to combine these references because doing so would allow for automatically updating and retraining machine learning models for higher accuracy results, as recognized by Zheng. Thus, the claimed invention would have been obvious to one having ordinary skill in the art before the effective filing date.
Claim 13 is rejected under 35 U.S.C. 103 as being unpatentable over Hiroaki Kingetsu (US 2022/0207307 A1) in view of Brownlee (“A Gentle Introduction to Mixture of Experts Ensembles”, November 7, 2021) further in view of Dapogny et al. (US 2024/0161483 A1, with Foreign priority to Application No. EP 4057184, filed March 11, 2021, US PGPub used herein as a translation and for mapping purposes) and Dighe et al. (US 2022/0093095 A1), as applied to claims 1, 6, 8-9 and 11 above, and further in view of Kale et al. (US 2023/0139682 A1, filed November 1, 2021).
Regarding claim 13, Kingetsu in view of Brownlee further in view of Dapogny and Dighe teaches the method of claim 11, as described above.
Although Kingetsu in view of Brownlee further in view of Dapogny and Dighe teaches retraining a model based on detected accuracy degradation (Kingetsu, Para. [0119]), they do not explicitly teach “iteratively retraining the plurality of trained image recognition models by allocating a time period of using a processing resource for retraining the plurality of trained image recognition models”. However, in an analogous field of endeavor, Kale teaches a scheduler module that may determine an amount of time to perform the retraining process and schedule the retraining process to be performed during a time period that corresponds to or is larger than the determined amount of time. Scheduler module may schedule the retraining process to be performed during a time period that corresponds to or is larger than the time period between time T(0) and time T(N) (Kale, Para. [0077]).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the method of Kingetsu in view of Brownlee further in view of Dapogny and Dighe with the teachings of Kale by including iteratively retraining the image recognition models by scheduling a time period during which to perform the retraining. One having ordinary skill in the art would have been motivated to combine these references, because doing so would allow for increasing overall system efficiency and decreasing system latency, as recognized by Kale. Thus, the claimed invention would have been obvious to one having ordinary skill in the art before the effective filing date.
Claim 14 is rejected under 35 U.S.C. 103 as being unpatentable over Hiroaki Kingetsu (US 2022/0207307 A1) in view of Dapogny et al. (US 2024/0161483 A1, with Foreign priority to Application No. EP 4057184, filed March 11, 2021, US PGPub used herein as a translation and for mapping purposes) further in view of Jacobs et al. (US 2020/0302784 A1) and Dighe et al. (US 2022/0093095 A1).
Regarding claim 14, Kingetsu teaches a system for reusing and retraining image recognition models for inferencing data (Kingetsu, Para. [0099]), configured to execute a method comprising:
receiving image data (Kingetsu, Para. [0066], training data is given and a correct answer label of “dog” is given to the training data. Para. [0090], the training data corresponds to data on email spam, electricity demand prediction, stock price prediction, data on poker hands, image data, or the like);
determining, (Kingetsu, Para. [0067], the computing system trains the parameters of the Student Model such that the output result obtained at the time of inputting the training data approaches the output result of the Teacher Model);
wherein the teacher model generates the reference label (Kingetsu, Para. [0067], the computing system trains the parameters of the teacher model such that the output result of the teacher model obtained at the time of inputting the training data approaches the correct answer label of “dog”);
based on the level of accuracy, selecting the first image recognition model for retraining (Kingetsu, Para. [0119], the detection unit may notify the training unit of information indicating that accuracy degradation has been detected and retrain the machine learning model data by using a training dataset that is newly designated); and
Although Kingetsu teaches a teacher and student model for labelling data (Kingetsu, Para. [0067]), Kingetsu does not explicitly teach “wherein the first image recognition model corresponds to an inference model”, the teacher model determines the reference label “by inferencing”, “comparing the first feature label of the image data to a predetermined threshold of a reference label of a sample image generated by a trained teacher model”, “based on the comparing, validating a level of accuracy of inferencing the image data by the first image recognition model”, and “updating, based on the retrained first image recognition model, a store of a plurality of trained image recognition models”. However, in an analogous field of endeavor, Dapogny teaches a method of inference of one or more predictive models to obtain one or more predictions by applying one of the one or more predictive models to the provided image (Dapogny, Para. [0041]). Dapogny further teaches computing a cost (or loss) function based on the prediction of the student model and the prediction of the teacher model (i.e., level of accuracy), and updating the parameters of the student model based on the computed cost function and by backpropagation (Dapogny, Para. [0064]). Dapogny further teaches the processing machine may perform one or more methods of machine learning as discussed above in order to train, calibrate, re-train, and/or re-calibrate any of the predictive models stored on the data storage unit (Dapogny, Para. [0101]).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the system of Kingetsu with the teachings of Dapogny by including the teacher model and first image recognition model generating the reference label based on inference, determining an accuracy (i.e., cost/loss function) based on comparison between the reference label and the first feature label, and updating the store of the plurality of models by re-training one or more of the models. One having ordinary skill in the art would have been motivated to combine these references because doing so would allow for a low-cost network that is trained by retraining a student network using knowledge distillation from the teacher network, as recognized by Dapogny.
Although Kingetsu in view of Dapogny teaches a processing machine for performing re-training (Dapogny, Para. [0101]), they do not explicitly teach the inferencing data is “captured by an edge device”. However, in an analogous field of endeavor, Jacobs teaches a server device for storing and/or processing a data feed, wherein the server device can be an edge device (Jacobs, Para. [0047]).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the system of Kingetsu in view of Dapogny with the teachings of Jacobs by including an edge device connected to a 5G network for storing and/or processing data. One having ordinary skill in the art would have been motivated to combine these references, because doing so would allow for reducing a latency of data communications and to conserve bandwidth, as recognized by Jacobs.
Although Kingetsu in view of Dapogny further in view of Jacobs teaches a teacher model determining a reference label (Kingetsu, Para. [0067]), they do not explicitly teach ”the teacher model performs inferencing of the first image data more accurately than the first image recognition model by consuming more memory resources than the first image recognition model”. However, in an analogous field of endeavor, Dighe teaches that student-teacher training is a training technique where a (typically) more accurate and computationally expensive teacher model trains a less computationally expensive student model to mimic the teacher model’s outputs and/or determinations (Dighe, Para. [0277]).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the method of Kingetsu in view of Dapogny further in view of Jacobs with the teachings of Dighe by including that the teacher model is more accurate by consuming more computation resources than the student model (i.e., first image recognition model). One having ordinary skill in the art would have been motivated to combine these references because doing so would allow for knowledge distillation from a teacher model to students that use fewer resources, as recognized by Dighe. Thus, the claimed invention would have been obvious to one having ordinary skill in the art before the effective filing date.
Claims 15-17 are rejected under 35 U.S.C. 103 as being unpatentable over Hiroaki Kingetsu (US 2022/0207307 A1) in view of Dapogny et al. (US 2024/0161483 A1, with Foreign priority to Application No. EP 4057184, filed March 11, 2021, US PGPub used herein as a translation and for mapping purposes) further in view of Jacobs et al. (US 2020/0302784 A1) and Dighe et al. (US 2022/0093095 A1), as applied to claim 14 above, and further in view of Talagala et al. (US 2019/0108417 A1).
Regarding claim 15, Kingetsu in view of Dapogny further in view of Jacobs and Dighe teaches the system of claim 14, the processor further configured to execute a method comprising:
Although Kingetsu in view of Dapogny further in view of Jacobs and Dighe teaches an edge device associated with the 5G telecommunication network (Jacobs, Para. [0031]), they do not explicitly teach “selecting, based on the first feature label by a selection model, a second image recognition model from a plurality of image recognition models for reuse” and “installing the second image recognition model in the edge device associated with the 5G telecommunication network”. However, in an analogous field of endeavor, Talagala teaches a model selection module that receives the machine learning models that the training pipelines generate and determines which of the machine learning models is the best fit for the objective that is being analyzed (Talagala, Para. [0066]). Talagala further teaches the machine learning models are pushed to the inference pipelines that comprise the logical pipeline grouping for the objective, each of which is executing on live data coming from an edge device, e.g., input data (Talagala, Para. [0062]).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the system of Kingetsu in view of Dapogny further in view of Jacobs and Dighe with the teachings of Talagala by including selecting a second image recognition model for reuse based on the variance (i.e., similarity) between the results of the machine learning models (i.e., first feature label and third feature label) and the actual results for the training data (i.e., reference label) and installing the second image recognition model in the edge device. One having ordinary skill in the art would have been motivated to combine these references because doing so would allow for a machine learning system that delivers accurate and relevant results, as recognized by Talagala. Thus, the claimed invention would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention.
Regarding claim 16, Kingetsu in view of Dapogny further in view of Jacobs, Dighe and Talagala teaches the system according to claim 15, the processor further configured to execute a method comprising:
determining a second feature label associated with the image data using the second image recognition model (Talagala, Para. [0066], the best-fitting machine learning model may be the machine learning model that produced results (i.e., feature label) most similar to the actual results for the training data);
selecting, based on variances of the first feature label and the second feature label from the reference label, the second image recognition model for reuse (Talagala, Para. [0066], a model selection module that receives the machine learning models that the training pipelines generate and determines which of the machine learning models is the best fit for the objective that is being analyzed. The best-fitting machine learning model may be the machine learning model that produced results most similar to the actual results for the training data); and
performing inferencing subsequently received image input using the second image recognition model (Talagala, Para. [0069], the selected machine learning model is pushed to the policy pipeline 202, for validation, verification, or the like, which then pushes it back to the inference pipelines. Para. [0061], the inference pipelines use the machine learning model and the corresponding analytics engine to generate machine learning results/predictions on input data).
The proposed combination as well as the motivation for combining the Kingetsu, Dapogny, Jacobs, Dighe and Talagala references presented in the rejection of Claim 15, apply to Claim 16 and are incorporated herein by reference. Thus, the system recited in Claim 16 is met by Kingetsu in view of Dapogny further in view of Jacobs, Dighe and Talagala.
Regarding claim 17, Kingetsu in view of Dapogny further in view of Jacobs, Dighe and Talagala teaches the system according to claim 15, the processor further configured to execute a method comprising:
iteratively retraining the first image recognition model using a combination of the reference label and the image data while a level of accuracy in inferring the image data is below a predetermined level of accuracy (Kingetsu, Para. [0119], the detection unit may notify the training unit of information indicating that accuracy degradation has been detected and retrain the machine learning model data by using a training dataset that is newly designated); and
updating the retrained first image recognition model in the plurality of trained image recognition models (Dapogny, Para. [0101], the processing machine may perform one or more methods of machine learning as discussed above in order to train, calibrate, re-train, and/or re-calibrate any of the predictive models stored on the data storage).
The proposed combination as well as the motivation for combining the Kingetsu, Dapogny, Jacobs, Dighe and Talagala references presented in the rejection of Claim 15, apply to Claim 17 and are incorporated herein by reference. Thus, the system recited in Claim 17 is met by Kingetsu in view of Dapogny further in view of Jacobs, Dighe and Talagala.
Claim 18 is rejected under 35 U.S.C. 103 as being unpatentable over Hiroaki Kingetsu (US 2022/0207307 A1) in view of Dapogny et al. (US 2024/0161483 A1, with Foreign priority to Application No. EP 4057184, filed Mar