DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Acknowledgement is made of Applicant’s claim of priority from U.S. Provisional Application No. 63/408,712, filed September 21, 2022.
Status of Claims
Claims 1-2, 4-5 and 8-23 are pending. Claims 3 and 6-7 have been canceled. Claims 21-23 are newly added.
Response to Arguments
Applicant's arguments filed May 11, 2026 have been fully considered but they are not persuasive.
Applicant argues that the references do not teach the limitation “responsive to the validating, adaptively installing the first image recognition model for reuse” in claim 1. Examiner respectfully disagrees. As described in Kingetsu, when accuracy degradation is detected (i.e., the model does not satisfy an accuracy test) the model is retrained and then reused for class prediction (see Kingetsu, Para. [0119] and Fig. 20). When accuracy degradation is not detected (i.e., the model satisfies an accuracy test), the model is not retrained and the process continues to using the model to predict classes and output a result (see Kingetsu, Para. [0121]). Examiner asserts that this process of Kingetsu satisfies the broadest reasonable interpretation of “responsive to the validating, adaptively installing the first image recognition model for reuse” because the model of Kingetsu is adaptively installed for reuse by either retraining or by continuing to use the model to predict classes when it satisfies an accuracy test. Thus, the 35 USC 103 rejection of claim 1 is maintained.
Applicant further argues that the references do not teach the limitation “comparing the first feature label of the image data to a predetermined threshold of a reference label of a sample image generated by a trained teacher model” in claim 14. Examiner respectfully disagrees. As described in the 35 USC 103 rejections below, the Dapogny reference teaches applying one or more predictive models to an image to determine one or more predictions and computing a cost (or loss) function based on comparison between the prediction of the student model and prediction of the teacher model (see Dapogny, Para. [0041]). The cost function comprises computing a metric for the error, i.e., the distance between the two predictions (see Dapogny, Para. [0058]). Examiner asserts that determining the distance between a first feature label and a reference label is sufficient to satisfy the broadest reasonable interpretation of “comparing the first feature label to a predetermined threshold of a reference label”. Thus, the 35 USC 103 rejection of claim 14 is maintained.
Applicant further argues that the references do not teach the limitation “causing, responsive to validation of accuracy of a first image recognition model based on a comparison between the reference image data and a first feature label of the sample image data determined using the first image recognition model, a selection of the first image recognition model from a plurality of image recognition models for installation and installation of the first image recognition model” in claim 19. Examiner respectfully disagrees. As described in the 35 USC 103 rejection below, Talagala teaches determining a best fit from a plurality of models where the best-fitting machine learning model may be the machine learning model that produced the results most similar to actual results for training data (see Talagala, Para. [0066]). Examiner asserts that this is sufficient to teach the broadest reasonable interpretation of the above recited limitation because Talagala selects a first image recognition model from a plurality of models for installation and uses this model based on validating that it is the most accurate model from a comparison of the results it produces to the actual results (i.e., reference data).
Applicant further argues that Claims 2-5, 7, 10 and 12-13 are not taught by the prior art by virtue of their dependence on Claim 1. The Examiner respectfully disagrees. Claim 1 is taught by the prior art of record as discussed above (see paragraph 4 on page 2-3) and as shown in the 35 USC 103 rejections below.
Applicant further argues that Claims 15-18 are not taught by the prior art by virtue of their dependence on Claim 14. The Examiner respectfully disagrees. Claim 14 is taught by the prior art of record as discussed above (see the second paragraph on page 3) and as shown in the 35 USC 103 rejections below.
Finally, Applicant argues that Claim 20 is not taught by the prior art by virtue of its dependence on Claim 19. The Examiner respectfully disagrees. Claim 19 is taught by the prior art of record as discussed above (see paragraph 1 on page 4) and as shown in the 35 USC 103 rejections below. Thus, the 35 USC 103 rejection of the claims is maintained, and consequently, THIS ACTION IS FINAL.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1, 6, 8-9 and 11 are rejected under 35 U.S.C. 103 as being unpatentable over Hiroaki Kingetsu (US 2022/0207307 A1) in view of Jason Brownlee (“A Gentle Introduction to Mixture of Experts Ensembles”, November 7, 2021) further in view of Dapogny et al. (US 2024/0161483 A1, with Foreign priority to Application No. EP 4057184, filed March 11, 2021, US PGPub used herein as a translation and for mapping purposes) and Dighe et al. (US 2022/0093095 A1)
Regarding claim 1, Kingetsu teaches a method for reusing and retraining image recognition models, comprising:
receiving first image data (Kingetsu, Para. [0066], training data is given and a correct answer label of “dog” is given to the training data. Para. [0090], the training data corresponds to data on email spam, electricity demand prediction, stock price prediction, data on poker hands, image data, or the like);
determining, based on the first image data, a reference label of the first image data using a trained teacher model inferencing the first image data, wherein the reference label represents a ground-truth inference of the first image data (Kingetsu, Para. [0067], the computing system trains the parameters of the teacher model such that the output result of the teacher model obtained at the time of inputting the training data approaches the correct answer label (i.e., ground-truth inference) of “dog”),
determining a first feature label of the first image data using the first image recognition model (Kingetsu, Para. [0067], the computing system trains the parameters of the Student Model such that the output result obtained at the time of inputting the training data approaches the output result of the Teacher Model);
responsive to the validating, adaptively installing the first image recognition model for reuse (Kingetsu, Para. [0119], the detection unit may notify the training unit of information indicating that accuracy degradation has been detected and retrain the machine learning model. Para. [0121], in the case where accuracy degradation of the machine learning model 50 is not detected, that inputs the operation data set, and that predicts a classification class of each of the pieces of operation data. The prediction unit 154 may output and display the prediction result onto the display unit 130, or may transmit the prediction result to an external device);
receiving second image data (Kingetsu, Para. [0119], the training unit retrains the machine learning model by using a training data set that is newly designated (i.e., second image data)); and
processing the second image data for inferencing (Kingetsu, Para. [0119], the training unit retrains the machine learning model by using a training data set that is newly designated (i.e., second image data)).
Although Kingetsu teaches a plurality of inspector models created based on the knowledge distillation of the machine learning model (Kingetsu, Para. [0166]), Kingetsu does not explicitly teach “selecting, based on the first image data using a selection model, a first image recognition model from a plurality of trained image recognition models, wherein the selection model comprises a gating network, wherein the gating network selects the first image recognition model based on level of confidence of the first image recognition model predicting a label of the first image data without further training the plurality of trained image recognition models based on the first image data”. However, in an analogous field of endeavor, Brownlee teaches a model is used to interpret the predictions made by each expert and to aid in deciding which expert to trust for a given input. This is called the gating model, or the gating network, given that it is traditionally a neural network model. The gating network takes as input the input pattern that was provided to the expert models and outputs the contribution that each expert should have in making a prediction for the input. The gating network might have a softmax output that gives a probability-like confidence score (i.e., level of confidence of the first image recognition model predicting a label of the first image data) for each expert. Finally, the mixture of expert models must make a prediction, and this is achieved using a pooling or aggregation mechanism. This might be as simple as selecting the expert with the largest output or confidence provided by the gating network. Alternatively, a weighted sum prediction could be made that explicitly combines the predictions made by each expert and the confidence estimated by the gating network (i.e., select the model based on level of confidence of prediction for input without further training the models) (Brownlee, “Gating Model” and “Pooling Method”).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the method of Kingetsu with the teachings of Brownlee by including selecting the first image recognition model using a gating network that outputs a probability-like confidence score for each model in making a prediction for the input and selects the model with the largest confidence. One having ordinary skill in the art would have been motivated to combine these references because doing so would allow for an automated approach to predictive modeling that allows for broad problem-solving, as recognized by Brownlee.
Although Kingetsu in view of Brownlee teaches a teacher model determining a reference label (Kingetsu, Para. [0067]) and retraining the machine learning model data based on accuracy degradation detection (Kingetsu, Para. [0119]), they do not explicitly teach “wherein the teacher model generates the reference label by inferencing” and “validating an accuracy of the first image recognition model in determining the first feature label of the first image data based on a comparison between the reference label and the first feature label”. However, in an analogous field of endeavor, Dapogny teaches a method of inference of one or more predictive models to obtain one or more predictions by applying one of the one or more predictive models to the provided image (Dapogny, Para. [0041]). Dapogny further teaches computing a cost (or loss) function based on the prediction of the student model and the prediction of the teacher model, and updating the parameters of the student model based on the computed cost function and by backpropagation (Dapogny, Para. [0064]).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the method of Kingetsu in view of Brownlee with the teachings of Dapogny by including the teacher model generating the reference label based on inference and validating an accuracy (i.e., cost/loss function) based on comparison between the reference label and the first feature label. One having ordinary skill in the art would have been motivated to combine these references because doing so would allow for a low-cost network that is trained by retraining a student network using knowledge distillation from the teacher network, as recognized by Dapogny.
Although Kingetsu in view of Brownlee further in view of Dapogny teaches a teacher model determining a reference label (Kingetsu, Para. [0067]), they do not explicitly teach ”the trained teacher model performs inferencing of the first image data more accurately than the first image recognition model by consuming more memory resources than the first image recognition model”. However, in an analogous field of endeavor, Dighe teaches that student-teacher training is a training technique where a (typically) more accurate and computationally expensive teacher model trains a less computationally expensive student model to mimic the teacher model’s outputs and/or determinations (Dighe, Para. [0277]).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the method of Kingetsu in view of Brownlee further in view of Dapogny with the teachings of Dighe by including that the teacher model is more accurate by consuming more computation resources than the student model (i.e., first image recognition model). One having ordinary skill in the art would have been motivated to combine these references because doing so would allow for knowledge distillation from a teacher model to students that use fewer resources, as recognized by Dighe. Thus, the claimed invention would have been obvious to one having ordinary skill in the art before the effective filing date.
Regarding claim 8, Kingetsu in view of Brownlee further in view of Dapogny and Dighe teaches the method of claim 1 wherein the receiving first image data comprises:
receiving, based on a predefined rule associated with a timing of capturing a frame of video data, the first image data (Dapogny, Para. [0069], the method of machine-learning may further comprise determining, based on the prior condition, whether the one or more images are to be provided to the first and second model. In other words, the prior condition may be a(n) (automatic) determining criteria whether a provided image from the first stream is to be used in the on-the-fly adaptation. Para. [0068], the prior condition may be computed based on one or more of: image metadata, time stamp, information extracted from the image's content. In examples, the information extracted from the image may comprise the time of the day, weather, luminosity, backlight, movement of the camera, and/or hazing).
The proposed combination as well as the motivation for combining the Kingetsu, Brownlee, Dapogny and Dighe references presented in the rejection of Claim 1, apply to Claim 8 and are incorporated herein by reference. Thus, the method recited in Claim 8 is met by Kingetsu in view of Brownlee further in view of Dapogny and Dighe.
Regarding claim 9, Kingetsu in view of Brownlee further in view of Dapogny and Dighe teaches the method of claim 1, wherein the receiving first image data comprises:
receiving, based on a change of scenery captured in a frame of video data, the first image data (Dapogny, Para. [0069], the method of machine-learning may further comprise determining, based on the prior condition, whether the one or more images are to be provided to the first and second model. In other words, the prior condition may be a(n) (automatic) determining criteria whether a provided image from the first stream is to be used in the on-the-fly adaptation. Para. [0068], the prior condition may be computed based on one or more of: image metadata, time stamp, information extracted from the image's content. In examples, the information extracted from the image may comprise the time of the day, weather, luminosity, backlight, movement of the camera, and/or hazing).
The proposed combination as well as the motivation for combining the Kingetsu, Brownlee, Dapogny and Dighe references presented in the rejection of Claim 1, apply to Claim 9 and are incorporated herein by reference. Thus, the method recited in Claim 9 is met by Kingetsu in view of Brownlee further in view of Dapogny and Dighe.
Regarding claim 11, Kingetsu in view of Brownlee further in view of Dapogny and Dighe teaches the method of claim 1, further comprising:
iteratively retraining the first image recognition model (Kingetsu, Para. [0119], the detection unit may notify the training unit of information indicating that accuracy degradation has been detected and retrain the machine learning model data by using a training dataset that is newly designated); and
adding the iteratively retrained first image recognition model to the plurality of trained image recognition models (Kingetsu, Para. [0126], the training unit retrains the machine learning model by using a new training data set and proceeds to Step S102. Para. [0123], In Step S102, the creating unit generates the inspector models from the distillation data table).
Claim 2 is rejected under 35 U.S.C. 103 as being unpatentable over Hiroaki Kingetsu (US 2022/0207307 A1) in view of Brownlee (“A Gentle Introduction to Mixture of Experts Ensembles”, November 7, 2021) further in view of Dapogny et al. (US 2024/0161483 A1, with Foreign priority to Application No. EP 4057184, filed March 11, 2021, US PGPub used herein as a translation and for mapping purposes) and Dighe et al. (US 2022/0093095 A1), as applied to claims 1, 6, 8-9 and 11 above, and further in view of Talagala et al. (US 2019/0108417 A1) and Jacobs et al. (US 2020/0302784 A1).
Regarding claim 2, Kingetsu in view of Brownlee further in view of Dapogny and Dighe teaches the method of claim 1, as described above.
Although Kingetsu in view of Brownlee further in view of Dapogny and Dighe teaches selecting a best model using a gating network (Brownlee, “A Gentle Introduction to Mixture of Experts Ensembles”), they do not explicitly teach “selecting, based on the first image data using the selection model, a second image recognition model from the plurality of trained image recognition models”, “determining a third feature label associated with the first image data using the selected second image recognition model”, “determining variances of the first feature label and the third feature label from the reference label” and “selecting, based on the variances of the first feature label and the third feature label from the reference label, the first image recognition model”. However, in an analogous field of endeavor, Talagala teaches a model selection module that determines which of the machine learning models is the best fit for the objective that is being analyzed. The best-fitting machine learning model may be the machine learning model that produced results most similar to the actual results for the training data (i.e., reference label) (e.g., the most accurate machine learning model) (Talagala, Para. [0066]).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the method of Kingetsu in view of Brownlee further in view of Dapogny and Dighe with the teachings of Talagala by including selecting the image recognition model based on the variance (i.e., similarity) between the results of the machine learning models (i.e., first feature label and third feature label) and the actual results for the training data (i.e., reference label). One having ordinary skill in the art would have been motivated to combine these references because doing so would allow for a machine learning system that delivers accurate and relevant results, as recognized by Talagala.
Although Kingetsu in view of Brownlee further in view of Dapogny, Dighe and Talagala teaches the machine learning models are pushed to the inference pipelines that comprise the logical pipeline grouping for the objective, each of which is executing on live data coming from an edge device, e.g., input data (Talagala, Para. [0062]), they do not explicitly teach “receiving the first image data from an edge device associated with a 5G telecommunication network”. However, in an analogous field of endeavor, Jacobs teaches a 5G network (or another network protocol) can connect the edge device with other devices and objects (Jacobs, Para. [0031]).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the method of Kingetsu in view of Brownlee further in view of Dapogny, Dighe and Talagala with the teachings of Jacobs by including the edge device being connected to a 5G telecommunication network. One having ordinary skill in the art would have been motivated to combine these references because doing so would allow for reducing a latency of data communications and to conserve bandwidth, as recognized by Jacobs. Thus, the claimed invention would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention.
Claims 4 and 22 are rejected under 35 U.S.C. 103 as being unpatentable over Hiroaki Kingetsu (US 2022/0207307 A1) in view of Brownlee (“A Gentle Introduction to Mixture of Experts Ensembles”, November 7, 2021) further in view of Dapogny et al. (US 2024/0161483 A1, with Foreign priority to Application No. EP 4057184, filed March 11, 2021, US PGPub used herein as a translation and for mapping purposes) and Dighe et al. (US 2022/0093095 A1), as applied to claims 1, 6, 8-9 and 11 above, and further in view of Takimoto et al. (US 2022/0129675 A1).
Regarding claim 4, Kingetsu in view of Brownlee further in view of Dapogny and Dighe teaches the method of claim 1, as described above.
Although Kingetsu in view of Brownlee further in view of Dapogny and Dighe teaches selecting a best model using a gating network (Brownlee, “A Gentle Introduction to Mixture of Experts Ensembles”), they do not explicitly teach “wherein the selecting the first image recognition model from the plurality of trained image recognition models comprises: selecting, based on the first image data using the selection model, a set of image recognition models from the plurality of trained image recognition models”, “ranking, based on probability values associated with a likelihood of respective image recognition models accurately recognizing the first image data, the set of image recognition models”, and “selecting, based on the ranked set of image recognition models, the first image recognition model”. However, in an analogous field of endeavor, Takimoto teaches the CPU obtains a score for the result of object detection processing for each of the P captured images in correspondence with each of the M candidate learning models. The CPU then performs ranking (ranking creation) of the M candidate learning models based on the scores, and selects N candidate learning models from the M candidate learning models (i.e., N=1) (Takimoto, Para. [0137]).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the method of Kingetsu in view of Brownlee further in view of Dapogny and Dighe with the teachings of Takimoto by including selecting a set of candidate learning models and ranking the models based on a score of the result, and selecting the first image recognition model based on the rank. One having ordinary skill in the art would have been motivated to combine these references because doing so would allow for ensuring high performance of a learning model for a new input with a low operation cost, as recognized by Takimoto. Thus, the claimed invention would have been obvious to one having ordinary skill in the art before the effective filing date.
Regarding claim 22, Kingetsu in view of Brownlee further in view of Dapogny and Dighe teaches the method of claim 1, wherein, for an edge device, the plurality of trained image recognition models includes
an image recognition model currently installed at the edge device (Kingetsu, Para. [0119], the detection unit detects accuracy degradation of the machine learning model (i.e., image recognition model currently installed at the device)), and
a most recently trained image recognition model (Kingetsu, Para. [0119], the training unit retrains the machine learning model by using a training data set that is newly designated (i.e., most recently trained image recognition model)).
Although Kingetsu in view of Brownlee further in view of Dapogny and Dighe teaches retraining the machine learning model (Kingetsu, Para. [0119]), they do not explicitly teach “wherein, for an edge device, the plurality of trained image recognition models includes k candidate image recognition models in a model store”. However, in an analogous field of endeavor, Takimoto teaches the CPU selects M learning models (candidate learning models) that are candidates in E (E is an integer of 2 or more) learning models stored in the external storage device (i.e., k candidate image recognition models in a model store) (Takimoto, Para. [0120]).
The proposed combination as well as the motivation for combining the Kingetsu, Brownlee, Dapogny, Dighe and Takimoto references presented in the rejection of Claim 4, apply to Claim 22 and are incorporated herein by reference. Thus, the system recited in Claim 22 is met by Kingetsu in view of Brownlee further in view of Dapogny, Dighe and Takimoto.
Claim 5 is rejected under 35 U.S.C. 103 as being unpatentable over Hiroaki Kingetsu (US 2022/0207307 A1) in view of Brownlee (“A Gentle Introduction to Mixture of Experts Ensembles”, November 7, 2021) further in view of Dapogny et al. (US 2024/0161483 A1, with Foreign priority to Application No. EP 4057184, filed March 11, 2021, US PGPub used herein as a translation and for mapping purposes) and Dighe et al. (US 2022/0093095 A1), as applied to claims 1, 6, 8-9 and 11 above, and further in view of Jacobs et al. (US 2020/0302784 A1).
Regarding claim 5, Kingetsu in view of Brownlee further in view of Dapogny and Dighe teaches the method of claim 1, as described above.
Although Kingetsu in view of Brownlee further in view of Dapogny and Dighe teaches a stream of images from a live video camera (Dapogny, Para. [0064]), they do not explicitly teach “wherein the receiving first image data comprises: receiving, by an edge server associated with the 5G telecommunication network, the first image data from an edge device via a wireless network of the 5G telecommunication network, wherein the edge device includes a camera for capturing the first image data”. However, in an analogous field of endeavor, Jacobs teaches a 5G network (or another network protocol) can connect the edge device with the augmented reality device, the centrally located components of the movement analytics platform, and/or other objects (Jacobs, Para. [0031]) and teaches augmented reality device includes a camera that can capture video data, image data, and/or the like (Jacobs, Para. [0046]).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the method of Kingetsu in view of Brownlee further in view of Dapogny and Dighe with the teachings of Jacobs by including an edge device connected to a 5G network and wherein the edge device includes a camera for capturing the first image data. One having ordinary skill in the art would have been motivated to combine these references, because doing so would allow for reducing a latency of data communications and to conserve bandwidth, as recognized by Jacobs. Thus, the claimed invention would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention.
Claim 10 is rejected under 35 U.S.C. 103 as being unpatentable over Hiroaki Kingetsu (US 2022/0207307 A1) in view of Brownlee (“A Gentle Introduction to Mixture of Experts Ensembles”, November 7, 2021) further in view of Dapogny et al. (US 2024/0161483 A1, with Foreign priority to Application No. EP 4057184, filed March 11, 2021, US PGPub used herein as a translation and for mapping purposes) and Dighe et al. (US 2022/0093095 A1), as applied to claims 1, 6, 8-9 and 11 above, and further in view of Atsushi Nogami (US 2022/0269996 A1) and Balasubramanian et al. (US 2022/0172100 A1).
Regarding claim 10, Kingetsu in view of Brownlee further in view of Dapogny and Dighe teaches the method of claim 1, as described above.
Although Kingetsu in view of Brownlee further in view of Dapogny and Dighe teaches selecting a best model using a gating network (Brownlee, “A Gentle Introduction to Mixture of Experts Ensembles”), they do not explicitly teach “counting a number of occasions of selecting the first image recognition model”. However, in an analogous field of endeavor, Nogami teaches recording the count at which a model is selected, and that this count of selection is recorded for each model, and the records are added up such that the model evaluation value rises as the selection count increases (Nogami, Para. [0135]).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the method of Kingetsu in view of Brownlee further in view of Dapogny and Dighe with the teachings of Nogami by including recording a count of the number of occasions of selecting the first image recognition model. One having ordinary skill in the art before the effective filing date would have been motivated to combine these references because doing so would allow for evaluating the selection of a model, as recognized by Nogami.
Although Kingetsu in view of Brownlee further in view of Dapogny, Dighe and Nogami teaches selecting a best model using a gating network (Brownlee, “A Gentle Introduction to Mixture of Experts Ensembles”), they do not explicitly teach “removing, based on the number of occasions of selecting the first image recognition model, the first image recognition model from the plurality of trained image recognition models”. However, in an analogous field of endeavor, Balasubramanian teaches that when the model metrics indicate that the machine learning model is performing poorly (for example, has an error rate above a threshold value, has a statistical distribution that is not an expected or desired distribution (for example, not a binomial distribution, a Poisson distribution, a geometric distribution, a normal distribution, Gaussian distribution, etc.), has an execution latency above a threshold value, has a confidence level below a threshold value)) and/or is performing progressively worse (for example, the quality metric continues to worsen over time), the model training system can instruct the virtual machine instance to delete the ML training container and/or to delete any model data stored in the training model data store (Balasubramanian, Para. [0206]).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the method of Kingetsu in view of Brownlee further in view of Dapogny, Dighe and Nogami with the teachings of Balasubramanian by deleting the first image training model from the data store when the number of occasions of model selection (as taught by Nogami) is above a threshold value. One having ordinary skill in the art would have been motivated to combine these references, because doing so would allow for removing a machine learning model that is performing incorrectly based on model metrics, as recognized by Balasubramanian. Thus, the claimed invention would have been obvious to one having ordinary skill in the art before the effective filing date.
Claim 12 is rejected under 35 U.S.C. 103 as being unpatentable over Hiroaki Kingetsu (US 2022/0207307 A1) in view of Brownlee (“A Gentle Introduction to Mixture of Experts Ensembles”, November 7, 2021) further in view of Dapogny et al. (US 2024/0161483 A1, with Foreign priority to Application No. EP 4057184, filed March 11, 2021, US PGPub used herein as a translation and for mapping purposes) and Dighe et al. (US 2022/0093095 A1), as applied to claims 1, 6, 8-9 and 11 above, and further in view of Zheng et al. (US 2018/0348781 A1).
Regarding claim 12, Kingetsu in view of Brownlee further in view of Dapogny and Dighe teaches the method of claim 11, as described above.
Although Kingetsu in view of Brownlee further in view of Dapogny and Dighe teaches retraining the machine learning model when accuracy degradation is detected (Kingetsu, Para. [0119]), they do not explicitly teach “selecting a plurality of candidate models for retraining from the plurality of trained image recognition models” and “iteratively processing the plurality of trained image recognition models until a change of a level of accuracy in labeling is less than a predetermined threshold”. However, in an analogous field of endeavor, Zheng teaches determining whether the model to be updated involves the global models. If the global models are to be updated, appropriate labeled training data are used to re-train. The re-trained global models are then tested, using benchmark testing data selected for testing the global models. If the testing result is satisfactory, the global models are updated. If the testing result is not satisfactory, the processing goes back to re-train (e.g., iteratively) the global models (Zheng, Para. [0156]).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the method of Kingetsu in view of Brownlee further in view of Dapogny and Dighe with the teachings of Zheng by including determining a set of candidate models to retrain iteratively until the testing result (i.e., change in level of accuracy) is satisfactory. One having ordinary skill in the art would have been motivated to combine these references because doing so would allow for automatically updating and retraining machine learning models for higher accuracy results, as recognized by Zheng. Thus, the claimed invention would have been obvious to one having ordinary skill in the art before the effective filing date.
Claim 13 is rejected under 35 U.S.C. 103 as being unpatentable over Hiroaki Kingetsu (US 2022/0207307 A1) in view of Brownlee (“A Gentle Introduction to Mixture of Experts Ensembles”, November 7, 2021) further in view of Dapogny et al. (US 2024/0161483 A1, with Foreign priority to Application No. EP 4057184, filed March 11, 2021, US PGPub used herein as a translation and for mapping purposes) and Dighe et al. (US 2022/0093095 A1), as applied to claims 1, 6, 8-9 and 11 above, and further in view of Kale et al. (US 2023/0139682 A1, filed November 1, 2021).
Regarding claim 13, Kingetsu in view of Brownlee further in view of Dapogny and Dighe teaches the method of claim 11, as described above.
Although Kingetsu in view of Brownlee further in view of Dapogny and Dighe teaches retraining a model based on detected accuracy degradation (Kingetsu, Para. [0119]), they do not explicitly teach “wherein the iteratively processing includes: iteratively retraining the plurality of trained image recognition models by allocating a time period of using a processing resource for retraining the plurality of trained image recognition models”. However, in an analogous field of endeavor, Kale teaches a scheduler module that may determine an amount of time to perform the retraining process and schedule the retraining process to be performed during a time period that corresponds to or is larger than the determined amount of time. Scheduler module may schedule the retraining process to be performed during a time period that corresponds to or is larger than the time period between time T(0) and time T(N) (Kale, Para. [0077]).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the method of Kingetsu in view of Brownlee further in view of Dapogny and Dighe with the teachings of Kale by including iteratively retraining the image recognition models by scheduling a time period during which to perform the retraining. One having ordinary skill in the art would have been motivated to combine these references, because doing so would allow for increasing overall system efficiency and decreasing system latency, as recognized by Kale. Thus, the claimed invention would have been obvious to one having ordinary skill in the art before the effective filing date.
Claim 14 is rejected under 35 U.S.C. 103 as being unpatentable over Hiroaki Kingetsu (US 2022/0207307 A1) in view of Dapogny et al. (US 2024/0161483 A1, with Foreign priority to Application No. EP 4057184, filed March 11, 2021, US PGPub used herein as a translation and for mapping purposes) further in view of Jacobs et al. (US 2020/0302784 A1) and Dighe et al. (US 2022/0093095 A1).
Regarding claim 14, Kingetsu teaches a system for reusing and retraining image recognition models for inferencing data wherein the system comprises a processor (Kingetsu, Para. [0099]), configured to execute operations comprising:
receiving image data (Kingetsu, Para. [0066], training data is given and a correct answer label of “dog” is given to the training data. Para. [0090], the training data corresponds to data on email spam, electricity demand prediction, stock price prediction, data on poker hands, image data, or the like);
determining, (Kingetsu, Para. [0067], the computing system trains the parameters of the Student Model such that the output result obtained at the time of inputting the training data approaches the output result of the Teacher Model);
wherein the teacher model generates the reference label (Kingetsu, Para. [0067], the computing system trains the parameters of the teacher model such that the output result of the teacher model obtained at the time of inputting the training data approaches the correct answer label of “dog”);
based on the level of accuracy, selecting the first image recognition model for retraining (Kingetsu, Para. [0119], the detection unit may notify the training unit of information indicating that accuracy degradation has been detected and retrain the machine learning model data by using a training dataset that is newly designated); and
Although Kingetsu teaches a teacher and student model for labelling data (Kingetsu, Para. [0067]), Kingetsu does not explicitly teach “wherein the first image recognition model corresponds to an inference model”, the teacher model determines the reference label “by inferencing”, “comparing the first feature label of the image data to a predetermined threshold of a reference label of a sample image generated by a trained teacher model”, “based on the comparing, determining a level of accuracy of inferencing the image data by the first image recognition model”, and “updating, based on the retrained first image recognition model, a store of a plurality of trained image recognition models”. However, in an analogous field of endeavor, Dapogny teaches a method of inference of one or more predictive models to obtain one or more predictions by applying one of the one or more predictive models to the provided image (Dapogny, Para. [0041]). Dapogny further teaches computing a cost (or loss) function based on the prediction of the student model and the prediction of the teacher model (i.e., level of accuracy), and updating the parameters of the student model based on the computed cost function and by backpropagation (Dapogny, Para. [0064]). Dapogny further teaches the processing machine may perform one or more methods of machine learning as discussed above in order to train, calibrate, re-train, and/or re-calibrate any of the predictive models stored on the data storage unit (Dapogny, Para. [0101]).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the system of Kingetsu with the teachings of Dapogny by including the teacher model and first image recognition model generating the reference label based on inference, determining an accuracy (i.e., cost/loss function) based on comparison between the reference label and the first feature label, and updating the store of the plurality of models by re-training one or more of the models. One having ordinary skill in the art would have been motivated to combine these references because doing so would allow for a low-cost network that is trained by retraining a student network using knowledge distillation from the teacher network, as recognized by Dapogny.
Although Kingetsu in view of Dapogny teaches a processing machine for performing re-training (Dapogny, Para. [0101]), they do not explicitly teach the inferencing data is “captured by an edge device”. However, in an analogous field of endeavor, Jacobs teaches a server device for storing and/or processing a data feed, wherein the server device can be an edge device (Jacobs, Para. [0047]).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the system of Kingetsu in view of Dapogny with the teachings of Jacobs by including an edge device connected to a 5G network for storing and/or processing data. One having ordinary skill in the art would have been motivated to combine these references, because doing so would allow for reducing a latency of data communications and to conserve bandwidth, as recognized by Jacobs.
Although Kingetsu in view of Dapogny further in view of Jacobs teaches a teacher model determining a reference label (Kingetsu, Para. [0067]), they do not explicitly teach ”the teacher model performs inferencing of the first image data more accurately than the first image recognition model by consuming more memory resources than the first image recognition model”. However, in an analogous field of endeavor, Dighe teaches that student-teacher training is a training technique where a (typically) more accurate and computationally expensive teacher model trains a less computationally expensive student model to mimic the teacher model’s outputs and/or determinations (Dighe, Para. [0277]).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the method of Kingetsu in view of Dapogny further in view of Jacobs with the teachings of Dighe by including that the teacher model is more accurate by consuming more computation resources than the student model (i.e., first image recognition model). One having ordinary skill in the art would have been motivated to combine these references because doing so would allow for knowledge distillation from a teacher model to students that use fewer resources, as recognized by Dighe. Thus, the claimed invention would have been obvious to one having ordinary skill in the art before the effective filing date.
Claims 15-17 are rejected under 35 U.S.C. 103 as being unpatentable over Hiroaki Kingetsu (US 2022/0207307 A1) in view of Dapogny et al. (US 2024/0161483 A1, with Foreign priority to Application No. EP 4057184, filed March 11, 2021, US PGPub used herein as a translation and for mapping purposes) further in view of Jacobs et al. (US 2020/0302784 A1) and Dighe et al. (US 2022/0093095 A1), as applied to claim 14 above, and further in view of Talagala et al. (US 2019/0108417 A1).
Regarding claim 15, Kingetsu in view of Dapogny further in view of Jacobs and Dighe teaches the system of claim 14, as described above.
Although Kingetsu in view of Dapogny further in view of Jacobs and Dighe teaches an edge device associated with the 5G telecommunication network (Jacobs, Para. [0031]), they do not explicitly teach “selecting, based on the first feature label by a selection model, a second image recognition model from a plurality of image recognition models for reuse” and “installing the second image recognition model in the edge device”. However, in an analogous field of endeavor, Talagala teaches a model selection module that receives the machine learning models that the training pipelines generate and determines which of the machine learning models is the best fit for the objective that is being analyzed (Talagala, Para. [0066]). Talagala further teaches the machine learning models are pushed to the inference pipelines that comprise the logical pipeline grouping for the objective, each of which is executing on live data coming from an edge device, e.g., input data (Talagala, Para. [0062]).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the system of Kingetsu in view of Dapogny further in view of Jacobs and Dighe with the teachings of Talagala by including selecting a second image recognition model for reuse based on the variance (i.e., similarity) between the results of the machine learning models (i.e., first feature label and third feature label) and the actual results for the training data (i.e., reference label) and installing the second image recognition model in the edge device. One having ordinary skill in the art would have been motivated to combine these references because doing so would allow for a machine learning system that delivers accurate and relevant results, as recognized by Talagala. Thus, the claimed invention would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention.
Regarding claim 16, Kingetsu in view of Dapogny further in view of Jacobs, Dighe and Talagala teaches the system according to claim 15, wherein the selecting the second image recognition model comprises:
determining a second feature label associated with the image data using the second image recognition model (Talagala, Para. [0066], the best-fitting machine learning model may be the machine learning model that produced results (i.e., feature label) most similar to the actual results for the training data); and
selecting, based on variances of the first feature label and the second feature label from the reference label, the second image recognition model for reuse (Talagala, Para. [0066], a model selection module that receives the machine learning models that the training pipelines generate and determines which of the machine learning models is the best fit for the objective that is being analyzed. The best-fitting machine learning model may be the machine learning model that produced results most similar to the actual results for the training data); and
wherein the operations further comprise: performing inferencing subsequently received image input using the second image recognition model (Talagala, Para. [0069], the selected machine learning model is pushed to the policy pipeline 202, for validation, verification, or the like, which then pushes it back to the inference pipelines. Para. [0061], the inference pipelines use the machine learning model and the corresponding analytics engine to generate machine learning results/predictions on input data).
The proposed combination as well as the motivation for combining the Kingetsu, Dapogny, Jacobs, Dighe and Talagala references presented in the rejection of Claim 15, apply to Claim 16 and are incorporated herein by reference. Thus, the system recited in Claim 16 is met by Kingetsu in view of Dapogny further in view of Jacobs, Dighe and Talagala.
Regarding claim 17, Kingetsu in view of Dapogny further in view of Jacobs, Dighe and Talagala teaches the system according to claim 15, wherein the operations further comprise:
iteratively retraining the first image recognition model using a combination of the reference label and the image data while a level of accuracy in inferring the image data is below a predetermined level of accuracy (Kingetsu, Para. [0119], the detection unit may notify the training unit of information indicating that accuracy degradation has been detected and retrain the machine learning model data by using a training dataset that is newly designated); and
updating the retrained first image recognition model in the plurality of trained image recognition models (Dapogny, Para. [0101], the processing machine may perform one or more methods of machine learning as discussed above in order to train, calibrate, re-train, and/or re-calibrate any of the predictive models stored on the data storage).
The proposed combination as well as the motivation for combining the Kingetsu, Dapogny, Jacobs, Dighe and Talagala references presented in the rejection of Claim 15, apply to Claim 17 and are incorporated herein by reference. Thus, the system recited in Claim 17 is met by Kingetsu in view of Dapogny further in view of Jacobs, Dighe and Talagala.
Claim 18 is rejected under 35 U.S.C. 103 as being unpatentable over Hiroaki Kingetsu (US 2022/0207307 A1) in view of Dapogny et al. (US 2024/0161483 A1, with Foreign priority to Application No. EP 4057184, filed March 11, 2021, US PGPub used herein as a translation and for mapping purposes) further in view of Jacobs et al. (US 2020/0302784 A1), Dighe et al. (US 2022/0093095 A1) and Talagala et al. (US 2019/0108417 A1), as applied to claims 15-17 above, and further in view of Lintereur (US 2021/0183522 A1).
Regarding claim 18, Kingetsu in view of Dapogny further in view of Jacobs, Dighe and Talagala teaches the system according to claim 15,
wherein the reference label is higher in accuracy in inferencing the image data than the first feature label associated with the first image recognition model (Kingetsu, Para. [0067], the computing system trains the parameters of the teacher model such that the output result of the teacher model approaches the correct answer label of “dog”. Furthermore, the computing system trains the parameters of the student model such that the output result of the student model approaches the output result of the teacher model. An output of the teacher model is referred to as a “soft target”. A correct answer label of the training data is referred to as a “hard target”).
Although Kingetsu in view of Dapogny further in view of Jacobs, Dighe and Talagala teaches a model selection module that receives the machine learning models that the training pipelines generate and determines which of the machine learning models is the best fit for the objective that is being analyzed (Talagala, Para. [0066]), they do not explicitly teach “wherein the selection model selects, based on the image data, the second image recognition model from the plurality of image recognition models including a gating network” and “wherein the gating network predicts the first image recognition model based on a likelihood of outputting image data matching with the image data”. However, in an analogous field of endeavor, Lintereur teaches a model selection algorithm is used to choose the best model for each problem and that gating is a generalization of cross-validation selection that involves training another learning model to decide which of the models in the bucket is best-suited to solve the problem. Often, a perceptron is used for the gating model, which can be used to pick the “best model” or to give a linear weight to the predictions from each model in the bucket (Lintereur, Para. [0139]).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the system of Kingetsu in view of Dapogny further in view of Jacobs, Dighe and Talagala with the teachings of Lintereur by including selecting a first image recognition model by using a gating model to train another learning model to decide the best model based on a likelihood of outputting matching image data. One having ordinary skill in the art would have been motivated to combine these references because doing so would allow for higher accuracy by selecting the best image recognition model, as recognized by Lintereur. Thus, the claimed invention would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention.
Claim 23 is rejected under 35 U.S.C. 103 as being unpatentable over Hiroaki Kingetsu (US 2022/0207307 A1) in view of Dapogny et al. (US 2024/0161483 A1, with Foreign priority to Application No. EP 4057184, filed March 11, 2021, US PGPub used herein as a translation and for mapping purposes) further in view of Jacobs et al. (US 2020/0302784 A1) and Dighe et al. (US 2022/0093095 A1), as applied to claim 14 above, and further in view of Mattar et al. (US 12,056,579 B1).
Regarding claim 23, Kingetsu in view of Dapogny further in view of Jacobs and Dighe teaches the system of claim 14, as described above.
Although Kingetsu in view of Dapogny further in view of Jacobs and Dighe teaches retraining a model based on detected accuracy degradation (Kingetsu, Para. [0119]), they do not explicitly teach “wherein the operations further comprise: as part of scheduling the first image recognition model for retraining, adding the first image recognition model to a queue of image recognition models to be retrained, wherein the image recognition models to be retrained are ordered, in the queue, according to levels of accuracy estimated for the respective image recognition models to be retrained”. However, in an analogous field of endeavor, Mattar teaches a scheduler can use a dynamic queue of models for retraining to generate the schedule. The queue can be dynamically adjusted in real time in response to various changing conditions. Dynamically adjusting the queue can also include recalculating the reward score for any queued items affected by changing conditions, and then rescheduling the queue based on the recalculated reward scores (Mattar, Col. 18, lines 43-51). The reward score can be based at least in part on a utility (as represented by a utility score) and a cost (as represented by a cost score) (Mattar, Col. 20, lines 9-17). Utility factors can include an accuracy of the model (Mattar, Col. 7, lines 23-30).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date to modify the system of Kingetsu in view of Dapogny further in view of Jacobs and Dighe with the teachings of Mattar by including a queue for models to be retrained where the models are ordered based on levels of accuracy of the models. One having ordinary skill in the art would have been motivated to combine these references because doing so would allow for prioritizing retraining of a model, as recognized by Mattar. Thus, the claimed invention would have been obvious to one having ordinary skill in the art before the effective filing date.
Claim 19 is rejected under 35 U.S.C. 103 as being unpatentable over Dapogny et al. (US 2024/0161483 A1, with Foreign priority to Application No. EP 4057184, filed March 11, 2021, US PGPub used herein as a translation and for mapping purposes) in view of Hiroaki Kingetsu (US 2022/0207307 A1) further in view of Talagala et al. (US 2019/0108417 A1) and Dighe et al. (US 2022/0093095 A1).
Regarding claim 19, Dapogny teaches a device comprising a processor (Dapogny, Para. [0132], the unit includes a processor) configured to execute operations comprising:
capturing a frame of video data, wherein the frame of video data includes image data (Dapogny, Para. [0064], a stream of images is provided comprising context-based images. The stream may come from a live video camera);
determining, based on predetermined conditions of sampling image data, the frame of video data including the image data as sample image data, wherein the predetermined conditions include the frame of video data representing a change of scenery or when a predetermined time lapses (Dapogny, Para. [0069], the method of machine-learning may further comprise determining, based on the prior condition, whether the one or more images are to be provided to the first and second model. In other words, the prior condition may be a(n) (automatic) determining criteria whether a provided image from the first stream is to be used in the on-the-fly adaptation. Para. [0068], the prior condition may be computed based on one or more of: image metadata, time stamp, information extracted from the image's content. In examples, the information extracted from the image may comprise the time of the day, weather, luminosity, backlight, movement of the camera, and/or hazing);
transmitting the sample image data for inferencing (Dapogny, Para. [0064], each of the images of the stream is forwarded to the teacher and student model);
causing based on the frame of video data, generating reference image data using a trained teacher model (Dapogny, Para. [0064], each of the images of the stream is forwarded to the models to obtain the first prediction by the teacher model), the reference label infers the frame of video data, and wherein the trained teacher model generates the reference label by inferencing (Dapogny, Para. [0041], the method of inference comprises providing a context-based image from a stream of images and obtaining one or more predictions each obtained by applying one of the one or more predictive models to the provided image).
Although Dapogny teaches a prediction by the teacher model (Dapogny, Para. [0064]), Dapogny does not explicitly teach “wherein the reference image data includes a reference label”. However, in an analogous field of endeavor, Kingetsu teaches the computing system trains the parameters of the teacher model such that the output result of the teacher model obtained at the time of inputting the training data approaches the correct answer label of “dog” (Kingetsu, Para. [0067]).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the system of Dapogny with the teachings of Kingetsu by including determining a reference label using the teacher model. One having ordinary skill in the art would have been motivated to combine these references, because doing so would allow for a system that retrains a machine-learning model to prevent inefficiency caused by accuracy degradation, as recognized by Kingetsu.
Although Dapogny in view of Kingetsu teaches predictions made by the teacher model and student models (Dapogny, Para. [0064]), they do not explicitly teach “causing, responsive to validation of accuracy of a first image recognition model based on a comparison between the reference image data and a first feature label of the sample image data determined using the first image recognition model, a selection of the first image recognition model from a plurality of image recognition models for installation and installation of the first image recognition model”. However, in an analogous field of endeavor, Talagala teaches a model selection module that receives the machine learning models that the training pipelines generate and determines which of the machine learning models is the best fit for the objective that is being analyzed. The best-fitting machine learning model may be the machine learning model that produced the results most similar to the actual results for the training data (i.e., validation of accuracy based on a comparison between reference image data and a first feature label of sample image data determined using the first image recognition model) (Talagala, Para. [0066]).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the system of Dapogny in view of Kingetsu with the teachings of Talagala by including a model selection module that selects a machine learning model that is the best fit for the objective being analyzed based on the comparison between reference data (i.e., actual results) and a first feature label (i.e., result of the machine learning model). One having ordinary skill in the art would have been motivated to combine these references because doing so would allow for a machine learning system that delivers accurate and relevant results, as recognized by Talagala.
Although Dapogny in view of Kingetsu further in view of Talagala teaches a teacher model determining a reference label (Kingetsu, Para. [0067]), they do not explicitly teach ”wherein the trained teacher model performs inferencing of the first image data more accurately than the first image recognition model by consuming more memory resources than the first image recognition model”. However, in an analogous field of endeavor, Dighe teaches that student-teacher training is a training technique where a (typically) more accurate and computationally expensive teacher model trains a less computationally expensive student model to mimic the teacher model’s outputs and/or determinations (Dighe, Para. [0277]).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the method of Dapogny in view of Kingetsu further in view of Talagala with the teachings of Dighe by including that the teacher model is more accurate by consuming more computation resources than the student model (i.e., first image recognition model). One having ordinary skill in the art would have been motivated to combine these references because doing so would allow for knowledge distillation from a teacher model to students that use fewer resources, as recognized by Dighe. Thus, the claimed invention would have been obvious to one having ordinary skill in the art before the effective filing date.
Claim 20 is rejected under 35 U.S.C. 103 as being unpatentable over Dapogny et al. (US 2024/0161483 A1, with Foreign priority to Application No. EP 4057184, filed March 11, 2021, US PGPub used herein as a translation and for mapping purposes) in view of Hiroaki Kingetsu (US 2022/0207307 A1) further in view of Talagala et al. (US 2019/0108417 A1) and Dighe et al. (US 2022/0093095 A1), as applied to claim 19 above, and further in view of Kale et al. (US 2023/0139682 A1, filed November 1, 2021) and Jacobs et al. (US 2020/0302784 A1).
Regarding claim 20, Dapogny in view of Kingetsu further in view of Talagala and Dighe teaches the device of claim 19, as described above.
Although Dapogny in view of Kingetsu further in view of Talagala and Dighe teaches a processing machine for performing re-training (Dapogny, Para. [0101]), they do not explicitly teach “wherein the device represents an edge device operations further comprise: causing retraining of the image recognition model in a cloud (Kale, Para. [0023]).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the system of Dapogny in view of Kingetsu further in view of Talagala and Dighe with the teachings of Kale by including an edge device and performing retraining at a remote computing system of the cloud computing environment. One having ordinary skill in the art would have been motivated to combine these references, because doing so would allow for increasing overall system efficiency and decreasing system latency, as recognized by Kale.
Although Dapogny in view of Kingetsu further in view of Talagala, Dighe and Kale teaches an edge device (Kale, Para. [0023]), they do not explicitly teach that the edge device and cloud are “associated with the 5G telecommunication network”. However, in an analogous field of endeavor Jacobs teaches an edge device associated with the 5G telecommunication network (Jacobs, Para. [0031]).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the system of Dapogny in view of Kingetsu further in view of Talagala, Dighe and Kale with the teachings of Jacobs by including an edge device connected to a 5G network for storing and/or processing data. One having ordinary skill in the art would have been motivated to combine these references, because doing so would allow for reducing a latency of data communications and to conserve bandwidth, as recognized by Jacobs. Thus, the claimed invention would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention.
Claim 21 is rejected under 35 U.S.C. 103 as being unpatentable over Dapogny et al. (US 2024/0161483 A1, with Foreign priority to Application No. EP 4057184, filed March 11, 2021, US PGPub used herein as a translation and for mapping purposes) in view of Hiroaki Kingetsu (US 2022/0207307 A1) further in view of Talagala et al. (US 2019/0108417 A1) and Dighe et al. (US 2022/0093095 A1), as applied to claim 19 above, and further in view of Takimoto et al. (US 2022/0129675 A1).
Regarding claim 21, Dapogny in view of Kingetsu further in view of Talagala and Dighe teaches the device of claim 19, wherein the plurality of image recognition models includes
an image recognition model currently installed at the device (Kingetsu, Para. [0119], the detection unit detects accuracy degradation of the machine learning model (i.e., image recognition model currently installed at the device)),
and a most recently trained image recognition model (Kingetsu, Para. [0119], the training unit retrains the machine learning model by using a training data set that is newly designated (i.e., most recently trained image recognition model)).
The proposed combination as well as the motivation for combining the Dapogny, Kingetsu, Talagala and Dighe references presented in the rejection of Claim 19, apply to Claim 21 and are incorporated herein by reference.
Although Dapogny in view of Kingetsu further in view of Talagala and Dighe teaches retraining the machine learning model (Kingetsu, Para. [0119]), they do not explicitly teach “wherein the plurality of image recognition models includes k candidate image recognition models in a model store”. However, in an analogous field of endeavor, Takimoto teaches the CPU selects M learning models (candidate learning models) that are candidates in E (E is an integer of 2 or more) learning models stored in the external storage device (i.e., k candidate image recognition models in a model store) (Takimoto, Para. [0120]).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the method of Dapogny in view of Kingetsu further in view of Talagala and Dighe with the teachings of Takimoto by including selecting a set of candidate learning models from k candidate models in an image store. One having ordinary skill in the art would have been motivated to combine these references because doing so would allow for ensuring high performance of a learning model for a new input with a low operation cost, as recognized by Takimoto. Thus, the claimed invention would have been obvious to one having ordinary skill in the art before the effective filing date.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Emma Rose Goebel whose telephone number is (703)756-5582. The examiner can normally be reached Monday - Friday 7:30-5.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Amandeep Saini can be reached at (571) 272-3382. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Emma Rose Goebel/Examiner, Art Unit 2662
/AMANDEEP SAINI/Supervisory Patent Examiner, Art Unit 2662
Prosecution Insights