Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
The arguments filed 01/29/2026 have been entered. Claims 1-18, 21-22 remain pending in the application.
Applicant’s amendment, with respect to the claim rejection(s) of claim 1-20 under 35 U.S.C 112b filed 10/29/2025 have been considered and are persuasive. Therefore, the previous rejections as set forth in the previous office action has been removed.
Applicant’s argument, with respect to the claim rejection(s) of claim 1-20 under 35 U.S.C 103 filed 10/29/2025 have been considered and are not persuasive. Therefore, the previous rejections as set forth in the previous office action will be maintained.
The applicant argues that the teaching combination fails to teach or at least suggest the amended limitations requiring storing and accessing a first encoder model checkpoint in the manner claimed. In particular, Applicant contends that the Office Action relies on Ahmad for storing an encoder checkpoint and for training a second encoder model, but Ahmad allegedly only describes resuming training from a previous checkpoint, and not accessing a model checkpoint in response to a user request to train a second encoder model to perform a second different AI function.
Applicant further argues that merely resuming training from a checkpoint is not analogous to the claimed “plug-and-play” use of previously created encoder model checkpoints to facilitate training of multiple encoder models for different AI functions. According to Applicant, the present application describes checkpoint reuse or recycling to reduce the need to repeat training tasks, whereas Ahmad’s relied-upon disclosure is limited to continuing or paused training operation of the machine learning model.
Applicant also disputes the Examiner’s BRI that an “AI function” may be interpreted as a trained machine learning model. Applicant asserts that the amended claim language clarifies the first and second AI function, which are expressly different image analysis tasks. Applicant contends that Ahmad, at best, discloses training-related functions or a second training operation, not an image analysis task. Therefore, Applicant argues that the cited art does not teach the claimed relationship between the first encoder model checkpoint, the supervised training of the second encoder model, and the different first and second image analysis tasks.
For dependent claim 10 and 22, Applicant additionally argues that the amended claims further specify the nature of the different image analysis tasks: the first task is associated with classification functions used to analyze images and supplement metadata for items offered through an electronic platform, while the second image analysis task is associated with performing visual similarity searches. Applicant contends that Ahmad does not teach or suggest accessing the first encoder model checkpoint to train the second encoder model for this distinct visual-similarity-search task, and therefore claims 10 and 22 are allowable for at least these reasons.
The examiner respectfully disagrees. Ahmad is relied upon for teaching the checkpoint and subsequent training features, while Hou/Parangi/London are relied upon for the image-analysis encoder model environment. Ahmad discloses that checkpoint information may be stored for a Machine learning model and that a second training operation may be initiated using the stored first checkpoint information. Thus, Ahmad teaches accessing stored checkpoint information to perform a later training operation, and is not limited to only resume or recovery training from an interruption.
Applicant’s argument regards the “user request” is also not persuasive. Ahmad discloses that checkpoints may be user-specified prior to initiating a training operation and further discloses initiating a second training operation using stored first checkpoint information. Therefore, Ahmad teaches or at least suggests a user-requested or user-specified training operation in which stored checkpoint information is accessed to perform the later training operation. In the proposed combination, this corresponds to accessing the first encoder model checkpoint in response to a user request to train the second encoder model.
Applicant’s arguments that Ahmad does not teach a second AI function different from the first AI function is also not persuasive. Claim 1 does not require any particular type of first or second image-analysis task. Ahmad discloses that the second training operation may use a second set of training data different from the first set of training data. Because training data defines what features, labels, examples, or outputs the trained model learns to recognize or produce, using different training data in a later training operation teaches or at least suggests adapting the model for a different learned image-analysis use. Accordingly, when Ahmad’s checkpoint technique is applied to the image-analysis-encoder model of Hou/Parangi/London, Ahmad teaches or at least suggests training the second encoder model to perform a second image-analysis task different from the first image-analysis task under the broadest reasonable interpretation.
Applicant’s argument regarding “plug-and-play” checkpoint is not persuasive because the claims do not require the phrase “plug-and-play”, a particular user interface, or any specific amount of training reduction. The claim broadly requires accessing the first encoder model checkpoint to execute a supervise training procedure for the second encoder model. Ahmad’s disclosure of using stored first checkpoint information to initiate a second training operation teaches or at least suggests this claimed checkpoint-based training feature.
With respect to claim 10 and 22, Tang, in combination with Hou/Parangi/London/Ahmad, also teaches or at least suggest training different image-analysis function, such as classification function to analyze image to supplement metadata or analyzing images to perform visual similarity search. In particular, Tang discloses that metadata associated with images and visual features of images may be used for classification as explained in the previous Office Action. Tang further discloses applying a visual classifier to visual feature data of images to generate scores indicating whether an image is likely associated with an event. Thus, Tang teaches classification functions that analyze images using visual feature. Tang discloses this at paragraph 47 “A visual classifier is applied to the visual feature data for the images to provide a classification output for each event. For example, the visual classifier may give a high score for the event(s) it determines the image is likely associated with, and a low score for events it determines the image is not likely associated with”. Although Tang describes the visual-feature analysis in the context of classification, a person ordinary skill in the art would have understood that the visual feature analysis used by Tang’s visual classifier could be applied to locate images having similar features, because Tang’s visual classifier determines whether an image’s visual features correspond to a learned visual feature of a trained model, which teaches or at least suggest the visual similarity search, as claim. Thus, the teaching combination, in view of Tang’s teaching still teaches or at least suggests the claimed limitation as amended.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-9, 11-19, 21 are rejected under 35 U.S.C. 103 as being unpatentable in view of Liang Hou, herein after Hou et.al (NPL: Representation learning via a semi-supervised stacked distance autoencoder for image classification), further in view of Parangi et.al (US 20210232920 A1), further in view of London et.al (US 11200511 B1), further in view of Ahmad et.al (US 20210398020 A1).
Regarding claim 1,
Hou teaches a part of the 3rd limitation “providing a semi-supervised learning (SSL) ... by executing a pre-training procedure and a supervised training procedure ...” ...”. (Page 2 section 2.1 column 1 “The AE is an unsupervised learning algorithm whose goal is to keep the difference between the input and output as small as possible. The traditional basic AE is a three-layer neural network: an input layer, a representation layer, and an output layer ... The entire network consists of two networks called the encoder and decoder, respectively”, Page 6 column 2 section 3.2 figure.4 “we use the unlabeled data to pre-train the network... Based on the discussion above, we propose a semi-supervised DAE”. Hou discloses a distance autoencoder network model comprising of an encoder which is first pre-trained and then further trained through a semi-supervised training. According to figure 4, the model is pre-trained using unlabeled data and then further trained using labeled data, thus suggesting a procedure of semi-supervised learning and training.)
Hou teaches a part of the 4th limitation “... pre-training parameters at least identifying (a) a first set of unlabeled images and (b) a first encoder model... executing the pre-training procedure for the first encoder model” (Page 2 section 2.1 column 1 “The AE is an unsupervised learning algorithm whose goal is to keep the difference between the input and output as small as possible. The traditional basic AE is a three-layer neural network: an input layer, a representation layer, and an output layer ... The entire network consists of two networks called the encoder and decoder, respectively”, and Page 6 column 2 section 3.2 figure 4 “we use the unlabeled data to pre-train the network” Hou discloses the autoencoder model is first pre-trained using the unlabeled data. Figure 4 also represent the training procedure, in which an encoder model is first initialized. The encoder is a part of the distance autoencoder network model. Therefore, the process of pre-training the AE by unlabeled data as indicated in Fig.4 is analogous to the process of pretraining with parameters of (a) a first set of unlabeled images and (b) a first encoder model and executing the pre-training procedure for the encoder model within the claim.)
Hou teaches a part of the 5th limitation “after receiving the pre-training parameters via the API, executing, via the API, the pre-training procedure that trains the first encoder model using the first set of unlabeled images ..., for pre-training of the first encoder model”. (Page 2 section 2.1 column 1 “The AE is an unsupervised learning algorithm whose goal is to keep the difference between the input and output as small as possible. The traditional basic AE is a three-layer neural network: an input layer, a representation layer, and an output layer ... The entire network consists of two networks called the encoder and decoder, respectively”, and Page 6 column 2 section 3.2 “we use the unlabeled data to pre-train the network”. Hou discloses the pre-training procedure of the autoencoder model comprises using unlabeled data as illustrated in figure 4 and the encoder model is a part of the autoencoder model, wherein the pre-training is performed with unlabeled data and encoder model of the VAE network model. The process of training the autoencoder is also demosntrated through algorithm 1. A person ordinary skilled in the art would have been able to configure the transferring of data such as unlabeled images and encoder model for the training procedure via an API as disclosed by Parangi based on the teaching combination below.)
Hou teaches a part of the 6th limitation “receiving, ..., supervised training parameters at least identifying (a) a second set of labeled images and (b) the first encoder model, as pretrained, using the pre-training procedure, ... for supervised training of the first encoder model, as pre-trained, ...” (Page 1 section Abstract “Each layer is first pre-trained in an unsupervised manner. In the subsequent supervised training, the optimized parameters are set as the initial values”, and Page 6 column 2 section 3.2 figure 4 “Train AE by labeled data”. Hou discloses the process of training the autoencoder is illustrated through figure 4. Hou discloses the next step of training occurs using the autoencoder including the encoder that has been pre-trained with unlabeled data, which is then further trained with labeled data and using supervised training procedure as well as the optimized parameters, suggesting that the pre-trained autoencoder model include a pre-trained encoder, and a set of labeled data such as labeled images and optimized parameters are obtained for the supervised training step, wherein a person ordinary skilled in the art would have been able to configure the transferring of data such as labeled images and the pre-trained encoder model of data through an API based on the teaching combination with Parangi below.)
Hou teaches a part of the 7th limitation “after receiving the supervised training parameters via the API, executing, via the API, the supervised training procedure that further trains the first encoder model using the second set of labeled images ... for supervised training of the first encoder model” (Page 1 section Abstract “Each layer is first pre-trained in an unsupervised manner. In the subsequent supervised training, the optimized parameters are set as the initial values”, and Page 6 column 2 section 3.2 Figure 4 “Train AE by labeled data”. Hou discloses the process of training the autoencoder network model is illustrated through figure 4, wherein the autoencoder model include an encoder model. In figure 4, the autoencoder model along with its encoder model is pre-trained with unlabeled data. The next step of training occurs with the pre-trained autoencoder along with its pre-trained encoder model is further supervised trained with labeled data. A person ordinary skilled in the art would have been able to configure the transferring of data and the training procedure via an API as disclosed by Parangi based on the teaching combination below.)
Hou does not teach the 1st limitation “one or more processors”. However, Parangi teaches this limitation (paragraph 0065, where Parangi discloses “one or more computer programs executing on a programmable computer including a processor”. Parangi discloses the program executing on a computer including a processor within the computer.)
Hou does not teach the 2nd limitation “one or more non-transitory computer-readable storage devices storing computing instructions configured to run on the one or more processors and perform acts”. However, Parangi teaches this limitation (paragraph 0061, where Parangi discloses “the system 100 includes non-transitory, computer-readable medium comprising computer program instructions tangibly stored on the non-transitory computer-readable medium, wherein the instructions are executable by at least one processor to perform each of the steps described above”. Parangi discloses the system includes a non-transitory computer-readable medium wherein the instructions are executable by at least one processor.)
Hou does not teach a part of the 3rd limitation “providing... abstraction model that provides an abstraction layer to facilitate training the set of encoder models ..., wherein the SSL abstraction model utilizes an API (application programming interface) to access an encoder library comprising the set of encoder models and to collect user-specified input parameters” However, Parangi teaches this part of the limitation (paragraph 0056 “A user may select how to interact with the generated model 107. The user may choose “API” to configure, deploy, and pass data in and out of the model 107 programmatically with an API”, paragraph 0027 “The method 200 includes selecting, by the machine learning engine, a plurality of encoders based upon the at least one characteristic of the user-specified data set and at least one characteristic of the user-specified task”. Parangi teaches a method and system for dynamically generating a plurality of machine learning models for processing a user data set. Parangi discloses the system comprise of an API for users to interact with data and perform generating and training of machine learning models, wherein the API suggested an abstraction layer or framework to perform the generating and training of machine learning models. Parangi discloses a user-specified data set, which suggested user-specified input parameters. Parangi also discloses a plurality of encoders for the user to select from, suggesting a storage such as a library to store these encoder models. Even though the encoder as disclosed within the reference is selected by users to encode input data prior to the generation of the machine learning model, it is well known in the art that an encoder is a part of a machine learning model such as in an autoencoder model. Therefore, during the process of generating a model for training, the user may obtain any models such as an autoencoder model which incorporate an encoder from the storage of encoders as part of the model. One of ordinary skilled in the art can select an encoder model during the training of the autoencoder through utilize the machine learning technique within the API by Parangi, in which the encoder can be selected for the pre-training and supervised training stage by Hou based on the motivation to combine below. )
Hou does not teach a part of the 4th limitation “receiving, via the API, the user-specified input parameters including pre-training parameters at least identifying ... (b) a first encoder model selected from the set of encoder models, ..., wherein the user-specified input parameters are received by the API and are utilized ...”. However, Parangi teaches this part of the limitation (paragraph 0027 “In some embodiments, the machine learning engine 103 may include or have access to a machine learning model executed to select an encoder for use with a particular data set”, paragraph 0044 “receiving, by a machine learning engine, a user-specified data set”, and paragraph 47 “As shown in FIG. 3B, an Input Type Selection page allows a user to select an input type of a dataset they'd like to work with ... The data types may include tables, text, images, audio, video, sequences, and more”. Parangi discloses the API for user to interact with the system, wherein the machine learning engine allow the selection of an encoder using a machine learning model and engine. The machine learning engine also receive from the user a user-specified data set, thus suggesting that a user can specify input data set including input parameters for the data set, and images data for the first set of data, wherein this images data may be the unlabeled image data as disclosed above by Hou based on the teaching combination below.)
Hou does not teach a part of the 6th limitation “receiving, via the API, supervised training parameters, ... wherein the supervised training parameters are received by the API and are utilized when executing the supervised training procedure” However, Parangi teaches this part of the limitation (paragraph 0056 “A user may select how to interact with the generated model 107. The user may choose “API” to configure, deploy, and pass data in and out of the model 107 programmatically with an API”. Parangi discloses the interaction of user to the system through an API, wherein the user can configure the API to receive or pass data in and out of the model programmatically, wherein a person ordinary skilled in the art would have been able to configure the passing data being unlabeled/labeled images and encoder model used for autoencoder model training based on the teaching combination below.)
Before the effective filing date of the invention, it would have been obvious to one of ordinary skill in the art to combine the teaching of a method of training an autoencoder with unlabeled and labeled data using semi-supervised that include pre-training and supervised training by Hou with the teaching of a system and method of an API that collect user-specified data and task as input to generate and train machine learning models as well as selection of encoder model by Parangi. The motivation to do so is referred to in Parangi’s disclosure (paragraph 21 “The method 200 includes selecting, by the machine learning engine, a plurality of encoders based upon the at least one characteristic of the user-specified data set and at least one characteristic of the user-specified task (206). The method 200 includes directing, by the machine learning engine, each of the selected plurality of encoders to encode the received user-specified data set”, paragraph 0039 “formulating, by the machine learning engine 103, the data in a way that increases a level of efficiency in generating a machine learning model 107 that has a higher level of accuracy, for example, by generating a machine learning model 107 that is better suited to completing one type of task over another. Therefore, in some embodiments, implementation of a method that includes generating and executing a plurality of machine learning models, each of which is suited to completing different types of tasks, increases a level of accuracy of the output”, and paragraph 47 “an Input Type Selection page allows a user to select an input type of a dataset they'd like to work with. On the left bar, they may see a visual representation of the steps, or Flow, they're building. The data types may include tables, text, images, audio, video, sequences, and more”. Parangi discloses the benefit of the system, including an API and a plurality of machine learning models within the system. The method by Parangi further provide a plurality of encoder model and using machine learning to select encoder model suitable for the user-specified data set, in which the utilization of the machine learning model to select a suitable encoder model provide an improvement toward training the autoencoder model as a suitable encoder model can be efficiently selected for training. The API by Parangi also provide an ease of use for user to select an input type of a dataset they'd like to work with such as image, wherein one of ordinary skilled in the art can utilize this feature to provide unlabeled and labeled image data set for training of the autoencoder. Therefore, a user can incorporate the teaching of Parangi with Hou to create a more completed and improved system which comprise of an API, machine learning to select encoder and user specified input data set to provide a better platform for easier user’s interaction in training the autoencoder model as disclosed by Hou.)
Hou/Parangi does not teach a part of the 4th limitation “receiving, via the API, the user-specified input parameters including... (c) first hyperparameters for pretraining of the first encoder model...” However, London teaches this limitation (Column 4, lines 16-21 “a knowledge base in which records of experiences with various hyperparameter settings used in previous model training exercises may be stored, and the contents of such a knowledge base may be used to select the hyperparameters for some model training phases”. London discloses hyperparameters will be stored in a knowledge base and can be selected for some model training phase, thus implying that at any training phases such as a pre-training phase in semi-supervised learning of a machine learning model, suitable hyperparameters can be selected by a user from a knowledge base incorporated within the system. In combination with the teaching by Hou/Parangi as explained below, the hyperparameters selected for the pre-training procedure may suggest the first hyperparameters within the claim.)
Hou/Parangi does not teach a part of the 5th limitation “... executing, via the API, the pre-training procedure ... based on the first hyperparameters as received, ...” However, London teaches this limitation (Column 4, lines 16-21 “a knowledge base in which records of experiences with various hyperparameter settings used in previous model training exercises may be stored, and the contents of such a knowledge base may be used to select the hyperparameters for some model training phases”. London discloses hyperparameters will be stored in a knowledge base and can be selected for some model training phase, such as a pre-training phase is executed based on hyperparameter as selected. A person ordinary skilled in the art would have been able to configure the pre-training procedure disclosed by Hou and further incorporate hyperparameter into the pre-training procedure based on the motivation to combine the teachings below.)
Hou/Parangi does not teach a part of the 6th limitation “receiving, via the API ... (c) second hyperparameters for supervised training ...” However, London teaches this limitation (Column 4, lines 16-21 “a knowledge base in which records of experiences with various hyperparameter settings used in previous model training exercises may be stored, and the contents of such a knowledge base may be used to select the hyperparameters for some model training phases”. London discloses hyperparameters will be stored in a knowledge base and can be selected for some model training phase, thus implying that at any training phases such as a supervised training phase in semi-supervised training of a machine learning model, suitable hyperparameters can be selected by a user from a knowledge base incorporated within the system. In combination with the teaching by Hou/Parangi as explained below, the hyperparameters selected for the supervised training procedure may suggest the second hyperparameters within the claim)
Hou/Parangi does not teach a part of the 7th limitation “... executing, via the API, the supervised training procedure ... based on the second hyperparameters, as received, ...” However, London teaches this limitation (Column 4, lines 16-21 “a knowledge base in which records of experiences with various hyperparameter settings used in previous model training exercises may be stored, and the contents of such a knowledge base may be used to select the hyperparameters for some model training phases”. London discloses hyperparameters will be stored in a knowledge base and can be selected for some model training phase, such as a supervised training phase is executed based on hyperparameter as selected. A person ordinary skilled in the art would have been able to configure the supervised training procedure disclosed by Hou and further incorporate hyperparameter into the supervised training procedure based on the motivation to combine the teachings below.)
Before the effective filing date of the invention, it would have been obvious to one of ordinary skill in the art to combine the teaching of a computer readable medium device with processor and an API to perform training of the plurality of machine learning models, including autoencoder using semi-supervised learning and function to store checkpoint for the model by Hou/Parangi with the teaching of a knowledge base containing hyperparameter settings and allow user to provide preferences to these hyperparameters by London. The motivation to do so is referred to in London’s disclosure (Column 4, lines 1-12 where London discloses “The programmatic interfaces may be used by clients of the machine learning service to submit requests, including model training requests, and receive corresponding responses from the service. In at least some embodiments, a model training request received from a client may trigger the adaptive sampling based training of a model…In at least some embodiments, clients may provide preferences regarding various hyperparameters to the machine learning service”. London discloses a programming interface, which is analogous to an API that allow user to initiate training of a machine learning model. Within that programming interface, a user can provide preference regarding various hyperparameters to the machine learning service. The user can further incorporate the feature of a knowledge base containing hyperparameters information and setting from the programming interface for training machine learning model by London into the API and training process for the autoencoder by Hou/Parangi for the benefit of allowing user to select preference of hyperparameters suitable for their training purpose.)
Hou/Parangi/London does not teach the 8th limitation “storing a first encoder model checkpoint for the first encoder model after executing the supervised training procedure, wherein the first encoder model checkpoint is configured to be accessed to facilitate performance a first AI function of the one or more AI functions;” However, Ahmad teaches this limitation (paragraph 0013 “The checkpoint information generated at each checkpoint also may be used to generate an operational machine learning model in the partially trained state achieved during the training operation up to the checkpoint. An operational machine learning model allows the training operation to be evaluated at the stage of the checkpoint. The evaluation may include passing new data not included in the training data as input data through the operational machine learning model to evaluate the performance of the training operation.”, paragraph 26 “The training operation may have one or more checkpoints set at different points during the operation. Upon reaching a checkpoint, training module 230 may signal checkpoint module 260 to generate checkpoint information capturing the machine learning model in a partially trained state achieved at the time of the checkpoint. Checkpoint module 260 may store the generated checkpoint information in repository 270.”, and paragraph 41 “In the style transfer example and/or other examples, an operational machine learning model may be generated based on checkpoint information ... In this example, a user-selected set of data (e.g., labeled or unlabeled data) may be passed through the operational machine learning model to generate output data ... In one or more examples, the data may be unlabeled data or may be labeled data (e.g., for performing a mathematical evaluation of the accuracy of the generated output data and/or for performing a supervised task)”. Ahmad discloses a system to generate checkpoint of an operational machine learning model after the model has executed a training procedure. The checkpoints can be set at different points during the training operation and the checkpoint module may store a generated checkpoint information in a repository as the checkpoint information is generated through the trained machine learning model upon reaching the set checkpoint. Ahmad then discloses an operational machine learning model may be generated based on checkpoint information, wherein this operational machine learning model is analogous to the first AI function within the claim, and a performance evaluation of the newly generated operational machine learning model based on the checkpoint information is analogous to the facilitating performance of a first AI function by accessing the model checkpoint within the claim. One of ordinary skilled in the art may configure to incorporate an encoder and generate checkpoint information of the encoder model after training based on the motivation to combine the teachings below.)
Hou/Parangi/London does not teach the 9th limitation “in response to a user request to train a second encoder model to perform a second Al function different from the first Al function, accessing the first encoder model checkpoint to execute the supervised training procedure to train the second encoder model to perform the second Al function different from the first Al function, wherein the first Al function is a first image analysis task and the second Al function is a second image analysis task different from the first image analysis task.” However, Ahmad teaches or at least suggest this limitation (paragraph 26 “The training operation may have one or more checkpoints set at different points during the operation. Upon reaching a checkpoint, training module 230 may signal checkpoint module 260 to generate checkpoint information capturing the machine learning model in a partially trained state achieved at the time of the checkpoint. Checkpoint module 260 may store the generated checkpoint information in repository 270”, paragraph 31 “The checkpoints may be ... user-specified prior to initiating a training operation”, paragraph 45 “At block 406, a second training operation may be initiated for training the machine learning model starting in the first partially trained state using the stored first checkpoint information”, paragraph 47 “the second training operation for training the machine learning model uses a second set of training data. The second set of training data may be the same as, or different from, the first set of training data in various implementations.” Ahmad discloses using the stored first checkpoint information to start a second training operation, and further discloses that the second training operation may use a second set of training data that is different from the first set of training data. This teaches accessing the previously stored checkpoint to perform later training, because the second training operation beings from the stored checkpoint rather than starting over from the beginning. Checkpoints may be user-specified prior to initiating a training operation, which teaches or at least suggests a user request associated with checkpoint-based training. The different set of training data further teaches or suggests that the later training is directed to a different learned use or task, because training data defines what image features, labels, or outputs the model is trained to recognize or produce. Accordingly, when Ahmad’s checkpoint technique is applied to the image-analysis encoder model of Hou/Parangi/London, the stored first checkpoint information corresponds to the claimed first encoder model checkpoint, the second training operation corresponds to executing the supervised training procedure to train the second encoder model, as claimed, and the use of different second training data teaches or at least suggests training the second encoder model to perform a second image analysis task different from the first image analysis task.)
Before the effective filing date of the invention, it would have been obvious to one of ordinary skill in the art to combine the teaching of a computer readable medium device with processor and an API to perform training of the plurality of machine learning models, including autoencoder using semi-supervised learning and function to store checkpoint for the model by Hou/Parangi/London with the teaching of the training module of machine learning model including the generation of a checkpoint as a representation of the machine learning model at a partially trained state by Ahmad. The motivation to do so is referred to in Ahmad’s disclosure (paragraph 0019, where Ahmad discloses “Inserting checkpoints into a training operation to capture representations of a machine learning model in partially trained states during the training operation improves efficiencies in the development and training of machine learning models. The generated and stored checkpoint information for one or more checkpoints provides effective backup to the training operation in the event of power loss or system failure before the training operation has completed all of its training iterations, thereby enabling resumption of the training operation after power or the system has been restored without expending processing resources to repeat the previously completed training iterations up to the last completed checkpoint” and paragraph 40 “The ability to check performance of the machine learning model at different stages of training (e.g., at optional block 358) may be beneficial in evaluating models having no objective criteria for evaluation” Ahmad discloses the overall benefit of inserting checkpoints into training operation, which include improving efficiencies in development and training of machine learning model, provides effective backup to the operation and enabling resumption of training operation without expending processing resources to repeat previously trained iteration of the model. The checkpoint may also be utilized to generate different machine learning models for evaluation or different purposes. The teaching combination of Hou/Parangi/London also utilize the feature of a checkpoint of an application state of the model. Therefore, by incorporating the teaching of checkpoint by Ahmad into the teaching of Hou/Parangi/London will help it obtains all the mentioned benefits as disclosed above.)
Regarding claim 2 depends on claim 1, thus the rejection of claim 1 is incorporated.
Ahmad teaches “the first encoder model checkpoint is stored by an AI training system” (paragraph 0011 “The subject technology provides a training platform for machine learning models that utilizes checkpoints set at different stages or milestones reached during training operations.” Ahmad discloses the invention provides a training platform for machine learning models suggesting an AI training system. The platform utilizes checkpoints at different stage of the trained model, suggesting that the checkpoint is stored by the training platform.)
Ahmad teaches a part of the second limitation “the first encoder model checkpoint is accessed via the AI training system...” (paragraph 0011 “The subject technology provides a training platform for machine learning models that utilizes checkpoints set at different stages or milestones reached during training operations.” Ahmad discloses the invention provides a training platform for machine learning models suggesting an AI training system. The platform utilizes checkpoints at different stage of the trained model, suggesting that the checkpoint can be accessed to facilitate further training by the training platform.)
Parangi teaches a part of the second limitation “...and loaded into one or more classifiers to perform one or more classification functions” (paragraph 0026 “...The machine learning engine 103 may execute one or more machine learning models 111 trained to classify data into one of several data types (e.g., dates, names, unique IDs, Categories, and so on) ... (e.g., by executing a data type classification machine learning model 111 shown in FIG. 1A)”. Parangi discloses the system include a classification model to classify data. Based on the teaching combination as disclosed above, one of ordinary skilled in the art may incorporate the encoder with the checkpoint operation within the teaching combination to be provided to the classification model by Parangi, such that the classification model can further utilize data that could have been encoded by the checkpointed encoder for classification.)
Regarding claim 3 depends on claim 1, thus the rejection of claim 1 is incorporated.
Hou teaches “the first set of unlabeled images identified by the pre-training parameters are retrieved from the electronic platform” (Page 7, section 4.1.2 where Hou discloses “The SVHN was obtained from the house numbers in Google street view images”. The set of SVHN data set is image data that is obtained through electronic platform of Google Image. At page 8, column 2, Hou also discloses “Specifically, when we took 50% unlabeled data (artificially erasing the label) from the original data”. Hou discloses after obtaining the set of image data, they artificially erase the label, thus a set of unlabeled image data will be received as parameters for pre-training.)
Hou teaches “the first set of unlabeled images include a plurality of images that are selected from one or more item categories on the electronic platform” (Page 7, column 2 where Hou discloses “The SVHN was obtained from the house numbers in Google street view images and it has 10 classes... There are 10 images in each category”. Hou discloses within the set of SHVN images, there are different image categories divided from the set and each category contains 10 images.)
Hou teaches “the first set of unlabeled images do not include labels” (Page 8, column 2 where Hou discloses “Specifically, when we took 50% unlabeled data (artificially erasing the label) from the original data”. Hou discloses after obtaining the set of image data, they artificially erase the label, thus indicating that the SVHN image dataset selected will have label erased and become unlabeled images.)
Hou teaches “the second set of labeled images identified by the supervised training parameters include a plurality of labeled images that include labels” (Page 7-8, section 4.1.4 where Hou discloses “The CIFAR-10 dataset consists of 60000 color images in 10 classes with 6000 images per class; it represents a labeled subset which includes 80 million tiny images”. Hou discloses another set of image data CIFAR-10 for training wherein this set of image data is labeled and can be used for the supervised training phase of the autoencoder.)
Regarding claim 4 depends on claim 3, thus the rejection of claim 3 is incorporated.
Parangi teaches “the API...” (paragraph 0056 “A user may select how to interact with the generated model 107. The user may choose “API” to configure, deploy, and pass data in and out of the model 107 programmatically with an API”. Parangi discloses the interaction of user to the system through an API, wherein the user can input data to generate a model for training.)
Hou teaches “...configures the pre-training procedure to use the first set of unlabeled images” (Page 6, fig. 4 where Hou discloses the algorithm process of the autoencoder, Hou discloses the step of “Pre-train AE by unlabeled data” within the figure, thus implying the unlabeled image dataset obtained such as SHVN images as disclosed from claim 3 can be used for the pre-training procedure of the autoencoder.)
Hou teaches “...configures the supervised training procedure to use the second set of labeled images” (Page 6, fig. 4 where Hou discloses the algorithm process of the autoencoder, Hou discloses the step of “train AE by labeled data” within the figure, thus implying the labeled image dataset obtained such as CIFAR-10 images as disclosed from claim 3 can be used for the supervised training procedure of the autoencoder.)
Regarding claim 5 depends on claim 1, thus the rejection of claim 1 is incorporated. The applicant is further directed to the rejections of claim 1 as set forth above, because the claim recites similar limitations, and they are rejected based on the same rationale.
Regarding claim 6 depends on claim 5, thus the rejection of claim 5 is incorporated. The applicant is further directed to the rejections of claim 1 as set forth above, because the claim recites similar limitations, and they are rejected based on the same rationale.
Regarding claim 7 depends on claim 1, thus the rejection of claim 1 is incorporated.
Ahmad teaches “after the pre-training procedure is executed, a second encoder model checkpoint of the first encoder model is stored” (paragraph 0046 where Ahmad discloses “second checkpoint information may be generated, the second checkpoint information including a representation of the machine learning model in a second partially trained state. The second checkpoint information may also be stored in the non-volatile storage medium”. Ahmad discloses the generation of a second checkpoint which can be stored in the storage medium. The second checkpoint information including a representation of the model in a partially trained state, thus suggesting that after the machine learning model has gone through some training stage such as a pre-training stage, a second checkpoint can be generated for the model at that stage.)
Ahmad teaches a part of the second limitation “the second encoder model checkpoint is configured, ..., as a basis for performing a set of different supervised training procedures” (paragraph 0028, where Ahmad discloses “In addition to initializing a training operation of an untrained machine learning model, training module 230 also may initialize a resumption of a training operation starting from a checkpoint set in the training operation where the training operation was either previously completed or interrupted (e.g., paused or cancelled) before completion. Training module 230 may provide access to a log of stored checkpoint information for a particular machine learning model for which the training operation was previously executed in whole or in part”. Ahmad discloses the checkpoint feature allow the training module to initiate a resumption of a training operation of the machine learning starting from where the operation was either previously completed or interrupted, thus indicating that after the pre-training stage of the autoencoder, a second checkpoint as disclosed above is generated, this checkpoint can be further access to initiate resumption of the training operation of the autoencoder with a plurality of different supervised training procedure.)
Parangi teaches a part of the second limitation “to be accessed...via the API...” (paragraph 0056 “A user may select how to interact with the generated model 107. The user may choose “API” to configure, deploy, and pass data in and out of the model 107 programmatically with an API”. Parangi discloses the interaction of user to the system through an API, wherein the user can interact with data to generate a model for training.)
Regarding claim 8 depends on claim 1, thus the rejection of claim 1 is incorporated.
Parangi teaches “the SSL abstraction model is configured to permit a user to indicate user-specified input parameters for training multiple encoder models” (paragraph 0021, where Parangi discloses “The method 200 includes receiving, by a machine learning engine, a user-specified data set and a user-specified task”. Parangi disclosed there is a method to receive user-specified data set and task, thus indicating that the semi-supervised learning model to train the autoencoder as disclosed in claim 1 can receive and allow for user-specified data set and task, wherein these user-specified data set and task can be identified as input parameters for training of the encoder model.)
Parangi teaches “after receiving the user-specified input parameters training the multiple encoder models, the SSL abstraction model automatically configures the multiple encoder models and training procedures for each of the multiple encoder models based on the user-specified input parameters” (paragraph 0021 “The method 200 includes analyzing, by the machine learning engine, at least one characteristic of the user-specified data set ... The method 200 includes selecting, by the machine learning engine, a plurality of encoders based upon the at least one characteristic of the user-specified data set” and paragraph 0031 “In some embodiments, the machine learning engine 103 executes a method for training the machine learning model 107”. Parangi discloses the method to analyze by the machine learning engine a characteristic of the user-specified data set thus to select a plurality of encoders based on the at least one characteristic, which is a similar interpretation of configure multiple encoder models. As disclosed in claim 1, since the encoder model can also a part of a machine learning model, the process of training machine learning model by the engine can be applied for any models that include an encoder within, wherein Parangi further discloses the machine learning engine able to execute a method for training the machine learning model, thus the engine can also execute a method for training a model with an encoder within based on the user-specified data set.)
Parangi teaches a part of the third limitation “the SSL abstraction model further provides the abstraction layer... to train the multiple encoder models” (paragraph 0056 “A user may select how to interact with the generated model 107. The user may choose “API” to configure, deploy, and pass data in and out of the model 107 programmatically with an API”, paragraph 0027 “The method 200 includes selecting, by the machine learning engine, a plurality of encoders based upon the at least one characteristic of the user-specified data set and at least one characteristic of the user-specified task”. Parangi teaches a method and system for dynamically generating a plurality of machine learning models for processing a user data set. Parangi discloses an API for users to interact with data and perform generating and training of machine learning models, wherein the API suggested an abstraction layer to perform the generating and training of machine learning models, wherein the models can be a model with an encoder as part of the model.)
Hou teaches a part of the third limitation “...for executing semi-supervised training procedures utilizing unlabeled images and labeled images...” (page 8, section 4.2, column 2 “Both the unlabeled and labeled data would be used in semi-supervised learning”. Hou discloses using both unlabeled and labeled images in the process of semi-supervised learning to train an autoencoder.)
Hou teaches “the SSL abstraction model identifies training images as input for pre-training stages and supervised training stages” (page 7, section 4.1.2, column 1 “The street view house numbers (SVHN) datasetis a real-world image dataset for developing machine learning and object recognition algorithms”. Hou discloses using a data set of images as input, which the engine as disclosed by Parangi can also utilize this image dataset based on the teaching combination and Hou further discloses in Fig. 4 the training process for the semi-supervised autoencoder. The training process includes the step pre-training and the step of semi-supervised training.
Regarding claim 9 depends on claim 8, thus the rejection of claim 1 is incorporated.
Parangi teaches “each of the multiple encoder models is trained using the pre-training procedure and the semi-supervised training procedure” (paragraph 0026, where Parangi discloses “The machine learning engine 103 may execute one or more machine learning models 111 trained”. Parangi disclosed one or more machine learning models stored in the machine learning engine is trained. As disclosed in claim 1 by Hou, upon which claim 8 that have dependent claim 9 depends on, a model of an autoencoder can be trained using the pre-training and semi-supervised training procedure. A user can incorporate the teaching by Hou into Parangi such that the machine learning engine can execute training of one or more models, wherein if the model contains an encoder, the model is trained in a similar procedure to how the autoencoder model is trained as disclosed by Hou.)
Ahmad teaches “a set of encoder model checkpoints is stored, each of which is associated with a respective one of the multiple encoder models” (paragraph 0011 “A training operation including one or more checkpoints may be initiated to perform a set number of training iterations to train a selected machine learning model using selected training data.”. Ahmad discloses on or more checkpoints may be initiated for a selected machine learning model that is being trained, suggesting that one or more checkpoints may be stored for one or more encoder model that is selected to be trained.)
Regarding claim 11, the applicant is directed to the rejections to claim 1 set forth above, because the claim recites similar limitations, and they are rejected based on the same rationale.
Regarding claim 12 depends on claim 11, thus the rejection of claim 11 is incorporated. The applicant is further directed to the rejections of claim 2 as set forth above, because the claim recites similar limitations, and they are rejected based on the same rationale.
Regarding claim 13 depends on claim 11, thus the rejection of claim 11 is incorporated. The applicant is further directed to the rejections of claim 3 as set forth above, because the claim recites similar limitations, and they are rejected based on the same rationale.
Regarding claim 14 depends on claim 13, thus the rejection of claim 13 is incorporated. The applicant is further directed to the rejections of claim 4 as set forth above, because the claim recites similar limitations, and they are rejected based on the same rationale.
Regarding claim 15 depends on claim 11, thus the rejection of claim 11 is incorporated. The applicant is further directed to the rejections of claim 1 as set forth above, as they are rejected based on the same rationale.
Regarding claim 16 depends on claim 11, thus the rejection of claim 11 is incorporated. The applicant is further directed to the rejections of claim 1 as set forth above, as they are rejected based on the same rationale.
Regarding claim 17 depends on claim 11, thus the rejection of claim 11 is incorporated. The applicant is further directed to the rejections of claim 7 as set forth above, as they are rejected based on the same rationale.
Regarding claim 18 depends on claim 11, thus the rejection of claim 11 is incorporated. The applicant is further directed to the rejections of claim 8 as set forth above, because the claim recites similar limitations, and they are rejected based on the same rationale.
Regarding claim 21, the applicant is directed to the rejections to claim 1 set forth above, because the claim recites similar limitations, and they are rejected based on the same rationale.
Claims 10, 22 are rejected under 35 U.S.C. 103 as being unpatentable in view of Liang Hou, herein after Hou et.al (NPL: Representation learning via a semi-supervised stacked distance autoencoder for image classification), further in view of Parangi et.al (US 20210232920 A1), further in view of London et.al (US 11200511 B1), further in view of Ahmad et.al (US 20210398020 A1), further in view of Tang et.al (US 20120106854 A1).
Regarding claim 10 depends on claim 1, thus the rejection of claim 1 is incorporated.
Parangi teaches “the one or more Al functions are configured to analyze images pertaining to the items offered through the electronic platform” (paragraph 0021, where Parangi discloses “The method 200 includes analyzing, by the machine learning engine, at least one characteristic of the user-specified data set”. Parangi discloses one of the features of the machine learning engine include analyzing the characteristic of the user-specified data set, thus implying this feature can analyze image data type input pertaining to items when obtaining this image data type through an electronic platform. The electronic platform as suggested by Parangi can be understood as the computer that the system as disclosed herein is performed on)
Hou/Parangi/London/Ahmad does not teach “the first image classification task is associated with one or more classification functions that are utilized to analyzed images to supplement metadata associated with the items offered through an electronic platform”. However, Tang teaches or at least suggests this limitation (paragraph 0026, where Tang discloses “Individual classifiers are built for classifying metadata features (a metadata classifier)”. Tang discloses creating an individual classifier for classifying metadata features of image data, thus implying that the classifier can perform classification function to the metadata associated with the obtained items through an electronic platform such as a computer that perform the system.)
Hou/Parangi/London/Ahmad does not teach “the second image analysis task is associated with analyzing images to perform visual similarity searches”. However, Tang teaches or at least suggests this limitation (paragraph 24 “in another example, a system and method herein can facilitate a user's navigation and search throughout an entire collection of images”, paragraph 28 “Visual features of an image can be obtained using the image forming elements of the image”, and paragraph 47 “A visual classifier is applied to the visual feature data for the images to provide a classification output for each event. For example, the visual classifier may give a high score for the event(s) it determines the image is likely associated with, and a low score for events it determines the image is not likely associated with”. Tang discloses applying a visual classifier to visual feature data of images to generate scores indicating whether an image is likely associated with an event, wherein the system facilitate can a user's navigation and search throughout an entire collection of images, suggesting that the user can search through collection of image using the visual classifier for image analysis. Although Tang describes the visual-feature analysis in the context of classification, a person ordinary skill in the art would have understood that the visual feature analysis used by Tang’s visual classifier analyzes whether visual features of an image correspond to learned visual characteristics associated with an event. In view of Tang’s teaching of searching thorough image collection, it would have been obvious to apply such visual-feature analysis to identify images having similar visual feature. Thus, Tang teaches or at least suggests analyzing images to perform visual similarity searching, as claimed.
Before the effective filing date of the invention, it would have been obvious to one of ordinary skill in the art to combine the teaching of a computer readable medium device with processor and an API to perform training of the plurality of machine learning models, including autoencoder using semi-supervised learning and function to store checkpoint for the model by Hou/Parangi/London/Ahmad with the teaching of a classifier for metadata by Tang. The motivation to do so is referred to in Tang’s disclosure (paragraph 0024, where Tang discloses “a system and method herein can facilitate a user's navigation and search throughout an entire collection of images. For example, the user can browse all the images according to their event labels. In an example scenario, the user wants to find a particular image, and cannot recall where the image is stored, but does remember that the image was taken during Halloween. That is, classifying the images according to the associated event can help a user narrow their search and find the desired image more quickly”. Tang discloses the benefit of using such classification system is to help users quickly finding the desired image that fit with the classification result based on meta data or visual feature. By incorporating another classifier of metadata or visual feature to the classifier of the teaching combination by Hou/Parangi/London/Ahmad will allow the extra classifier built-in to work as an enhancement to the classifier as disclosed in claim 1 to help further classify data and obtain the benefit mentioned above.)
Regarding claim 22, depends on claim 21, thus the rejection of claim 21 is incorporated.
The applicant is directed to the rejections of claim 10 set forth above, because the claim recites similar limitations, and they are rejected based on the same rationale.
The motivation to combine the teaching combination with the teaching by Tang is similar to the teaching combination in claim 10 because claim 22 depends on claim 21, wherein claim 21 recites similar limitations to claim 1.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DUY TU DIEP whose telephone number is (703)756-1738. The examiner can normally be reached M-F 8-4:30.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Alexey Shmatov can be reached at (571) 270-3428. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DUY T DIEP/Examiner, Art Unit 2123
/ALEXEY SHMATOV/Supervisory Patent Examiner, Art Unit 2123
Prosecution Insights