Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR
1.17(e), was filed in this application after final rejection. Since this application is eligible for continued
examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the
finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's
submission filed on 01/15/2026 has been entered.
Response to Amendment
The amendments filed 01/15/2026 have been entered. Claims 1-7, 9-16, 18-20 remain pending in the application.
Applicant’s amendments and arguments, with respect to claim rejections of claims 1-7, 9-16, 18-20 under 35 U.S.C 103 filed 05/19/2025 have been considered but some of them are not persuasive. Therefore, the previous rejections as set forth in the previous office action will be maintain.
Applicant argues that the cited combination fails to teach or suggest the amended limitations of the claimed invention. Specifically, Applicant contends that although Appalaraju discloses training using image pairs labeled as similar or dissimilar, such labels are merely similarity-based labels and do not correspond to the claimed “sufficient label based on a comparison of a first set of original labels and a second set of original labels corresponding to actual classes” Applicant further asserts that Appalaraju does not teach generating labeled example-pairs based on original class labels.
Additionally, Applicant argues that the Office Action acknowledges that Appalaraju does not disclose “sequentially after training the hidden module of the machine learning model, training an output module using a reduced portion of a plurality of labeled examples” and relies on Jung for this teaching. Applicant contends that Jung merely discloses sampling data for training and validation from a common dataset and does not teach or suggest training an output module using a reduced portion of labeled examples or that such reduction is performed in accordance with minimization of a loss function. Applicant further argues that Jung’s sampling does not constitute reducing the training dataset relative to the plurality of labeled examples as claimed.
Accordingly, Applicant asserts that none of the cited references, alone or in combination, teach or suggest the claimed invention, and therefore the rejection under 35 U.S.C 103 is improper.
Applicant’s arguments have been considered but are not persuasive. With respect to Applicant’s contention that Appalaraju fails to teach generating labeled example-pairs based on original class labels, the examiner disagrees. Appalaraju discloses forming pair of images and assigning label indicating similarity or dissimilarity between the images based on the original semantic criteria of the image. Such similarity/dissimilarity labels inherently reflect whether the respective images correspond to the same class or different classes and are used for training the disclosed machine learning model, and therefore constitute labels derived from a comparison of labeled associated with the example pair under the broadest reasonable interpretation. Accordingly, Appalaraju teaches generating labeled-example pairs as recited in the claim.
The examiner however, agrees with Applicant that Jung does not explicitly teach “sequentially after training the hidden module of the machine learning model, training an output module using a reduced portion of a plurality of labeled examples ... wherein the reduced portion ... is in accordance with minimization of the loss function”. However, However, upon further consideration, new ground(s) of rejections have been raised (See Below.)
Hagen teaches a two-stage training process in which a machine learning model is initially trained using a training dataset and subsequently further trained using a reduced subset of the training data after eliminating certain data points, thereby generating a trained prediction model or classifier. Such disclosure corresponds to training an output module using a reduced portion of labeled examples as recited in the claim.
It would have been obvious to a person ordinary skill in the art to combine the reduced training data technique of Hagen into the system of Appalaraju in order to improve model accuracy and efficiency, as taught by Hagen. Accoridngly, the combination of Applaraju, Hagen and Tan teaches or suggests the claimed invention.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-5, 7, 9-14, 16, 18-20 are rejected under 35 U.S.C. 103 as being unpatentable over Appalaraju et.al (US 10467526 B1), in view of Hagen et.al (US 20240070554 A1), further in view of Tan et.al (US 20210232909 A1)
Regarding claim 1,
Appalaraju teaches the 1st limitation “generating, by one or more processors, sufficiently-labeled data from comprising a plurality of example-pairs by generating, for an example-pair of the plurality of example- pairs, a sufficient label based on a comparison of a first set of original labels that corresponds to a first example of the example-pair and a second set of original labels that corresponds to a second example of the example-pair, wherein the sufficient label indicates whether the first set of original labels and the second set of original labels correspond to one or more same or a plurality of different classes” (“Column 3 lines 24-33 “Individual ones of the training examples may comprise a pair of images I1 and I2 and a similarity-based label (e.g., the logical equivalent of a 0/1 indicating similarity/dissimilarity respectively) corresponding to the pair. A given pair of images (I1, I2) may be labeled either as a positive pair or a negative pair in some embodiments, with the two images of a positive pair being designated as similar to one another based on one or more criteria, and the two images of a negative pair being designated as dissimilar to one another”, Column 5 lines 8-13 “a given pair of images may be labeled, for the purposes of model training, as a positive or similar pair based on a variety of criteria. Visual similarity, in which for example the shapes, colors, patterns and the like of the objects are taken into account, may be used to classify images as similar in various embodiments. In addition, for the purposes of some applications, it may be useful to classify images as similar based on semantic criteria ... For example, with respect to logos or symbols representing an organization, some symbols may be graphics-oriented while others may be more text-oriented. For the purposes of identifying images that may both represent a given organization, an image pair consisting of one graphical logo and one image of a company's name may be considered a positive image pair in some embodiments” Appalaraju discloses an Artificial Intelligence system for image similarity analysis using optimized image pair and neural network. Within the disclosure, Appalaraju discloses using a pair of images, wherein each image may have their own classification of shapes, colors, patterns and the like of the objects. In other words, each image within the image pair may be provided as pre-labeled images based on their own classification and each image within the image pair is analogous to the first and second input example of the sufficiently-labeled example-pair. Appalaraju further discloses labeling the pair of images as a positive pair or a negative pair based on their similarity, wherein the similarity determination is based on the classified/pre-labeled images, which is analogous to the process of generating a sufficient label to indicate a same or different class between two input examples based on relative information of original label to obtain a sufficiently-labeled example-pair, wherein the positive/negative label corresponds to the sufficient label the indicate a same or different class, and the positive/negative pair resulted from the positive/negative label corresponds to the sufficiently-labeled example-pair within the claim.)
Appalaraju teaches the 2nd limitation “training, by one or more processor and using the sufficiently-labeled data, a hidden module of a machine learning model by determining a hidden representation in accordance with a minimization of a loss function that corresponds to training the machine learning model with a sufficiently-labeled data sample size corresponding to the sufficiently-labeled data relative to a fully-labeled training data sample size” (Column 6 lines 17-22 “As shown, system 100 includes various components and artifacts of an image analytics service (IAS) 102, which may be logically subdivided into a training subsystem 104 (at which machine learning models for image processing may be trained”, Column 8 lines 56-62 “The training of a neural network-based model of the kind discussed above may comprise a plurality of iterations or epochs in some embodiments. In a given training iteration, a plurality of candidate training image pairs may be initially be identified at the image pair selection subsystem, including for example one or more positive pairs and one or more negative pairs”, Column 10 lines 47-50 “A loss function 270 may be calculated based on the output vectors and a label y indicating whether the input image pair was a positive (similar) pair or a negative (dissimilar) pair in some embodiment”, and Column 11 lines 28-39 “... the loss becomes the distance between the embeddings of two similar images. As such, the model learns to reduce the distance between similar images, which is a desirable result ... If the two images are very dissimilar, the maximum function returns zero, so no minimization may be required. However, depending on the selected value of the m hyperparameter, if the images are not very dissimilar, a non-zero error value may be computed as the loss” Appalaraju discloses the training of the machine learning models, wherein the training is performed at the training subsystem, which corresponds to the hidden module, as claimed. The training subsystem by Appalaraju further comprises training the model using the similarity-based image pair (sufficiently-labeled data), wherein the training further comprises a loss function to determine the loss in similarity determination by the training subsystem. The loss function is calculated based on the output vectors of the image pair and the corresponding similarity label; thus, the loss corresponds to the distance between embeddings of similar and dissimilar image pairs. Accordingly, the model is trained in a manner that minimizes the loss function, thereby correspond to the claimed determining a hidden representation in accordance with a minimization of a loss function.)
Appalaraju teaches the 4th limitation “(i) the plurality of labeled examples comprises a plurality of original labels that correspond to one or more actual classes” (Column 5 lines 8-13 “a given pair of images may be labeled, for the purposes of model training, as a positive or similar pair based on a variety of criteria. Visual similarity, in which for example the shapes, colors, patterns and the like of the objects are taken into account, may be used to classify images as similar in various embodiments”, Column 7 lines 65-66 “Some image sources may provide pre-labeled images”, and Column 20, lines 7-12 “A number of image sources, such as publicly-accessible web sites, online image collections, and the like may be used in different embodiments. Optionally, in some embodiments, metadata associated with the collected images may be used to classify images into categories with associated labels” Appalaraju discloses using a pair of images, wherein each image may have their own classification of shapes, colors, patterns and the like of the objects. In other words, each image within the image pair may be provided as pre-labeled images based on their own classification and is analogous to the plurality of original labels that correspond to one or more actual classes.)
Appalaraju teaches the 5th limitation “(ii) the reduced portion of the plurality of labeled examples is less than an entirety of the plurality of labeled examples by an amount that is in accordance with the minimization of the loss function, and” (Column 9-10 lines 60-67 “The total amount of computing, storage and/or network resources used to train the model may, for example, be reduced by selecting and ordering training image pairs based on the difficulty of discriminating among the images in the pairs. Instead of having to train the model with a training data comprising hundreds of millions of image pairs, for example, a training data set of a few thousand image pairs may be sufficient when the described training data selection optimization techniques are used”, and Column 11 lines 9-12 “A contrastive loss function may be used in some embodiments to update the shared parameters of the model after each image pair or mini-batch of image pairs is processed” (Appalaraju discloses selecting and ordering training image pairs based on the difficulty of discrimination, such that only a subset of available image pairs is utilized for training, thereby reducing the total amount of training data required while maintaining model performance. Further, Appalaraju discloses that a contrastive loss function is used to updated the model during training, where the loss function reflects the degree of error associated with respective training image pairs based on their similarity or dissimilarity. Training image pairs that are more difficult to discriminate contribute to a higher loss and thus have a greater impact on the model training, whereas less informative or easily classified pairs contribute less to the loss. As such the disclosed reducing training data based on difficulty corresponds to prioritizing training samples according to their contribution to the loss function, such that less informative samples may be omitted without degrading performance. As such, the selection and reduction of training image pairs may be determined in accordance with the loss-based optimization of the model, thereby corresponds to the claimed reduced portion of the plurality of labeled examples is less than an entirety of the plurality of labeled examples by an amount that is in accordance with the minimization of the loss function.)
Appalaraju teaches the 7th limitation “providing, by the one or more processors, the machine learning model for use in one or more prediction tasks” (Column 6, lines 47-62“The run-time subsystem 105 may comprise, for example, one or more run-time coordinators 135, image pre-processors 128, as well as resources 155 that may be used to execute trained versions 124 of the models produced at the training subsystem 104 in the depicted embodiment ... Class information stored regarding the previously-processed images may be used to select a subset of the images with which the similarity analysis for a given source image is to be performed at run-time in some embodiments. In at least one embodiment, a given run-time request submitted by a client may include two images for which a similarity score is to be generated using the trained model 124” Appalaraju discloses the implementation of the machine learning model at the run-time subsystem, wherein the trained machine learning model produced by the training subsystem is provided to and executed by a run-time subsystem to perform similarity analysis on input images. Such output constitutes a prediction regarding the similarity or dissimilarity between the images. The run-time subsystem utilizes this predicted similarity to perform tasks such as identifying related images. Accordingly, Appalaraju teaches the claimed process of machine learning model for use in one or more prediction tasks, wherein generating an output of a similarity score corresponds performing a prediction.)
Appalaraju does not teach a part of the 3rd limitation “training, by the one or more processors and using a reduced portion of a plurality of labeled examples, an output module of the machine learning model ...” However, Hagen teaches this part of the limitation (paragraph 15 “In some embodiments, the training module 106 may be configured to train a machine learning tool in two stages. A first training stage may be conducted based on a full set of training data, and a second stage may be conducted on a reduced set of training data that is a subset of the full training data set, after identifying and eliminating certain data points from the full training data set”, and paragraph 27 “The method 200 may further include a step 210 that includes continuing to train the machine learning algorithm with the reduced training data set to create a trained prediction model or classifier”. Hagen discloses a method to optimize training data for image classification. Within the disclosure, Hagen discloses training a classifier model in a second stage using reduced training data set, which corresponds to the claimed output module of the machine learning model trained using the reduced portion of examples under the broadest reasonable interpretation. The reduced training data set by Hagen may corresponds to the reduced training data of image pairs disclosed by Appalaraju above.)
Before the effective filing date, it would have been obvious to a person ordinary skilled in the art to combine the teaching of an Artificial Intelligence system for image similarity analysis using optimized image pair and neural network by Appalaraju, with the teaching of training a classifier in a second stage using reduced training data by Hagen. The motivation to do so is recited in Hagen’s disclosure (paragraph 54 “Further experiments illustrated that the use of second-stage classifiers (e.g., a Siamese network), instead of only a single advanced neural network, can improve classification accuracy”, and paragraph 27 “The method 200 may further include a step 210 that includes continuing to train the machine learning algorithm with the reduced training data set to create a trained prediction model or classifier. Step 210 may include training the first state of the model with the reduced training data set generated at step 208 to generate a second model state. As a result of training on an improved, reduced training data set, the second model state may be a more accurate classifier than the first model state.” Hagen discloses the benefit of training a classifier using an improved reduced training data set such that the model may be more accurate. Given that Appalaraju also discloses a machine learning model and method to classify image using reduced training data, one of ordinary skill in the art would recognize that incorporating the teaching by Hagen into Appalaraju would further improve the reduce training data technique by Appalaraju, and a classifier may be incorporated at a second stage of training to further classify the image pair and improve the model accuracy. Thus, the combined teaching results in a training subsystem by Appalaraju that learns from positive/negative image pair (sufficient labeled data), which corresponds to the claimed hidden module, to obtain output vector representation of image similarity. These learned representations may be used as input features to the classifier disclosed by Hagen, which may be further trained in a second stage using a reduced training dataset, wherein the classifier by Hagen corresponds to the claimed output module that generates outputs based on the learned hidden representation as claimed.)
Appalaraju/Hagen does not teach a part of the 3rd limitation “training ... an output module of the machine learning model, freezing the hidden module” However, Tan teaches this limitation (paragraph 30 “Freeze-out provides an improved regularization technique as it eliminates the need to update the weights of output connections... the freeze-out technique involves randomly freezing a certain percentage of hidden units... The output of frozen units is not included for a training run but the weights of output connections from the frozen units to units of the following layer or layers are not changed for the training run. Thus, there is no need to update the weights of output connections from the frozen units to units of the following layer or layers during each training run.” Tan discloses systems and techniques that facilitate freeze-out as a regularizer in training neural networks. Within the disclosure, Tan discloses the freeze-out technique involve freezing a certain percentage of hidden units such as the output such that another unit at the following layer may not receive the output of the frozen hidden unit but still receive weights of output connections such that there is no need to update the weights, which is analogous to the claimed process of freezing the hidden module, wherein the hidden module corresponds to the training subsystem by Appalaraju.)
Appalaraju/ Hagen does not teach the 6th limitation “(iii) the hidden representation in accordance with the minimization of the loss function is unchanged during the training of the output module” However, Tan teaches this limitation (paragraph 30 “Freeze-out provides an improved regularization technique as it eliminates the need to update the weights of output connections... the freeze-out technique involves randomly freezing a certain percentage of hidden units... The output of frozen units is not included for a training run but the weights of output connections from the frozen units to units of the following layer or layers are not changed for the training run. Thus, there is no need to update the weights of output connections from the frozen units to units of the following layer or layers during each training run.” Tan discloses systems and techniques that facilitate freeze-out as a regularizer in training neural networks. Within the disclosure, Tan discloses the freeze-out technique involve freezing a certain percentage of hidden units such as the output such that another unit at the following layer may not receive the output of the frozen hidden unit but still receive weights of output connections such that there is no need to update the weights, which is analogous to the claimed process of keeping the hidden representation in accordance with the minimization of the loss function unchanged during the training of the output module, wherein a person ordinary skilled in the art would have been able to configure the freezing technique to freeze-out the representation of the minimization of the loss function at the machine learning model in the training subsystem by Appalaraju.)
Before the effective filing date, it would have been obvious to a person ordinary skilled in the art to combine the teaching of an Artificial Intelligence system for image similarity analysis using optimized image pair and neural network by Appalaraju, and the teaching of training by using a reduced training data by Hagen, with the teaching of systems and techniques that facilitate freeze-out as a regularizer in training neural networks by Tan. The motivation to do so is referred to in Tan’s disclosure (paragraph 30 “Freeze-out provides an improved regularization technique as it eliminates the need to update the weights of output connections... Additionally, the reduction in steps and elimination of the need to update weights of output connections can reduce amount of time and effort required in training a neural network, thus optimizing training. Furthermore, the reduction of steps and elimination of need to update weights of output connections can mitigate reduction of errors as well as improve accuracy prediction by the neural network as evidenced by results of experiments shown in this specification below.” Tan discloses the benefit of the freeze-out regulation technique in improving the training of a machine learning model such as reduce amount of time and effort required in training a neural network, mitigate reduction of errors and improve accuracy in prediction. Since Appalaraju trains a neural network using similarity-based example pairs and Hagen discloses a two-stage training framework with reduced training data, a person ordinary skilled in the art would have recognized that incorporating Tan’s technique into the combine Appalaraju/Hagen model would have been a routine optimization to further reduce overfitting and computational cost. Therefore, a person ordinary skilled in the art may integrate the teaching by Tan into the teaching combination of Appalaraju in view of Hagen for predictable improvement in applying the freeze-out regulation technique.)
Regarding claim 2 depends on claim1, thus the rejection of claim 1 is incorporated.
Appalaraju teaches the limitation “The method of claim 1, wherein reduced portion of the plurality of labeled examples is obtained from fully-labeled data used to generate the sufficiently-labeled data” (Column 9-10 lines 60-67 “The total amount of computing, storage and/or network resources used to train the model may, for example, be reduced by selecting and ordering training image pairs based on the difficulty of discriminating among the images in the pairs. Instead of having to train the model with a training data comprising hundreds of millions of image pairs, for example, a training data set of a few thousand image pairs may be sufficient when the described training data selection optimization techniques are used.”, Column 5 lines 8-13 “a given pair of images may be labeled, for the purposes of model training, as a positive or similar pair based on a variety of criteria. Visual similarity, in which for example the shapes, colors, patterns and the like of the objects are taken into account, may be used to classify images as similar in various embodiments.” Appalaraju discloses reducing the amount of training image pairs data, for example, a training data set of a few thousand image pairs may be sufficient for training the model, wherein the training image pairs (positive/negative) may comprise pre-labeled images (e.g., shapes, colors, patterns) for the purposes of model training, which corresponds to plurality of labeled examples is obtained from fully-labeled data used to generate the sufficiently-labeled data.)
Regarding claim 3 depends on claim1, thus the rejection of claim 1 is incorporated.
Appalaraju teaches the limitation “The method of claim 1, wherein the machine learning model is configured to, for the one or more prediction tasks, identify an original label from the plurality of original labels for an unseen input provided to the machine learning model.” (Column 7 lines 38-65 “Images that can be used to train the model(s) may be acquired from a variety of image sources 110 ... Such semantic information may be used to classify some pairs of images (such as images of company logos and company names) as similar based on the meaning or semantics associated with the images in some embodiments, even if the images of the pair do not necessarily look like one another ... Some image sources may provide pre-labeled images from which positive and negative pairs can be generated” Appalaraju discloses obtaining images used to train the model, wherein the image comprises semantic information (pre-labeled images) that may be used to classify some pairs of images such as company names or logos. These images training data corresponds to the unseen input with identified original label for the one or more prediction tasks, as recited in the claim.)
Regarding claim 5 depends on claim1, thus the rejection of claim 1 is incorporated.
Appalaraju teaches the limitation “obtaining fully-labeled data comprising a plurality of input examples each having one of the plurality of original labels” (Column 5 lines 8-13 “a given pair of images may be labeled, for the purposes of model training, as a positive or similar pair based on a variety of criteria. Visual similarity, in which for example the shapes, colors, patterns and the like of the objects are taken into account, may be used to classify images as similar in various embodiments.” Appalaraju discloses the training image pairs (positive/negative) may comprise pre-labeled images (e.g., shapes, colors, patterns) for the purposes of model training, which corresponds to fully-labeled data comprising a plurality of input examples each having one of the plurality of original labels)
Appalaraju teaches the limitation “generating the plurality of example-pairs, each example-pair comprising a first input example selected from the fully-labeled data and a second input example selected from the fully-labeled data” (Page 16 column 5 “images to be used for training may be collected from a variety of image sources (such as web sites), at least some of which may also comprise semantic metadata”, Page 23 column 20 “metadata associated with the collected images may be used to classify images into categories with associated labels. In at least one embodiment”, and Page 17 column 8“In a given training iteration, a plurality of candidate training image pairs may be initially be identified at the image pair selection subsystem, ... A number of operations may be performed to optimize the selection of training image pairs from among the candidates in some embodiments” Appalaraju discloses a selection subsystem to select image pairs from among the candidates, wherein images in image pairs to be used for training may be collected from a variety of image sources, wherein images from image sources may comprise semantic metadata to classify images into categories with associated labels, which corresponds to a first and second input example as claimed.)
Appalaraju teaches the limitation “generating one or more sufficient labels for the plurality of example-pairs based at least in part on summarizing a first original label of the plurality of original labels and a second original label of the plurality of original labels” (page 15 column 3 “individual ones of the training examples may comprise a pair of images I1 and I2 and a similarity-based label (e.g., the logical equivalent of a 0/1 indicating similarity/dissimilarity respectively) corresponding to the pair. A given pair of images (I1, I2) may be labeled either as a positive pair or a negative pair in some embodiments, with the two images of a positive pair being designated as similar to one another based on one or more criteria, and the two images of a negative pair being designated as dissimilar to one another”, and page 16 column 5 “it may be useful to classify images as similar based on semantic criteria ... images to be used for training may be collected from a variety of image sources (such as web sites), at least some of which may also comprise semantic metadata associated with the images ... In at least some embodiments, such semantic metadata may be analyzed to determine whether two images are to be designated as similar, even if they may not look alike from a purely visual perspective.”. Appalaraju discloses generating the similarity-based label for the image pair based on the criteria, wherein the criteria may be semantic criteria of semantic data associated with each image within the image pair, wherein the positive/negative label of each image pair corresponds to the generating of sufficient labels for the plurality of example-pairs based at least in part on summarizing a first original label of the plurality of original labels and a second original label of the plurality of original labels, as claimed.)
Regarding claim 7 depends on claim 5, thus the rejection of claim 5 is incorporated.
Appalaraju teaches the limitation “The method of claim 1, further comprising storing the sufficiently-labeled data in a storage medium as an encrypted representation of the first input example and the second input example” (Page 25 column 24 “Various embodiments may further include receiving, sending or storing instructions and/or data implemented in accordance with the foregoing description upon a computer-accessible medium.” Appalaraju discloses the embodiment may include storing data in a computer-accessible medium, wherein the data can be the similarity-based label representing the relationship between the image pair input data. A person ordinary skilled in the art would have been able to configure such label to be encrypted to be stored in the computer storage medium.)
Regarding claim 9 depends on claim1, thus the rejection of claim 1 is incorporated.
Appalaraju teaches the limitation “The method of claim 1, wherein the machine learning model comprises a neural network configured as a classifier” (Page 24 column 22 “In at least some embodiments, a server that implements a portion or all of one or more of the technologies described herein, including the various components of the training and run-time subsystems of an image analytics service, including for example ... classifiers”, and Page 24 column 21 “In the depicted embodiment, the images available at the analytics service may be analyzed, e.g., with the help of models similar to the CNN models discussed earlier, and placed into various classes”. Appalaraju discloses the computer system of the method may comprise of classifier, wherein the classifier may be the classifier model as disclosed by Frandsen based on the teaching combination, and the images with semantic metadata may be placed into classes.)
Regarding claim 10 depends on claim 1, thus the rejection of claim 1 is incorporated.
Appalaraju teaches the limitation “The method of claim 1, wherein the hidden module is trained using one of a hinge loss function, a negative cosine similarity function, a contrastive function, or a mean squared error” (Page 18 column 10 “A loss function 270 may be calculated based on the output vectors and a label y indicating whether the input image pair was a positive (similar) pair or a negative (dissimilar) pair in some embodiments”, and Page 17 column 8 “A contrastive loss function may be used for the overall neural network model in at least some embodiments”. Appalaraju discloses calculating a loss function for the training of the machine learning model, wherein the loss function can be a contrastive loss function.)
Regarding claim 11,
Appalaraju teaches limitations “one or more processors”, and “at least one memory storing processors-executable instructions that, when executed by any of the one or more processors, cause the one or more processors to perform operations” (page 24 column 22 “In various embodiments, computing device 9000 may be a uniprocessor system including one processor 9010, or a multiprocessor system”, and “System memory 9020 may be configured to store instructions and data accessible by processor(s) 9010... In the illustrated embodiment, program instructions and data implementing one or more desired functions, such as those methods, techniques, and data described above, are shown stored within system memory 9020 as code 9025 and data 9026.” Appalaraju discloses the embodiment may be a computing device with one or more processors, and memory configured to store coded instructions to be executed by the one or more processor.)
The applicant is further directed to the rejection of claim 1, because claim 11 recites similar limitation to claim 1, thus the claim is rejected under the same rationale.
Regarding claim 12 depends on claim 11 thus the rejection of claim 11 is incorporated. The applicant is further directed to the rejection of claim 3, because claim 12 recites similar limitation to claim 3, thus the claim is rejected under the same rationale.
Regarding claim 14 depends on claim 11 thus the rejection of claim 11 is incorporated. The applicant is further directed to the rejection of claim 5, because claim 14 recites similar limitation to claim 5, thus the claim is rejected under the same rationale.
Regarding claim 16 depends on claim 14 thus the rejection of claim 14 is incorporated. The applicant is further directed to the rejection of claim 7, because claim 16 recites similar limitation to claim 7, thus the claim is rejected under the same rationale.
Regarding claim 18 depends on claim 11 thus the rejection of claim 11 is incorporated. The applicant is further directed to the rejection of claim 9, because claim 18 recites similar limitation to claim 9, thus the claim is rejected under the same rationale.
Regarding claim 19 which recites a machine, one of the four statutory categories of patentable subject matter.
Appalaraju teaches the limitation: “One or more non-transitory computer storage media storing instructions that, when executed by one or more processors, cause the one or more processors to perform operations” (page 25 column 23 “Generally speaking, a computer-accessible medium may include non-transitory storage media or memory media”, and page 24 column 22 “System memory 9020 may be configured to store instructions and data accessible by processor(s) 9010... In the illustrated embodiment, program instructions and data implementing one or more desired functions, such as those methods, techniques, and data described above, are shown stored within system memory 9020 as code 9025 and data 9026.” Appalaraju discloses a computer-accessible medium may include non-transitory storage media or memory media, wherein the storage media may comprise of memories which contain instructions to cause program operations executed by one or more processors.)
The applicant is further directed to the rejection of claim 1, because claim 19 recites similar limitation to claim 1, thus the claim is rejected under the same rationale.
Regarding claim 20 depends on claim 19 thus the rejection of claim 19 is incorporated. The applicant is further directed to the rejection of claim 5, because claim 20 recites similar limitation to claim 5, thus the claim is rejected under the same rationale.
Claims 4, 13 are rejected under 35 U.S.C. 103 as being unpatentable over Appalaraju et.al (US 10467526 B1), in view of Hagen et.al (US 20200387755 A1), further in view of Tan et.al (US 20210232909 A1), further in view of Frandsen et.al (NPL: Machine Learning for Disease Prediction).
Regarding claim 4 depends on claim1, thus the rejection of claim 1 is incorporated.
Frandsen teaches the limitation “The method of claim 1, wherein the plurality of original labels comprises a first original label classifying an individual as contracting a disease and a second original label classifying the individual as not contracting the disease, and wherein the one or more prediction tasks includes identification of either the first original label or the second original label for an unseen individual to indicate a likelihood of the unseen individual having contracted the disease” (Chapter 2 Page 13 section 2.1 “The aim of classification is to assign labels to objects. More formally, assume there is a set X of objects and a finite set Y of labels”, Chapter 2 Page 15 section 2.1 “Example 2.5 (Disease Prediction). Suppose we have a soft classifier that takes values in the unit interval. This classifier has been created to predict if someone is at risk of developing 14 late stage CKD; the label 0 indicates no risk, and the label 1 indicates high risk”, and Page 24 section 2.4 “... Since the effectiveness of a classifier resides in its ability to make useful predictions even on unseen examples, it is important to investigate its generalization. We measure the generalization by gathering a new collection of labeled data, which we call the test set, and reporting possibly multiple numerical quantities that evaluate how well the classifier was able to predict the labels in the test set.” Frandsen discloses the the label 1 indicates high risk of disease and label 0 indicates no risk of disease, which is analogous to the claimed fourth and fifth label within the claim respectively, wherein the classifier may obtain image data from Appalaraju with semantic metadata being these 0 and 1 labels to perform its training and after the training, the classifier may be able to perform prediction to assign either 0 or 1 label for respectively no risk of contracting disease and high risk of contracting disease for unseen input, wherein the unseen input can be data associated with unseen individuals.)
Before the effective filing date, it would have been obvious to a person ordinary skilled in the art to combine the teaching of an Artificial Intelligence system for image similarity analysis using optimized image pair and neural network by Appalaraju, the teaching of training by using a reduced training data by Hagen, and the teaching of systems and techniques that facilitate freeze-out as a regularizer in training neural networks by Tan, with the teaching of machine learning for disease prediction by Frandsen. The motivation to do so is referred to in Frandsen’s disclosure (page 14 example 2.5 “This classifier has been created to predict if someone is at risk of developing late stage CKD; the label 0 indicates no risk, and the label 1 indicates high risk. Should an individual be flagged as high risk, the doctor may order potentially costly tests and treatments”, and page 24 section 2.4 “However, our ultimate goal is to have a classifier that performs well on all appropriate data, not just the particular examples in the training set. We use the term generalization to denote the performance of a classifier on new, unseen data. Since the effectiveness of a classifier resides in its ability to make useful predictions even on unseen examples, it is important to investigate its generalization.” Frandsen discloses the utilization of the trained machine learning model in predicting task such as predicting diseases within the medical field and the trained classifier model is evaluated on unseen labeled data to measure generalization performance and to produce prediction outputs. Frandsen provides examples such as in example 2.5, wherein the classifier machine learning model within the example is capable to provide label of 0 or 1, which is similar to the machine learning model by Appalaraju as disclosed above, wherein the model has been trained on similarity-based example pairs. Frandsen further discloses evaluating the performance of a trained classifier model on unseen labeled data to determine how well the model perform. This post training evaluation step is a standard component of supervised machine learning workflows. Once the model has been trained and stabilized as taught by Appalaraju/Hagen/Tan, one of ordinary skilled in the art would have found it obvious to apply the trained model to predict the similarity in unseen labeled data using the testing procedure in Frandsen in order to measure the model generalization and enable prediction. Therefore, the incorporation of the teaching by Frandsen into the teaching combination would have represented a predictable use of known machine learning techniques to perform their established functions.)
Regarding claim 13 depends on claim 11 thus the rejection of claim 11 is incorporated. The applicant is further directed to the rejection of claim 4, because claim 13 recites similar limitation to claim 4, thus the claim is rejected under the same rationale.
Claims 6, 15 are rejected under 35 U.S.C. 103 as being unpatentable over Appalaraju et.al (US 10467526 B1), in view of Hagen et.al (US 20200387755 A1), further in view of Tan et.al (US 20210232909 A1), further in view of Frandsen et.al (NPL: Machine Learning for Disease Prediction), further in view of Esteva et.al (US 20220222484 A1)
Regarding claim 6 depends on claim 5, thus the rejection of claim 5 is incorporated.
Appalaraju/Hagen/Tan/Frandsen does not teach “The method of claim 5, wherein the one or more sufficient labels are generated using an annotation machine learning model”. However, Esteva teaches this limitation (paragraph 28 “The labeling interface module 220 generates information for displaying subsets of the segmented portions of the image data to an operator for annotation. In one embodiment, the labeling interface module 220 identifies a region within the image that includes one or more portions to be annotated”, and paragraph 29 “When a session begins, the portions to be annotated may be presented to the operator for labeling without recommendations. The labeling interface module 220 receives data indicating the labels assigned to portions by the operator. The classifier training module 230 trains a classifier using the assigned labels”. Esteva discloses an AI-enhanced data labeling tool with annotation. Within the disclosure, Esteva discloses the tool comprises of modules for annotation operation of images, such that the annotated image may then be labeled. A person ordinary skilled in the art would have been able to configure the labeling process to assign similarity-based label based on the annotation image based on the teaching combination as explained below.)
Before the effective filing date, it would have been obvious to a person ordinary skilled in the art to combine the teaching of an Artificial Intelligence system for image similarity analysis using optimized image pair and neural network by Appalaraju, the teaching of training by using a reduced training data by Hagen, the teaching of systems and techniques that facilitate freeze-out as a regularizer in training neural networks by Tan, and the teaching of machine learning for disease prediction by Frandsen, with the teaching of the AI-enhanced data labeling tool with annotation by Esteva. The motivation to do so is referred to in Esteva’s disclosure (paragraph 20 “the AI-enhanced labeling tool may enable the operator to annotate more examples in a given time period, reducing the cost of data annotation. The tool may also increase accuracy of the labels assigned to image portions.”, paragraph 53 “FIG. 7 includes experimental data demonstrating efficiency improvements that may be realized using one embodiment of the AI-enhanced data labeling tool.”, and paragraph 56 “FIG. 8 is an example comparison plot of a dataset annotated with the AI-enhanced data labeling tool versus one annotated without... a model trained with 50, 75, and 100 training examples benefits from an 11%, 11%, and 5% boost in model validation accuracy, respectively. In benchmark machine learning competitions, top performing models typically win by fractions of a percent to single percentage points. Thus, this improvement is effectiveness is significant”. Esteva discloses the benefit of the AI-enhanced data labeling tool with annotation feature, which increase accuracy of the labels assigned to image portions. The tool provides labeled annotated image training data that help boosting in model validation accuracy, which is demonstrated through experiment and example in fig. 7 and fig. 8. Given that Appalaraju discloses neural network model training based on similarity-based example pairs of images, thus the teaching combination by Appalaraju/Hagen/Tan/Frandsen may further incorporate the teaching of annotation image data for further improvement on obtaining training data for better training result and better machine learning model performance.)
Regarding claim 15 depends on claim 14 thus the rejection of claim 14 is incorporated. The applicant is further directed to the rejection of claim 6, because claim 15 recites similar limitation to claim 6, thus the claim is rejected under the same rationale.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DUY TU DIEP whose telephone number is (703)756-1738. The examiner can normally be reached M-F 8-4:30.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Alexey Shmatov can be reached at (571) 270-3428. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DUY T DIEP/Examiner, Art Unit 2123
/ALEXEY SHMATOV/Supervisory Patent Examiner, Art Unit 2123