DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 05/22/2023 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-2, 7, 10, 12 and 14-15 are rejected under 35 U.S.C. 103 as being unpatentable over Chaitanya et al. ("Contrastive learning of global and local features for medical image segmentation with limited annotations") in view of Chen et al. ("Generative Pretraining from Pixels").
Regarding Claim 1, Chaitanya teaches a computer-implemented method comprising: receiving a plurality of unlabeled images (Page 10, 6 Broader Impact, Paragraph 2, “We demonstrate on three medical datasets for the segmentation task that the proposed pre-training with unlabeled images can yield a good initialization, and reduce the need for large annotated sets to yield high performance when fine-tuned to the segmentation task”, in this section of the prior art, the dataset for training the network are unlabeled images.); generating an augmented training data set from the plurality of unlabeled images, wherein the augmented training data set comprises a first set of augmented images and a second set of augmented images(Page 3, 3.1 Global contrastive loss , Paragraph 1, “Here, x˜ and xˆ are two differently transformed versions of the same image x, i.e. x˜ = t˜(x)”, as disclosed in this section of the prior art, augmentation is performed on a set of images to create 2 different versions of the image.), wherein the first set of augmented images is generated from the unlabeled images by applying one or more spatial augmentation techniques to the unlabeled images (Page 3, 3.1 Global contrastive loss , Paragraph 1, “Here, x˜ and xˆ are two differently transformed versions of the same image x, i.e. x˜ = t˜(x) and xˆ = tˆ(x) where t,˜ tˆ ∈ T are simple transformations as used in [24, 12, 4, 18, 56], such as crop followed by color transformations, and T is the set of such transformations. These two images are treated as similar and their representations are encouraged to be similar. In contrast, the set Λ − consists of images that are dissimilar to x and its transformed versions”, in this section of the prior art, a data augmentation of transforming the unlabeled images is performed to create a first and second set of images.)training a first machine learning model on the first set of augmented images and the second set of augmented images (Page 6, Experimental setup: “We split each dataset into a pre-training set Xpre and a test set Xts, each consisting of volumetric images and corresponding segmentation labels. We pre-train a UNet architecture using only the images from Xpre without their labels, then fine-tune the pre-trained network with a small number of labeled examples chosen from Xpre.”, this section of the prior art, discloses training a machine learning model using two sets of volumetric images. )wherein the first machine learning model comprises an encoder-decoder structure with a contrastive output at the end of the encoder (Page 5, 3.3 Local contrastive loss, Paragraph 2, “With this motivation, we propose a self-supervised learning strategy that encourages the decoder of an encoder-decoder network (Fig. 1.ii) to extract distinctive local representations to complement global representations extracted by the encoder”, and a reconstruction output at the end of the decoder (Page 5, 3.3 Local contrastive loss, Equation 3 shows the contrastive loss at the end of the decoder and equation 4 represents the total local contrastive loss in the set of images.)wherein the first machine learning model is trained: to output, for each image of the second set of augmented images, the respective image of the first set of augmented images via the reconstruction output (Page 14, 7.1 Network Architecture, Paragraph 1, “We use a UNet [49] based encoder (e) - decoder (d) architecture. e consists of 6 convolutional blocks, each consisting of two 3 × 3 convolutions followed by a 2 × 2 maxpooling layer with stride 2. While training with global contrastive loss, we append a small network, g1, on top of e. g1 consists of two dense layers with output dimensions 3200 and 128. While training with local contrastive loss, we add a partial decoder dl, with l convolutional blocks, on top of the pre-trained e. Each block consists of one upsampling layer with a factor of 2, followed by concatenation from corresponding level of e via a skip connection, followed by two 3 × 3 convolutions. Similar to g1, we have an additional small network, g2, on top of last decoder block with two 1 × 1 convolutions to obtain the feature map that is used for the local loss computation. Lastly, we take the pre-trained e and dl and append the remaining convolutional blocks of d along with skip connections such that we a sufficient number of layers d to output a segmentation with same dimensions as the input image. W”, as disclosed in this section, the prior art discloses the architecture of the model and how the model is reconstructing the image such that the dimensions are the same as the input image.)and to discriminate augmented images which originate from the same unlabeled image from augmented images which do not originate from the same unlabeled image via the contrastive output. (Page 3, 3.1 Global contrastive loss, Paragraph 1, “For a given encoder network e(·), the contrastive loss is defined as: l(˜x, xˆ) = − log e sim(˜z,zˆ)/τ e sim(˜z,zˆ)/τ + P x¯∈Λ− e sim(˜z,g1(e(¯x)))/τ , z˜ = g1(e(˜x)), zˆ = g1(e(ˆx)). (1) Here, x˜ and xˆ are two differently transformed versions of the same image x, i.e. x˜ = t˜(x) and xˆ = tˆ(x) where t,˜ tˆ ∈ T are simple transformations as used in [24, 12, 4, 18, 56], such as crop followed by color transformations, and T is the set of such transformations. These two images are treated as similar and their representations are encouraged to be similar. In contrast, the set Λ − consists of images that are dissimilar to x and its transformed versions. This set may include all images other than x, including their possible transformations. Minimizing the loss l(˜x, xˆ) increases the similarity between the representations of x˜ and xˆ, denoted by z˜ and zˆ, while increasing the dissimilarity between the representation of x and those of dissimilar images.”, in this section of the prior art, a similarity between the images is calculated and a loss is calculated as well and the model is retrained to minimize the loss such that the similar images are obtained and dissimilar images are filtered out.)
Chaitanya does not explicitly teach wherein the second set of augmented images is generated from the images of the first set of augmented images by applying one or more masking augmentation techniques to the images of the first set of augmented images;
Chen teaches wherein the second set of augmented images is generated from the images of the first set of augmented images by applying one or more masking augmentation techniques to the images of the first set of augmented images (4.6 BERT, Paragraph 3, “Finally, because inputs to the BERT model are masked at training time, we must also mask them at evaluation time to keep inputs in-distribution. This masking corruption may hinder the BERT model’s ability to correctly predict image classes.”, in this section of the prior art, the input images are disclosed to be generated by masking the image during training.)
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify the claimed invention as taught by Chaitanya with Chen in order to apply masking augmentation on a set of augmented images. One skilled in the art would have been motivated to modify Chaitanya in this manner in order to auto-regressively predict pixels, without incorporating knowledge of the 2D input structure.(Chen, Abstract)
Regarding Claim 2, the combination of Chaitanya and Chen teach the method of claim 1, where Chaitanya further teaches wherein the unlabeled images are medical images (Abstract, In this work, we propose strategies for extending the contrastive learning framework for segmentation of volumetric medical images in the semi-supervised setting with limited annotations)
Regarding Claim 4, the combination of Chaitanya and Chen teach the method of claim 1, where Chaitanya further teaches wherein one or more of the following techniques are applied to the unlabeled images: rotation, elastic deformation, flipping, scaling, stretching, shearing, cropping, resizing and/or combinations thereof.(Page 8, III. Comparison with other methods:, Paragraph 1, As a baseline, we trained a network (randomly initialized) with extensive data augmentation: rotation, cropping, flipping, scaling [13], elastic deformations [53], random contrast and brightness changes [29, 47].)
Regarding Claim 7, the combination of Chaitanya and Chen teach the method of claim 1, where Chaitanya further teaches wherein a neural network projection head is introduced at the end of the encoder, wherein the projection head maps the representations to a space where contrastive loss is applied, wherein the projection head performs a learnable nonlinear transformation. (Page 3, 3.1 Global contrastive loss, Paragraph 1, “Minimizing the loss l(˜x, xˆ) increases the similarity between the representations of x˜ and xˆ, denoted by z˜ and zˆ, while increasing the dissimilarity between the representation of x and those of dissimilar images. Note that the representations used in the loss are extracted after appending e(·) with g1(·), a shallow fully-connected network with limited capacity, also referred to as “projection head” in [12]. The presence of g1(·) allows e() some flexibility to also retain information regarding the transformations, as was empirically shown in [12]. Lastly, similarity in the representation space is defined via the cosine similarity between two vectors, i.e. sim(a, b) = a T b/kakkbk, and τ is a temperature scaling parameter”, as disclosed in this section a projection head is introduced at the end of the encoder and a contrastive loss is determined in order to minimize the loss.)
Regarding Claim 10, the combination of Chaitanya and Chen teach the method of claim 1, where Chaitanya further teaches comprising generating a pre-trained neural network by executing the steps of receiving the plurality of unlabeled images(Page 10, 6 Broader Impact, Paragraph 2, “We demonstrate on three medical datasets for the segmentation task that the proposed pre-training with unlabeled images can yield a good initialization, and reduce the need for large annotated sets to yield high performance when fine-tuned to the segmentation task”, in this section of the prior art, the dataset for training the network are unlabeled images.), generating the augmented training data set from the plurality of unlabeled images(Page 3, 3.1 Global contrastive loss , Paragraph 1, “Here, x˜ and xˆ are two differently transformed versions of the same image x, i.e. x˜ = t˜(x) and xˆ = tˆ(x) where t,˜ tˆ ∈ T are simple transformations as used in [24, 12, 4, 18, 56], such as crop followed by color transformations, and T is the set of such transformations. These two images are treated as similar and their representations are encouraged to be similar. In contrast, the set Λ − consists of images that are dissimilar to x and its transformed versions”, in this section of the prior art, a data augmentation of transforming the unlabeled images is performed to create a first and second set of images.), and training the first machine learning model on the first set of augmented images and the second set of augmented images generate a pre-trained neural network. (Page 6, Experimental setup: “We split each dataset into a pre-training set Xpre and a test set Xts, each consisting of volumetric images and corresponding segmentation labels. We pre-train a UNet architecture using only the images from Xpre without their labels, then fine-tune the pre-trained network with a small number of labeled examples chosen from Xpre.”, this section of the prior art, discloses training a machine learning model using two sets of volumetric images. )
Regarding Claim 12, the combination of Chaitanya and Chen teach the method of claim 10, where Chaitanya further teaches comprising generating a classifier using the pre-trained neural network, wherein generating the classifier comprises extracting the encoder from the encoder-decoder structure of the first machine model training and training the extracted encoder on a set comprising labeled images. (Page 3, 3,1 Contrastive Loss, Last Paragraph, “In the following discussion, we focus on 2D encoder-decoder architectures that are used for image segmentation, such as the UNet architecture [49]. In such architectures, we use the encoder to extract the global representation and the decoder layers to extract the complementary local representation.” In this section of the prior art, an encoder-decoder architecture is used to segment the images to extract certain features.)
Regarding Claim 14, claim 14 is considered an apparatus claim substantially corresponding to claim 1. Please see the discussion of claim 1 above for a discussion of similar limitations. Furthermore, Chaitanya teaches a computer system (Fig. 1 shows the architecture of the neural network) a processor; and a memory storing an application program configured to perform, when executed by the processor, an operation (Page 14, 7.4 Details of training time and convergence, Paragraph 1, discloses a GPU which is a processor and would inherently be coupled to memory.)
Regarding Claim 15, claim 15 is considered an apparatus claim substantially corresponding to claim 1. Please see the discussion of claim 1 above for a discussion of similar limitations. Furthermore, Chaitanya teaches a non-transitory computer readable medium storing software instruction that, when executed by a processor of a computer system, cause the computer system to (Page 14, 7.4 Details of training time and convergence, Paragraph 1, discloses a GPU which is a processor and would inherently be coupled to memory.)
Claim 3 is rejected under 35 U.S.C. 103 as being unpatentable over Chaitanya et al. ("Contrastive learning of global and local features for medical image segmentation with limited annotations") in view of Chen et al. ("Generative Pretraining from Pixels") in further view of Wang et al. US PG-Pub(US 20220067451 A1).
Regarding Claim 3, while the combination of Chaitanya and Chen teach the method of claim 1, they do not explicitly teach wherein the unlabeled images are photos of plants or parts thereof.
Wang teaches wherein the unlabeled images are photos of plants or parts thereof. (¶[0041], “training module 246 may also perform semi-supervised training of the machine learning model using unlabeled ground truth imagery 245 to increase the machine learning model's accuracy further. In such a context, the machine learning model may be referred to as the “base” model, which is further trained using semi-supervised training. For example, an unlabeled ground truth image 245 from the same domain—i.e. an image that depicts the same plant trait that the machine learning model is being trained to detect—may be processed by training module 246 using the machine learning model to generate a first prediction”)
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify the claimed invention as taught by Chaitanya and Chen with Wang in order to use unlabeled photos of plants. One skilled in the art would have been motivated to modify Chaitanya and Chen in this manner in order to generate a plurality of quasi-realistic synthetic training images, each depicting quasi-realistic and labeled instance(s) of the targeted plant trait. (Wang, Abstract)
Claims 5, 8-9, 11, 13 and 16-17 are rejected under 35 U.S.C. 103 as being unpatentable over Chaitanya et al. ("Contrastive learning of global and local features for medical image segmentation with limited annotations") in view of Chen et al. ("Generative Pretraining from Pixels") in further view of Chen 2 et al. US PG-Pub(US 20210319266 A1).
Regarding Claim 5, while the combination of Chaitanya and Chen teach the method of claim 1, they do not explicitly teach wherein one or more of the following techniques are applied to the images of the first set of augmented images: inner cutouts, outer cutouts, erasing and/or combinations thereof.
Chen 2 teaches wherein one or more of the following techniques are applied to the images of the first set of augmented images: inner cutouts, outer cutouts, erasing and/or combinations thereof. (¶[0065], “FIG. 4 visualizes the augmentations were considered and can optionally be included in implementations of the present disclosure, which include the following examples visualized relative to the original image: crop and resize; crop, resize (and flip); color distortion (drop); color distortion (jitter); rotate; cutout; Gaussian noise; Gaussian blur; and Sobel filtering.”, in this section of the prior art, discloses performing cutout augmentation on the images.)
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify the claimed invention as taught by Chaitanya and Chen with Chen 2 in order to perform certain data augmentation techniques. One skilled in the art would have been motivated to modify Chaitanya and Chen in this manner in order to leverage data augmentation and a learnable nonlinear transformation between the representation and the contrastive loss to provide improved visual representations. (Chen 2, ¶[0002])
Regarding Claim 8, while the combination of Chaitanya and Chen teach the method of claim 1, they do not explicitly teach further comprising: generating a second machine learning model from the first machine learning model, wherein generating the second machine learning model comprises creating a classifier based on the encoder from the encoder-decoder structure; and training the classifier on a training set comprising labeled images.
Chen 2 teaches further comprising: generating a second machine learning model from the first machine learning model (¶[0123], “Classification model 1204 also reuses a portion of multiple layers of a projection head neural network that also was pretrained with contrastive learning using unlabeled training data (i.e., projection head layer(s) 1208). For example, instead of discarding a projection head neural network (e.g., projection head neural network 206) entirely after pretraining, a portion of the layers of the projection head neural network (i.e., projection head layer(s) 1208) may be retained and incorporated with the pretrained based encoder neural network during fine-tuning”, in this section of the prior art, a second neural network is generated based off the encoder network.)
wherein generating the second machine learning model comprises creating a classifier based on the encoder from the encoder-decoder structure; and training the classifier on a training set comprising labeled images. (¶[0006], “ generating an image classification model from the base encoder neural network and the projection head neural network, the image classification model comprising some but not all of the plurality of layers of the projection head neural network, performing fine-tuning of the image classification model based on a set of labeled images”, as disclosed in this section, the prior art generates a classifier based off the encoder and trains the model based off a set of labeled images.)
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify the claimed invention as taught by Chaitanya and Chen with Chen 2 in order to generate a second model from the first model. One skilled in the art would have been motivated to modify Chaitanya and Chen in this manner in order to leverage data augmentation and a learnable nonlinear transformation between the representation and the contrastive loss to provide improved visual representations. (Chen 2, ¶[0002])
Regarding Claim 9, while the combination of Chaitanya and Chen teach the method of claim 1, where Chaitanya further teaches extracting the encoder-decoder structure from the first machine learning model and training the encoder-decoder structure based on labeled images to segment images (Page 3, 3,1 Contrastive Loss, Last Paragraph, “In the following discussion, we focus on 2D encoder-decoder architectures that are used for image segmentation, such as the UNet architecture [49]. In such architectures, we use the encoder to extract the global representation and the decoder layers to extract the complementary local representation.” In this section of the prior art, an encoder-decoder architecture is used to segment the images to extract certain features.)
However, they do not explicitly teach generating a second machine learning model from the first machine learning model,
Chen 2 teaches generating a second machine learning model from the first machine learning model(¶[0123], “Classification model 1204 also reuses a portion of multiple layers of a projection head neural network that also was pretrained with contrastive learning using unlabeled training data (i.e., projection head layer(s) 1208). For example, instead of discarding a projection head neural network (e.g., projection head neural network 206) entirely after pretraining, a portion of the layers of the projection head neural network (i.e., projection head layer(s) 1208) may be retained and incorporated with the pretrained based encoder neural network during fine-tuning”, in this section of the prior art, a second neural network is generated based off the encoder network.), It would have been obvious to one of ordinary skill in the art before the effective filing date to modify the claimed invention as taught by Chaitanya and Chen with Chen 2 in order to generate a second model from the first model. One skilled in the art would have been motivated to modify Chaitanya and Chen in this manner in order to leverage data augmentation and a learnable nonlinear transformation between the representation and the contrastive loss to provide improved visual representations. (Chen 2, ¶[0002])
Regarding Claim 11, while the combination of Chaitanya and Chen teach the method of claim 8, where Chaitanya further teaches comprising generating a trained neural network by executing the steps of receiving the plurality of unlabeled images(Page 10, 6 Broader Impact, Paragraph 2, “We demonstrate on three medical datasets for the segmentation task that the proposed pre-training with unlabeled images can yield a good initialization, and reduce the need for large annotated sets to yield high performance when fine-tuned to the segmentation task”, in this section of the prior art, the dataset for training the network are unlabeled images.), generating the augmented training data set from the plurality of unlabeled images(Page 3, 3.1 Global contrastive loss , Paragraph 1, “Here, x˜ and xˆ are two differently transformed versions of the same image x, i.e. x˜ = t˜(x) and xˆ = tˆ(x) where t,˜ tˆ ∈ T are simple transformations as used in [24, 12, 4, 18, 56], such as crop followed by color transformations, and T is the set of such transformations. These two images are treated as similar and their representations are encouraged to be similar. In contrast, the set Λ − consists of images that are dissimilar to x and its transformed versions”, in this section of the prior art, a data augmentation of transforming the unlabeled images is performed to create a first and second set of images.),training the first machine learning model on the first set of augmented images and the second set of augmented images generate a pre-trained neural network(Page 6, Experimental setup: “We split each dataset into a pre-training set Xpre and a test set Xts, each consisting of volumetric images and corresponding segmentation labels. We pre-train a UNet architecture using only the images from Xpre without their labels, then fine-tune the pre-trained network with a small number of labeled examples chosen from Xpre.”, this section of the prior art, discloses training a machine learning model using two sets of volumetric images.),
However, they do not explicitly teach generating the second machine learning model from the first machine learning model, and training the classifier on the training set comprising labeled images.
Chen 2 teaches generating a second machine learning model from the first machine learning model (¶[0123], “Classification model 1204 also reuses a portion of multiple layers of a projection head neural network that also was pretrained with contrastive learning using unlabeled training data (i.e., projection head layer(s) 1208). For example, instead of discarding a projection head neural network (e.g., projection head neural network 206) entirely after pretraining, a portion of the layers of the projection head neural network (i.e., projection head layer(s) 1208) may be retained and incorporated with the pretrained based encoder neural network during fine-tuning”, in this section of the prior art, a second neural network is generated based off the encoder network.)
and training the classifier on a training set comprising labeled images. (¶[0006], “ generating an image classification model from the base encoder neural network and the projection head neural network, the image classification model comprising some but not all of the plurality of layers of the projection head neural network, performing fine-tuning of the image classification model based on a set of labeled images”, as disclosed in this section, the prior art generates a classifier based off the encoder and trains the model based off a set of labeled images.)
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify the claimed invention as taught by Chaitanya and Chen with Chen 2 in order to generate a second model from the first and train a classifier from the labeled images. One skilled in the art would have been motivated to modify Chaitanya and Chen in this manner in order to leverage data augmentation and a learnable nonlinear transformation between the representation and the contrastive loss to provide improved visual representations. (Chen 2, ¶[0002])
Regarding Claim 13, the combination of Chaitanya, Chen and Chen 2 teach the method of claim 11, where Chaitanya further teaches comprising classifying and/or segmenting images using the trained neural network, in particular classifying and/or segmenting medical images or photos of diseased plants or pest-infected plants or parts thereof. (Page 6, 4 Experiments and Results, Experimental setup: “We split each dataset into a pre-training set Xpre and a test set Xts, each consisting of volumetric images and corresponding segmentation labels. We pre-train a UNet architecture using only the images from Xpre without their labels, then fine-tune the pre-trained network with a small number of labeled examples chosen from Xpre. Fine-tuned model’s segmentation performance is used to assess the pre-training procedure.”, in this section of the prior art, a trained machine learning model is used to segment the medical images.)
Regarding Claim 16, while the combination of Chaitanya and Chen teach the method of claim 9, where Chaitanya further teaches comprising generating a trained neural network by executing the steps of receiving the plurality of unlabeled images(Page 10, 6 Broader Impact, Paragraph 2, “We demonstrate on three medical datasets for the segmentation task that the proposed pre-training with unlabeled images can yield a good initialization, and reduce the need for large annotated sets to yield high performance when fine-tuned to the segmentation task”, in this section of the prior art, the dataset for training the network are unlabeled images.), generating the augmented training data set from the plurality of unlabeled images(Page 3, 3.1 Global contrastive loss , Paragraph 1, “Here, x˜ and xˆ are two differently transformed versions of the same image x, i.e. x˜ = t˜(x) and xˆ = tˆ(x) where t,˜ tˆ ∈ T are simple transformations as used in [24, 12, 4, 18, 56], such as crop followed by color transformations, and T is the set of such transformations. These two images are treated as similar and their representations are encouraged to be similar. In contrast, the set Λ − consists of images that are dissimilar to x and its transformed versions”, in this section of the prior art, a data augmentation of transforming the unlabeled images is performed to create a first and second set of images.),training the first machine learning model on the first set of augmented images and the second set of augmented images generate a pre-trained neural network(Page 6, Experimental setup: “We split each dataset into a pre-training set Xpre and a test set Xts, each consisting of volumetric images and corresponding segmentation labels. We pre-train a UNet architecture using only the images from Xpre without their labels, then fine-tune the pre-trained network with a small number of labeled examples chosen from Xpre.”, this section of the prior art, discloses training a machine learning model using two sets of volumetric images.),
Chen 2 teaches generating the second machine learning model from the first machine learning model (¶[0123], “Classification model 1204 also reuses a portion of multiple layers of a projection head neural network that also was pretrained with contrastive learning using unlabeled training data (i.e., projection head layer(s) 1208). For example, instead of discarding a projection head neural network (e.g., projection head neural network 206) entirely after pretraining, a portion of the layers of the projection head neural network (i.e., projection head layer(s) 1208) may be retained and incorporated with the pretrained based encoder neural network during fine-tuning”, in this section of the prior art, a second neural network is generated based off the encoder network.)
wherein generating the second machine learning model comprises creating a classifier based on the encoder from the encoder-decoder structure; and training the classifier on a training set comprising labeled images. (¶[0006], “ generating an image classification model from the base encoder neural network and the projection head neural network, the image classification model comprising some but not all of the plurality of layers of the projection head neural network, performing fine-tuning of the image classification model based on a set of labeled images”, as disclosed in this section, the prior art generates a classifier based off the encoder and trains the model based off a set of labeled images.)
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify the claimed invention as taught by Chaitanya and Chen with Chen 2 in order to generate a second model from the first and train a classifier from the labeled images. One skilled in the art would have been motivated to modify Chaitanya and Chen in this manner in order to leverage data augmentation and a learnable nonlinear transformation between the representation and the contrastive loss to provide improved visual representations. (Chen 2, ¶[0002])
Regarding Claim 17, the combination of Chaitanya, Chen and Chen 2 teach the method of claim 16, where Chaitanya further teaches comprising classifying and/or segmenting images using the trained neural network, in particular classifying and/or segmenting medical images or photos of diseased plants or pest-infected plants or parts thereof. (Page 6, 4 Experiments and Results, Experimental setup: “We split each dataset into a pre-training set Xpre and a test set Xts, each consisting of volumetric images and corresponding segmentation labels. We pre-train a UNet architecture using only the images from Xpre without their labels, then fine-tune the pre-trained network with a small number of labeled examples chosen from Xpre. Fine-tuned model’s segmentation performance is used to assess the pre-training procedure.”, in this section of the prior art, a trained machine learning model is used to segment the medical images.)
Allowable Subject Matter
Claim 6 is objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to HAN D HOANG whose telephone number is (571)272-4344. The examiner can normally be reached Monday-Friday 8-5.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, JOHN M VILLECCO can be reached at 571-272-7319. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/HAN HOANG/Examiner, Art Unit 2661