Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Applicants Arguments/Amendments
Applicants arguments/amendments filed on 4/28/2026 have been entered and made of record.
Applicant’s arguments, see page 11, filed 4/28/2026, with respect to 35 U.S.C. 112b have been fully considered and are persuasive. The claim amendments have overcome the rejection. The rejection of claims 21-45 has been withdrawn.
Applicant's arguments filed 4/28/202 have been fully considered but they are not persuasive.
Re claim 39 applicant argues:
Without acquiescing to the rejection, claims have been amended to at least further expedite prosecution. Support for the amendments can be found in the specification, e.g., paragraphs [0016]-[0018], [0032]-[0035], and [0044]-[0048] and no new matter has been added by these amendments.
Applicant respectfully submits that Karki and Yasutomi at least fail to teach or suggest claim 39. For example, Karki states in paragraph [0050] that "[t]o generate the synthetic normal image 112 from the abnormal image 106, the generator 102 modifies the abnormal image 106. It does so by removing a distinct region of the abnormal image 106 that corresponds to the abnormality. A residual part of the abnormal image 106 that has thus been removed from the abnormal image 106 forms the basis of the abnormal segmentation map 110." Karki further states that "The abnormal images 106 showing lesions are passed to the encoder 132, which provides them to the first decoder 136. The first decoder 136 creates the synthetic normal image 112 and an abnormal segmentation map 104, or lesion mask. A combiner 122 combines the synthetic normal image 112 and the lesion mask 104 to yield the reconstructed image 124." The cited portions of Karki, however, at least fail to teach "decoding the one or more latent representations to generate (1) a translated image from a first decoder that decodes one or more first samples corresponding to one or more features classified as common relative to training data and (2) a residual image from a second decoder that decodes one or more second samples that correspond to the one or more features classified as uncommon relative to training data; and using the second decoder to generate the segmentation mask, wherein one or more normalization parameters associated with the second decoder are adjusted based, at least in part, on the one or more features classified as common and the one or more features classified as uncommon," as claimed.
Yasutomi fails to remedy the deficiencies of Karki. For example, paragraphs [0029]- [0030] state that "[t]he input unit 131 inputs an output from the encoder to which an input image is input to the shade decoder and the subject decoder. Using the combining function, the combining unit 132 synthesizes a shade image that is an output from the shade decoder and a subject image that is an output from the subject decoder. The shade image is an example of the first image. The subject image is an example of the second image." and "The learning unit 133 executes learning for the encoder, the shade decoder and the subject decoder, based on a reconstruction error, a first likelihood function and a second likelihood function. The reconstruction error is an error between the input image and an output image obtained using the combining function to synthesize the shade image that is an output of the shade decoder and the subject image that is an output of the subject decoder. The first likelihood function is a likelihood function for the shade image relating to shades in ultrasound images. The second likelihood function is a likelihood function for the subject image relating to subjects in ultrasound images." However, similar to Karki, the cited portions of Yasutomi similarly at least fail to teach the missing elements of Karki as provided above.
The examiner notes that the newly amended subject matter has not been previously considered. Applicant argues that “decoding the one or more latent representations to generate (1) a translated image from a first decoder that decodes one or more first samples corresponding to one or more features classified as common relative to training data and (2) a residual image from a second decoder that decodes one or more second samples that correspond to the one or more features classified as uncommon relative to training data; and using the second decoder to generate the segmentation mask, wherein one or more normalization parameters associated with the second decoder are adjusted based, at least in part, on the one or more features classified as common and the one or more features classified as uncommon,” is not taught by Karki or Yasutomi. Applicant fails to particularly point out how the features are distinguished from the prior art and merely states the prior art does not teach a list of features but merely cites a large block of the claim and states that it is not taught. The examiner is unsure why applicant believes that these features are not taught by Yasutomi. The examiner disagrees and notes that Yasutomi teaches many (but not all) of the argued features as stated in the new grounds of rejection below. Further the examiner has applied additional new grounds of rejection to demonstrate the missing features.
Regarding claims 21, 27 33 and 45 Applicant argues:
Applicant respectfully submits that claims 21, 27, 33, and 45 are allowable at least for reasons including some of those discussed above in connection with claim 39. For example, claim 21 recites "encoding the one or more images to generate one or more latent representations; decoding the one or more latent representations to generate (1) a translated image from a first decoder that decodes one or more first samples corresponding to one or more features classified as common relative to training data and (2) a residual image from a second decoder that decodes one or more second samples that correspond to the one or more features classified as uncommon relative to training data; and updating one or more parameters of the second decoder to generate the segmentation mask, wherein one or more normalization parameters associated with the second decoder are adjusted based, at least in part, on the one or more features classified as common and the one or more features classified as uncommon." explained above, Karki and Yasutomi fail to teach or suggest at least these elements. Claim 27 recites "encoding the one or more images to generate one or more latent representations; decoding the one or more latent representations to generate (1) a translated image from a first decoder that decodes one or more first samples corresponding to one or more features classified as common relative to training data and (2) a residual image from a second decoder that decodes one or more second samples that correspond to the one or more features classified as uncommon relative to training data; and updating one or more parameters of the second decoder to generate the segmentation mask, wherein one or more normalization parameters associated with the second decoder are adjusted based, at least in part, on the one or more features classified as common and the one or more features classified as uncommon." As explained above, Karki and Yasutomi fail to teach or suggest at least these elements. Claim 33 recites "encoding the one or more images to generate one or more latent representations; decoding the one or more latent representations to generate (1) a translated image from a first decoder that decodes one or more first samples corresponding to one or more features classified as common relative to training data and (2) a residual image from a second decoder that decodes one or more second samples that correspond to the one or more features classified as uncommon relative to training data; and updating one or more parameters of the second decoder to generate the segmentation mask, wherein one or more normalization parameters associated with the second decoder are adjusted based, at least in part, on the one or more features classified as common and the one or more features classified as uncommon." As explained above, Karki and Yasutomi fail to teach or suggest at least these elements. Claim 45 recites "encoding the one or more images to generate one or more latent representations; decoding the one or more latent representations to generate(1) a translated image from a first decoder that decodes one or more first samples corresponding to one or more features classified as common relative to training data and (2) a residual image from a second decoder that decodes one or more second samples that correspond to the one or more features classified as uncommon relative to training data; and updating one or more parameters of the second decoder to generate the segmentation mask, wherein one or more normalization parameters associated with the second decoder are adjusted based, at least in part, on the one or more features classified as common and the one or more features classified as uncommon." As explained above, Karki and Yasutomi fail to teach or suggest at least these elements
The examiner notes that applicant applies similar reasoning and references the arguments to claim 39 above with regards to these claims. The examiner notes that these arguments are unpersuasive for similar reasoning as explained above. Applicant fails to particularly point out how the features are distinguished from the prior art and merely states the prior art does not teach a list of features but merely cites a large block of the claim and states that it is not taught. The examiner is unsure why applicant believes that these features are not taught by Yasutomi. The examiner disagrees and notes that Yasutomi teaches many (but not all) of the argued features as stated in the new grounds of rejection below. Further the examiner has applied additional new grounds of rejection to demonstrate the missing features.
Applicant's arguments filed 4/282026 with respect to 35 U.S.C. 103 and clasim 22-25, 28, 29 31, 32, 34-38, 40-44 and 46-50 have been fully considered but they are not persuasive.
Applicant argues:
Claims 22-26, 28-32, 34-38, 40-44, and 46-50 each depend from one of claims 21, 27, 33, 39, and 45 described above. Accordingly, Applicant respectfully submits that claims 22-26, 28- 32, 34-38, 40-44, and 46-50 are allowable at least for depending from an allowable independent claim. Hope Simpson, Ceccaldi, Wang, Andermatt, and Ioffe fail to cure the deficiencies of Karki and Yasutomi. In addition, Applicant respectfully submits that at least some of claims 22-26, 28- 32, 34-38, 40-44, and 46-50 additionally recite patentable subject matter not taught or otherwise rendered obvious by Karki, Yasutomi, Hope Simpson, Ceccaldi, Wang, Andermatt, and Ioffe, individually or in combination.
Applicant's arguments fail to comply with 37 CFR 1.111(b) because they amount to a general allegation that the claims define a patentable invention without specifically pointing out how the language of the claims patentably distinguishes them from the references.
The examiner notes that amended claims 26 and 30 are not rejected under prior art below and contain allowable subject matter.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 21, 23 25, 27, 31-33, 35-37, 39, 41, 42, 44, 45, 47, 49, and 50 is/are rejected under 35 U.S.C. 103 as being unpatentable over Yasutomi US 2020/0226796 in view of Xun Huang, Ming-Yu Liu, Serge Belongie, Jan Kautz; Proceedings of the European Conference on Computer Vision (ECCV), 2018, pp. 172-189 “Multimodal Unsupervised Image-to-Image Translation”.
Re claim 21 Yatsutomi discloses One or more processors, comprising circuitry to (see paragraph 80-82): train a neural network to generate a segmentation mask corresponding to a feature depicted in one or more images (see paragraph 31 and 39 note that and figure 2 note that a shade image which segments the shade from the subject is generated, this shade image could be considered a segmentation mask )
by encoding the one or more images to generate one or more latent representations ( see paragraph 32 35 note that the output from the encoder could be considered a latent representation);
decoding the one or more latent representations to generate (1) a translated image from a first decoder that decodes one or more first samples corresponding to one or more features classified as common relative to training data (see figure 2 not the subject decoder generates a subject image which is an medical image without a shade artifact see paragraph 29 and 30)
and (2) a residual image from a second decoder that decodes one or more second samples that correspond to one or more features classified as uncommon relative to training data (see paragraph 28 and 30 note that a shade image is output from a shade decoder which contains the shade artifact.);
and updating one or more parameters of the second decoder to generate the segmentation mask, (see paragraph 30 31 “] The calculator 133a calculates a loss function from the likelihood based on the first likelihood function and the likelihood based on the second likelihood function. The updating unit 133b updates the model parameters of the encoder, the shade decoder, and the subject decoder such that the loss function reduces”) wherein parameters associated with the second decoder are adjusted ( see paragraph 48 49 31 note that parameters of the second decoder are adjusted.) based, at least in part, on the one or more features classified as common and the one or more features classified as uncommon (figure 2 note that reconstructed subject and shade images are combined and then subtracted from the first image to determine reconstruction error which is used to update the parameters, see also paragraph 48 and 49 )
Yatsutomi does not expressly disclose second decoder has normalization parameters. Huang discloses disclose decoder has normalization parameters (see section 5.1 decoder section note that Scale parameter gamma and Shift parameter Beta are dynamically generated by the MLP). The motivation to combine is “Extensive experiments demonstrate the effectiveness of our method in modeling multimodal output distributions and its superior image quality compared with state-of-the-art approaches”. One of ordinary skill in the art could have used the image to image translation of Huang to implement the neural networks of Yatsutomi. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Yatsutomi and Huang to reach the aforementioned advantage.
Re claim 23 Yatsutomi discloses wherein the circuitry is to such that the second decoder outputs the segmentation mask indicating a location of the feature that is classified as uncommon (see figure 2 shade image note that the shade decoder output a shade image which discloses the location of the uncommon features i.e. shade).
Yatsutomi further does not expressly disclose use a multi-layer perceptron to modify the second decoder Huang discloses disclose se a multi-layer perceptron to modify the second decoder (see section 5.1 decoder note that Scale parameter gamma and Shift parameter Beta are dynamically generated by the MLP) the motivation to combine is “Extensive experiments demonstrate the effectiveness of our method in modeling multimodal output distributions and its superior image quality compared with state-of-the-art approaches”. One of ordinary skill in the art could have used the image-to-image translation of Huang to implement the neural networks of Yatsutomi. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Yatsutomi and Huang to reach the aforementioned advantage.
Re claim 25 Yatsutomi discloses wherein the circuitry is to cause the neural network to be trained by at least updating the second decoder (see figure 2 see paragraph 48 -50 note that the loss function is based on the reconstruction error between reconstructed images and the input image) based, at least in part, on differences between the translated image generated by the first decoder and the residual image generated by the second decoder ( see figure 2 see also paragraph; see paragraph 37 using the combining function to add the subject image and subtract the shade image, the combining unit 132 synthesizes the images to generate the output image” note).
Re claim 27 Yatsutomi discloses A system, comprising: one or more processors to (see paragraph 80-82): train a neural network to generate a segmentation mask corresponding to a feature depicted in one or more images (see paragraph 31 and 39 note that and figure 2 note that a shade image which segments the shade from the subject is generated, this shade image could be considered a segmentation mask)
by encoding the one or more images to generate one or more latent representations (see paragraph 32 35 note that the output from the encoder could be considered a latent representation);
decoding the one or more latent representations to generate (1) a translated image from a first decoder that decodes one or more first samples corresponding to one or more features classified as common relative to training data (see figure 2 not the subject decoder generates a subject image which is an medical image without a shade artifact see paragraph 29 and 30)
and (2) a residual image from a second decoder that decodes one or more second samples that correspond to one or more features classified as uncommon relative to training data (see paragraph 28 and 30 note that a shade image is output from a shade decoder which contains the shade artifact.);
and updating one or more parameters of the second decoder to generate the segmentation mask, (see paragraph 30 31 “] The calculator 133a calculates a loss function from the likelihood based on the first likelihood function and the likelihood based on the second likelihood function. The updating unit 133b updates the model parameters of the encoder, the shade decoder, and the subject decoder such that the loss function reduces”) wherein parameters associated with the second decoder are adjusted ( see paragraph 48 49 31 note that parameters of the second decoder are adjusted.) based, at least in part, on the one or more features classified as common and the one or more features classified as uncommon (figure 2 note that reconstructed subject and shade images are combined and then subtracted from the first image to determine reconstruction error which is used to update the parameters, see also paragraph 48 and 49 )
Yatsutomi does not expressly disclose second decoder has normalization parameters. Huang discloses disclose decoder has normalization parameters (see section 5.1 decoder section note that Scale parameter gamma and Shift parameter Beta are dynamically generated by the MLP) the motivation to combine is “Extensive experiments demonstrate the effectiveness of our method in modeling multimodal output distributions and its superior image quality compared with state-of-the-art approaches”. One of ordinary skill in the art could have used the image to image translation of Huang to implement the neural networks of Yatsutomi. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Yatsutomi and Huang to reach the aforementioned advantage.
Re claim 31 Yatsutomi discloses herein the one or more processors are to generate the segmentation mask the second decoder, (see figure 2 shade image note that the shade decoder output a shade image which discloses the location of the uncommon features i.e. shade).
Yatsutomi does not disclose discloses using a multi-layer perceptron between an encoder and the decoder, wherein the one or more normalization parameters are to be adjusted based at least in part on one or more scale parameters. Huang discloses using a multi-layer perceptron between an encoder and the decoder, wherein the one or more normalization parameters are to be adjusted based at least in part on one or more scale parameters. (see section 5.1 decoder section note that Scale parameter gamma and Shift parameter Beta are dynamically generated by the MLP note that parameter gamma is multiplied which corresponds to a scaling). The motivation to combine is “Extensive experiments demonstrate the effectiveness of our method in modeling multimodal output distributions and its superior image quality compared with state-of-the-art approaches”. One of ordinary skill in the art could have used the image to image translation of Huang to implement the neural networks of Yatsutomi. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Yatsutomi and Huang to reach the aforementioned advantage.
Re claim 32 Yatsutomi discloses wherein the feature depicted in one or more images corresponds to a first region of a latent space comprising one or more samples corresponding to the one or more features classified as uncommon (see figure 2 The examiner notes that the output of the encoder corresponds to the latent space [note a latent space merely and intermediate output of a neural network] and at least some region (possibly all) of the latent space must correspond each of the shade portion [uncommon features] and subject portion[common feature] as it is used by the decoder to output the subject image and a shade image].)
Re claim 33 Yatsutomi discloses A non-transitory machine-readable medium having stored thereon a set of instructions, which if performed by one or more processors, cause the one or more processors to (see paragraph 80-82)
cause a neural network to be trained to generate a segmentation mask corresponding to a feature depicted in one or more images (see paragraph 31 and 39 note that and figure 2 note that a shade image which segments the shade from the subject is generated, this shade image could be considered a segmentation mask)
by encoding the one or more images to generate one or more latent representations (see paragraph 32 35 note that the output from the encoder could be considered a latent representation);
decoding the one or more latent representations to generate (1) a translated image from a first decoder that decodes one or more first samples corresponding to one or more features classified as common relative to training data (see figure 2 not the subject decoder generates a subject image which is an medical image without a shade artifact see paragraph 29 and 30)
and (2) a residual image from a second decoder that decodes one or more second samples that correspond to one or more features classified as uncommon relative to training data (see paragraph 28 and 30 note that a shade image is output from a shade decoder which contains the shade artifact.);
and updating one or more parameters of the second decoder to generate the segmentation mask, (see paragraph 30 31 “] The calculator 133a calculates a loss function from the likelihood based on the first likelihood function and the likelihood based on the second likelihood function. The updating unit 133b updates the model parameters of the encoder, the shade decoder, and the subject decoder such that the loss function reduces”) wherein parameters associated with the second decoder are adjusted ( see paragraph 48 49 31 note that parameters of the second decoder are adjusted.) based, at least in part, on the one or more features classified as common and the one or more features classified as uncommon (figure 2 note that reconstructed subject and shade images are combined and then subtracted from the first image to determine reconstruction error which is used to update the parameters, see also paragraph 48 and 49 )
Yatsutomi does not expressly disclose second decoder has normalization parameters. Huang discloses a decoder which has normalization parameters (see section 5.1 decoder section note that Scale parameter gamma and Shift parameter Beta are dynamically generated by the MLP) the motivation to combine is “Extensive experiments demonstrate the effectiveness of our method in modeling multimodal output distributions and its superior image quality compared with state-of-the-art approaches”. One of ordinary skill in the art could have used the image to image translation of Huang to implement the neural networks of Yatsutomi. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Yatsutomi and Huang to reach the aforementioned advantage.
Re claim 35 Yatsutomi discloses wherein the neural network further comprises one or more encoders that encode the one or more features classified as common relative to other training data to a first region of a latent space and the one or more features classified as uncommon relative to other training data to a second region of a latent space (see figure 2 The examiner notes that the output of the encoder corresponds to the latent space [note a latent space merely and intermediate output of a neural network] and at least some region (possibly all) of the latent space must correspond each of the shade portion [uncommon features] and subject portion[common feature] as it is used by the decoder to output the subject image and a shade image].)
Re claim 36 Yatsutomi discloses wherein the set of instructions further include instructions, which if performed by the one or more processors, cause the one or more processors to train the neural network by at least using one or more objective functions. (see paragraph 31 47 and 48 note that a loss function is used to train the encoder)
Re claim 37 Yatsutomi discloses generate the segmentation mask by at least identifying a location of the feature in the one or more images (see figure 2 note that the shade image is generated which identifies the location of the shade.)
Re claim 39 Yatsutomi discloses One or more processors, comprising circuitry to (see paragraph 80-82): cause a neural network to be used to generate a segmentation mask corresponding to a feature depicted in one or more images (see paragraph 31 and 39 note that and figure 2 note that a shade image which segments the shade from the subject is generated, this shade image could be considered a segmentation mask)
by encoding the one or more images to generate one or more latent representations see paragraph 32 35 note that the output from the encoder could be considered a latent representation);
decoding the one or more latent representations to generate (1) a translated image from a first decoder that decodes one or more first samples corresponding to one or more features classified as common relative to training data (see figure 2 not the subject decoder generates a subject image which is an medical image without a shade artifact see paragraph 29 and 30)
and (2) a residual image from a second decoder that decodes one or more second samples that correspond to one or more features classified as uncommon relative to training data (see paragraph 28 and 30 note that a shade image is output from a shade decoder which contains the shade artifact.);
and using the second decoder to generate the segmentation mask, (see figure 2 note that shade image corresponding to the segmentation mask is generated by the features of the encoder) wherein parameters associated with the second decoder are adjusted ( see paragraph 48 49 31 note that parameters of the second decoder are adjusted.) based, at least in part, on the one or more features classified as common and the one or more features classified as uncommon (figure 2 note that reconstructed subject and shade images are combined and then subtracted from the first image to determine reconstruction error which is used to update the parameters, see also paragraph 48 and 49 )
Yatsutomi does not expressly disclose second decoder has normalization parameters. Huang discloses a decoder with normalization parameters (see section 5.1 decoder section note that Scale parameter gamma and Shift parameter Beta are dynamically generated by the MLP). The motivation to combine is “Extensive experiments demonstrate the effectiveness of our method in modeling multimodal output distributions and its superior image quality compared with state-of-the-art approaches”. One of ordinary skill in the art could have used the image to image translation of Huang to implement the neural networks of Yatsutomi. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Yatsutomi and Huang to reach the aforementioned advantage.
Re claim 41 Yasutomi dislcloses wherein the neural network includes one or more encoders for generating the one or more latent representations comprising the one or more first samples and the one or more second samples (see figure 2 The examiner notes that the output of the encoder corresponds to the latent space [note a latent space merely and intermediate output of a neural network] and at least some region (possibly all) of the latent space must correspond each of the shade portion [ second samples i.e. uncommon features] and subject portion [ first samples i.e.common feature] as it is used by the decoders to output the subject image and a shade image].)
Re claim 42 Yatsutomi discloses generate the segmentation mask by at least identifying a location of the feature in the one or more images (see figure 2 note that the shade image is generated which identifies the location of the shade.)
Re claim 44 Yatsutomi discloses generate the segmentation mask based on the decoder. Yatsutomi does not expressly disclose based at least in part on one or more shift parameters Huang discloses based at least in part on one or more shift parameters. (see section 5.1 decoder section note that Scale parameter gamma and Shift parameter Beta are dynamically generated by the MLP). The motivation to combine is “Extensive experiments demonstrate the effectiveness of our method in modeling multimodal output distributions and its superior image quality compared with state-of-the-art approaches”. One of ordinary skill in the art could have used the image to image translation of Huang to implement the neural networks of Yatsutomi. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Yatsutomi and Huang to reach the aforementioned advantage.
Re claim 45 Yasutomi discloses A method, comprising: causing a neural network to be used to generate a segmentation mask corresponding to a feature depicted in one or more images (see paragraph 31 and 39 note that and figure 2 note that a shade image which segments the shade from the subject is generated, this shade image could be considered a segmentation mask )
by encoding the one or more images to generate one or more latent representations ( see paragraph 32 35 note that the output from the encoder could be considered a latent representation);
decoding the one or more latent representations to generate (1) a translated image from a first decoder that decodes one or more first samples corresponding to one or more features classified as common relative to training data (see figure 2 not the subject decoder generates a subject image which is an medical image without a shade artifact see paragraph 29 and 30)
and (2) a residual image from a second decoder that decodes one or more second samples that correspond to one or more features classified as uncommon relative to training data (see paragraph 28 and 30 note that a shade image is output from a shade decoder which contains the shade artifact.);
and using the second decoder to generate the segmentation mask, (see figure 2 note that shade image corresponding to the segmentation mask is generated by the features of the encoder) wherein parameters associated with the second decoder are adjusted ( see paragraph 48 49 31 note that parameters of the second decoder are adjusted.) based, at least in part, on the one or more features classified as common and the one or more features classified as uncommon (figure 2 note that reconstructed subject and shade images are combined and then subtracted from the first image to determine reconstruction error which is used to update the parameters, see also paragraph 48 and 49 )
Yatsutomi does not expressly disclose second decoder has normalization parameters. Huang discloses decoder has normalization parameters (see section 5.1 decoder section note that Scale parameter gamma and Shift parameter Beta are dynamically generated by the MLP). The motivation to combine is “Extensive experiments demonstrate the effectiveness of our method in modeling multimodal output distributions and its superior image quality compared with state-of-the-art approaches”. One of ordinary skill in the art could have used the image to image translation of Huang to implement the neural networks of Yatsutomi. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Yatsutomi and Huang to reach the aforementioned advantage.
Re claim 47 Yatsutomi discloses generate the segmentation mask based on the decoder. Yatsutomi does not expressly disclose using one or more multi-layer perceptrons (MLPs). Huang discloses using one or more multi-layer perceptrons (MLPs). (see section 5.1 decoder section note that Scale parameter gamma and Shift parameter Beta are dynamically generated by the MLP). The motivation to combine is “Extensive experiments demonstrate the effectiveness of our method in modeling multimodal output distributions and its superior image quality compared with state-of-the-art approaches”. One of ordinary skill in the art could have used the image to image translation of Huang to implement the neural networks of Yatsutomi. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Yatsutomi and Huang to reach the aforementioned advantage.
Re claim 49 Yatsutomi discloses comprising generating the segmentation mask based at least in part the one or more features classified as common and the one or more features classified as uncommon. (figure 2 note that the output of the encoder corresponds to the features, this output is used to generate both shade image [corresponding to the uncommon features and the segmentation mask] and the subject mast corresponding to the common features). Yatsutomi does not expressly disclose on one or more scale and shift parameters associated with scale and shift operations to be performed with the features. Huang discloses one or more scale and shift parameters associated with scale and shift operations to be performed with the features (see section 5.1 decoder section note that Scale parameter gamma and Shift parameter Beta are dynamically generated by the MLP see in particular equation 6 note that the gamma parameter scales the normalization function and parameter beta shifts the normalization function). The motivation to combine is “Extensive experiments demonstrate the effectiveness of our method in modeling multimodal output distributions and its superior image quality compared with state-of-the-art approaches”. One of ordinary skill in the art could have used the image to image translation of Huang to implement the neural networks of Yatsutomi which would perform the scale and shift function on the uncommon and common features of Yatsutomi. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Yatsutomi and Huang to reach the aforementioned advantage.
Re claim 50 Yatsutomi discloses wherein the one or more images comprise a medical image.( see paragraph 6 note that ultrasound images are used)
Claim(s) 22, 29 38 and 40 is/are rejected under 35 U.S.C. 103 as being unpatentable over Yasutomi US 2020/0226796 in view of Xun Huang, Ming-Yu Liu, Serge Belongie, Jan Kautz; Proceedings of the European Conference on Computer Vision (ECCV), 2018, pp. 172-189 “Multimodal Unsupervised Image-to-Image Translation” in view of Hope Simpson et al US 2019/0336108.
Re claim 22 Yasutomi and Huang discloses all the elements of claim 21. Yasutomi does not disclose wherein the one or more processors include one or more parallel processing units (PPUs) Hope Simpson discloses wherein the one or more processors include one or more parallel processing units (PPUs ) (note that a Neural network may implemented by multiple processors arranged for parallel processing. see paragraph 32). The motivation to combine is to achieve parallel processing (see paragraph 32). Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Huang, Yasutomi and Hope Simpson to reach the aforementioned advantage.
Re claim 29 Yasutomi and Huang discloses all the elements of claim 27. Yasutomi does not disclose wherein the one or more processors include one or more parallel processing units (PPUs) Hope Simpson discloses wherein the one or more processors include one or more parallel processing units (PPUs ) (note that a Neural network may implemented by multiple processors arranged for parallel processing. see paragraph 32). The motivation to combine is to achieve parallel processing (see paragraph 32). Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Huang, Yasutomi and Hope Simpson to reach the aforementioned advantage.
Re claim 38 Yasutomi and Huang discloses all the elements of claim 33. Yasutomi does not disclose wherein the one or more processors include one or more parallel processing units (PPUs) Hope Simpson discloses wherein the one or more processors include one or more parallel processing units (PPUs ) (note that a Neural network may implemented by multiple processors arranged for parallel processing. see paragraph 32). The motivation to combine is to achieve parallel processing (see paragraph 32). Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Huang, Yasutomi and Hope Simpson to reach the aforementioned advantage.
Re claim 40 Yasutomi and Huang discloses all the elements of claim 39. Yasutomi does not disclose wherein the one or more processors include one or more parallel processing units (PPUs) Hope Simpson discloses wherein the one or more processors include one or more parallel processing units (PPUs ) (note that a Neural network may implemented by multiple processors arranged for parallel processing. see paragraph 32). The motivation to combine is to achieve parallel processing (see paragraph 32). Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Huang, Yasutomi and Hope Simpson to reach the aforementioned advantage.
Claim(s) 24, 28, 34, 43, 46 and 48 is/are rejected under 35 U.S.C. 103 as being unpatentable over Yasutomi US 2020/0226796 in view of Xun Huang, Ming-Yu Liu, Serge Belongie, Jan Kautz; Proceedings of the European Conference on Computer Vision (ECCV), 2018, pp. 172-189 “Multimodal Unsupervised Image-to-Image Translation” in view of in view of Ceccaldi US 2019/0046068 .
Re claim 24 Yatsutomi discloses wherein the neural network includes a first encoder coupled to at least one of the first decoder and the second decoder (see figure 2 note that encoder is coupled to the shade decoder and subject encoder) that are to receive the one or more features classified as common and the one or more features classified as uncommon (see figure 2 note that the output of the encoder is used to decode the uncommon features i.e. the shade and the subject image the output of the encoder must include features included as common and uncommon)
Yatsutomi does not expressly the encoder and decoder disclose coupled via a set of long-skip connections to generate at least one or more compressed feature maps.
Ciccaldi discloses wherein the one or more neural networks include the encoder and decoders connected via a set of long-skip connections. (see paragraph 38 “In an embodiment, the encoder and decoder are symmetrical, using the same number of pooling (downsampling/upsampling) layers. The symmetrical structures provide for connections between encoding and decoding stages referred to as skip connections. The skip connections help against vanishing gradients and help maintain the high frequency components of the images” see paragraph 38 note that the output of the layers are features maps see paragraph 65 “One segment (i.e., encoder) of layers or units applies convolution to increase abstractness or compression” note that each layer of the encoder increases compression). Ceccaldi is in a similar art of segmentation using and decoder encoder architecture. One of ordinary skill in the art could have modified the encoder and decoders of Yatsutomi to include the long skip connection as described in Ceccaldi. The motivation to combine is to “maintain the high frequency components of the images” (see paragraph 38). Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Yasutomi Huang and Ceccaldi to reach the aforementioned advantage.
Re claim 28 Yatsutomi discloses wherein the neural network includes a first encoder coupled to at least one of the first decoder and the second decoder (see figure 2 note that encoder is coupled to the shade decoder and subject encoder). Yatsutomi does not expressly the encoder and decoder disclose coupled via a set of long-skip connections.
Ciccaldi discloses wherein the one or more neural networks include the encoder and decoders connected via a set of long-skip connections. (see paragraph 38 “In an embodiment, the encoder and decoder are symmetrical, using the same number of pooling (downsampling/upsampling) layers. The symmetrical structures provide for connections between encoding and decoding stages referred to as skip connections. The skip connections help against vanishing gradients and help maintain the high frequency components of the images”. Ceccaldi is in a similar art of segmentation using and decoder encoder architecture. One of ordinary skill in the art could have modified the encoder and decoders of Yatsutomi to include the long skip connection as described in Ceccaldi. The motivation to combine is to “maintain the high frequency components of the images” (see paragraph 38). Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Yasutomi Huang and Ceccaldi to reach the aforementioned advantage.
Re claim 34 Yatsutomi discloses wherein the neural network includes a first encoder coupled to at least one of the first decoder and the second decoder (see figure 2 note that encoder is coupled to the shade decoder and subject encoder). Yatsutomi does not expressly the encoder and decoder disclose coupled via a set of long-skip connections.
Ciccaldi discloses wherein the one or more neural networks include the encoder and decoders connected via a set of long-skip connections. (see paragraph 38 “In an embodiment, the encoder and decoder are symmetrical, using the same number of pooling (downsampling/upsampling) layers. The symmetrical structures provide for connections between encoding and decoding stages referred to as skip connections. The skip connections help against vanishing gradients and help maintain the high frequency components of the images”. Ceccaldi is in a similar art of segmentation using and decoder encoder architecture. One of ordinary skill in the art could have modified the encoder and decoders of Yatsutomi to include the long skip connection as described in Ceccaldi. The motivation to combine is to “maintain the high frequency components of the images” (see paragraph 38). Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Yasutomi Huang and Ceccaldi to reach the aforementioned advantage.
Re claim 43 Yatsutomi does not expressly wherein the neural network includes one or more long-skip connections.
Ciccaldi discloses wherein the neural network includes one or more long-skip connections. (see paragraph 38 “In an embodiment, the encoder and decoder are symmetrical, using the same number of pooling (downsampling/upsampling) layers. The symmetrical structures provide for connections between encoding and decoding stages referred to as skip connections. The skip connections help against vanishing gradients and help maintain the high frequency components of the images”. Ceccaldi is in a similar art of segmentation using and decoder encoder architecture. One of ordinary skill in the art could have modified the encoder and decoders of Yatsutomi to include the long skip connection as described in Ceccaldi. The motivation to combine is to “maintain the high frequency components of the images” (see paragraph 38). Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Yasutomi Huang and Ceccaldi to reach the aforementioned advantage.
Re claim 46 Yatsutomi does not expressly wherein the neural network includes one or more long-skip connections.
Ciccaldi discloses wherein the neural network includes one or more long-skip connections. (see paragraph 38 “In an embodiment, the encoder and decoder are symmetrical, using the same number of pooling (downsampling/upsampling) layers. The symmetrical structures provide for connections between encoding and decoding stages referred to as skip connections. The skip connections help against vanishing gradients and help maintain the high frequency components of the images”. Ceccaldi is in a similar art of segmentation using and decoder encoder architecture. One of ordinary skill in the art could have modified the encoder and decoders of Yatsutomi to include the long skip connection as described in Ceccaldi. The motivation to combine is to “maintain the high frequency components of the images” (see paragraph 38). Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Yasutomi Huang and Ceccaldi to reach the aforementioned advantage.
Re claim 48 Yatsutomi discloses wherein the neural network includes a first encoder coupled to at least one of the first decoder and the second decoder (see figure 2 note that encoder is coupled to the shade decoder and subject encoder). Yatsutomi does not expressly the encoder and decoder disclose coupled via a set of long-skip connections.
Ciccaldi discloses wherein the one or more neural networks include the encoder and decoders connected via a set of long-skip connections. (see paragraph 38 “In an embodiment, the encoder and decoder are symmetrical, using the same number of pooling (downsampling/upsampling) layers. The symmetrical structures provide for connections between encoding and decoding stages referred to as skip connections. The skip connections help against vanishing gradients and help maintain the high frequency components of the images”. Ceccaldi is in a similar art of segmentation using and decoder encoder architecture. One of ordinary skill in the art could have modified the encoder and decoders of Yatsutomi to include the long skip connection as described in Ceccaldi. The motivation to combine is to “maintain the high frequency components of the images” (see paragraph 38). Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Yasutomi Huang and Ceccaldi to reach the aforementioned advantage.
Allowable Subject Matter
Claim 26 and 30 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Re claim 26 Haung discloses Using an MLP to generate features and scale and shift parameters (See section 5.1) Huang does not expressly disclose “wherein the one or more normalization parameters comprise one or more scale and shift parameters that are to be generated using a multi-layer perceptron that is to receive the one or more features classified as common and the one or more features classified as uncommon”.
Re claim 30 the prior art of record does not disclose “wherein the one or more processors are further to use one or more encoders connected to a multilayer perceptron that is to adjust the one or more normalization parameters based, at least in part, on the one or more features classified as common and the one or more features classified as uncommon”
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SEAN T MOTSINGER whose telephone number is (571)270-1237. The examiner can normally be reached 9AM-5PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Chineyere Wills-Burns can be reached at (571) 272-9752. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SEAN T MOTSINGER/Primary Examiner, Art Unit 2673
Prosecution Insights