DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 9/2/2026 has been entered.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim 1 is rejected under 35 U.S.C. 103 as being unpatentable over Zhang et al. (US 2021/0264235) in view of Fu et al. (US 2019/0295302).
Regarding claim 1, Zhang teaches a processor, comprising:
one or more circuits to identify one or more objects in an input image by using one or more generative adversarial networks (GANs), the one or more circuits to:
use the one or more GANs to generate a synthetic version of the input image based, at least in part, on a latent code associated with the input image (see figure 2, para. 0051-0054, Zhang discusses GAN generating a synthetic image from a target image using latent vector data from the input image);
generate an updated latent code of the input image based, at least in part, on a similarity between the input image and the generated synthetic version of the input image (see figure 2, para. 0051-0054, Zhang discusses updating latent vector data based on the difference between the input image and generated synthetic image);
use the updated latent code as an input to the one or more GANs to generate one or more labels corresponding to the one or more objects within the synthetic version of the input image (see para. 0053, Zhang discusses generating a synthetic image with updated final latent vector and class vector).
Zhang does not expressly disclose use the updated latent code as an input to the one or more GANs to generate one or more pixel-level labels indicating one or more classifications for one or more pixels of the synthetic version of the input image corresponding to the one or more objects within the synthetic version of the input image.
However, Fu teaches use the updated latent code as an input to the one or more GANs to generate one or more pixel-level labels indicating one or more classifications for one or more pixels of the synthetic version of the input image corresponding to the one or more objects within the synthetic version of the input image (see figure 3, para . 0053-0054, Fu discusses classification results for pixels of a generated fake image; see para. 0076, 0182-0183, Fu discusses pixel semantic segmentation on both real images and generated fake images; see para. 0134, Fu discusses multi-label classification that provides attribute information).
Motivation to combine may be gleaned from the prior art considered. It would have been obvious before the effective filing date of the claimed invention to one of ordinary skill in the art to modify the invention of Zhang with Fu to derive at the invention of claim 1. The result would have been expected, routine, and predictable in order to perform generative adversarial network image generation.
The determination of obviousness is predicated upon the following: One skilled in the art would have been motivated to modify Zhang in this manner in order to improve image generation using a generative adversarial network to produce image data with object labels to properly identify object regions. Furthermore, the prior art collectively includes each element claimed (though not all in the same reference), and one of ordinary skill in the art could have combined the elements in this manner explained using known engineering design, interface and/or programming techniques, without changing a fundamental operating principle of Zhang, while the teaching of Fu continues to perform the same function as originally taught prior to being combined, in order to produce the repeatable and predictable result of labeling objects in a synthetic image generated by a GAN to properly classify image regions. The Zhang and Fu systems perform image generation using a generative adversarial network ("GAN"). It is for at least the aforementioned reasons that the examiner has reached a conclusion of obviousness with respect to the claim in question.
Claims 2-6 are rejected under 35 U.S.C. 103 as being unpatentable over Zhang et al. (US 2021/0264235) in view of Fu et al. (US 2019/0295302) in view of Fang (US 2019/0251612).
Regarding claim 2, Zhang and Fu do not expressly disclose wherein to generate the synthetic version of the input image, a generator network of the GAN is to: determine an optimized latent code that, when input into the generator network, causes the generator network to generate the synthetic version of the input image.
However, Fang teaches wherein to generate the synthetic version of the input image, a generator network of the GAN is to: determine an optimized latent code (see para. 0123, 0203, Fang discusses optimized latent code generation) that, when input into the generator network, causes the generator network to generate the synthetic version of the input image (see para. 0123, 0136-0137, Fang discusses GAN-generated synthesized images).
Motivation to combine may be gleaned from the prior art considered. It would have been obvious before the effective filing date of the claimed invention to one of ordinary skill in the art to modify the invention of Zhang and Fu with Fang to derive at the invention of claim 2. The result would have been expected, routine, and predictable in order to perform generative adversarial network image generation.
The determination of obviousness is predicated upon the following: One skilled in the art would have been motivated to modify Zhang and Fu in this manner in order to improve image generation using a generative adversarial network to produce image data with object labels to properly identify object regions. Furthermore, the prior art collectively includes each element claimed (though not all in the same reference), and one of ordinary skill in the art could have combined the elements in this manner explained using known engineering design, interface and/or programming techniques, without changing a fundamental operating principle of Zhang and Fu, while the teaching of Fang continues to perform the same function as originally taught prior to being combined, in order to produce the repeatable and predictable result of labeling objects in a synthetic image generated by a GAN to properly classify image regions. The Zhang, Fu, and Fang systems perform image generation using a generative adversarial network ("GAN"). It is for at least the aforementioned reasons that the examiner has reached a conclusion of obviousness with respect to the claim in question.
Regarding claim 3, Fang teaches wherein the optimized latent code is determined using an inverse optimization process (see para. 0123, 0203, Fang discusses optimized latent code generation; see para. 0149-0150, 0153-0155, Fang discusses generating latent code and performing back-propagation (inverse optimization) to minimize the loss function).
The same motivation of claim 2 is applied to claim 3. Motivation to combine may be gleaned from the prior art considered. It would have been obvious before the effective filing date of the claimed invention to one of ordinary skill in the art to modify the invention of Zhang and Fu with Fang to derive at the invention of claim 3. The result would have been expected, routine, and predictable in order to perform generative adversarial network image generation.
Regarding claim 4, Fang teaches wherein to use the inverse optimization process the processor is to perform one or more inverse optimization cycles, wherein each inverse optimization cycle comprises: using a latent code to generate a version of the input image; determining differences between the version and the input image; and determining a new latent code based on the differences (see para. 0062, Fang discusses the error loss in the feedback indicates how different the latent visual features of the synthesized image are from that of real images), wherein the new latent code is usable for a subsequent inverse optimization cycle (see para. 0062, Fang discusses generator uses the error loss to tune weights and parameters (e.g., learn) such that the generator generates realistic synthesized images; see para 0119-0123, Fang discusses latent code optimization).
The same motivation of claim 2 is applied to claim 4. Motivation to combine may be gleaned from the prior art considered. It would have been obvious before the effective filing date of the claimed invention to one of ordinary skill in the art to modify the invention of Zhang and Fu with Fang to derive at the invention of claim 4. The result would have been expected, routine, and predictable in order to perform generative adversarial network image generation.
Regarding claim 5, Fang teaches wherein responsive to determining that the similarity between the input image and the synthetic version of the input image reaches a threshold, the processor is to designate the new latent code as the optimized latent code (see para. 0149-0150, 0153-0155, Fang discusses generating latent code and performing back-propagation (inverse optimization) to minimize the loss function; see para . 0177, Fang discusses GAN 710 learns and uses latent code used as input to generate synthesized images via the generator, which satisfy a realness threshold determined by the discriminator).
The same motivation of claim 2 is applied to claim 5. Motivation to combine may be gleaned from the prior art considered. It would have been obvious before the effective filing date of the claimed invention to one of ordinary skill in the art to modify the invention of Zhang and Fu with Fang to derive at the invention of claim 5. The result would have been expected, routine, and predictable in order to perform generative adversarial network image generation.
Regarding claim 6, Fang teaches wherein the generator network of the GAN is further to: use the optimized latent code (see para. 0123, Fang discusses optimized latent code generation) as an input to generate the synthetic version of the input image (see para. 0123, Fang discusses GAN-generated synthesized images).
Fu teaches and the one or more labels corresponding to the one or more objects within the synthetic version of the input image (see figure 3, para . 0053-0054, Fu discusses classification results for pixels of a generated fake image; see para. 0076, 0182-0183, Fu discusses pixel semantic segmentation on both real images and generated fake images; see para. 0134, Fu discusses multi-label classification that provides attribute information).
The same motivation of claim 2 is applied to claim 6. Motivation to combine may be gleaned from the prior art considered. It would have been obvious before the effective filing date of the claimed invention to one of ordinary skill in the art to modify the invention of Zhang and Fu with Fang to derive at the invention of claim 6. The result would have been expected, routine, and predictable in order to perform generative adversarial network image generation.
Claim 7 is rejected under 35 U.S.C. 103 as being unpatentable over Zhang et al. (US 2021/0264235) in view of Fu et al. (US 2019/0295302) in view of Park et al. (US 2020/0193269).
Regarding claim 7, Zhang and Fu do not expressly disclose wherein each GAN of the one or more GAN s comprises a generator network and each of the first and second discriminators, wherein the first discriminator takes as an input the synthetic version of the input image and outputs a first score for the synthetic version of the input image, wherein the second discriminator takes as a first input the synthetic version of the input image and as a second input a generated label associated with the synthetic version of the input image, and wherein the second discriminator outputs a second score for the synthetic version of the input image and the generated label.
However, Park teaches wherein each GAN of the one or more GAN s comprises a generator network and each of the first and second discriminators, wherein the first discriminator takes as an input the synthetic version of the input image and outputs a first score for the synthetic version of the input image, wherein the second discriminator takes as a first input the synthetic version of the input image and as a second input a generated label associated with the synthetic version of the input image, and wherein the second discriminator outputs a second score for the synthetic version of the input image and the generated label (see figure 1, 0058-0067, Park discusses a GAN with two discriminators to determine an accuracy of object synthetic images and an accuracy of generated labels).
Motivation to combine may be gleaned from the prior art considered. It would have been obvious before the effective filing date of the claimed invention to one of ordinary skill in the art to modify the invention of Zhang and Fu with Park to derive at the invention of claim 7. The result would have been expected, routine, and predictable in order to perform generative adversarial network image generation.
The determination of obviousness is predicated upon the following: One skilled in the art would have been motivated to modify Zhang and Fu in this manner in order to improve image generation using a generative adversarial network to produce image data with object labels to properly identify object regions. Furthermore, the prior art collectively includes each element claimed (though not all in the same reference), and one of ordinary skill in the art could have combined the elements in this manner explained using known engineering design, interface and/or programming techniques, without changing a fundamental operating principle of Zhang and Fu, while the teaching of Park continues to perform the same function as originally taught prior to being combined, in order to produce the repeatable and predictable result of labeling objects in a synthetic image generated by a GAN to properly classify image regions. The Zhang, Fu, and Park systems perform image generation using a generative adversarial network ("GAN"). It is for at least the aforementioned reasons that the examiner has reached a conclusion of obviousness with respect to the claim in question.
Claims 8, 9, 19, 20 are rejected under 35 U.S.C. 103 as being unpatentable over Zhang et al. (US 2021/0264235) in view of Park et al. (US 2020/0193269) in view of Fu et al. (US 2019/0295302).
Regarding claim 8, Zhang teaches processor, comprising: one or more circuits to train one or more generative adversarial networks (GAN)s to: generate a synthetic version of an input image based, at least in part, on a latent code associated with the input image (see para. 0022-0024, 0051-0054, Zhang discusses generating a synthetic image with latent vector and class vector), and
the updated latent code generated based, at least in part, on a similarity between the input image and the generated synthetic version of the input image (see figure 2, para. 0051-0054, Zhang discusses updating latent vector data based on the difference between the input image and generated synthetic image).
Zhang does not expressly disclose use an updated latent code to generate one or more pixel-level labels indicating one or more classifications for one or more pixels of the synthetic version of the input image corresponding to one or more objects within the synthetic version of the input image, wherein the one or more GAN s are trained using a training dataset comprising a plurality of images and a plurality of labels corresponding to at least some of the plurality of images, and wherein each GAN of the one or more GAN s comprises a generator network and two discriminator networks.
However, Park teaches wherein the one or more GAN s are trained using a training dataset comprising a plurality of images and a plurality of labels corresponding to at least some of the plurality of images, and wherein each GAN of the one or more GANs comprises a generator network and two discriminator networks (see figure 1, 0058-0067, Park discusses a GAN with two discriminators to determine an accuracy of object synthetic images and an accuracy of generated labels).
Motivation to combine may be gleaned from the prior art considered. It would have been obvious before the effective filing date of the claimed invention to one of ordinary skill in the art to modify the invention of Zhang with Park to derive at the invention of claim 8. The result would have been expected, routine, and predictable in order to perform generative adversarial network image generation.
The determination of obviousness is predicated upon the following: One skilled in the art would have been motivated to modify Zhang in this manner in order to improve image generation using a generative adversarial network to produce image data with object labels to properly identify object regions. Furthermore, the prior art collectively includes each element claimed (though not all in the same reference), and one of ordinary skill in the art could have combined the elements in this manner explained using known engineering design, interface and/or programming techniques, without changing a fundamental operating principle of Zhang, while the teaching of Park continues to perform the same function as originally taught prior to being combined, in order to produce the repeatable and predictable result of labeling objects in a synthetic image generated by a GAN to properly classify image regions. The Zhang and Park systems perform image generation using a generative adversarial network ("GAN"). It is for at least the aforementioned reasons that the examiner has reached a conclusion of obviousness with respect to the claim in question.
Fu teaches use an updated latent code to generate one or more pixel-level labels indicating one or more classifications for one or more pixels of the synthetic version of the input image corresponding to one or more objects within the synthetic version of the input image (see figure 3, para . 0053-0054, Fu discusses classification results for pixels of a generated fake image; see para. 0076, 0182-0183, Fu discusses pixel semantic segmentation on both real images and generated fake images; see para. 0134, Fu discusses multi-label classification that provides attribute information).
Motivation to combine may be gleaned from the prior art considered. It would have been obvious before the effective filing date of the claimed invention to one of ordinary skill in the art to modify the invention of Zhang and Park with Fu to derive at the invention of claim 8. The result would have been expected, routine, and predictable in order to perform generative adversarial network image generation.
The determination of obviousness is predicated upon the following: One skilled in the art would have been motivated to modify Zhang and Park in this manner in order to improve image generation using a generative adversarial network to produce image data with object labels to properly identify object regions. Furthermore, the prior art collectively includes each element claimed (though not all in the same reference), and one of ordinary skill in the art could have combined the elements in this manner explained using known engineering design, interface and/or programming techniques, without changing a fundamental operating principle of Zhang and Park, while the teaching of Fu continues to perform the same function as originally taught prior to being combined, in order to produce the repeatable and predictable result of labeling objects in a synthetic image generated by a GAN to properly classify image regions. The Zhang, Park, and Fu systems perform image generation using a generative adversarial network ("GAN"). It is for at least the aforementioned reasons that the examiner has reached a conclusion of obviousness with respect to the claim in question.
Regarding claim 9, Park teaches wherein during training: a first discriminator network of the two discriminator networks is to: receive a plurality of synthetic images generated by the generator network (see figure 1, 0058-0067, Park discusses a GAN with two discriminators to determine an accuracy of object synthetic images and an accuracy of generated labels); and
determine a respective first score for each respective synthetic image of the plurality of synthetic images, wherein the respective first score is indicative of an extent to which the respective synthetic image resembles a real image (see figure 1, 0058-0067, Park discusses a GAN with two discriminators to determine an accuracy of object synthetic images and an accuracy of generated labels);
a second discriminator network of the two discriminator networks is to: receive a plurality of pairs of a synthetic image and corresponding synthetic labels for the synthetic image (see figure 1, 0058-0067, Park discusses a GAN with two discriminators to determine an accuracy of object synthetic images and an accuracy of generated labels); and
determine a respective second score for each pair of the plurality of pairs of the synthetic image and the corresponding synthetic labels, wherein the respective second score for a pair is indicative of an extent to which a) the synthetic image in the pair resembles a real image and an extent to which the synthetic labels in the pair resemble real labels (see figure 1, 0058-0067, Park discusses a GAN with two discriminators to determine an accuracy of object synthetic images and an accuracy of generated labels).
The same motivation of claim 8 is applied to claim 9. Motivation to combine may be gleaned from the prior art considered. It would have been obvious before the effective filing date of the claimed invention to one of ordinary skill in the art to modify the invention of Zhang and Park with Fu to derive at the invention of claim 9. The result would have been expected, routine, and predictable in order to perform generative adversarial network image generation.
Claim 19 is rejected as applied to claim 8 as pertaining to a corresponding system.
Claim 20 is rejected as applied to claim 9 as pertaining to a corresponding system.
Claims 10, 21 are rejected under 35 U.S.C. 103 as being unpatentable over Zhang et al. (US 2021/0264235) in view of Park et al. (US 2020/0193269) in view of Fu et al. (US 2019/0295302) in view of Yao et al. (US 2021/0201078).
Regarding claim 10, Zhang, Park, and Fu do not expressly disclose wherein the training dataset comprises a first quantity of images that lack labels and a second quantity of images that have pixel-level labels, wherein the first quantity is greater than the second quantity. However, Yao teaches wherein the training dataset comprises a first quantity of images that lack labels (see para. 0199, Yao discusses “FIGS. 16A-16B illustrate exemplary embodiments of using a modified generative adversarial network (GAN) which is trained on … and unlabeled real images”), and a second quantity of images that have pixel-level labels, wherein the first quantity is greater than the second quantity (see para. 0199, Yao discusses “FIGS. 16A-16B illustrate exemplary embodiments of using a modified generative adversarial network (GAN) which is trained on labeled synthetic images.”; pixel level).
Motivation to combine may be gleaned from the prior art considered. It would have been obvious before the effective filing date of the claimed invention to one of ordinary skill in the art to modify the invention of Zhang, Park, and Fu with Yao to derive at the invention of claim 10. The result would have been expected, routine, and predictable in order to perform generative adversarial network image generation.
The determination of obviousness is predicated upon the following: One skilled in the art would have been motivated to modify Zhang, Park, and Fu in this manner in order to improve image generation using a generative adversarial network to produce image data with object labels to properly identify object regions. Furthermore, the prior art collectively includes each element claimed (though not all in the same reference), and one of ordinary skill in the art could have combined the elements in this manner explained using known engineering design, interface and/or programming techniques, without changing a fundamental operating principle of Zhang, Park, and Fu, while the teaching of Yao continues to perform the same function as originally taught prior to being combined, in order to produce the repeatable and predictable result of labeling objects in a synthetic image generated by a GAN to properly classify image regions. The Zhang, Park, Fu, and Yao systems perform image generation using a generative adversarial network ("GAN"). It is for at least the aforementioned reasons that the examiner has reached a conclusion of obviousness with respect to the claim in question.
Claim 21 is rejected as applied to claim 10 as pertaining to a corresponding system.
Claims 11, 22 are rejected under 35 U.S.C. 103 as being unpatentable over Zhang et al. (US 2021/0264235) in view of Park et al. (US 2020/0193269) in view of Fu et al. (US 2019/0295302) in view of Fang (US 2019/0251612).
Regarding claim 11, Zhang, Park, and Fu do not expressly disclose wherein the trained one or more GANs are trained to perform operations comprising: determining an optimized latent code that, when input into the generator network, causes the generator network to generate the synthetic version of the input image, wherein the optimized latent code is determined using an inverse optimization process, and wherein to use the inverse optimization process the processor is to perform one or more inverse optimization cycles, wherein each inverse optimization cycle comprises: using a latent code to generate a version of the input image; determining differences between the version and the input image; and determining a new latent code based on the differences, wherein the new latent code is usable for a subsequent inverse optimization cycle.
However, Fang teaches wherein the trained one or more GANs are trained to perform operations comprising: determining an optimized latent code that, when input into the generator network, causes the generator network to generate the synthetic version of the input image, wherein the optimized latent code is determined using an inverse optimization process, and wherein to use the inverse optimization process the processor is to perform one or more inverse optimization cycles (see para. 0201, Fang discusses training the generator neural network and the discriminator neural network using objective functions via back propagation (inverse optimization cycles) and least squares loss), wherein each inverse optimization cycle comprises: using a latent code to generate a version of the input image (see para. 0200, 0202, Fang discusses latent code of images)
determining differences between the version and the input image; and determining a new latent code based on the differences, wherein the new latent code is usable for a subsequent inverse optimization cycle (see para. 0201, Fang discusses training the generator neural network and the discriminator neural network using objective functions via back propagation (inverse optimization cycles) and least squares loss. The least square loss is the difference between synthetic version of the input image and input image. Back propagation in GANs iteratively updates the latent code to minimize loss function).
Motivation to combine may be gleaned from the prior art considered. It would have been obvious before the effective filing date of the claimed invention to one of ordinary skill in the art to modify the invention of Zhang, Park, and Fu with Fang to derive at the invention of claim 11. The result would have been expected, routine, and predictable in order to perform generative adversarial network image generation.
The determination of obviousness is predicated upon the following: One skilled in the art would have been motivated to modify Zhang, Park, and Fu in this manner in order to improve image generation using a generative adversarial network to produce image data with object labels to properly identify object regions. Furthermore, the prior art collectively includes each element claimed (though not all in the same reference), and one of ordinary skill in the art could have combined the elements in this manner explained using known engineering design, interface and/or programming techniques, without changing a fundamental operating principle of Zhang, Park, and Fu, while the teaching of Fang continues to perform the same function as originally taught prior to being combined, in order to produce the repeatable and predictable result of labeling objects in a synthetic image generated by a GAN to properly classify image regions. The Zhang, Park, Fu, and Fang systems perform image generation using a generative adversarial network ("GAN"). It is for at least the aforementioned reasons that the examiner has reached a conclusion of obviousness with respect to the claim in question.
Claim 22 is rejected as applied to claim 11 as pertaining to a corresponding system.
Claims 12, 18 are rejected under 35 U.S.C. 103 as being unpatentable over Zhang et al. (US 2021/0264235) in view of Kearney et al. (US 2020/0405242) in view of Fu et al. (US 2019/0295302).
Regarding claim 12, Zhang teaches a method, comprising: identifying one or more objects in an input medical image, comprising: using one or more generative adversarial networks (GANs) to generate a synthetic version of the input medical image based, at least in part, on a latent code associated with the input image (see figure 2, para. 0051-0054, Zhang discusses updating latent vector data based on the difference between the input image and generated synthetic image);
generating an updated latent code of the input medical image based, at least in part, on a similarity between the input medical image and the generated synthetic version of the input (medical) image (see figure 2, para. 0051-0054, Zhang discusses updating latent vector data based on the difference between the input image and generated synthetic image); and
Zhang does not disclose identifying one or more objects in an input medical image, comprising: an input medical image into a one or more generative adversarial networks (GANs) to generate a synthetic version of the input medical; using the updated latent code as an input to the one or more GANs to generate one or more pixel-level labels indicating one or more classifications for one or more pixels of the synthetic version of the input medical image corresponding to the one or more objects within the synthetic version of the input medical image.
However, Kearney teaches identifying one or more objects in an input medical image, comprising: an input medical image into a one or more generative adversarial networks (GANs) to generate a synthetic version of the input medical (see figure 1, figure 11, para. 0137, 0157, Kearney discusses input dental and bone images into a GAN).
Motivation to combine may be gleaned from the prior art considered. It would have been obvious before the effective filing date of the claimed invention to one of ordinary skill in the art to modify the invention of Zhang with Kearney to derive at the invention of claim 12. The result would have been expected, routine, and predictable in order to perform generative adversarial network image generation.
The determination of obviousness is predicated upon the following: One skilled in the art would have been motivated to modify Zhang in this manner in order to improve image generation using a generative adversarial network to produce image data with object labels to properly identify object regions. Furthermore, the prior art collectively includes each element claimed (though not all in the same reference), and one of ordinary skill in the art could have combined the elements in this manner explained using known engineering design, interface and/or programming techniques, without changing a fundamental operating principle of Zhang, while the teaching of Kearney continues to perform the same function as originally taught prior to being combined, in order to produce the repeatable and predictable result of labeling objects in a synthetic image generated by a GAN to properly classify image regions. The Zhang and Kearney systems perform image generation using a generative adversarial network ("GAN"). It is for at least the aforementioned reasons that the examiner has reached a conclusion of obviousness with respect to the claim in question.
Fu teaches using the updated latent code as an input to the one or more GANs to generate one or more pixel-level labels indicating one or more classifications for one or more pixels of the synthetic version of the input medical image corresponding to the one or more objects within the synthetic version of the input medical image (see figure 3, para . 0053-0054, Fu discusses classification results for pixels of a generated fake image; see para. 0076, 0182-0183, Fu discusses pixel semantic segmentation on both real images and generated fake images; see para. 0134, Fu discusses multi-label classification that provides attribute information).
Motivation to combine may be gleaned from the prior art considered. It would have been obvious before the effective filing date of the claimed invention to one of ordinary skill in the art to modify the invention of Zhang and Kearney with Fu to derive at the invention of claim 12. The result would have been expected, routine, and predictable in order to perform generative adversarial network image generation.
The determination of obviousness is predicated upon the following: One skilled in the art would have been motivated to modify Zhang and Kearney in this manner in order to improve image generation using a generative adversarial network to produce image data with object labels to properly identify object regions. Furthermore, the prior art collectively includes each element claimed (though not all in the same reference), and one of ordinary skill in the art could have combined the elements in this manner explained using known engineering design, interface and/or programming techniques, without changing a fundamental operating principle of Zhang and Kearney, while the teaching of Fu continues to perform the same function as originally taught prior to being combined, in order to produce the repeatable and predictable result of labeling objects in a synthetic image generated by a GAN to properly classify image regions. The Zhang, Kearney, and Fu systems perform image generation using a generative adversarial network ("GAN"). It is for at least the aforementioned reasons that the examiner has reached a conclusion of obviousness with respect to the claim in question.
Claim 18 is rejected as applied to claim 7 as pertaining to a corresponding method processing a medical image.
Claims 13-17 are rejected under 35 U.S.C. 103 as being unpatentable over Zhang et al. (US 2021/0264235) in view of Kearney et al. (US 2020/0405242) in view of Fu et al. (US 2019/0295302) in view of Fang (US 2019/0251612).
Regarding claim 13, Kearney teaches wherein to generate the synthetic version of the input medical image, a generator network of the GAN is to: causes the generator network to generate the synthetic version of the input medical image (see figure 1, figure 11, para. 0137, 0157, Kearney discusses input dental and bone images into a GAN).
Zhang, Kearney, and Fu do not expressly disclose determine an optimized latent code that, when input into the generator network. However, Fang teaches determine an optimized latent code that, when input into the generator network(see para. 0123, 0203, Fang discusses optimized latent code generation).
Motivation to combine may be gleaned from the prior art considered. It would have been obvious before the effective filing date of the claimed invention to one of ordinary skill in the art to modify the invention of Zhang, Kearney, and Fu with Fang to derive at the invention of claim 13. The result would have been expected, routine, and predictable in order to perform generative adversarial network image generation.
The determination of obviousness is predicated upon the following: One skilled in the art would have been motivated to modify Zhang, Kearney, and Fu in this manner in order to improve image generation using a generative adversarial network to produce image data with object labels to properly identify object regions. Furthermore, the prior art collectively includes each element claimed (though not all in the same reference), and one of ordinary skill in the art could have combined the elements in this manner explained using known engineering design, interface and/or programming techniques, without changing a fundamental operating principle of Zhang, Kearney, and Fu, while the teaching of Fang continues to perform the same function as originally taught prior to being combined, in order to produce the repeatable and predictable result of labeling objects in a synthetic image generated by a GAN to properly classify image regions. The Zhang, Kearney, Fu, and Fang systems perform image generation using a generative adversarial network ("GAN"). It is for at least the aforementioned reasons that the examiner has reached a conclusion of obviousness with respect to the claim in question.
Claim 14 is rejected as applied to claim 3 as pertaining to a corresponding method.
Claim 15 is rejected as applied to claim 4 as pertaining to a corresponding method.
Claim 16 is rejected as applied to claim 5 as pertaining to a corresponding method.
Regarding claim 17, Kearney and Fang teach wherein the generator network of the GAN is further to: use the optimized latent code as an input to generate the synthetic version of the input medical image (see figure 1, figure 11, para. 0137, 0157, Kearney discusses input dental and bone images into a GAN) and the one or more pixel-level labels corresponding to the one or more objects within the synthetic version of the input medical image (see para. 0123, Fang discusses optimized latent code generation).
The same motivation of claim 13 is applied to claim 17. Motivation to combine may be gleaned from the prior art considered. It would have been obvious before the effective filing date of the claimed invention to one of ordinary skill in the art to modify the invention of Zhang, Kearney, and Fu with Fang to derive at the invention of claim 17. The result would have been expected, routine, and predictable in order to perform generative adversarial network image generation.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Zhu et al. (US 2022/0335689) discusses training of GANs through the optimization of noisy Z-vector in the latent space.
Lai et al. (US 11,315,231) discusses calculating a second loss function according to the pixel value difference between the generated image and the test image, and updating the test latent vector of the test image using the stochastic gradient descent (SGD).
Munoz Delgado (US 2021/0125061) discusses computing gradients with respect to an input code z for a loss (objective) function to be optimized, e.g. to be maximized or minimized.
Contact Information
Any inquiry concerning this communication or earlier communications from the examiner should be directed to KENNY A CESE whose telephone number is (571) 270-1896. The examiner can normally be reached on Monday – Friday, 9am – 4pm.
If attempts to reach the primary examiner by telephone are unsuccessful, the examiner’s supervisor, Gregory Morse can be reached on (571) 272-3838. The fax phone number for the organization where this application or proceeding is assigned is (571) 273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Kenny A Cese/
Primary Examiner, Art Unit 2663
