DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 6/17/2026 has been entered.
Response to Arguments
Applicant's arguments, in the amendment filed 6/17/2026 with respect to the rejections of Claim(s) 1,7,13,19 and 25 is/are rejected under 35 U.S.C. 103 as being unpatentable over Takahashi et al. (US 11,686,721 B2) in view of Tan et al ("ArtGAN: Artwork Synthesis with Conditional Categorical GANs ") have been fully considered but are moot in view of the new ground(s) of rejection necessitated by the amendments. Therefore, the rejection has been withdrawn. However, upon further consideration, a new ground(s) of rejection is made in view of Feng et al (Disaasembling object representation without labels)
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1,7,13,19 and 25 is/are rejected under 35 U.S.C. 103 as being unpatentable over Feng et al (Disaasembling object representation without labels) in view of Tan et al ("ArtGAN: Artwork Synthesis with Conditional Categorical GANs ").
Regarding claim 1, Feng et al teaches the one or more processors, comprising: circuitry to use one or more neural networks to: receive one or more reference images depicting one or more objects and one or more first images(input image I, section 4.1 and figure 2);
encode, using one or more variational autoencoders (VAEs) trained to encode image features of different classes of the one or more objects into a latent space (we adopt a double AE architecture, including an Image Reconstruction AE (IR-AE) and a Representation Reconstruction AE (RR-AE), shown in Fig. 2. The IR-AE is composed of an encoder f / and a decoder fw. Through reconstructing the input image I, the IR-AE ensures that the latent representation R contains all the features of the input image. With R as input, the RR-AE recon structs the representation R0, which enhances the consistency between the input image I and the representation R, section 4.1 ) , one or more object features for the one or more objects depicted in the reference images into the latent space (section 4.2 The features of the objects are still intertwined upon the deriva tion of the initial representation R. We thus propose the Fuzzy Classification component to disassemble each object’s features into different parts of the representation.);
and modify, using a generative adversarial network (GAN) from the one or more neural networks, the one or more first images to remove, from the one or more first images, at least one depiction of at least one object corresponding to the object classifications based, at least in part, on the latent space as a constraint, wherein the GAN is to perform inpainting for a region of the one or more first images previously corresponding to the at least one depiction of at least one object (The framework of UDOR. It comprises three components: double AutoEncoder (AE), Fuzzy Classification, and Object-removing Operation. f/;fw;fh;fu and Pd denote the encoder, decoder, discriminator, classifier, and original data distribution, respectively. The double AE includes an Image Reconstruction AE (IR-AE) and a Representation Reconstruction AE (RR-AE), which are used to reconstruct the input image I and the latent representation R ( r1;r2;...;rk;...;rn ½ ), respectively. The Fuzzy Classification constrains that the k-th part of the representation, denoted by rk, only contain features of k-th object category (label lk) or the empty features (label l0). Here, n denotes the number of object categories in the dataset. The Object-removing Operation replaces a randomly-selected part rk to an empty vector r0 k . The same RR-AE is adopted to reconstruct the object-removed representation R0 ( r1;r2;...;r0 k ;...;rn ), and the WGAN-GP [6] is adopted to enhance the quality of the generated image I00, figure 2) .
While Feng meets the limitations above Feng fails to teach “ determine object classifications for the one or more objects based, at least in part, on the encoded one or more object features represented in the latent space”
However , Tan et al teaches generate object classifications indicating one or more object features for the one or more objects depicted in the reference images Section 2.3, Paragraph 1 and Figure 2 shows the discriminator network can consume the output image x and run it through the classifier clsNet).Tan teaches in Paragraph 2 shows computing a discriminator loss LD through equation 2. It also mentions updating parameters in the discriminator, D, based on the discriminator loss. Tan further teaches the method further comprising: tuning the GAN by using the latent vector as a pivot for a latent space of the GAN (Section 2.3, Paragraph 3, Figure 2 shows the input image's latent feature Z and the latent vector, output from the zNet, being passed into the decoder phase of the GAN to train the GAN. This can be considered using the latent vector as a pivot).It would have been obvious to one skilled in the art before filing of the claimed invention to depict a classification from the one or more first images in order to learn faster and achieve better generated image quality. Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
b. Regarding claim 7, claim 7 is analogous and corresponds to claim 1. See rejection of claim 1 for further explanation.
c. Regarding claim 13, claim 13 is analogous and corresponds to claim 1. See rejection of claim 1 for further explanation.
d. Regarding claim 19, claim 19 is analogous and corresponds to claim 1. See rejection of claim 1 for further explanation.
e. Regarding claim 25, claim 25 is analogous and corresponds to claim 1. See rejection of claim 1 for further explanation.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 2-4, 8-10, 14-16, 20-22, and 26-28 are rejected under 35 U.S.C. 103 as being unpatentable over Feng et al (Disaasembling object representation without labels) in view of Tan et al ("ArtGAN: Artwork Synthesis with Conditional Categorical GANs "), and further in view of Bao et al. (“CVAE-GAN: Fine-Grained Image Generation through Asymmetric Training”).
a. Regarding claim 12, Feng discloses wherein the one or more neural networks.
However, Feng does not explicitly disclose gating network to select the one or more VAEs from a plurality of variational autoencoders (VAEs)Bao discloses wherein the one or more neural networks 2include a plurality of variational autoencoders (VAEs) trained to encode image features of 3different classes of objects into a latent space (Bao discloses that “[t]he function of networks E and G is the same as that in conditional variational auto-encoder (CVAE) [34]. The encoder network E maps the data sample x to a latent representation z through a learned distribution P (z|x, c), where c is the category of the data” at chapter 3).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to utilize CVAE of Bao to Fengs’s neural network.
The suggestion/motivation would have been to “[estimate] a good representation of the input image, and the generated image appears to be more realistic” (Bao; chapter 1).
b. Regarding claim 13, the combination applied in claim 2 discloses wherein the one or more neural networks 2further include a gating network to select one of the VAEs to encode image features of the one or more objects into the latent space (Bao discloses that “[t]he function of networks E and G is the same as that in conditional variational auto-encoder (CVAE) [34]. The encoder network E maps the data sample x to a latent representation z through a learned distribution P (z|x, c), where c is the category of the data” at chapter 3).
c. Regarding claim 14, the combination applied in claim 2 discloses wherein the one or more neural networks 2further include a generative adversarial network (GAN) for generating an output image 3based on image content of the one or more first image and using the latent space as a constraint to cause 4the output image to not include image content corresponding to the one or more object, wherein the 5GAN is to perform inpainting for a region of the one or more first image previously corresponding to the one or more 6objects (Bao discloses that “[t]he generative network G generates image x’ by sampling from a learned distribution P (x|z, c). The function of network G and D is the same as that in the generative adversarial network (GAN). The network G tries to learn the real data distribution by the gradients given by the discriminative network D which learns to distinguish between “real” and “fake” samples. The function of network C is to measure the posterior P (c|x)” at chapter 3).
d. Regarding claims 8-10, claims 8-10 are analogous and correspond to claims 2-4, respectively. See rejection of claims 2-4 for further explanation.
e. Regarding claims 14-16, claims 14-16 are analogous and correspond to claims 2-4, respectively. See rejection of claims 2-4 for further explanation.
f. Regarding claims 20-22, claims 20-22 are analogous and correspond to claims 2-4, respectively. See rejection of claims 2-4 for further explanation.
g. Regarding claims 26-28, claims 26-28 are analogous and correspond to claims 2-4, respectively. See rejection of claims 2-4 for further explanation.
Claims 5-6, 11-12, 17-18, 23-24, and 29-30 are rejected under 35 U.S.C. 103 as being unpatentable over Feng et al (Disaasembling object representation without labels) in view of Tan et al ("ArtGAN: Artwork Synthesis with Conditional Categorical GANs "), and further in view of Huang et al. (US 2021/0048881 A1).
a. Regarding claim 5, Feng discloses wherein the one or more neural networks (fully neural network at Fig. 6, col. 12, lines 39-55).
However, Feng does not explicitly disclose herein the one or more objects features are depicted in the one or more objects from the one or more first images and are absent in the modified one or more first images .
Huang discloses further to detect one or more anomalies in the one or more first images after the one or more first objects are removed and cause the one or more first images to be regenerated to attempt to remove the one or more anomalies (Huang discloses that “the processor removes a specific object in the training screens to generate a plurality of preprocessed training images. Here, the manner in which the processor 130 removes the specific object from the training screens is the same as the manner in which the processor 130 removes the specific object from the first person view screen in step S202. In other words, the processor 130 can also respectively cut the training screens into a plurality of sub training screens and take at least one sub training screen of each training screen to generate a plurality of preprocessed training images” at Fig. 7-S702 and ¶ 0039. Here it is inherent and readily apparent that S702 would have to erase the one or more first objects from the one or more first images).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to removing process of Huang to Feng ’s neural network.
The suggestion/motivation would have been to enable “the recognition accuracy of the neural network model can be significantly improved” (Huang; ¶ 0027) and “improve a processing efficiency” (Huang; ¶ 0034).
b. Regarding claim 16, Takahashi discloses wherein the one or more neural networks (fully neural network at Fig. 6, col. 12, lines 39-55).
However, Feng does not explicitly disclose further to detect one or more instances of an object in the one or more first images after removal of the at least one depiction of the at least one object corresponding to the object classifications and cause the one or more first images to be regenerated to attempt to remove the one or more instances.
Huang discloses wherein the one or more neural networks are further to detect one or more instances of an object in the one or more first images after removal of the one or more first objects and cause the one or more first images to be regenerated to attempt to remove the one or more instances (Huang discloses that “the processor removes a specific object in the training screens to generate a plurality of preprocessed training images. Here, the manner in which the processor 130 removes the specific object from the training screens is the same as the manner in which the processor 130 removes the specific object from the first person view screen in step S202. In other words, the processor 130 can also respectively cut the training screens into a plurality of sub training screens and take at least one sub training screen of each training screen to generate a plurality of preprocessed training images” at Fig. 7-S702 and ¶ 0039).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to removing process of Huang to Feng’s neural network.
The suggestion/motivation would have been to enable “the recognition accuracy of the neural network model can be significantly improved” (Huang; ¶ 0027) and “improve a processing efficiency” (Huang; ¶ 0034).
c. Regarding claims 11-12, claims 11-12 are analogous and correspond to claims 5-6, respectively. See rejection of claims 5-6 for further explanation.
d. Regarding claims 17-18, claims 17-18 are analogous and correspond to claims 5-6, respectively. See rejection of claims 5-6 for further explanation.
e. Regarding claims 23-24, claims 23-24 are analogous and correspond to claims 5-6, respectively. See rejection of claims 5-6 for further explanation.
f. Regarding claims 29-30, claims 29-30 are analogous and correspond to claims 5-6, respectively. See rejection of claims 5-6 for further explanation.
Conclusion
Following is a list of references pertinent to the claimed invention:
Frolova et al. (US 2021/0081754 A1): Systems and methods are disclosed for error correction in convolutional neural networks. In one implementation, a first image is received. A first activation map is generated with respect to the first image within a first layer of the convolutional neural network. A correlation is computed between data reflected in the first activation map and data reflected in a second activation map associated with a second image. Based on the computed correlation, a linear combination of the first activation map and the second activation map is used to process the first image within a second layer of the convolutional neural network. An output is provided based on the processing of the first image within the second layer of the convolutional neural network.
Shechtman et al. (US 2019/0251401 A1): The present disclosure relates to an image composite system that employs a generative adversarial network to generate realistic composite images. For example, in one or more embodiments, the image composite system trains a geometric prediction neural network using an adversarial discrimination neural network to learn warp parameters that provide correct geometric alignment of foreground objects with respect to a background image. Once trained, the determined warp parameters provide realistic geometric corrections to foreground objects such that the warped foreground objects appear to blend into background images naturally when composited together.
Contact Information
Any inquiry concerning this communication or earlier communications from the examiner should be directed to NANCY BITAR whose telephone number is (571)270-1041. The examiner can normally be reached Mon-Friday from 8:00 am to 5:00 p.m..
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Ms. Jennifer Mahmoud can be reached on 571-272-2976. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
NANCY . BITAR
Examiner
Art Unit 2664
/NANCY BITAR/Primary Examiner, Art Unit 2664
Prosecution Insights