DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The two information disclosure statements (IDS) submitted on March 13, 2026 are in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statements are being considered by the examiner.
Priority
The present application is a continuation of application number 16/261,395 filed on January 29, 2019.
Specification
The disclosure is objected to because of the following informalities:
p. 3, line 15, “come” should read “some”.
p. 9, line 8 (paragraph [0029]), reference number “602” denotes a “client device”, while the corresponding drawing (FIG. 3) and references to “client device” found throughout the remainder of the disclosure use reference number “302”.
p. 10, line 20 (paragraph [0031]), reference number “324” denotes a “training manager”, while the corresponding drawing (FIG. 3) and references to a “training manger” found throughout the remainder of the disclosure use reference number “322”.
Applicant is required to amend the specification to provide consistent terminology and/or numerical representation corresponding to “client device” and “training manager”, specifically with regard to the element numbers used when referring to the respective elements found in FIG. 3. Corresponding amendments to the drawing may be required to ensure compliance with 37 CFR 1.84.
Appropriate correction is required.
Claim Objections
Claim 5-7 and 12-14 are objected to because of the following informalities:
Claims 5-7 have limitations that read “…one or more circuits are further to train the one or more neural networks…”. “Further to train” is grammatically incorrect. Applicant is required to resolve grammatical issue, such as amending the corresponding subsections of the limitations to read “…one or more circuits are configured to train the one ore more neural networks…” or any other possible appropriate correction that maintains the scope and consistency of the limitations.
Claim 12 and 14 have limitations that read “…one ore more processors are further to train the one or more neural networks..”, and claim 13 has a limitation that reads “….one or more processors are further to cause the one ore more neural networks to…”. “Further to train” and “further to cause” are grammatically incorrect. Applicant is required to resolve grammatical issue, such as amending the corresponding subsections of the limitations (similar to claims 5-7) to maintain the scope and consistency of the limitations.
Appropriate correction is required.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Examiner notes, while Bouhnik has a publication date of Jan. 21, 2020 (after the effective filing date of the instant application), Bouhnik qualifies as prior art under 35 U.S.C. 102(a)(2) based on its effective filing date of Mar. 12, 2018, which precedes the effective filing date of the claimed invention, and is therefore available as prior art in the following 35 U.S.C. 103 rejection.
Claim(s) 1-2, 8-9, and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Bouhnik (US 10,540,757 B1) in view of Tran (“Disentangled Representation Learning GAN for Pose-Invariant Face Recognition”; provided by applicant in IDs; examiner provided a copy of the version used by examiner, with corresponding citations mentioned in the present office action).
Regarding claim 1, Bouhnik teaches: A processor, comprising: one or more circuits (FIG. 1, processor 130; FIG. 11, processor 1124; column 2, lines 22-27) use one or more neural networks to generate one or more images of one or more first objects (Bouhnik teaches using neural networks to generate one or more images, including of first objects like clothing (column 3, lines 3-14
PNG
media_image1.png
238
451
media_image1.png
Greyscale
; person in generated image 223 and clothes based on person found in images 203 & 211, refer to column 4, line 4 to column 5, line 5 for more detail))
based at least in part, on one or more (see Abstract; see column 4, line 4 to column 5, line 5 and FIG. 2 that shows generated image 223 based on a first person’s (from image 203) non-pose dependent clothing (e.g. top) information and a second person’s (from image 211) pose-dependent leg pose information.).
While Bouhnik does teach using an encoder-decoder process for inpainting using a neural network, e.g. GAN (column 14, lines 7-12), Bouhnik fails to explicitly disclose using a first-encoder and second-encoder to encode pose-independent information and pose-dependent information.
In a related art, Tran teaches: inputting images of any pose into an encoder (i.e. one or more first and/or second encoders) to learn and encode objects, exemplified by faces (p. 1283, right column “The input to the encoder Genc is a face image of any pose, the output of the decoder Gdec is a synthetic face at a target pose, and the learnt representation bridges Genc and Gdec”; p. 1286, left column, first paragraph.).
It would have been obvious to a person of ordinary skill in the art before the effective filing date to have modified the teachings of Bouhnik, including the encoding techniques taught by Bouhnik, to incorporate encoding techniques of Tran in order to provide a more efficient framework of processing data (e.g. learning and creating synthetic images in a compact manner), subsequent retrieval practices, and, as Tran points out, to enhance recognition performance (Tran p. 1284, left column, last paragraph). Both references lie in the field of image analysis with an aim at pose related recognition and synthesizing of images.
Regarding claim 8, Bouhnik and Tran teach a system (see Bouhnik, column 1, lines 17-20, FIG. 1 “a system diagram of system for generating combined images utilizing image processing of multiple-images according to examples of the present disclosure.”). The limitations found in claim 8 equally mirror the limitations established in claim 1. Claim 8 is rejected based on the prior art taught by Bouhnik and Tran in claim 1.
Regarding claim 15, Bouhnik and Tran teach a method (see Bouhnik, Abstract). The limitations found in claim 15 equally mirror the limitations established in claim 1. Claim 15 is rejected based on the prior art taught by Bouhnik and Tran in claim 1.
Regarding claim 2, Bouhnik and Tran teach the processor of claim 1, including a first encoder to encode pose-independent information about the one or more first objects.
Tran further teaches: generating a class-specific representation of the one or more first objects based, at least in part, on one or more target images of the one or more first objects (Tran p. 1285, right column, last paragraph through p. 1286, left column, first and second paragraphs, discloses an encoder and decoder aimed to learn an identity representation from a face image (i.e. target image of one or more objects), including through the use of identity classification. Specifically, Tran teaches learning “pose-invariant identity representation for PIFR” (p. 1285, right column, third paragraph), a well-known technique based on pose-independent information.).
Regarding claim 9, Bouhnik and Tran teach the system of claim 8, including a first encoder to encode pose-independent information about the one or more first objects.
Tran further teaches: generating a class-specific representation of the one or more first objects based, at least in part, on one or more target images of the one or more first objects (These limitations equally mirror the limitations found in claim 2. For sake of brevity, refer to claim 2’s 103 rejection, found above, for Tran’s teachings of the limitation.), and
wherein the one or more processors are to generate the one or more images based, at least in part, on the class-specific representation (Tran teaches generating a new image based on the class-specific identity information from input images (“3.3 Multi-Image DR-GAN” found on p. 1286, right column through p. 1287 left column). Examiner interprets this to be equivalent to generating the one or more images based at least in part, on the class-specific representation.).
Claim(s) 3, 10, and 16 are rejected under 35 U.S.C. 103 as being unpatentable over Bouhnik (US 10,540,757 B1) in view of Tran (“Disentangled Representation Learning GAN for Pose-Invariant Face Recognition”), in further view of Huang (“Arbitrary Style Transfer in Real-time with Adaptive Instance Normalization”; provided by applicant in IDs; examiner provided a copy of the version used by examiner, with corresponding citations mentioned in the present office action).
Regarding claim 3, Bouhnik and Tran teach the processor of claim 1, including one or more circuits generating one or more images based, at least in part, on pose-independent information.
Bouhnik further teaches using vectors for an “image discriminator” that determines whether or not an image should be displayed through the use of a trained model (Bouhnik, column 12, lines 12-31), and Tran teaches transforming concatenated vectors into a synthetic image (Tran, p. 1286, lines 1-6 of right column).
Bouhnik and Tran fail to explicitly disclose: wherein the one or more circuits are to generate the one or more images based, at least in part, on a set of mean and variance vectors generated according to the pose-independent information.
In a related art, Huang teaches: using adaptive instance normalization layer for the purpose of decoding feature representation in order to reconstruct images, including the use of mean and variance (Abstract; Section 1, paragraph 3, lines 6-10; Section 3.2 lines 1-6; Section 5.1 paragraph 1, lines 4-7).
It would have been obvious to a person of ordinary skill in the art before the effective filing date to have modified the teachings of Bouhnik and Tran to incorporate the teaching of Huang in order to improve image reconstruction by achieving “speed comparable to the fastest existing approach, without the restriction to a pre-defined set of style features” (Huang, Abstract). All inventions lie in the field of image analysis with an aim at reconstructing images.
Regarding claim 10, Bouhnik and Tran teach the system of claim 8, including one or more processors to generate one or more images of one or more first objects based, at least in part on pose-independent information.
Tran further teaches: using a decoder throughout the process of synthesizing images (Abstract; p. 1283, right column second paragraph, lines 7-12; p. 1285 section 3.2 lines 5-7).
Bouhnik and Tran fail to explicitly disclose: wherein the one or more processors are to generate the one or more images based, at least in part, on one or more decoders to decode the pose-independent information as a set of mean and variance vectors.
In a related art, Huang teaches: using adaptive instance normalization layer for the purpose of decoding feature representation in order to reconstruct images, including the use of mean and variance (Abstract; Section 1, paragraph 3, lines 6-10; Section 3.2 lines 1-6; Section 5.1 paragraph 1, lines 4-7), while specifically citing the use of a decoder network to generate the final stylized image (Section 1, paragraph 3, third to last sentence).
It would have been obvious to a person of ordinary skill in the art before the effective filing date to have modified the teachings of Bouhnik and Tran to incorporate the teaching of Huang in order to improve image reconstruction by achieving “speed comparable to the fastest existing approach, without the restriction to a pre-defined set of style features” (Huang, Abstract). All inventions lie in the field of image analysis with an aim at reconstructing images.
Regarding claim 16, Bouhnik and Tran teach the method of claim 15.
Tran further teaches: wherein the one or more first encoders are to encode the pose-independent information at least by generating a class-specific representation of the one or more first objects based, at least in part, on one or more target images of the one or more first objects (Tran p. 1285, right column, last paragraph through p. 1286, left column, first and second paragraphs, discloses an encoder and decoder aimed to learn an identity representation from a face image (i.e. target image of one or more objects), including through the use of identity classification. Specifically, Tran teaches learning “pose-invariant identity representation for PIFR” (p. 1285, right column, third paragraph), a well-known technique based on pose-independent information.). Tran also teaches using a decoder throughout the process of synthesizing images (Abstract; p. 1283, right column second paragraph, lines 7-12; p. 1285 section 3.2 lines 5-7).
However, Tran fails to explicitly disclose: wherein generating the one or more images is further based, at least in part, on one or more decoders to decode the class-specific representation as a set of spatially invariant means and variances.
In a related art, Huang teaches: using adaptive instance normalization layer for the purpose of decoding feature representation in order to reconstruct images, including the use of mean and variance computed across spatial dimensions independently for each channel, (Abstract; Section 1, paragraph 3, lines 6-10; Section 3.2 lines 1-10; Section 5.1 paragraph 1, lines 4-7), while specifically citing the use of a decoder network to generate the final stylized image (Section 1, paragraph 3, third to last sentence).
It would have been obvious to a person of ordinary skill in the art before the effective filing date to have modified the teachings of Bouhnik and Tran to incorporate the teaching of Huang in order to improve image reconstruction by achieving “speed comparable to the fastest existing approach, without the restriction to a pre-defined set of style features” (Huang, Abstract), while also accounting for features, regardless of their special position in an image. All inventions lie in the field of image analysis with an aim at reconstructing images.
Claim(s) 4-7, 11-14, and 17-20 are rejected under 35 U.S.C. 103 as being unpatentable over Bouhnik (US 10,540,757 B1) in view of Tran (“Disentangled Representation Learning GAN for Pose-Invariant Face Recognition”), in further view of Kim (US 20180314716 A1).
Regarding claim 4, Bouhnik and Tran teach the processor of claim 1, including a first encoder and second encoders used in the generation of images.
Bouhnik further teaches: wherein the one or more images are of a target class (see FIG. 2, “first human/first subject” and “second human/ second person/ second subject” (column 4, lines 10-25). Examiner interprets human to be the target class.), and
wherein the one or more circuits are to generate the one or more images based, at least in part, on a class-specific representation, (see Abstract; see column 4, line 4 to column 5, line 5 and FIG. 2 that shows generated image 223 based on a first person’s (from image 203) non-pose dependent clothing (e.g. top) information and a second person’s (from image 211) pose-dependent leg pose information. The first person and second person used to create the generated image of a person are interpreted as class-specific and class-invariant representations.).
Bouhnik and Tran fail to explicitly disclose: using a class-invariant representation different from the target class for generating the one or more images.
In a related art, Kim teaches: a method for learning cross domain relations using a model for direct transfer of attributes between images of different classes/domains (see paragraphs [0011]-[0016], [0060]).
It would have been obvious to a person of ordinary skill in the art before the effective filing date to have modified the teachings of Bouhnik and Tran to incorporate the teachings of Kim in order to improve relation extraction techniques with more image inputs of different domains/classes. Doing so would make the system more robust by allowing for a variety of objects and their corresponding images to be input and processed. All three references are image analysis based with a focus on reconstructing images.
Regarding claim 11, Bouhnik and Tran teach the system of claim 8.
The limitations found in claim 11 equally mirror the limitations found in claim 4 and taught by Bouhnik, Tran, and Kim, with the exception of the use of latent representation. For the sake of brevity, please refer to claim 4’s 103 rejection for Bouhnik, Tran, and Kim’s teachings of: wherein the one or more images are of a target class, and
wherein the one or more processors are to generate the one or more images based, at least in part, on a class-specific
Tran further teaches: a relevant GAN variant, known as an “Adversarial Autoencoder (AAE)” that uses an autoencoder to reconstruct the input image and a latent vector is generated by the encoder to be used in subsequent tasks like training the discriminator model (p. 1287, seciont 3.4, subsection “AAdversarial Autoencoder (AAE)”). Tran further specifies the autoencoder is trained to learn a latent representation (p. 1287, seciont 3.4, subsection “AAdversarial Autoencoder (AAE)”).
It would have been obvious to a person of ordinary skill in the art before the effective filing date to have modified the teachings of Bouhnik and Tran, previously modified by Kim, to incorporate the teachings of a variant GAN, taught by Tran, that uses latent representation information in order to help train the model to distinguish between real/fake distributions (Tran (p. 1287, seciont 3.4, subsection “AAdversarial Autoencoder (AAE)”), thereby making the system more accurate.
Regarding claim 7 and 20, Bouhnik and Tran teach: The processor of claim 1 and the method of claim 15.
Bouhnik teaches training one or more neural networks and translating images (Bouhnik column 3, lines 1-11), and Tran teaches the use of neural networks (e.g. CNN) for identity classification and training (Tran section 3.2.1, paragraph 2, lines 1-5), and inputting random noise vectors in the encoder to learn mapping of feature representation (Tran p. 1284, left column, lines 4-8).
However, Bouhnik and Tran fail to explicitly disclose: training the one or more neural networks to translate images between two randomly sampled source classes (claim 20: sampled source object classes).
In a related art, Kim teaches: training the one or more neural networks to translate images between two randomly sampled source classes (Paragraphs [0011]-[0016] and [0060] disclose training to translate between two different domains using a GAN. Kim further discloses standard GAN models take random Gaussian noise, while the model taught by Kim uses images instead of noise.).
It would have been obvious to a person of ordinary skill in the art before the effective filing date to have modified the teachings of Bouhnik and Tran to incorporate the teachings of Kim in order to improve relation extraction techniques, thereby improving the accuracy of the processor.
Regarding claim 14, Bouhnik and Tran teach the system of claim 8. The limitations found in claim 14 equally mirror the limitations found in claim 7, with the exception of translating pose images. Therefore, claim 14 is rejected based on the prior art of Bouhnik, Tran, and Kim taught in claim 7 of the present office action. The additional use of pose images fails to overcome the rejection because Bouhnik teaches using pose dependent information, as exemplified in claim 8’s 103 rejection.
Regarding claims 6, 13, and 19, Bouhnik and Tran teach the processor of claim 1, the system of claim 8, and the method of claim 15. Examiner notes, claims 6 and 19 of the instant application mirror or have broader scopes than claim 13, while all claim techniques of training the one or more neural networks in an unsupervised manner, therefore the rejection for claim 13 applies equally to claims 6 and 19.
Bouhnik further teaches: “Various techniques 20 may be used to train the models including backpropagation, statistical learning, supervised learning, semi-supervised learning, stochastic learning, or other known techniques” (column 19, lines 20-23), without explicitly citing unsupervised training.
Thus, Bouhnik fails to explicitly disclose: wherein the one or more processors are further to cause the one or more neural networks to be trained in an unsupervised manner based, at least in part, on a training data set comprising images of different classes (as stated in claim 13 of the instant office action).
In a related art, Kim teaches a method for learning cross domain relations using a model for direct transfer of attributes between images of different classes/domains (see paragraphs [0011]-[0016], [0060]). Kim further teaches the method utilizes a GAN to perform unsupervised training of a first generative model for translating images in domain A (i.e. class A) to images in domain B (i.e. class B) (see paragraphs [0015]-[0016]) and training the system using a set of images from domain A and a set of images representing domain B (see paragraphs [0055]-[0057]).
It would have been obvious to a person of ordinary skill in the art before the effective filing date to have modified the teachings of Bouhnik and Tran to incorporate the teachings of Kim in order to improve relation extraction techniques, thereby improving the accuracy of the model for reconstructing image(s).
Regarding claim 5, Bouhnik and Tran teach the processor of claim 1, including the use of one or more neural networks.
Bouhnik and Tran fail to explicitly disclose: wherein the one or more circuits are further to train the one or more neural networks based, at least in part, on one or more second images of different objects from the one or more first objects.
In a related art, Kim teaches: wherein the one or more circuits are further to train the one or more neural networks based, at least in part, on one or more second images of different objects from the one or more first objects (see paragraphs [0055]-[0057], training the system using a set of images from domain A (i.e. first objects) and a second set of images representing domain B (i.e. second objects that are different objects than the domain A objects).).
It would have been obvious to a person of ordinary skill in the art before the effective filing date to have modified the teachings of Bouhnik and Tran to incorporate the teachings of Kim in order to improve relation extraction techniques, thereby improving the accuracy of the model for reconstructing image(s).
Regarding claim 12, Bouhnik and Tran teach the system of claim 8.
Bouhnik and Tran fail to disclose: wherein the one or more processors are further to train the one or more neural networks based, at least in part, on one or more second images of the one or more second objects, and wherein the one or more second objects are one or more objects other than the one or more first objects.
In a related art, Kim teaches: wherein the one or more processors are further to train the one or more neural networks based, at least in part, on one or more second images of the one or more second objects, and wherein the one or more second objects are one or more objects other than the one or more first objects (see paragraphs [0055]-[0057], training the system using a set of images from domain A (i.e. first objects) and a second set of images representing domain B (i.e. second objects that are different objects than the domain A objects).).
It would have been obvious to a person of ordinary skill in the art before the effective filing date to have modified the teachings of Bouhnik and Tran to incorporate the teachings of Kim in order to improve relation extraction techniques, thereby improving the accuracy of the model for reconstructing image(s).
Regarding claim 17, Bouhnik and Tran teach the method of claim 15, including generating one or more images and using one or more first and second encoders.
Bouhnik and Tran further disclose: wherein generating the one or more images is based, at least in part, on a class-specific latent representation (see Tran p. 1287, seciont 3.4, subsection “AAdversarial Autoencoder (AAE)” teaching of the use of latent representation information), generated by the one or more first encoders, of a target object class (see Bouhnik FIG. 2, “first human/first subject” and “second human/ second person/ second subject” (column 4, lines 10-25). Examiner interprets human to be the target class.) and a class-invariant latent representation, generated by the one or more second encoders (see Bouhnik Abstract; see column 4, line 4 to column 5, line 5 and FIG. 2 that shows generated image 223 based on a first person’s (from image 203) non-pose dependent clothing (e.g. top) information and a second person’s (from image 211) pose-dependent leg pose information. The first person and second person used to create the generated image of a person are interpreted as class-specific and class-invariant representations.; Tran p. 1287, seciont 3.4, subsection “AAdversarial Autoencoder (AAE)”, teaching of the use of latent representation information).
It would have been obvious to a person of ordinary skill in the art before the effective filing date to have modified the teachings of Bouhnik and Tran, including first and second encoders for generating images, to incorporate the further teachings of Bouhnik and Tran to make the model more accurate by accounting for latent representative information.
Bouhnik and Tran fail to explicitly disclose: a source object class different from the target object class, wherein the one or more first objects are of the target object class, and
wherein the one or more second objects are of the source object class.
In a related art, Kim teaches: a source object class different from the target object class, wherein the one or more first objects are of the target object class, and wherein the one or more second objects are of the source object class (see paragraphs [0011]-[0016], [0060], and FIG. 8A-8C representing a learning model trained with various handbag and shoe images (FIG. 8A) and generated output of shoe images (FIG. 8B) based on shoe images and second images of different objects, specifically exemplified by handbags.).
It would have been obvious to a person of ordinary skill in the art before the effective filing date to have modified the teachings of Bouhnik and Tran to incorporate the teachings of Kim in order to improve relation extraction techniques, thereby improving the accuracy of the processor.
Regarding claim 18, Bouhnik and Tran teach the method of claim 15.
Bouhnik and Tran fail to explicitly disclose: further comprising training the one or more neural networks based, at least in part, on one or more second images of a different object class from an object class of the one or more generated images.
In a related art, Kim teaches: further comprising training the one or more neural networks based, at least in part, on one or more second images of a different object class from an object class of the one or more generated images (see paragraphs [0055]-[0059], training the system using a set of images from domain A (i.e. first objects) and a second set of images representing domain B (i.e. second objects that are different objects than the domain A objects) and FIG. 8A-8C representing a learning model trained with various handbag and shoe images (FIG. 8A) and generated output of shoe images (FIG. 8B) based on shoe images and second images of different objects, specifically exemplified by handbags.).
It would have been obvious to a person of ordinary skill in the art before the effective filing date to have modified the teachings of Bouhnik and Tran to incorporate the teachings of Kim in order to improve relation extraction techniques, thereby improving the accuracy of the model for reconstructing image(s).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SAMUEL DAVID BAYNES whose telephone number is (571)272-0607. The examiner can normally be reached Monday - Friday 8:00 am - 5:00 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Stephen R Koziol can be reached at (408)918-7630. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/S.D.B./
Samuel D. Baynes
Examiner, Art Unit 2665
/Stephen R Koziol/Supervisory Patent Examiner, Art Unit 2665
Prosecution Insights