DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 4, 11, 13, and 18-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Jie (US 2023/0360357 A1) in view of Gottemukkula (US 2020/0302149 A1) and further in view of Soborski et al. (US 2023/0289952 A1).
Regarding claim 1, Jie discloses a computer-implemented method for performing style transfer, the method comprising: training a neural network based on (i) one or more supervised losses computed between a first set of training output produced by the neural network from a first set of training content samples and a set of stylized samples corresponding to the first set of training content samples, (Paragraphs 0003, 0129, and 0136, supervised learning for training a network model using a loss function with test style images and training style images output from the network model using inputted training images). Jie does not clearly disclose (ii) one or more unsupervised losses computed using a second set of training output produced by the neural network from a second set of training content samples to generate a trained neural network. Gottemukkula discloses training a neural network using supervised and unsupervised loss functions (Figure 4 and paragraph 0060). Gottemukkula’s technique of training a neural network using supervised and unsupervised loss functions would have been recognized by one of ordinary skill in the art to be applicable to the training of a neural network using a supervised loss function with output style images from the neural network based on input training images of Jie and the results would have been predictable in the training of a neural network using a supervised and unsupervised loss function with output style images from the neural network based on input training Images. Therefore, the claimed subject matter would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention. Jie in view of Gottemukkula does not clearly disclose inputting a content sample into the trained neural network; and generating, via execution of the trained neural network, a style transfer result that comprises one or more content-based attributes of the content sample and one or more style-based attributes of the set of stylized samples.
Soborski discloses a trained style transfer neural network that generates from an input content image, a synthetic image from that represents one or more desired features of a style image while preserving content data of the content image (Paragraph 0046). Soborski’s technique of using a trained neural network to generate a synthetic image from an input content image with preserved content data and desired features of a style image would have been recognized by one of ordinary skill in the art to be applicable to the trained neural network using style images of Jie in view of Gottemukkula and the results would have been predictable in providing an input content image to a neural network trained using style images to generate a synthetic image with preserved content data of the input content image with desired features of the style images. Therefore, the claimed subject matter would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention.
Regarding claim 4, Jie disclose wherein training the neural network comprises: generating a first version of the trained neural network via a first set of training iterations that train the neural network using a first subset of the first set of training content samples and a first subset of the set of stylized samples corresponding to the first subset of the first set of training content samples; (Paragraphs 0129 and 0136, training the network model using the loss function with test style images and training style images output from the network model using the inputted training images) determining a second subset of the first set of training content samples based on a style transfer performance associated with the first version of the trained neural network; (Paragraph 0039, each training image from a training image set can be used as the input) and generating a second version of the trained neural network via a second set of training iterations that further train the first version of the trained neural network using the second subset of the first set of training content samples and a second subset of the set of stylized samples corresponding to the second subset of the first set of training content samples (Paragraphs 0039, 0129, and 0136, the network model can be trained based on each training image from the image training set as subsequent input to the network model that is trained with each input).
Regarding claims 11 and 20, similar reasoning as discussed in claim 1 is applied. Furthermore, Jie discloses one or more memories that store instructions, and one or more processors that are coupled to the one or more memories (Paragraphs 0018-0019, memory storing program code to be read by a processor).
Regarding claim 13, similar reasoning as discussed in claim 4 is applied.
Regarding claim 18, Gottemukkula discloses wherein the neural network is trained using a weighted combination of the one or more supervised losses and the one or more unsupervised losses (Figure 4, training the neural network on the combination of unsupervised and supervised losses where each component can be of equal weight and if not equal it can be considered to have different weights).
Regarding claim 19, Soborski discloses wherein the trained neural network comprises a feedforward image-to-image translation model (Paragraph 0046, neural network for generating the output synthetic image from the input content image).
Claim(s) 2, 3, and 12 is/are rejected under 35 U.S.C. 103 as being unpatentable over Jie (US 2023/0360357 A1) in view of Gottemukkula (US 2020/0302149 A1) in view of Soborski et al. (US 2023/0289952 A1) and further in view of Sturlesi et al. (US 2025/0086781 A1).
Regarding claim 2, Jie in view of Gottemukkula and further in view of Soborski discloses all limitations as discussed in claim 1. Jie in view of Gottemukkula and further in view of Soborski does not clearly disclose wherein training the neural network comprises: converting, via a trained variational autoencoder, a first set of features associated with the second set of training output into a second set of features from a feature space associated with the set of stylized samples; and computing the one or more unsupervised losses based on the first set of features and the second set of features. Sturlesi et al. discloses a variational auto-encoder (VAE) using unsupervised learning for mapping an input nominal image to a latent space for producing variations to features to output a predicted reference image and computing a reconstruction loss between the images (Paragraphs 0063 and 0095, and 0098-0099). Sturlesi’s technique of using a VAE to produce out images that are variations of an input image and computing losses between the images would have been recognized by one of ordinary skill in the art to be applicable to the neural network for generating a synthetic image with a style transferred to an input image of Jie in view of Gottemukkula and further in view of Soborski and the results would have been predictable in the generation of synthetic images with variations in style as output from input images and calculating losses between the images. Therefore, the claimed subject matter would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention.
Regarding claim 3, Jie discloses wherein training the neural network further comprises extracting the first set of features from a plurality of layers included in a feature extractor neural network (Paragraph 0004, feature extracted by a network model).
Regarding claim 12, similar reasoning as discussed in claims 2 and 3 is applied.
Claim(s) 8 and 9 is/are rejected under 35 U.S.C. 103 as being unpatentable over Jie (US 2023/0360357 A1) in view of Gottemukkula (US 2020/0302149 A1) in view of Soborski et al. (US 2023/0289952 A1) and further in view of He (US 2021/0312195 A1).
Regarding claim 8, Jie in view of Gottemukkula and further in view of Soborski discloses all limitations as discussed in claim 1. Jie in view of Gottemukkula and further in view of Soborski does not clearly disclose wherein the first set of training content samples and the second set of training content samples each comprise a sequence of video frames. He discloses obtaining video frames for use in style transfer (Paragraphs 0090 and 0097). He’s technique of obtaining video frames for use in style transfer would have been recognized by one of ordinary skill in the art to be applicable to the input of images for training a neural network for style transfer of Jie in view of Gottemukkula and further in view of Soborski and the results would have been predictable in the input of video frames for the training of a neural network for style transfer. Therefore, the claimed subject matter would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention.
Regarding claim 9, Jie in view of Gottemukkula in view of Soborski and further in view of He discloses wherein the set of stylized samples comprise stylizations of one or more key frames that are included in the sequence of video frames and correspond to the first set of training content samples (He, Paragraph 0090, key frames that can be extracted from the video that can be provided as input to the network model, Jie, paragraph 0129).
Claim(s) 10 is/are rejected under 35 U.S.C. 103 as being unpatentable over Jie (US 2023/0360357 A1) in view of Gottemukkula (US 2020/0302149 A1) in view of Soborski et al. (US 2023/0289952 A1) and further in view of Erol et al. (US 2025/0117920 A1).
Regarding claim 10, Jie in view of Gottemukkula and further in view of Soborski discloses all limitations as discussed in claim 1. Jie in view of Gottemukkula and further in view of Soborski does not clearly disclose wherein the one or more unsupervised losses comprise at least one of a style loss, a content loss, a perceptual loss, a cosine distance, or a Euclidean distance. Erol discloses unsupervised learning that minimizes reconstruction losses including a perceptual loss (Paragraph 0009). Erol’s technique of unsupervised learning that minimizes reconstruction losses including a perceptual loss would have been recognized by one of ordinary skill in the art to be applicable to the training of a neural network using unsupervised loss functions of Jie in view of Gottemukkula and further in view of Soborski and the results would have been predictable in the training of a neural network using unsupervised loss functions that include perceptual losses. Therefore, the claimed subject matter would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention.
Allowable Subject Matter
Claims 5-7 and 14-17 objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Regarding claim 5, the prior art does not clearly disclose the computer-implemented method of claim 1, wherein generating the style transfer result comprises: determining a control map associated with the content sample, wherein the control map comprises a plurality of values for a plurality of locations in the content sample; and combining, via execution of the trained neural network, the control map and the content sample into the style transfer result, wherein the style transfer result comprises a plurality of style variants corresponding to the plurality of values in the plurality of locations.
Regarding claim 14, similar reasoning as discussed in claim 5 is applied.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Ryu et al. (US 2025/0216927 A1) style transfer that merges content features of one image with style features of another image.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PHI HOANG whose telephone number is (571)270-3417. The examiner can normally be reached Mon-Fri 8:00-5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, JASON CHAN can be reached at (571)272-3022. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/PHI HOANG/Primary Examiner, Art Unit 2619
Prosecution Insights