DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Examiner’s Note
Claim 20 is patent eligible because applicant excluded signal from the definition of “computer-readable storage media”. See paragraph [0104] “Computer-readable storage media” refers to media and/or devices that enable persistent and/or non-transitory information storage in contrast to mere signal transmission, carrier waves, or signals per se. Thus, computer-readable storage media refers to non-signal-bearing media.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 6-8 and 17-18 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claims 6 and 17 recite “the product of the resolution of the first image” and “ the product of the resolution of the second image”. Here there is a lack of antecedent basis for these underlined portions. Also “product” needs at least two parameters but here it is not clear what the two parameters are.
Claims 7-8 and 18 are also rejected by virtue of dependency.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-3, 15 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Guo et al. (CN 121563784 “Guo”) in view of LEVIATHAN et al. (WO 2024163624 “Leviathan”).
Regarding claim 15 Guo teaches A computing device comprising: a memory; and a processor coupled to the memory (“Page 17 lines 76-Page 18 lines 3 “FIG. 6 is a structural diagram of an electronic device according to an embodiment of the present application. As shown in FIG. 6, the electronic device includes: At least one processor 610 and memory 620. The electronic device also includes a communication component 630. Wherein, the processor 610, the memory 620 and the communication component 630 are connected through a bus 640”) and configured to:
receive a first image of a subject person, a second image of a clothing item, and a first video depicting a person moving with reference movements (“Page 2 lines 42, 43, 50 “obtaining a try-on clothes image; obtaining the user face image; obtaining the user video”; Page 5 lines 32-33 “a second extracting module for extracting the user gesture image, the clothes mask image, the clothes shielding image, the noise image and the first user skeleton image from the user video”);
generate, by one or more conditioning networks and based on the first image, the second image, and the first video, conditioning sequences (Page 6 lines 39-41 “coding the try-on clothes image by the VAE coder, generating low-dimensional potential space characteristic, obtaining low-dimensional first clothes characteristic”;
Page 12 lines 6-18 “S402, obtaining the user face image, and extracting the semantic feature from the try-on clothing image to obtain the second clothing feature, and extracting the semantic feature of the user face image to obtain the face feature. Specifically, after the server obtains the try-on clothing image and the user face image, it can perform the pre-training through the contrast language image (Contrastive Language-Image Pre-Training, CLIP) image encoder extracting semantic feature from the try-on clothes image to obtain the second clothes feature of the try-on clothes image; and extracting the semantic feature of the user face image by the CLIP image coder to obtain the face feature”;
Page 12 lines 54-68 “S407, obtaining the user video and extracting the user gesture image, the clothes mask image, the clothes shielding image, the noise image and the first user skeleton image from the user video. In this embodiment, after obtaining the user video, the server extracts a user gesture image, a clothing mask image, a clothing occlusion image, a noise image, and a first user bone image from the user video, comprising: according to the DensePose algorithm, identifying the position of the important part of the user body from each video frame of the user video, and predicting the user gesture according to the position, obtaining the user gesture image, so as to realize the precise location of the dynamic character and the user gesture estimation”);
generate, by a diffusion transformer with multiple transformer layers, a second video of the clothing item virtually fitted on the subject person moving with the reference movements based on the conditioning sequences (Page 8 lines 55-59 “Based on this, the embodiment of the application provides an improved virtual fitting video generating method based on diffusion model, for reserving the details of the face, hand and fitting clothes of the user in the video frame, and enhancing the continuity between the video frames, so as to generate the real and smooth virtual fitting video. Page 10 lines 9-16 In this embodiment, the fusion model includes a reference model and a de-noising model, and the reference model includes: M cascaded reference units, M is an integer greater than 1, and the de noising model comprises: M denoising units connected end to end, the M denoising units are connected in cascade; The reference unit includes: a first spatial attention layer and a first cross attention layer, which are sequentially cascaded, the de-noising unit comprises: a second space attention layer, a second cross attention layer and a time alignment layer which are cascaded in turn”);
However Guo is silent about parameters of a pre-trained text-to-video diffusion model,;
Leviathan teaches generate, by a diffusion transformer with multiple transformer layers and parameters of a pre-trained text-to-video diffusion model, a second video (“[00012] The video editing system generates the output video conditioned on the system input by using a video diffusion model. Some implementations of the system can use a video diffusion model which, rather than predicting one frame after another to output the video, jointly models entire videos, or blocks of frames to improve temporal coherence between the input video and the generated frames. [00013] Prior to using the video diffusion model to generate output videos, the video editing system fine-tunes the model, i.e., determines updates to the pre-trained parameter values of the video diffusion model, with respect to the input video based on optimizing a mixed fine- tuning objective to improve the quality of motion edits to the input video. In particular, by holding the pre-trained parameter values of the temporal attention layers in the model fixed, e.g., through masking, while allowing the pre-trained parameter values of the spatial attention layers in the model to be updated, the video editing system fine-tunes the video diffusion model to reconstruct individual frames of the input video while discarding information about the temporal order of these frames”);
Guo and Leviathan are analogous art as both of them are from the field of video editing.
Therefore it would have been obvious for an ordinary skilled person in the art before the effective filing date of claimed invention to have modified Guo by having parameters of a pre-trained text-to-video diffusion model as taught by Leviathan and use this with Guo’s diffusion model.
The motivation for the above is to get faster generation of optimized video.
Guo modified by Leviathan teaches present, via a display, the second video (Guo Page 8 lines 15-17 “The virtual fitting video can display the effect of texture, lustre, fold and sport of the clothes in a dynamic manner so that the user can know the characteristics of the clothes more comprehensively”).
Claim 1 is directed to a method claim and its steps are similar in scope and functions of the elements of device claim 15 and therefore claim 1 is also rejected with the same rationale as specified in the rejection of claim 15.
Claim 20 is directed to “One or more computer-readable storage media” (Guo Page 18 lines 58-60 “The present application also provides a computer-readable storage medium in which computer executable instructions are stored for implementing the above-described virtual fitting video generation method when the processor executes the computer-executable instructions”) and its elements are similar in scope and functions of the elements of device claim 15 and therefore claim 20 is also rejected with the same rationale as specified in the rejection of claim 15.
Regarding claim 2 Guo modified by Leviathan teaches generating, using an auto-regressive video refiner to enhance a visual quality or a temporal consistency of the second video, a refined second video; and presenting, by the processing device via the display, the refined second video (“Leviathan “[00048] of the video diffusion model 102, e.g., whether it has a Transformer-based architecture or a convolutional architecture, or whether it generates one video frame after another autoregressively or jointly outputs the entire video, the fine-tuning system 120 can incorporate different techniques when fine-tuning the video diffusion model 102”.
Guo Page 8 lines 15-17 “The virtual fitting video can display the effect of texture, lustre, fold and sport of the clothes in a dynamic manner so that the user can know the characteristics of the clothes more comprehensively”).
Regarding claim 3 Guo modified by Leviathan teaches the first machine-learning model comprises one or more conditioning networks; and the second machine-learning model comprises a diffusion transformer with multiple transformer layers and parameters of a pre-trained text-to-video diffusion model (Guo Page 6 lines 39-41 “coding the try-on clothes image by the VAE coder, generating low-dimensional potential space characteristic, obtaining low-dimensional first clothes characteristic”;
Page 8 lines 55-59 “Based on this, the embodiment of the application provides an improved virtual fitting video generating method based on diffusion model, for reserving the details of the face, hand and fitting clothes of the user in the video frame, and enhancing the continuity between the video frames, so as to generate the real and smooth virtual fitting video. Page 10 lines 9-16 In this embodiment, the fusion model includes a reference model and a de-noising model, and the reference model includes: M cascaded reference units, M is an integer greater than 1, and the de noising model comprises: M denoising units connected end to end, the M denoising units are connected in cascade; The reference unit includes: a first spatial attention layer and a first cross attention layer, which are sequentially cascaded, the de-noising unit comprises: a second space attention layer, a second cross attention layer and a time alignment layer which are cascaded in turn”).
Claim(s) 4 and 12 are rejected under 35 U.S.C. 103 as being unpatentable over Guo modified by Leviathan as applied to claim 1 above, and further in view of MISHRA et al. (US Pat. Pub. No. 20260051101 “Mishra”).
Regarding claim 4 Guo modified by Leviathan is silent about receiving a text prompt providing edits to the reference movements or the clothing item to be included in the second video.
Mishra teaches receiving a text prompt providing edits to a reference movements or a clothing item to be included in a second video (“[0019] In some embodiments, a user may provide inputs 102 including an input frame 102A, animation specifics 102B, and a text prompt 102C…….. The text prompt 102C can be a natural language description of a type of garment (e.g., shirt, dress, pants), a texture (e.g., leopard spotted, flower print, plain), and an object adorning the garment (e.g., a person). Example text prompt 102C can include “a man wearing a striped shirt” or “a women in a red satin dress with flowers.”);
Mishra and Guo modified by Leviathan are analogous art as both of them are from the field of video editing.
Therefore it would have been obvious for an ordinary skilled person in the art before the effective filing date of claimed invention to have modified Guo modified by Leviathan by receiving a text prompt providing edits to a reference movements or a clothing item to be included in a second video as taught by Mishra;
The motivation for the above is to generate garment animation according to user’s choice and provide better controllability to user.
Regarding claim 12 Guo modified by Leviathan is silent about receiving, via a text prompt, user input to modify one or more attributes of the clothing item;
Mishra teaches receiving, via a text prompt, user input to modify one or more attributes of a clothing item (“[0019] In some embodiments, a user may provide inputs 102 including an input frame 102A, animation specifics 102B, and a text prompt 102C…….. The text prompt 102C can be a natural language description of a type of garment (e.g., shirt, dress, pants), a texture (e.g., leopard spotted, flower print, plain), and an object adorning the garment (e.g., a person). Example text prompt 102C can include “a man wearing a striped shirt” or “a women in a red satin dress with flowers.”);
Mishra and Guo modified by Leviathan are analogous art as both of them are from the field of video editing.
Therefore it would have been obvious for an ordinary skilled person in the art before the effective filing date of claimed invention to have modified Guo modified by Leviathan by receiving, via a text prompt, user input to modify one or more attributes of a clothing item as taught by Mishra;
The motivation for the above is to generate garment animation according to user’s choice and provide better controllability to user.
Guo modified by Leviathan and Mishra teaches generating, based on the text prompt and using the second machine-learning model, a third video of the modified clothing item virtually fitted on the subject person moving with the reference movements (Mishra “[0019] In some embodiments, a user may provide inputs 102 including an input frame 102A, animation specifics 102B, and a text prompt 102C. [0030] The text-to-image generative model 108 can be any generative model configured to receive a text prompt (e.g., text prompt 102C) at numeral 1C and generate frames (e.g., images) which when combined, become video 120”).
Claim(s) 5 and 16 are rejected under 35 U.S.C. 103 as being unpatentable over Guo modified by Leviathan as applied to claim 1 above, and further in view of Yang et al. (US Pat. Pub. No. 20260195587 “Yang”).
Regarding claims 5 and 16 Guo modified by Leviathan is silent about receiving an input of a noisy video with a first number (F) of frames and a resolution of a second number (H) by a third number (W); generating, using a text encoder, a prompt sequence of the text prompt in a latent space having a fourth number (L) of elements; encoding, using a first variational autoencoder (VAE), the noisy video into a video sequence of a fifth number of elements in the latent space, the fifth number of elements being equal to a product of the first number (F), the second number (H), and the third number (W); concatenating, at each transformer layer of the diffusion transformer, the video sequence with the prompt sequence to generate a main sequence; and applying, at each transformer layer, self-attention to the main sequence to generate the second video.
Yang teaches receiving an input of a noisy video with a first number (F) of frames and a resolution of a second number (H) by a third number (W); generating, using a text encoder, a prompt sequence of the text prompt in a latent space having a fourth number (L) of elements; encoding, using a first variational autoencoder (VAE), the noisy video into a video sequence of a fifth number of elements in the latent space, the fifth number of elements being equal to a product of the first number (F), the second number (H), and the third number (W); concatenating, at each transformer layer of the diffusion transformer, the video sequence with the prompt sequence to generate a main sequence; and applying, at each transformer layer, self-attention to the main sequence to generate the second video (“[0043] Cross-attention maps in diffusion models can serve as strong descriptors of object positions. The disclosed processes can extract CA maps within the T2V diffusion model, which can be viewed as spatial-temporal motion trajectories across video frames. In T2V diffusion models, the CA maps can be softmax-normalized matrices M.sub.CA∈[0, 1].sup.F×H×W×|c|, calculated by first reshaping the 3-dimensional (3D) intermediate activations Φϵ[AltContent: rect] into the shape (H×W×F)×D, where the activation parameters are F (frames), H (height), W (width), and D (depth). Cross-attention can then be performed between these activations (P and word embeddings τ(y) as shown in Equation 5. [0044] Equation 5: Example activations and word embeddings [0101]….. . In some aspects, T2V system 1000 or T2V controller 1100 can include an encoder configured to encode noisy videos using the diffusion motion features and spatio-temporal parameters encoded in attention maps of the pretrained T2V diffusion model, where the noisy videos can be utilized by the feature matching objective”);
Yang and Guo modified by Leviathan are analogous art as both of them are from the field of image processing.
Therefore it would have been obvious for an ordinary skilled person in the art before the effective filing date of claimed invention to have modified Guo modified by Leviathan by receiving an input of a noisy video with a first number (F) of frames and a resolution of a second number (H) by a third number (W); generating, using a text encoder, a prompt sequence of the text prompt in a latent space having a fourth number (L) of elements; encoding, using a first variational autoencoder (VAE), the noisy video into a video sequence of a fifth number of elements in the latent space, the fifth number of elements being equal to a product of the first number (F), the second number (H), and the third number (W); concatenating, at each transformer layer of the diffusion transformer, the video sequence with the prompt sequence to generate a main sequence; and applying, at each transformer layer, self-attention to the main sequence to generate the second video as taught by Yang.
The motivation for the above is to produce unique and high quality output video.
Claim(s) 9 is rejected under 35 U.S.C. 103 as being unpatentable over Guo modified by Leviathan as applied to claim 1 above, and further in view of Song et al. (US Pat. Pub. No. 20200090039 “Song”).
Regarding claim 9 Guo modified by Leviathan is silent about wherein the diffusion transformer is trained using unpaired user images as training data, the unpaired user images being generated using image try-on models configured to generate an image try-on on random frames of a video with random garments.
Song teaches diffusion transformer is trained using unpaired user images as training data, the unpaired user images being generated using image try-on models configured to generate an image try-on on random frames of a video with random garments (“[0037] Learning operation 112 includes system 100 using the neural network of architecture 102 to generate model training data. The model training data can correspond to embedding outputs that are produced by system 100 when the system is trained to generate a particular specialized model.
[0040] In some implementations, system 100 trains a neural network of architecture 102 using image data for apparel items that are each associated with different apparel categories. [0041] As indicated above, a particular set of embedding outputs can correspond to a particular specialized model. For example, in apparel recognition, a first set of embedding outputs can correspond to neural network training data for learned inferences used to generate a first model for recognizing certain shirts/blouses or tops (e.g., see operation 114)”);
Song and Guo modified by Leviathan are analogous art as both of them are from the field of image processing.
Therefore it would have been obvious for an ordinary skilled person in the art before the effective filing date of claimed invention to have modified Guo modified by Leviathan by having diffusion transformer is trained using unpaired user images as training data, the unpaired user images being generated using image try-on models configured to generate an image try-on on random frames of a video with random garments as taught by Song.
The motivation for the above is to provide wide variety training data for better performance of unseen scenario.
Claim(s) 13 is rejected under 35 U.S.C. 103 as being unpatentable over Guo modified by Leviathan as applied to claim 1 above, and further in view of ILARDI et al. (US Pat. Pub. No. 20260024262 “Ilardi”) and Materzynska et al. (US Pat. Pub. No. 20250142182 “Materzynska”).
Regarding claim 13 Guo modified by Leviathan is silent about wherein the first video depicting the reference movements is selected from a predefined set of movement sequences, and wherein the method further comprises: providing an interface for a user to select or customize the reference movements to be performed in the second video via a text prompt in a natural language format.
Ilardi teaches first video depicting reference movements is selected from a predefined set of movement sequences (“[0052]……. in some embodiments, any number of animations can be applied, such as a predefined series of animations, multiple animations that a user is allowed to select among and/or switch between, etc”);
Ilradi and Guo modified by Leviathan are analogous art as both of them are from the field of image processing.
Therefore it would have been obvious for an ordinary skilled person in the art before the effective filing date of claimed invention to have modified Guo modified by Leviathan by having first video depicting reference movements is selected from a predefined set of movement sequences as taught by Ilradi.
The motivation for the above is to limit the types of input video.
Even though Guo modified by Leviathan and Ilradi teaches providing an interface for a user to select or customize additional characteristics to be performed in the second video via a text prompt in a natural language format (Ilradi “[0034]….. s another example, in some embodiments, the prompt generator 302 can modify text in the user input 202 to generate a prompt that includes text specifying additional characteristics the generated images should include. Although described herein primarily with respect to generating a prompt as a reference example, in some embodiments”) but is silent about providing an interface for a user to select or customize the reference movements to be performed in the second video via a text prompt in a natural language format.
Materzynska teaches providing an interface for a user to select or customize the reference movements to be performed in the second video via a text prompt in a natural language format (“[0050] According to some aspects, embedding optimizer 220 optimizes the custom motion token. In some examples, embedding optimizer 220 contextualizes the custom motion token based on the text prompt, where the synthetic video is based on the contextualized custom motion token”);
Materzynska and Guo modified by Leviathan and Ilradi are analogous art as both of them are from the field of image processing.
Therefore it would have been obvious for an ordinary skilled person in the art before the effective filing date of claimed invention to have modified Guo modified by Leviathan by providing an interface for a user to select or customize the reference movements to be performed in the second video via a text prompt in a natural language format as taught by Materzynska.
The motivation for the above is to generate animation according to user’s choice and provide better controllability to user.
Claim(s) 14 is rejected under 35 U.S.C. 103 as being unpatentable over Guo modified by Leviathan as applied to claim 1 above, and further in view of Szeli et al. (US Pat. Pub. No. 20220188897 “Szeli”).
Regarding claim 14 Guo modified by Leviathan is silent about analyzing the second video to detect potential fitting issues or style mismatches between the clothing item and the subject person; and generating suggestions for alternative clothing items or sizes based on the second video.
Szeli teaches analyzing second video to detect potential fitting issues or style mismatches between clothing item and subject person; and generating suggestions for alternative clothing items or sizes based on the second video (“Abstract: The present invention provides at least a method for determining at least one body measurement by obtaining a plurality of images, in particular a video, of a user. [0048] “Wrong size” may indicate that the person cannot fit (optionally differentiating for at least one body measurement where the person does not fit), whereas “poor fit” may indicate that the person fits but (for at least one body measurement) desires a different fit, for example because the person desires a looser or tighter fit for the at least one body measurement for medical, anatomical and/or aesthetic reasons. [0050] For example, the person may have bought a shirt and then have returned it, indicating the degree or quality of fitting of the person at the shoulders, chest, sleeve length and/or the like. Such information is not only highly useful for providing an improved clothing size recommendation to the person in the future, it may also be used to train at least one of the artificial neural networks used in the method. [0051] Preferably, the clothing size recommendation is generated also based on the return information”);
Szeli and Guo modified by Leviathan are analogous art as both of them are from the field of image processing.
Therefore it would have been obvious for an ordinary skilled person in the art before the effective filing date of claimed invention to have modified Guo modified by Leviathan by analyzing second video to detect potential fitting issues or style mismatches between clothing item and subject person; and generating suggestions for alternative clothing items or sizes based on the second video as taught by Szeli.
The motivation for the above is to provide an automatic way of getting recommended garments.
Allowable Subject Matter
Claims 6-8 and 17-18 would be allowable if rewritten to overcome the rejection(s) under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), 2nd paragraph, set forth in this Office action and to include all of the limitations of the base claim and any intervening claims.
Claims 10-11 and 19 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Polyak et al. (US Pat. Pub. No. 20260127792) teaches Generates output image/video based on input image/video and text prompt using neural network.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SAPTARSHI MAZUMDER whose telephone number is (571)270-3454. The examiner can normally be reached 8 am-4 pm PST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Said Broome can be reached at (571)272-2931. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SAPTARSHI MAZUMDER/ Primary Examiner, Art Unit 2612
Prosecution Insights