Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-5, 7-14, and 16-19 are rejected under 35 U.S.C. 103 as being unpatentable over Yu et al. (U.S. Patent Application Publication No. 2024/0104809), referred herein as Yu, in view of Li et al. (U.S. Patent Application Publication No. 2025/0078369), referred herein as Li.
Regarding claim 1, Yu teaches a computer-implemented method comprising: receiving, from a client device, a plurality of input elements for generating a digital design (figs 1 and 2; fig 8, client device 810; paragraph 68; paragraphs 79 and 81; receiving a plurality of input elements from a client device for a digital design);
generating, using an encoder of a multi-domain neural network having an image branch and a vector branch, embeddings representing visual characteristics (fig 1; paragraph 21, lines 1-11; paragraph 22, lines 1-12; paragraph 23; paragraph 24, lines 1-11; paragraph 82; embeddings are generated using the encoder that represent visual characteristics such as image, text, etc.) and bounding box characteristics of the plurality of input elements (paragraph 24, lines 1-7; paragraph 27, lines 1-6; paragraph 86; bounding box characteristics for the input elements are also generated);
generating, using at least one of the image branch or the vector branch of the multi-domain neural network, a layout for the digital design from the visual characteristics and bounding box characteristics of the embeddings (paragraph 23; paragraph 24, lines 7-10; paragraph 87, lines 1-4; the layout is generated based on the visual and bounding box characteristics); and
providing the layout for display on the client device (paragraph 24, lines 10-11; paragraph 87, the last 6 lines; the layout is provided to the client device for display).
Yu does not teach a diffusion network or diffusion branches.
However, in a similar field of endeavor, Li teaches a computer-implemented method comprising receiving input elements from a client device for generating a digital design, generating a layout for the design using an image editing network, and providing the layout for display on the client device (paragraph 43, lines 1-19; paragraphs 46-48; paragraphs 118 and 119), wherein representations of the image elements are generated by exchanging image noise and vector noise between branches (fig 3; paragraph 88; paragraphs 100-103; paragraphs 115 and 116), and wherein the network is a diffusion network with diffusion branches (figs 3 and 4; paragraph 87; paragraph 114; paragraph 115, lines 1-11).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the diffusion network and branches of Li with the network and branches of Yu because this improves the efficiency and quality of multi-model image processing comprising text, image, video, etc., while reducing the operational complexity for the user and improving the user experience (see, for example, Li, paragraphs 4 and 26).
Regarding claim 2, Yu in view of Li teaches the computer-implemented method of claim 1, further comprising generating a plurality of image elements from the plurality of input elements by rendering at least one image element from at least one input element of the plurality of input elements using a corresponding rendering engine, the at least one input element comprising a text element or a vector element (Yu, figs 1 and 2; paragraph 22; paragraph 24, lines 7-11; paragraph 27; paragraph 28, the last 4 lines; Li, paragraph 88; the motivation to combine Li is substantially similar to that discussed above in the rejection of claim 1),
wherein generating, using the encoder, the embeddings representing the visual characteristics and bounding box characteristics of the plurality of input elements comprises generating the embeddings from the plurality of image elements using the encoder (Yu, paragraph 21, lines 1-21; paragraph 22, lines 1-12; paragraphs 83 and 84).
Regarding claim 3, Yu in view of Li teaches the computer-implemented method of claim 1, further comprising determining an image domain noise input and a vector domain noise input for generating the digital design, wherein generating, using at least one of the image diffusion branch or the vector diffusion branch of the multi-domain diffusion neural network, the layout for the digital design from the visual characteristics and bounding box characteristics of the embeddings comprises generating, using at least one of the image diffusion branch or the vector diffusion branch of the multi-domain diffusion neural network, the layout for the digital design from the embeddings, the image domain noise input, and the vector domain noise input (Yu, paragraph 23; paragraph 24, lines 1-11; Li, paragraphs 87 and 88; paragraphs 114 and 115; the motivation to combine Li is substantially similar to that discussed above in the rejection of claim 1).
Regarding claim 4, Yu in view of Li teaches the computer-implemented method of claim 3, wherein generating, using at least one of the image diffusion branch or the vector diffusion branch of the multi-domain diffusion neural network, the layout for the digital design from the embeddings, the image domain noise input, and the vector domain noise input comprises: determining one or more diffusion conditions using the embeddings, the image domain noise input, and the vector domain noise input; and generating, using at least one of the image diffusion branch or the vector diffusion branch of the multi-domain diffusion neural network, the layout for the digital design from the one or more diffusion conditions (Yu, paragraphs 20 and 23; paragraph 24, lines 1-11; paragraphs 29 and 30; Li, paragraphs 87 and 88; paragraphs 114 and 115; the motivation to combine Li is substantially similar to that discussed above in the rejection of claim 1).
Regarding claim 5, Yu in view of Li teaches the computer-implemented method of claim 4, wherein determining the one or more diffusion conditions using the embeddings, the image domain noise input, and the vector domain noise input comprises determining, using a transformer neural network of the multi-domain diffusion neural network, the one or more diffusion conditions from the embeddings, the image domain noise input, and the vector domain noise input (Yu, paragraph 21, lines 1-15; paragraph 23; paragraph 24, lines 1-11; paragraph 27, lines 1-9; paragraphs 83 and 84; Li, paragraphs 88 and 115; the motivation to combine Li is substantially similar to that discussed above in the rejection of claim 1).
Regarding claim 7, Yu in view of Li teaches the computer-implemented method of claim 1, wherein generating the layout for the digital design using at least one of the
image diffusion branch or the vector diffusion branch of the multi-domain diffusion neural network comprises generating an image domain layout for the digital design using the image diffusion branch of the multi-domain diffusion neural network, the image domain layout comprising a digital image portraying one or more image elements (Yu, figs 3, 4, and 6; paragraph 21, the last 14 lines; paragraph 24, lines 7-11; paragraph 87; Li, paragraphs 87 and 88; paragraphs 114 and 115; the motivation to combine Li is substantially similar to that discussed above in the rejection of claim 1).
Regarding claim 8, Yu in view of Li teaches the computer-implemented method of claim 1, wherein generating the layout for the digital design using at least one of the image diffusion branch or the vector diffusion branch of the multi-domain diffusion neural network comprises generating a vector domain layout for the digital design using the vector diffusion branch of the multi-domain diffusion neural network, the vector domain layout comprising a plurality of bounding boxes for the plurality of input elements arranged on a canvas (Yu, paragraph 23; paragraph 24, lines 1-11; Li, paragraphs 87 and 88; paragraphs 114 and 115; the motivation to combine Li is substantially similar to that discussed above in the rejection of claim 1).
Regarding claim 9, Yu in view of Li teaches the computer-implemented method of claim 8, further comprising: detecting, via a graphical user interface of the client device, a user interaction with a bounding box from the plurality of bounding boxes; and modifying the vector domain layout by moving the bounding box within the vector domain layout in accordance with the user interaction (Yu, paragraph 24; paragraph 56; Li, paragraph 43, lines 1-19; paragraph 46; the motivation to combine Li is substantially similar to that discussed above in the rejection of claim 1).
Regarding claim 10, Yu in view of Li teaches the computer-implemented method of claim 1, further comprising: receiving, from the client device, a style template for generating the digital design; and generating, using an additional encoder of the multi-domain diffusion neural network, a style embedding from the style template, wherein generating, using at least one of the image diffusion branch or the vector diffusion branch of the multi-domain diffusion neural network, the layout for the digital design from the visual characteristics and bounding box characteristics of the embeddings comprises generating, using at least one of the image diffusion branch or the vector diffusion branch of the multi-domain diffusion neural network, the layout for the digital design from the embeddings and the style embedding (Yu, figs 3, 4, and 6; paragraph 21, lines 1-21; paragraph 22, lines 1-12; paragraph 24; paragraph 27, lines 1-9; paragraphs 83-85; Li, paragraph 70; paragraphs 86 and 88; paragraph 114; the motivation to combine Li is substantially similar to that discussed above in the rejection of claim 1).
Regarding claim 11, Yu teaches a system comprising: one or more memory devices; and one or more processors (fig 7; paragraph 60) configured to cause the system to:
receive a plurality of image elements for generating a digital design (figs 1 and 2; fig 8, client device 810; paragraph 68; paragraphs 79 and 81; receiving a plurality of input elements from a client device for a digital design); and
generate layouts for the digital design from the plurality of image elements by using a multi-domain neural network (fig 1; paragraph 21, lines 1-11; paragraph 24, lines 10-11; paragraph 87, the last 6 lines) to:
generate embeddings representing visual characteristics (paragraph 21, lines 1-11; paragraph 22, lines 1-12; paragraph 82; embeddings are generated using the encoder that represent visual characteristics such as image, text, etc.) and bounding box characteristics of the plurality of image elements (paragraph 24, lines 1-7; paragraph 27, lines 1-6; paragraph 86; bounding box characteristics for the input elements are also generated);
determine conditions for generating the digital design from the embeddings; generate, via an image branch of the multi-domain neural network, an image domain layout for the digital design from the conditions; and generate, via a vector branch of the multi-domain neural network, a vector domain layout for the digital design from the conditions (figs 1 and 2; paragraphs 20 and 23; paragraph 24, lines 1-11; paragraphs 29 and 30).
Yu does not teach a diffusion network, diffusion branches, or diffusion conditions.
However, in a similar field of endeavor, Li teaches a computer-implemented method comprising receiving input elements from a client device for generating a digital design, generating a layout for the design using an image editing network, and providing the layout for display on the client device (paragraph 43, lines 1-19; paragraphs 46-48; paragraphs 118 and 119), wherein the network is a diffusion network with diffusion branches (figs 3 and 4; paragraph 87; paragraph 114; paragraph 115, lines 1-11), and wherein diffusion conditions are determined for generating the digital design (paragraph 88; paragraphs 114 and 115).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the diffusion network, branches, and conditions of Li with the network, branches, and conditions of Yu because this improves the efficiency and quality of multi-model image processing comprising text, image, video, etc., while reducing the operational complexity for the user and improving the user experience (see, for example, Li, paragraphs 4 and 26).
Regarding claim 12, Yu in view of Li teaches the system of claim 11, wherein the one or more processors are configured to cause the system to: generate the image domain layout via the image diffusion branch by generating the image domain layout using a convolution-based diffusion neural network; and generate the vector domain layout via the vector diffusion branch by generating the vector domain layout using a transformer-based diffusion neural network (Yu, figs 1 and 2; paragraph 21, lines 1-11; paragraph 23; paragraph 24, lines 1-11; paragraphs 83 and 84; Li, paragraphs 87 and 88; paragraphs 114 and 115; the motivation to combine Li is substantially similar to that discussed above in the rejection of claim 1).
Regarding claim 13, Yu in view of Li teaches the system of claim 11, wherein the one or more processors are further configured to cause the system to: determining an image domain noise input for the image diffusion branch and a vector domain noise input for the vector diffusion branch (Yu, paragraph 23; paragraph 24, lines 1-11; Li, paragraphs 87 and 88; paragraphs 114 and 115; the motivation to combine Li is substantially similar to that discussed above in the rejection of claim 1); and
determine the diffusion conditions from the embeddings by determining the diffusion conditions from the embeddings, the image domain noise input, and the vector domain noise input (Yu, paragraphs 20 and 23; paragraph 24, lines 1-11; paragraphs 29 and 30; Li, paragraph 88; paragraphs 114 and 115; the motivation to combine Li is substantially similar to that discussed above in the rejection of claim 1).
Regarding claim 14, Yu in view of Li teaches the system of claim 11, wherein the one or more processors are further configured to cause the system to: determine an image domain noise input for the image diffusion branch and a vector domain noise input for the vector diffusion branch; and generate, via the image diffusion branch, the image domain layout from the diffusion conditions by generating, via the image diffusion branch, the image domain layout from the diffusion conditions, the image domain noise input, and the vector domain noise input (Yu, paragraph 23; paragraph 24, lines 1-11; Li, paragraphs 87 and 88; paragraphs 114 and 115; the motivation to combine Li is substantially similar to that discussed above in the rejection of claim 1).
Regarding claim 16, Yu in view of Li teaches the system of claim 15, wherein: the one or more processors are further configured to cause the system to generate, using an encoder of the image diffusion branch, a first set of intermediate features from the diffusion conditions and the image domain noise input; generating, using the vector diffusion branch, the intermediate features from the diffusion conditions and the vector domain noise input comprises generating, using the vector diffusion branch, a second set of intermediate features from the diffusion conditions, the vector domain noise input, and the first set of intermediate features; and generating, using the image diffusion branch, the image domain layout from the intermediate features comprises generating, using a decoder of the image diffusion branch, the image domain layout from the second set of intermediate features (Yu, figs 1 and 2; paragraph 23; paragraph 24, lines 1-11; paragraphs 83 and 84; Li, paragraphs 87 and 88; paragraphs 114 and 115; the motivation to combine Li is substantially similar to that discussed above in the rejection of claim 1).
Regarding claim 17, Yu in view of Li teaches the system of claim 11, wherein the one or more processors are further configured to cause the system to: determine a vector domain noise input for the vector diffusion branch and an image domain noise input for the image diffusion branch; and generate, via the vector diffusion branch, the vector domain layout from the diffusion conditions by generating, via the vector diffusion branch, the vector domain layout from the diffusion conditions, the vector domain noise input, and the image domain noise input (Yu, paragraph 23; paragraph 24, lines 1-11; Li, paragraphs 87 and 88; paragraphs 114 and 115; the motivation to combine Li is substantially similar to that discussed above in the rejection of claim 1).
Regarding claim 18, Yu teaches a non-transitory computer-readable medium storing executable instructions which, when executed by a processing device, cause the processing device to perform operations (fig 7; paragraph 60) comprising:
generating, using an encoder of a multi-domain neural network, embeddings representing visual characteristics (fig 1; paragraph 21, lines 1-11; paragraph 22, lines 1-12; paragraph 82; embeddings are generated using the encoder that represent visual characteristics such as image, text, etc.) and bounding box characteristics of a plurality of image elements for generating a digital design (paragraph 24, lines 1-7; paragraph 27, lines 1-6; paragraph 86; bounding box characteristics for the input elements are also generated);
determining an image domain noise input for an image branch of the multi-domain neural network and a vector domain noise input for a vector branch of the multi-domain neural network (figs 1 and 2; paragraph 23; paragraph 24, lines 1-11);
generating, from the embeddings, one or more cross-domain representations of the plurality of image elements by exchanging the image domain noise input and the vector domain noise input between the image branch and the vector branch (figs 1 and 2; paragraph 23; paragraph 24, lines 1-11; paragraphs 27 and 39; see also the additional citations to Li, below, regarding exchanging noise input); and
generating, using the image branch or the vector branch of the multi-domain neural network, a layout for the digital design from the one or more cross-domain representations of the plurality of image elements (paragraph 23; paragraph 24, lines 1-11; paragraph 87).
Yu does not teach a diffusion network or diffusion branches.
However, in a similar field of endeavor, Li teaches a computer-implemented method comprising receiving input elements from a client device for generating a digital design, generating a layout for the design using an image editing network, and providing the layout for display on the client device (paragraph 43, lines 1-19; paragraphs 46-48; paragraphs 118 and 119), wherein representations of the image elements are generated by exchanging image noise and vector noise between branches (fig 3; paragraph 88; paragraphs 100-103; paragraphs 115 and 116), and wherein the network is a diffusion network with diffusion branches (figs 3 and 4; paragraph 87; paragraph 114; paragraph 115, lines 1-11).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the diffusion network and branches of Li with the network and branches of Yu because this improves the efficiency and quality of multi-model image processing comprising text, image, video, etc., while reducing the operational complexity for the user and improving the user experience (see, for example, Li, paragraphs 4 and 26).
Regarding claim 19, Yu in view of Li teaches the non-transitory computer-readable medium of claim 18, wherein generating the one or more cross-domain representations of the plurality of image elements comprises generating one or more diffusion conditions for the image diffusion branch and the vector diffusion branch, the one or more diffusion conditions incorporating the image domain noise input and the vector domain noise input (Yu, paragraphs 20 and 23; paragraph 24, lines 1-11; paragraphs 29 and 30; Li, paragraph 88; paragraphs 114 and 115; the motivation to combine Li is substantially similar to that discussed above in the rejection of claim 1).
Allowable Subject Matter
Claims 6, 15, and 20 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims. The following is a statement of reasons for the indication of allowable subject matter:
Regarding claim 6, the prior art teaches a multi-domain diffusion neural network to generate a layout of a digital design, as described in claim 1, among other claim features. In the context of claims 1, 3, and 6 as a whole, however, the prior art does not appear to teach the method, further comprising determining an image domain noise input and a vector domain noise input for generating the digital design, wherein generating, using at least one of the image diffusion branch or the vector diffusion branch of the multi-domain diffusion neural network, the layout for the digital design from the visual characteristics and bounding box characteristics of the embeddings comprises generating the layout for the digital design from the embeddings, the image domain noise input, and the vector domain noise input, wherein generating the layout for the digital design from the embeddings, the image domain noise input, and the vector domain noise input comprises generating, using the vector diffusion branch of the multi-domain diffusion neural network, intermediate features from the embeddings and the vector domain noise input, and generating, using the image diffusion branch of the multi-domain diffusion neural network, the layout for the digital design from the embeddings, the image domain noise input, and the intermediate features.
Regarding claim 15, in the context of claims 11, 14, and 15 as a whole, this claim comprises allowable subject matter for similar reasons as those discussed above with respect to claim 6.
Regarding claim 20, in the context of claims 18 and 20 as a whole, this claim comprises allowable subject matter for similar reasons as those discussed above with respect to claim 6.
Response to Arguments
On pages 14 and 15 of the Applicant’s Remarks, with respect to the 103 rejection of claim 1, the Applicant argues that 1) Yu discloses a multi-modal neural network, but does not disclose multiple domains, or an image diffusion branch and a vector diffusion branch, and 2) Li discloses a diffusion neural network, but fails to teach a multi-domain diffusion neural network. The Examiner respectfully disagrees with these arguments.
Regarding the first argument, it is respectfully submitted that Yu and Li clearly disclose multiple domains in several reasonable interpretations, such as vector and image domains, text and image domains, or source and target domains, and so on. Additionally, Yu teaches that the multi-modal neural network comprises, inter alia, a vector branch (as one example, in figure 1, the branch including encoders 114-120) and an image branch (as one example, in figure 1, the branch including encoders 126 and 128). Thus Yu clearly teaches the broad term “multi-domain neural network” and discloses that the network includes vector and image branches. As discussed in the Office Actions, it is agreed that Yu does not disclose a diffusion network or diffusion branches; however, Yu is not relied upon to teach the diffusion network and branches, as this is disclosed by Li. Similarly, and with respect to the second argument, Li is not relied upon to teach a multi-domain neural network, as this is clearly disclosed in Yu. Accordingly, while neither reference individually teaches a multi-domain diffusion neural network, it is respectfully submitted that the combination of Yu and Li clearly teaches this feature. It is noted that one cannot show nonobviousness by attacking references individually where the rejections are based on combinations of references. See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981); In re Merck & Co., 800 F.2d 1091, 231 USPQ 375 (Fed. Cir. 1986).
On pages 15 and 16 of the Applicant’s Remarks, with respect to the 103 rejection of claim 11, the Applicant argues that the combination of Yu and Li does not teach the claim for reasons similar to those discussed above with respect to claim 1. The Examiner respectfully disagrees with this argument, for the reasons discussed above.
On pages 16 and 17 of the Applicant’s Remarks, with respect to the 103 rejection of claim 1, the Applicant argues that 1) the combination of Yu and Li does not teach the claim for reasons similar to those discussed above with respect to claim 1, 2) neither Yu nor Li teach an image noise input and a vector noise input, and 3) the models in Yu and Li appear to operate in a single domain such as an image domain. The Examiner respectfully disagrees with these arguments.
Regarding the first argument, the Examiner respectfully disagrees with this argument, for the reasons discussed above. Regarding the second argument, as discussed above, Yu teaches the vector branch and image branch, and each of those branches is disclosed as including its own noise input (as one example, in figure 1, the vector branch noise input 120 and the image branch noise input 126). Further, Li discloses that its diffusion models each include a noise input (as one example, figure 3, models 300 and noise input 320). Thus, it is respectfully submitted that Yu in view of Li teaches this feature of the claims. Regarding the third argument, as discussed above, Yu and Li clearly disclose multiple domains in several reasonable interpretations, such as vector and image domains, text and image domains, or source and target domains, and so on. It is respectfully submitted that any potential differences between the prior art and the broad term “multi-domain” in the claims are not currently reflected in the claims.
Conclusion
The following prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
Ren (U.S. Patent Application Publication No. 2026/0094371); 3D scene reconstruction using voxelized gaussian splat representations.
Drolet (U.S. Patent Application Publication No. 2025/0131928); AI-generated music derivative works.
Xiang (U.S. Patent Application Publication No. 2025/0292455); Image processing method and apparatus, computer device, and computer-readable storage medium.
Zhu (U.S. Patent Application Publication No. 2026/0004475); Image processing method and apparatus, device, medium, and program product.
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DAVID T WELCH whose telephone number is (571)270-5364. The examiner can normally be reached Monday-Thursday, 8:30-5:30 EST, and alternate Fridays, 9:00-2:30 EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Xiao Wu can be reached at 571-272-7761. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
DAVID T. WELCH
Primary Examiner
Art Unit 2613
/DAVID T WELCH/Primary Examiner, Art Unit 2613
Prosecution Insights