DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1, 3-7, 9, 11, 13-16 and 18-20 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Liu et al. (“3DALL-E: Integrating Text-to-Image AI in 3D Design Workflows”, arXiv:2210.11603v1 [cs.HC], URL: https://arxiv.org/pdf/2210.11603v1).
Regarding claim 19, Liu teaches a computing system, comprising:
one or more processors; and one or more non-transitory computer-readable media storing instructions that are executable to cause the one or more processors to perform operations, the operations comprising:
Obtaining, by a prompt element suggestion model, from a client device, user input data comprising initial prompt data (“Users begin at the starting state shown in Fig. 2-I, where they can describe what they want to make by typing in their goal (Fig. 2A)” (Liu: 3.1, para 1));
Selecting, by the prompt element suggestion model, one or more suggested prompt elements based at least in part on the initial prompt data (“Once they do that, different prompt suggestions populate the sections with 3D keywords, designs/styles, and parts (Fig. 2-II)” (Liu: 3.1, para 1));
Transmitting, by the prompt element suggestion model, to the client device, the one or more suggested prompt elements to be presented for display as selectable user interface elements via a user interface (“Prompt suggestions (Fig. 2C–E) are color-coded with a color for the group they belong to (blue for designs, green for styles, orange for parts) and varied in opacity to indicate how strongly their text aligns with the image prompt (see Fig. 4 for implementation details)” (Liu: 3.1, para 2). Also, refer to FIG.2-II of Liu.);
Obtaining, by the prompt element suggestion model, from the client device, second user input data comprising data indicative of a selection of the one or more suggested prompt elements (“Once users select a set of prompt suggestions (e.g. “3d render, isometric, plant stool, wrought-iron”), an automatically rephrased prompt appears in the final prompt box (e.g. “isometric 3d render of a wrought-iron plant stool“) as shown in Fig. 2-III.” (Liu: 3.1, para 1));
And responsive to obtaining the second user input data, providing the second user input to an image generation model to generate an output image comprising a visual representation associated with the one or more suggested prompt elements (the results shown Fig.2-V of Liu).
Claim(s) 1 is a corresponding method claim(s) of claim(s) 19. The limitations of claim(s) 1 are substantially similar to the limitations of claim(s) 19. Therefore, it has been analyzed and rejected substantially similar to claim(s) 1.
Claim(s) 20 is a corresponding computer readable media claim(s) of claim(s) 19. The limitations of claim(s) 20 are substantially similar to the limitations of claim(s) 19. Therefore, it has been analyzed and rejected substantially similar to claim(s) 20.
Regarding claim 3, Liu teaches the computer-implemented method of claim 1, wherein the prompt element suggestion model comprises a machine learning language model (we integrated three large AI models—DALL-E, GPT-3, and CLIP (Liu: 1, para 3)).
Regarding claim 4, Liu teaches the computer-implemented method of claim 1, wherein the image generation model is configured to perform operations comprising:
Generating, based on the obtained second user input data, an output image (the resulting image shown in Fig.2-V);
Performing a validation operation based on the output image (user determining whether to use the shuffle icon when the user wishes to a different result);
And updating the image generation model based on the performed validation operation (the resulting image after pressing the shuffle icon).
Regarding claim 5, Liu teaches the computer-implemented method of claim 1, comprising a prompt element generation component comprising a machine learning language model configured to generate one or more prompt elements (Prompt suggestions were populated by querying the GPT-3 API (Liu: 4, para 2)).
Regarding claim 6, Liu teaches the computer-implemented method of claim 5, wherein generating the suggested prompt elements comprises:
Obtaining the initial prompt data and context data (Users can also choose to include an image that is automatically extracted from their current 3D modeling workspace (“context data”) in addition to their text prompt (Liu: 3.2));
Obtaining one or more content elements from a content provider inventory database (keywords are obtained from a Fusion 360 Screencast dataset (Liu: 4, para 3));
And generating, based on the initial prompt data, the context data, and the one or more content elements from the content provider inventory database, one or more prompt elements (the resulting image based on the parameters determined by the user above).
Regarding claim 7, Liu teaches the computer-implemented method of claim 6, wherein generating the one or more prompt elements comprises:
Determining a similarity between a first characteristic associated with a first content element from the content provider inventory database that aligns with the obtained context data (CLIP produces logit scores that suggest how similar each text option was to the image (Liu: 4, para 4));
And generating a first prompt element based on the determined similarity and the first content element (the resulting prompt as shown in Fig.2-II of Liu).
Regarding claim 9, Liu teaches the computer-implemented method of claim 1, wherein the one or more suggested prompt elements are generated in real-time (“Users begin at the starting state shown in Fig. 2-I, where they can describe what they want to make by typing in their goal (Fig. 2A). Once they do that, different prompt suggestions populate the sections with 3D keywords, designs/styles, and parts (Fig. 2-II)” (Liu: 3.1, para 1). The Examiner submits that in the context of text-to-image AI the “once they do that” corresponds to “real-time” generation of prompts as presently claimed).
Regarding claim 13, Liu teaches the computer-implemented method of claim 1, wherein selecting, by the prompt element suggestion model, the one or more suggested prompt elements is based on at least one of a quality associated with a suggested prompt element, a relevance of a suggested prompt element, or a ranking of a suggested prompt element (allows the plugin (3DALL-E) to highlight what prompt suggestions may work best, which is implemented in the system by using CLIP (Liu: 1, para 3)).
Regarding claim 14, Liu teaches the computer-implemented method of claim 1, wherein selecting, by the prompt element suggestion model, one or more suggested prompt elements comprises:
Determining, based on the obtained initial prompt data, one or more keywords (“Ten3Dkeywords are sampled from a set of high frequency words (n=121) in a Fusion 360 Screencast dataset.” (Liu: 4, para 3). “After a designer inputs their goals (i.e. to design a "truck"), the plugin provides a number of related parts, styles, and designs that help users craft text prompts” (Liu: 1, para 3)); transmitting a request for data comprising one or more prompt elements associated with the one or more keywords (Prompt suggestions were populated (“transmitting”) by querying the GPT-3 API (Liu: 4, para 2)); and obtaining the one or more prompt elements associated with the one or more keywords (the resulting prompt suggestions as taught by Liu above).
Regarding claim 15, Liu teaches the computer-implemented method of claim 1, wherein the prompt element suggestion model is a component of an image generation model (prompt suggestions from GPT-3 that is part of the 3DALL-E system (Liu: Fig.2)).
Regarding claim 16, Liu teaches the computer-implemented method of claim 15, wherein the image generation model is a generative machine learning model (text-to-image AI systems such as DALL-E (Liu: 2.1, 3)).
Regarding claim 18, Liu teaches the computer-implemented method of claim 1, wherein the selecting, by the prompt element suggestion model, the one or more suggested prompt elements, is performed based at least in part on a bidding process (refer to the process of selecting the final keywords set and the determination of logit scores (Liu: 4, para 2-3)).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 2 and 17 is/are rejected under 35 U.S.C. 103 as being unpatentable over Liu as applied to the claim(s) above, and further in view of Qazvinian et al. (PGPUB Document No. US 2025/0013963).
Regarding claim 2, Liu does not expressly teach but Qazvinian teaches the computer-implemented method of claim 1, comprising: updating the prompt element suggestion model based on the second user input data (updating the suggested categories based on user input prompts (Qazvinian: 0122)).
Therefore, before the effective filing date of the claimed invention, it would have been obvious to one of an ordinary skill in the art to modify the teachings of Liu such as to utilize the teaching of Qazvinian, because this enables improved accuracy in better guiding the user towards the desired output.
Regarding claim 10, Liu does not expressly teach but Qazvinian teaches the computer-implemented method of claim 1, wherein the one or more suggested prompt elements and associated data are stored in a cache (this engine may utilize prompt optimization techniques, such as, for example, prompt rewriting, indexing, and caching, to expedite the retrieval and processing of relevant data (Qazvinian: 0082)).
Therefore, before the effective filing date of the claimed invention, it would have been obvious to one of an ordinary skill in the art to modify the teachings of Liu such as to utilize a cache as suggested by Qazvinian, because this enables expedited retrieval and processing of relevant data.
Regarding claim 17, Liu does not expressly teach but Qazvinian teaches the computer-implemented method of claim 1, wherein the prompt element data comprises a token (the generative AI model may tokenize the received prompt into subword units or word-level tokens (Qazvinian: 0057)).
Therefore, before the effective filing date of the claimed invention, it would have been obvious to one of an ordinary skill in the art to modify the teaching of Liu such as to apply the token teaching of Qazvinian, because this enable an effective and efficient method of processing prompts.
Allowable Subject Matter
Claims 8, 11 and 12 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to David H Chu whose telephone number is (571)272-8079. The examiner can normally be reached M-F: 9:30 - 1:30pm, 3:30-8:30pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Daniel F Hajnik can be reached at (571) 272-7642. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DAVID H CHU/Primary Examiner, Art Unit 2616
Prosecution Insights