DETAILED ACTION
The action is responsive to the amendment filed on 06/08/2026. Claims 1-20 are pending in the case. Claims 1, 19 and 20 are independent claims.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The filing of a terminal disclaimer by itself is not a complete reply to a nonstatutory double patenting (NSDP) rejection. A complete reply requires that the terminal disclaimer be accompanied by a reply requesting reconsideration of the prior Office action. Even where the NSDP rejection is provisional the reply must be complete. See MPEP § 804, subsection I.B.1. For a reply to a non-final Office action, see 37 CFR 1.111(a). For a reply to final Office action, see 37 CFR 1.113(c). A request for reconsideration while not provided for in 37 CFR 1.113(c) may be filed after final for consideration. See MPEP §§ 706.07(e) and 714.13.
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer.
Claims 1, 2, 5, 7, 19 and 20 are rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1, 3, 8 and 13 of U.S. Patent No. 12282755 B2. Although the claims at issue are not identical, they are not patentably distinct from each other because the claims are an obvious variant of the claim set from the '755 patent only including minor differences in structure and design styles having affinities for distances, see chart below where bolded portions are similar. It would have been obvious to allow the user to specify design styles that include an affinity associated with distances between elements and determining / adjusting the distance so that the user has more control over the visual style of the generated GUI.
Application No. 19/459256
U.S. Patent No. 12282755 B2
1. A non-transitory computer readable medium storing computer implementable instructions that when executed by at least one processor cause the at least one processor to perform operations for generation of user interfaces from natural language inputs, the operations comprising:
receiving an input in a natural language, the input is associated with a desire of an individual to create a new user interface;
obtaining an indication of a particular design style, wherein the particular design style includes affinity associated with distances between elements;
obtaining at least two user interface elements;
analyzing the input in the natural language to obtain a layout for the at least two user interface elements by determining a distance between the at least two user interface elements based on the particular design style that includes the affinity associated with distances between elements;
generating a particular draft of the new user interface that includes the at least two user interface elements based on the layout; and
presenting the particular draft of the new user interface to the individual.
2. The non-transitory computer readable medium of claim 1, wherein the operations further comprise analyzing at least one other user interface associated with a specific designer persona to determine the particular design style that includes the affinity associated with distances between elements.
5. The non-transitory computer readable medium of claim 1, wherein the operations further comprise:
after the presenting the particular draft of the new user interface to the individual, receiving a modification to the input from the individual; and
analyzing the modified input to modify the new user interface by at least adjusting the distance between the at least two user interface elements.
7. The non-transitory computer readable medium of claim 1, wherein the operations further comprise analyzing the input in the natural language to determine at least one of a background music or a sound effect compatible with a description included in the input for the new user interface, and wherein the generation of the particular draft of the new user interface includes the at least one of the background music or the sound effect.
1. A non-transitory computer readable medium storing a software program comprising data and computer implementable instructions that when executed by at least one processor cause the at least one processor to perform operations for generation of user interfaces from free text, the operations comprising:
receiving a textual input in a natural language, the textual input is associated with a desire of an individual to create a new user interface;
analyzing at least one other user interface to determine a particular design style;
analyzing the textual input to obtain at least two user interface elements;
analyzing the textual input to obtain a layout for the at least two user interface elements based on the particular design style determined by analyzing the at least one other user interface;
generating a particular draft of the new user interface that includes the at least two user interface elements based on the layout; and
presenting the particular draft of the new user interface to the individual in a form that enables the individual to modify the new user interface.
3. (Original) The non-transitory computer readable medium of claim 1, wherein the operations further comprise:
after the presenting the particular draft of the new user interface to the individual, receiving an additional textual input from the individual; and
analyzing the additional textual input to modify the new user interface.
8. The non-transitory computer readable medium of claim 1, wherein the operations further comprise analyzing the textual input to determine background music for the new user interface, and wherein the generation of the particular draft of the new user interface is further based on the background music.
Claims 1, 2, 4-16, 19 and 20 provisionally rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1-20 of copending Application No. 19/098533 (reference application). Although the claims at issue are not identical, they are not patentably distinct from each other because the claims are an obvious variant of the claim set from the 19/098533 application only including minor differences in structure and design styles having affinities for distances, see chart below where bolded portions are similar. It would have been obvious to allow the user to specify design styles that include an affinity associated with distances between elements and determining / adjusting the distance so that the user has more control over the visual style of the generated GUI. It would have been obvious to determine animations and proportions of elements so that the visual style of the generated GUI could be more varied. It would have been obvious to use a large language model to analyze natural language input so that more kinds of devices could be better supported by the finished product including those that include LLM accelerators.
Application No. 19/459256
Application No. 19/098533
1. A non-transitory computer readable medium storing computer implementable instructions that when executed by at least one processor cause the at least one processor to perform operations for generation of user interfaces from natural language inputs, the operations comprising:
receiving an input in a natural language, the input is associated with a desire of an individual to create a new user interface;
obtaining an indication of a particular design style, wherein the particular design style includes affinity associated with distances between elements;
obtaining at least two user interface elements;
analyzing the input in the natural language to obtain a layout for the at least two user interface elements by determining a distance between the at least two user interface elements based on the particular design style that includes the affinity associated with distances between elements;
generating a particular draft of the new user interface that includes the at least two user interface elements based on the layout; and
presenting the particular draft of the new user interface to the individual.
2. The non-transitory computer readable medium of claim 1, wherein the operations further comprise analyzing at least one other user interface associated with a specific designer persona to determine the particular design style that includes the affinity associated with distances between elements.
4. The non-transitory computer readable medium of claim 1, wherein the presentation of the particular draft of the new user interface to the individual enables the individual to modify a type of a user interface element in the new user interface.
5. The non-transitory computer readable medium of claim 1, wherein the operations further comprise:
after the presenting the particular draft of the new user interface to the individual, receiving a modification to the input from the individual; and
analyzing the modified input to modify the new user interface by at least adjusting the distance between the at least two user interface elements.
6. The non-transitory computer readable medium of claim 1, wherein the operations further comprise analyzing the input in the natural language to obtain an animation for a visual background for the new user interface, and wherein the generation of the particular draft of the new user interface is further based on the visual background.
7. The non-transitory computer readable medium of claim 1, wherein the operations further comprise analyzing the input in the natural language to determine at least one of a background music or a sound effect compatible with a description included in the input for the new user interface, and wherein the generation of the particular draft of the new user interface includes the at least one of the background music or the sound effect.
8. The non-transitory computer readable medium of claim 1, wherein the operations further comprise using a Large Language Model to analyze the input in the natural language to determine a type of a user interface element of the at least two user interface elements.
9. The non-transitory computer readable medium of claim 1, wherein the particular design style further includes and affinity associated with sizes of user interface elements, and a size of a user interface element in the new user interface is determined based on the affinity associated with sizes.
10. The non-transitory computer readable medium of claim 1, wherein the operations further comprise using a Large Language Model to analyze the input in the natural language to determine a content of a user interface element in the new user interface.
11. The non-transitory computer readable medium of claim 1, wherein the operations further comprise using a Large Language Model to analyze the input in the natural language to determine a font associated with a user interface element in the new user interface.
12. The non-transitory computer readable medium of claim 1, wherein the input includes at least five of a noun, a verb, a proposition, a pronoun, an adverb, an adjective, a conjunction or an interjection.
13. The non-transitory computer readable medium of claim 1, wherein the input includes an adjective that describes a shade of a color, and the operations further comprise determining a shade of a color associated with at least part of a particular user interface element in the new user interface based on the adjective.
14. The non-transitory computer readable medium of claim 1, wherein the input includes an adjective that describes shape, and the operations further comprise determining a proportion between height and width of a shape of a particular user interface element in the new user interface based on the adjective.
15. The non-transitory computer readable medium of claim 1, wherein the input includes an adverb that describes change, and the operations further comprise determining a a pace of a change associated with a particular user interface element in the new user interface based on the adverb.
16. The non-transitory computer readable medium of claim 1, wherein at least part of the at least two user interface elements are based on a demographic characteristic of the individual.
1. A non-transitory computer readable medium storing a software program comprising data and computer implementable instructions that when executed by at least one processor cause the at least one processor to perform operations for generation of user interfaces from natural language inputs, the operations comprising:
receiving an input in a natural language, the input is associated with a desire of an individual to create a new user interface;
determining a particular design style;
analyzing the input in the natural language to obtain at least two user interface elements based on the particular design style;
analyzing the input in the natural language to obtain a layout for the at least two user interface elements;
generating a particular draft of the new user interface that includes the at least two user interface elements based on the layout; and
presenting the particular draft of the new user interface to the individual.
2. The non-transitory computer readable medium of claim 1, wherein the operations further comprise analyzing at least one other user interface to determine the particular design style.
4. The non-transitory computer readable medium of claim 1, wherein the presentation of the particular draft of the new user interface to the individual enables the individual to perform at least four of adding a new user interface element to the new user interface, removing a user interface element from the new user interface, modifying a type of a user interface element in the new user interface, modifying a position of a user interface element in the new user interface, modifying a size of a user interface element in the new user interface, modifying a content of a user interface element in the new user interface, or modifying a font associated with a user interface element in the new user interface.
5. The non-transitory computer readable medium of claim 1, wherein the operations further comprise:
after the presenting the particular draft of the new user interface to the individual, receiving a modification to the input from the individual; and
analyzing the modified input to modify the new user interface.
6. The non-transitory computer readable medium of claim 1, wherein the operations further comprise analyzing the input in the natural language to obtain a visual background for the new user interface, and wherein the generation of the particular draft of the new user interface is further based on the visual background.
7. The non-transitory computer readable medium of claim 1, wherein the operations further comprise analyzing the input in the natural language to determine a sound effect for the new user interface, and wherein the generation of the particular draft of the new user interface is further based on the sound effect.
8. The non-transitory computer readable medium of claim 1, wherein the operations further comprise analyzing the input in the natural language to determine a type of a user interface element in the new user interface.
9. The non-transitory computer readable medium of claim 1, wherein the operations further comprise analyzing the input in the natural language to determine a size of a user interface element in the new user interface.
10. The non-transitory computer readable medium of claim 1, wherein the operations further comprise analyzing the input in the natural language to determine a content of a user interface element in the new user interface.
11. The non-transitory computer readable medium of claim 1, wherein the operations further comprise analyzing the input in the natural language to determine a font associated with a user interface element in the new user interface.
12. The non-transitory computer readable medium of claim 1, wherein the input includes at least five of a noun, a verb, a proposition, a pronoun, an adverb, an adjective, a conjunction or an interjection.
13. The non-transitory computer readable medium of claim 1, wherein the input includes an adjective that describes color, and the operations further comprise determining a color associated with at least part of a particular user interface element in the new user interface based on the adjective.2
14. The non-transitory computer readable medium of claim 1, wherein the input includes an adjective that describes shape, and the operations further comprise determining a shape of a particular user interface element in the new user interface based on the adjective.
15. The non-transitory computer readable medium of claim 1, wherein the input includes an adverb that describes change, and the operations further comprise determining a change pattern associated with a particular user interface element in the new user interface based on the adverb.
16. The non-transitory computer readable medium of claim 1, wherein at least part of the at least two user interface elements are based on a characteristic of the individual.
This is a provisional nonstatutory double patenting rejection because the patentably indistinct claims have not in fact been patented.
Claims 1-3 and 5-11 are rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1-3, 5, 7, 14, 17 and 18 of U.S. Patent No. 12536003 B2. Although the claims at issue are not identical, they are not patentably distinct from each other because the claims are an obvious variant of the claim set from the '003 patent only including minor differences in structure and analyzing text input to determine UI elements and images, see chart below where bolded portions are similar. It would have been obvious to allow the user to use text input to determine UI elements and images so that more kinds of user-inputted data could be used for the determination thus allowing the user greater control over the generation of the draft UI. It would have been obvious to use a large language model to analyze natural language input so that more kinds of devices could be better supported by the finished product including those that include LLM accelerators.
Application No. 19/459256
U.S. Patent No. 12536003 B2
1. A non-transitory computer readable medium storing computer implementable instructions that when executed by at least one processor cause the at least one processor to perform operations for generation of user interfaces from natural language inputs, the operations comprising:
receiving an input in a natural language, the input is associated with a desire of an individual to create a new user interface;
obtaining an indication of a particular design style, wherein the particular design style includes affinity associated with distances between elements;
obtaining at least two user interface elements;
analyzing the input in the natural language to obtain a layout for the at least two user interface elements by determining a distance between the at least two user interface elements based on the particular design style that includes the affinity associated with distances between elements;
generating a particular draft of the new user interface that includes the at least two user interface elements based on the layout; and
presenting the particular draft of the new user interface to the individual.
2. The non-transitory computer readable medium of claim 1, wherein the operations further comprise analyzing at least one other user interface associated with a specific designer persona to determine the particular design style that includes the affinity associated with distances between elements.
5. The non-transitory computer readable medium of claim 1, wherein the operations further comprise:
after the presenting the particular draft of the new user interface to the individual, receiving a modification to the input from the individual; and
analyzing the modified input to modify the new user interface by at least adjusting the distance between the two user interface elements.
6. The non-transitory computer readable medium of claim 1, wherein the operations further comprise analyzing the input in the natural language to obtain an animation for a visual background for the new user interface, and wherein the generation of the particular draft of the new user interface is further based on the visual background.
7. The non-transitory computer readable medium of claim 1, wherein the operations further comprise analyzing the input in the natural language to determine at least one of a background music or a sound effect compatible with a description included in the input for the new user interface, and wherein the generation of the particular draft of the new user interface includes the at least one of the background music or the sound effect.
8. The non-transitory computer readable medium of claim 1, wherein the operations further comprise using a large language model to analyze the input in the natural language to determine a type of a user interface element of the at least two user interface elements.
9. The non-transitory computer readable medium of claim 1, wherein the particular design style includes an affinity associated with sizes of user interface elements, and the operations further comprise analyzing the input in the natural language to determine a size of a user interface element in the new user interface.
10. The non-transitory computer readable medium of claim 1, wherein the operations further comprise using a large language model to analyze the input in the natural language to determine a content of a user interface element in the new user interface.
11. The non-transitory computer readable medium of claim 1, wherein the operations further comprise using a large language model to analyze the input in the natural language to determine a font associated with a user interface element in the new user interface.
1. A non-transitory computer readable medium storing a software program comprising data and computer implementable instructions that when executed by at least one processor cause the at least one processor to perform operations for generation of user interfaces from sketches, the operations comprising:
receiving a sketch, the sketch is associated with a desire of an individual to create a new user interface;
determining a particular design style;
obtaining at least two user interface elements;
analyzing the sketch to obtain a layout for the at least two user interface elements based on the particular design style, wherein the particular design style constrains distance between elements in the layout;
generating a particular draft of the new user interface that includes the at least two user interface elements based on the layout; and
presenting the particular draft of the new user interface to the individual in a form that enables the individual to modify the new user interface.
2. The non-transitory computer readable medium of claim 1, wherein the operations further comprise analyzing at least one other user interface associated with a specific brand to determine the particular design style including affinity information related to specific design elements and layout constraints.
5. The non-transitory computer readable medium of claim 1, wherein the operations further comprise analyzing the sketch to obtain the at least two user interface elements.
7. The non-transitory computer readable medium of claim 1, wherein the operations further comprise analyzing the sketch to determine at least one of a background music or a sound effect for the new user interface, and wherein the particular draft of the new user interface includes the at least one of the background music or the sound effect.
14. The non-transitory computer readable medium of claim 1, wherein the operations further comprise:
after the presenting the particular draft of the new user interface to the individual, receiving a modification to the sketch from the individual; and
analyzing the modified sketch to modify the new user interface.
17. The non-transitory computer readable medium of claim 1, wherein the operations further comprise analyzing the sketch to obtain a visual background including an animation for the new user interface, and wherein he particular draft of the new user interface includes the visual background including the animation.
18. The non-transitory computer readable medium of claim 1, wherein the operations further comprise analyzing the sketch to determine at least one of a type of a user interface element in the new user interface, a size of a user interface element in the new user interface, a content of a user interface element in the new user interface, or a font associated with a user interface element in the new user interface.
Allowable Subject Matter
The following is a statement of reasons for the indication of allowable subject matter but for the double patenting rejection:
Turek et al. (US Patent Pub. No. 20190317739 A1) discloses receiving an input in a natural language, the input is associated with a desire of an individual to create a new user interface, obtaining an indication of a particular design style, wherein the particular design style includes affinity associated with style properties, obtaining at least two user interface elements; analyzing the input in the natural language to obtain a layout for the at least two user interface elements based on the particular design style that includes the affinity associated with style properties, generating a particular draft of the new user interface that includes the at least two user interface elements based on the layout and presenting the particular draft of the new user interface to the individual.
Sollami et al. (US Patent Pub. No. 20230129431 A1) discloses an affinity associated with distances between elements.
Li et al. (US Patent Pub. No. 20210271815 A1) discloses input that includes an adverb that describes change and determining a change pattern associated with a particular user interface element in a new user interface based on the adverb.
Co et al. (US Patent Pub. No. 20210182468 A1) discloses at least part of at least two user interface elements are based on a characteristic of a target audience.
Vinod (US Patent Pub. No. 20200174776 A1) discloses using a user’s natural language input to generate a graphical user interface.
Westberg et al. (US Patent Pub. No. 20200150981 A1) discloses using a user’s natural language input to generate graphical user interface code.
Raghavan (US Patent Pub. No. 20210303318 A1) discloses using text input from a user to generate HTML code for a user interface.
Nikumb et al. (US Patent Pub. No. 20210397418 A1) discloses using natural language input to generate an application user interface.
Huang et al. (US Patent Pub. No. 20230115185 A1) discloses using a textual description of a graphical user interface to create graphical user interface code.
Price et al. (US Patent No. 9106812 B1) discloses an artificial intelligence that processes text to create a storyboard that includes sound effects and background music.
However the features of analyzing the input in the natural language to obtain a layout for the at least two user interface elements by determining a distance between the at least two user interface elements when taken in the context of the claims as a whole, were not found in the prior art teachings.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DANIEL SAMWEL whose telephone number is (313) 446-6549. The examiner can normally be reached Monday through Thursday 8:00-6:00 EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kieu Vu can be reached at (571) 272-4057. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DANIEL SAMWEL/ Primary Examiner, Art Unit 2171
Prosecution Insights