DETAILED ACTION
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This Action is responsive to the Application filed on 5/6/2024. Claims 1-20 are pending in the case.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-9 and 11-20 are rejected under 35 U.S.C. 103 as being unpatentable over Kraus et al. (US 20220138402 A1, hereinafter Kraus) in view of Sharifi et al. (US 20250148217 A1, hereinafter Sharifi).
As to independent claim 1, Kraus teaches a data processing system comprising:
a processor (processor 814), and a machine-readable storage medium storing executable instructions (“Computing device 800 typically includes a variety of computer-readable media” paragraph 0077), which when executed by the processor, cause the processor alone or in combination with other processors to perform the following operations:
receiving, via a user interface of a client device, an indication of a user marking of a textual area in a graphic design image, the textual area showing one or more characters (“a user may invoke a text style suggestion service by selecting a particular text” paragraph 0020, Fig. 2 textual area 215 selected);
constructing, via a prompt construction unit, a first prompt by appending the graphic design image (graphic design image 210) and the user marking of the textual area (textual area 215) in the graphic design image to a first instruction string, the first instruction string including instructions to a semantic role of the text element in the design, and/or other features. The text style tool sends the extracted metadata or a representation thereof to the text style suggestion service,” paragraph 0020, “Additionally or alternative, in some embodiments, style selector 182 includes one or more neural networks or some other machine-learning model configured to predict a compatibility score for each candidate style based on encoded contextual metadata from the request)” paragraph 0048, 0082-0083);
providing as an input the first prompt to the convolutional neural networks, recurrent neural networks, deep neural networks, and deep stacking networks, to name a few examples. Although some implementations are described with respect to neural networks, such as generative adversarial, and or other type of machine learning models” Paragraph 0082-0083);
providing the one or more characters, the one or more design context attributes, and the new graphic design image to the client device (“The identified compatible text styles are mapped to different positions of style wheel 116 (e.g., which may correspond to style wheel 250 of FIG. 2), and the user may scroll through the positions of style wheel 116 to iterate through the compatible style suggestions.” Paragraph 0045); and
causing the user interface of the client device to display at least one of the new graphic design image or an editable text box over the textual area in the graphic design image, wherein the editable text box shows the one or more characters based on the one or more design context attributes (“Upon selection of a particular text style suggestion (e.g., corresponding to the position of indicator 255 of style wheel 250 of FIG. 2), style preview component previews and/or applies the text suggestion to the user's design.” Paragraph 0045).
Kraus does not appear to expressly teach the first instruction string including instructions to a generative model and providing as an input the first prompt to the generative model.
Sharifi teaches LLM-based text editing assistant (“a text prompt can be generated based on the user request to generate content and the first generative model output, where the text prompt can further include a request for one or more focused edits. The request for one or more focused edits can be a request for generating a single focused edit (e.g., that edits a single text segment such as a sentence, a paragraph or other types of text segment) in each iteration out of one or more iterations.” Paragraph 0008).
Accordingly, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify the system of Kraus to comprise the first instruction string including instructions to a generative model and providing as an input the first prompt to the generative model. One would have been motivated to make such a combination to improve user experience.
As to dependent claim 2, Kraus teaches the data processing system of claim 1, wherein the machine-readable storage medium further includes instructions configured to cause the processor alone or in combination with other processors to perform operations of:
receiving, via the editable text box, one or more new characters; and causing the editable text box to display the one or more new characters based on the one or more design context attributes (“Upon selection of a particular text style suggestion (e.g., corresponding to the position of indicator 255 of style wheel 250 of FIG. 2), style preview component previews and/or applies the text suggestion to the user's design.” Paragraph 0045).
As to dependent claim 3, Kraus teaches the data processing system of claim 2, Krause further teaches wherein the first instruction string further includes instructions to identify one or more new white spaces with a consistent color or style in the graphic design image (“Upon selection, the text style tool looks up, extracts, or otherwise collects metadata about the selected text element and the user's design, such as position in the design, font features (e.g., density, ornateness, etc.), derived features like variance in background color around the text element,” paragraph 0020) and the machine-readable storage medium further includes instructions configured to cause the processor alone or in combination with other processors to perform operations of:
when determining that a length of the one or more new characters exceeds a length of the editable text box, causing the user interface of the client device to display an editable text box over the textual area and the one or more new white spaces in the graphic design image, wherein the editable text box shows the one or more new characters (As shown in Fig. 2, the editable text box over textual area 215 is expandable).
As to dependent claim 4, Kraus teaches the data processing system of claim 1, Krause further taches wherein the machine-readable storage medium further includes instructions configured to cause the processor alone or in combination with other processors to perform operations of:
causing the user interface of the client device to display the editable text box over the textual area in the new graphic design image (Fig. 2, editable text box over textual area 215).
As to dependent claim 5, Krause teaches the data processing system of claim 1, Krause further teaches wherein the design context attributes further include at least one of a visual hierarchy of the one or more characters in the graphic design image, a balance of the one or more characters with respect to white space in the graphic design image, a theme of the graphic design image, a formality of the graphic design image, or a color scheme of the graphic design image (“Upon selection, the text style tool looks up, extracts, or otherwise collects metadata about the selected text element and the user's design, such as position in the design, font features (e.g., density, ornateness, etc.), derived features like variance in background color around the text element, the semantic role of the text element in the design, and/or other features.” Paragraph 0020, 0034).
As to dependent claim 6, Krause teaches the data processing system of claim 1, Krause further teaches wherein the first instruction string further includes instructions to change the character design of the one or more characters based on the semantics of the one or more characters in the new graphic design image, and wherein the character design includes at least one of a font, color, size, style, angle, or transparency level (“Generally, any given phrase may have multiple ways to emphasize different words in a way that focuses the audience's attention on different semantic aspects. For example, “The BEACH you should be sitting on now” focuses attention on the subject of the phrase, whereas “The beach you should be sitting on NOW” focuses attention on the temporal aspect of the phrase.” Paragraph 0059).
As to dependent claim 7, Krause teaches the data processing system of claim 1, Krause further teaches wherein the first instruction string further includes instructions to change the position of the textual area in the graphic design image based on the semantics of the one or more characters in the new graphic design image (“There are many elements of visual stylization that can be applied to text or a text field, including font face, font color, point size, line spacing (leading), letter spacing (tracking and kerning), text position, line or field length, text background, and other effects, to name a few.” Paragraph 0015).
As to dependent claim 8, Krause teaches the data processing system of claim 1, Krause further teaches wherein the first instruction string further includes instructions to identify a background style of the graphic design image, and to change the character design of the one or more characters to match with the background style (“Upon selection, the text style tool looks up, extracts, or otherwise collects metadata about the selected text element and the user's design, such as position in the design, font features (e.g., density, ornateness, etc.), derived features like variance in background color around the text element, the semantic role of the text element in the design, and/or other features. The text style tool sends the extracted metadata or a representation thereof to the text style suggestion service,” paragraph 0020), and
wherein the editable text box shows the one or more characters in the changed character design that matches the background style (“There are many elements of visual stylization that can be applied to text or a text field, including font face, font color, point size, line spacing (leading), letter spacing (tracking and kerning), text position, line or field length, text background, and other effects, to name a few.” Paragraph 0015,0020).
As to dependent claim 9, Krause teaches the data processing system of claim 1, Krause further teaches wherein the first instruction string further includes instructions to identify a background of the graphic design image, and to change the background of the graphic design image to a higher contrast to the character design (“Upon selection, the text style tool looks up, extracts, or otherwise collects metadata about the selected text element and the user's design, such as position in the design, font features (e.g., density, ornateness, etc.), derived features like variance in background color around the text element, the semantic role of the text element in the design, and/or other features. The text style tool sends the extracted metadata or a representation thereof to the text style suggestion service,” paragraph 0020), and
wherein the user interface of the client device displays the at least one of the new graphic design image or the editable text box over the textual area in the changed background (“The text style suggestion service returns the selected text styles, and in some embodiments, a contextual analysis is performed to curate style suggestions that are compatible with the context of the user's design, for example, by selecting or modifying styles to avoid collisions with other design elements, match an existing design alignment, and ensure a threshold amount of readability (e.g., quantified based on font density, background color variance, contrast, point size). The user may iterate through the selected style suggestions to preview and/or apply the suggestions to the user's design.” Paragraph 0020, 0052).
As to dependent claim 11, Krause teaches the data processing system of claim 1, Krause further teaches wherein the first instruction string further includes instructions to re-write the one or more characters based on a user preference, and
wherein the user interface of the client device displays the at least one of the new graphic design image or the editable text box showing the rewritten one or more characters (“The user may iterate through the selected style suggestions to preview and/or apply the suggestions to the user's design.” Paragraph 0020, Fig. 2).
As to dependent claim 12, Krause teaches the data processing system of claim 1, Krause further teaches wherein the machine-readable storage medium further includes instructions configured to cause the processor alone or in combination with other processors to perform operations of:
causing the user interface of the client device to display an instruction for a user to edit the one or more characters displayed in the editable text box (“The user may iterate through the selected style suggestions to preview and/or apply the suggestions to the user's design.” Paragraph 0020, Fig. 2, characters 215 edited within the editable text box).
As to dependent claim 13, Krause teaches the data processing system of claim 12, Krause further teaches wherein the machine-readable storage medium further includes instructions configured to cause the processor alone or in combination with other processors to perform operations of:
upon receiving a character edited or added in the editable text, retrieving a corresponding character in the character design and displaying the corresponding character in the character design in the editable text in real-time or substantially real-time (“The user may iterate through the selected style suggestions to preview and/or apply the suggestions to the user's design.” Paragraph 0020, Fig. 2).
As to dependent claim 14, Krause teaches the data processing system of claim 1, Krause does not appear to expressly teach wherein the generative model is a multimodal model.
Sharifi teaches wherein the generative model is multimodal model (“utilizing both the first generative model and the second generative model,” paragraph 0005, 0004).
Accordingly, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify the system of Kraus to comprise wherein the generative model is multimodal model. One would have been motivated to make such a combination to improve user experience.
Claims 15-20 are substantially the same as claims 1-3 and are therefore rejected under similar rational.
Claim 10 is rejected under 35 U.S.C. 103 as being unpatentable over Kraus et al. in view of Sharifi et al. and further in view of Agarwal et al. (US 20210073340 A1, hereinafter Agarwal).
As to dependent claim 10, Krause teaches the data processing system of claim 1, Krause does not appear to expressly teach wherein the first instruction string further includes instructions to determine a language of the one or more characters, and to translate the one or more characters into another language based on a user preference, and
wherein the user interface of the client device displays the at least one of the new graphic design image or the editable text box showing the translated one or more characters.
Agarwal teaches wherein the first instruction string further includes instructions to determine a language of the one or more characters, and to translate the one or more characters into another language based on a user preference (“The present disclosure describes design-time tools that assist a document designer in designing a document that is ready for translation into multiple target languages.” Abstract) and
wherein the user interface of the client device displays the at least one of the new graphic design image or the editable text box showing the translated one or more characters (“In particular, techniques are described that enable a user or designer of a document to, at design time itself, check and verify that text elements included in the document for displaying text content are properly sized for displaying translations of the text content in one or more desired target languages.” Abstract).
Accordingly, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify the system of Kraus to comprise wherein the first instruction string further includes instructions to determine a language of the one or more characters, and to translate the one or more characters into another language based on a user preference, and wherein the user interface of the client device displays the at least one of the new graphic design image or the editable text box showing the translated one or more characters. One would have been motivated to make such a combination to improve user experience.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
LI US 20220335205 A1 teaches context based visual enhancement suggestion.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MAHELET SHIBEROU whose telephone number is (571)270-7493. The examiner can normally be reached Monday-Friday 9:00 AM-5:00 PM Eastern Time.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kieu Vu can be reached at 571-272-4057. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MAHELET SHIBEROU/Primary Examiner, Art Unit 2171