DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA . This action is made final.
Claims 1-20 are pending in the case. Claims 1, 14, and 17 are independent claims.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-11 and 13-20 under 35 U.S.C. 103 as being unpatentable over Peleg et al. (US 2023/0153546 A1), in view of De Ridder (US 2021/0271823 A1).
Regarding claim 1, Peleg teaches a method of modifying a document (FIGS. 4a-g and [0091-0097]), the method comprising:
receiving a first user input from a user via an editing window of a graphical user interface (FIGS. 4a-b and [0091-0092]: as seen in the transition from FIG. 4a-b, a first user input of a user entering text input 425 is received via an editing window/workspace 410 of a GUI 400), wherein
the first user input represents a request to a generative neural network model for drafting assistance with the document ([0005], [0182], [0246-0254]: text is generated via a neural network-based language model; FIGS. 4b-f and [0091-0097]: see how text like text output options 430a-c and adjusted text 485 are generated, via a neural network-based language model as supported in the preceding citation, as drafting assistance with the document),
the editing window displays content of the document during an editing session of the document for the user (FIGS. 4b-f and [0091-0097]: as supported in [0091], “In some cases, workspace 410 may include preexisting text 415 already entered by the user (or which may already appear as part of a preexisting document, such as a Word file, etc.)”. Editing window displays content including text 415 during an editing session of the document for the user);
obtaining, while maintaining the editing session, an output generated by the generative neural network model based on the first user input and the content of the document (FIGS. 4b-g and [0084-0097]: As supported in FIG. 3i and [0084], “The writing assistant can use both the user input 340 and the inserted text 330 in generating the text output options provided in fields 345a and 345b. In some cases, contextual information may be determined by the writing assistant analyzing inserted text 330 and/or user input 340.”As also supported in [0089], “In response to the enter words, etc., the writing assistant can provide text output options (e.g., in the form of complete sentences, etc.) that convey a meaning, idea, and/or information of the user input and that agree with preexisting text.” Thus, while the editing session is maintained, an output, such as text output options seen in FIG. 4b, is generated, by the generative neural network model based on the first user input and the content of the document, and obtained. Note that other text output options may be alternatively or additionally included as part of the generated output. For example, alternatively, adjusted text 485 as seen in FIGS. 4e-f may correspond to the output generated by the generated neural network model);
modifying the content of the document during the editing session to include the output generated by the generative neural network model (FIGS. 4b-g and [0091-0097]: as seen in FIG. 4g, the output/adjusted text 485 in FIGS. 4e-g may be added to the content of the document as inserted text 450); and
causing the modified content to be displayed within the editing window during the editing session (FIG. 4g and [0091-0097]: as seen in FIG. 4g, the output/adjusted text 485 in FIGS. 4e-g is displayed within the editing window/workspace 410 during the editing session).
Peleg does not explicitly teach obtaining, asynchronously while maintaining the editing session and receiving other user inputs in the editing session editing the content of the document.
De Ridder teaches obtaining, asynchronously while maintaining the editing session and receiving other user inputs in the editing session editing the content of the document, an output generated by the generative neural network model based on the first user input and the content of the document (FIG. 1, [0019-0021], [0024-0026], [0004], [0054], and [0088-0089]: for example, a user may perform a first user input to edit a document. Then, while an editing session is maintained and other user inputs are received in the editing session, an output is asynchronously and generated by the one or more content models 106 based on the first user input and the content of the document.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Peleg by incorporating the teachings of De Ridder so as to include obtaining, asynchronously while maintaining the editing session and receiving other user inputs in the editing session editing the content of the document. Doing so would allow the user to obtain the most updated and relevant feedback, or output, on their text entry while composing the document. By not having to manually perform an explicit non-editing input after each edit to call on the content model, or generative neural network model, nor having to wait between edits given the real-time generation, the user can more efficiently leverage the content model to edit the document.
Regarding claim 2, Peleg in view of De Ridder teaches the method of claim 1. Peleg further teaches wherein obtaining the output comprises:
generating a prompt, for the generative neural network model to generate the output, using the first user input, the content of the document, and at least one grounding context ([0057], FIGS. 4b-f and [0091-0097]: The user uses the first user input, the content of the document, and at least one grounding context/lowering formality by using toggles 480 or modifier window 442 to instruct, or generate a prompt for, the model to generate the output/adjusted text 485 as seen in FIGS. 4e-f);
sending the prompt to the generative neural network model ([0065-0066], [0005], [0182], [0246-0254]: prompt, instructions based on at least user input, is sent to the generative neural network model of the writing assistant; [0101]: writing assistant generates text options in response to “received text input from a user”; [0057], FIGS. 4b-f and [0091-0097]: prompt is sent to writing assistant to generate adjusted text 485); and
receiving the output generated by the generative neural network model ([0065-0066], [0005], [0182], [0246-0254]: output generated by generative neural network model of the writing assistant is received; [0101]: writing assistant generates text options in response to “received text input from a user”; [0057], FIGS. 4b-f and [0091-0097]: output of adjusted text 485 is received).
Regarding claim 3, Peleg in view of De Ridder teaches the method of claim 2. Peleg further teaches further comprising:
receiving a second user input from the user via the graphical user interface (FIGS. 4d-f and [0093-0097]: for example, a second user input corresponds to adjusting the level of formality 440b down to “-1” via toggles 480 or modifier window 442);
wherein:
the second user input identifies one or more of a textual context or a document context for the generative neural network model as the at least one grounding context (FIGS. 4d-f and [0093-0097]: the second user input identifies a less formal textual context), and
the textual context is a description in a natural language format of a simulated perspective for output generation by the generative neural network model (FIGS. 4d-f and [0093-0097]: the user reduces the text’s “formality” described in a natural language format, of a simulated less formal perspective for output generation by the generative neural network model of the writing assistant).
Regarding claim 4, Peleg in view of De Ridder teaches the method of claim 3. Peleg further teaches wherein the second user input comprises a text description that defines the textual context in the natural language format (FIGS. 4d-f and [0093-0097]: the second user input comprises a text description such as “formality” that defines the textual context in the natural language format).
Regarding claim 5, Peleg in view of De Ridder teaches the method of claim 3. Peleg further teaches further comprising generating the textual context using the generative neural network model based on the second user input (FIGS. 4d-f and [0093-0097]: based on the second user input, a less formal textual context is generated).
Regarding claim 6, Peleg in view of De Ridder teaches the method of claim 3.
In another embodiment, Peleg teaches receiving a second user input from the user via the editing window during the editing session, wherein the second user input represents a change in the content of the document; and
modifying the content of the document during the editing session to include the change in the content while the output is asynchronously obtained (FIGS. 2j-p and [0076-0080]: content of the document includes at least text 215, text in second field 250, and text in additional field 265. A second user input is received from the editing window/workspace 210 representing a change via adding text in additional field 265. Content is modified as seen in the transition from FIG. 2l to FIG. 2m while the output is asynchronously obtained as seen in the change of text output options seen in FIGS. 2l-2n. In particular, note the loading for updated text options seen in FIG. 2m; For an additional example regarding the drafting process, see FIGS. 3g-i and [0082-0084]. As generally described in [0082], “This drafting process, augmented by the writing assistant application may continue as long as the user has additional concepts or information to convey”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Peleg in view of De Ridder by incorporating the teachings of Peleg from another embodiment so as to include receiving a second user input from the user via the editing window during the editing session, wherein the second user input represents a change in the content of the document; and modifying the content of the document during the editing session to include the change in the content while the output is asynchronously obtained. Doing so would grant the user more flexibility and convenience as the user is able to continuously modify the content of the document to obtain the most relevant output without having to restart and retype a new input. The user would be able to seamlessly perform a second input to add details to the content that would help generate a more effective output.
Regarding claim 7, Peleg in view of De Ridder teaches the method of claim 1. Peleg further teaches further comprising generating an editing sub-window within the graphical user interface as a placeholder for the output generated by the generative neural network model (FIGS. 4d-f and [0093-0097]: for example, an editing sub-window may be the nested window in which adjusted text 485 will be or is displayed as seen in FIGS. 4d-f).
Regarding claim 8, Peleg in view of De Ridder teaches the method of claim 7. Peleg further teaches further comprising dynamically updating the editing sub-window to display received sub-portions of the output generated by the generative neural network model (FIGS. 4d-f and [0093-0097]: the editing sub-window is updated to display received sub-portions of the output).
Regarding claim 9, Peleg in view of De Ridder teaches the method of claim 7. Peleg further teaches further comprising merging the output generated by the generative neural network model from the editing sub-window into the content displayed in the editing window (FIGS. 4f-g and [0097]: “the writing assistant can automatically insert the adjusted/refined text into the document or email workspace 410 as inserted text 450”).
Regarding claim 10, Peleg in view of De Ridder teaches the method of claim 7. Peleg further teaches further comprising displaying the editing sub-window within the editing window and inline with the content of the document (FIGS. 4d-f and [0093-0097]: the editing sub-window is within the editing window/workspace 410 and inline with the content of the document).
Regarding claim 11, Peleg in view of De Ridder teaches the method of claim 1. Peleg further teaches wherein:
the output generated by the generative neural network model comprises one or more of source code, object notation, plain text ([0065-0066], [0005], [0182], [0246-0254]: output generated by generative neural network model of the writing assistant is received; [0101]: writing assistant generates text options, in plain text, in response to “received text input from a user”; [0057], FIGS. 4b-f and [0091-0097]: output of adjusted text 485 is received); and
the editing window is a text editing window and the generative neural network model is a large language model (FIGS. 4b-g and [0091-0097]: editing window/workspace 410 is a text editing window; [0058]: “In some cases, the disclosed writing assistant system may be based on trained machine learning language models trained to recognize complex contextual elements in text. For example, as alluded to above, such models may be trained, for example, using large corpuses of text, masking different segments of text (e.g., tokens), and one or more reward functions that penalize the system during training for generating text replacements that do not match the masked text and reward the system for generating a text replacement that matches the masked text.”; [0154], [0202-0204]: a Masked Language Model, which is an LLM, may be used).
Regarding claim 13, Peleg in view of De Ridder teaches the method of claim 1. Peleg further teaches wherein modifying the content of the document during the editing session comprises formatting the output generated by the generative neural network model to match a writing style of existing content within the document ([0098]: “Contextual agreement may have various meanings. In some cases, however, an agreement between two or more text elements may refer to grammatical agreement (e.g., the insertion of the generated text output option does not result in a grammar error relative to the preexisting text). In other cases, agreement between text elements may be achieved by the generated text output options being generated to include in the same or similar style as the text around it (e.g., preexisting text in a document workspace). Another contextual agreement may exist where a generated text output option connects coherently to the text around it once inserted into a document workspace. This form of agreement may include, but is not limited to, the generated text being related to the same general subject as the context and/or events or facts referenced in a generated text output options being consistent with events or facts referenced by preexisting text in a document workspace, for example. The consistency may be relative to a relationship (e.g., temporal, causal, teleological, explanatory, etc.) existing between generated text output options and preexisting text or user input.”).
Regarding claims 14-16, the claims recite a computing device for modifying a document, the computing device comprising a processor and a non-transitory computer-readable memory, wherein the processor is configured to carry out instructions from the memory that configure the computing device to perform operations corresponding to the method of claims 1-3, respectively, and are therefore rejected on the same premises.
Regarding claim 17, Peleg teaches a method of modifying a document (FIGS. 4a-g and [0091-0097]), the method comprising:
receiving a first user input from a user via an editing window of a graphical user interface (FIGS. 4a-b and [0091-0092]: as seen in the transition from FIG. 4a-b, a first user input of a user entering text input 425 is received via an editing window/workspace 410 of a GUI 400), wherein:
the first user input represents a request to a generative neural network model for drafting assistance with the document ([0005], [0182], [0246-0254]: text is generated via a neural network-based language model; FIGS. 4b-f and [0091-0097]: see how text like text output options 430a-c and adjusted text 485 are generated, via a neural network-based language model as supported in the preceding citation, as drafting assistance with the document),
the editing window displays content of the document during an editing session of the document for the user (FIGS. 4b-f and [0091-0097]: as supported in [0091], “In some cases, workspace 410 may include preexisting text 415 already entered by the user (or which may already appear as part of a preexisting document, such as a Word file, etc.)”. Editing window displays content including text 415 during an editing session of the document for the user);
selecting a textual context from a plurality of textual contexts, wherein each of the plurality of textual contexts is a description in a natural language format of a different simulated perspective for output generation by the generative neural network model (FIGS. 4b-c and [0091-0093]: a textual content/text output option 430a is selected from a plurality of textual contexts/text output options. Each textual context is a description in a natural language format of a simulated perspective for output generation by the generative neural network model);
obtaining an output generated by the generative neural network model based on the content of the document and the selected textual context (FIGS. 4b-g and [0084-0097]: As supported in FIG. 3i and [0084], “The writing assistant can use both the user input 340 and the inserted text 330 in generating the text output options provided in fields 345a and 345b. In some cases, contextual information may be determined by the writing assistant analyzing inserted text 330 and/or user input 340.”As also supported in [0089], “In response to the enter words, etc., the writing assistant can provide text output options (e.g., in the form of complete sentences, etc.) that convey a meaning, idea, and/or information of the user input and that agree with preexisting text.” Thus, while the editing session is maintained, an output, such as adjusted text 485 as seen in FIGS. 4e-f, is generated by the generative neural network model based on the content of the document and the selected textual context);
modifying the content of the document during the editing session to include the output generated by the generative neural network model (FIGS. 4b-g and [0091-0097]: as seen in FIG. 4g, the output/adjusted text 485 in FIGS. 4e-g may be added to the content of the document as inserted text 450); and
causing the modified content to be displayed within the editing window during the editing session (FIG. 4g and [0091-0097]: as seen in FIG. 4g, the output/adjusted text 485 in FIGS. 4e-g is displayed within the editing window/workspace 410 during the editing session).
Peleg does not explicitly teach obtaining, asynchronously while maintaining the editing session and receiving other user inputs in the editing session editing the content of the document.
De Ridder teaches obtaining, asynchronously while maintaining the editing session and receiving other user inputs in the editing session editing the content of the document, an output generated by the generative neural network model based on the first user input and the content of the document (FIG. 1, [0019-0021], [0024-0026], [0004], [0054], and [0088-0089]: for example, a user may perform a first user input to edit a document. Then, while an editing session is maintained and other user inputs are received in the editing session, an output is asynchronously and generated by the one or more content models 106 based on the first user input and the content of the document.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Peleg by incorporating the teachings of De Ridder so as to include obtaining, asynchronously while maintaining the editing session and receiving other user inputs in the editing session editing the content of the document. Doing so would allow the user to obtain the most updated and relevant feedback, or output, on their text entry while composing the document. By not having to manually perform an explicit non-editing input after each edit to call on the content model, or generative neural network model, nor having to wait between edits given the real-time generation, the user can more efficiently leverage the content model to edit the document.
Regarding claim 18, Peleg in view of De Ridder teaches the method of claim 17. Peleg further teaches further comprising generating a first textual context of the plurality of textual contexts based on the content of the document using the generative neural network model ([0084], [0089], FIG. 4b and [0091]: As supported in FIG. 3i and [0084], “The writing assistant can use both the user input 340 and the inserted text 330 in generating the text output options provided in fields 345a and 345b. In some cases, contextual information may be determined by the writing assistant analyzing inserted text 330 and/or user input 340.”As also supported in [0089], “In response to the enter words, etc., the writing assistant can provide text output options (e.g., in the form of complete sentences, etc.) that convey a meaning, idea, and/or information of the user input and that agree with preexisting text.” Thus, a first textual context/text output option 430a is generated based on the content of the document using the neural network model of the writing assistant).
Regarding claim 19, Peleg in view of De Ridder teaches the method of claim 18. Peleg further teaches further comprising: displaying the first textual context within the editing window during the editing session; and
modifying the first textual context, before obtaining the output, based on a second user input received via the editing window during the editing session (FIGS. 4c-e and [0093-0096]: the first textual context is displayed within the editing window/workspace 410 during the editing session. The first textual context is modified, before obtaining the output, via a second user input of adjusting the formality level via toggles 480 or modifier window 442).
Regarding claim 20, Peleg in view of De Ridder teaches the method of claim 17.
In another embodiment, Peleg teaches further comprising generating a first textual context of the plurality of textual contexts by receiving a corresponding description as a second user input via the editing window during the editing session (FIGS. 2j-p and [0076-0080]: a second user input is received from the editing window/workspace 210, the second user input being added text describing “When it is possible for you?” This causes generation of a first textual context such as any of the updated text outputs 275a-c, in contrast with the previous second text outputs 260a-c; For an additional example regarding the drafting process, see FIGS. 3g-i and [0082-0084]. As generally described in [0082], “This drafting process, augmented by the writing assistant application may continue as long as the user has additional concepts or information to convey”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Peleg in view of De Ridder by incorporating the teachings of Peleg from another embodiment so as to include generating a first textual context of the plurality of textual contexts by receiving a corresponding description as a second user input via the editing window during the editing session. Doing so would grant the user more flexibility and convenience as the user is able to continuously add context to obtain the most relevant textual context without having to restart and retype a new input. The user would be able to seamlessly perform a second input to add details to the content that would help generate a more effective first textual context.
Claim 12 is/are rejected under 35 U.S.C. 103 as being unpatentable over Peleg et al. (US 2023/0153546 A1), De Ridder (US 2021/0271823 A1), and in view of Cheung et al. (US 12124524 B1).
Regarding claim 12, Peleg in view of De Ridder teaches the method of claim 1.
Peleg in view of De Ridder does not explicitly teach wherein: the output generated by the generative neural network model is an image portion; the editing window is an image editing window; and the generative neural network model is one of a generative adversarial network, a variational autoencoder, a diffusion model, or a text-to-image model.
Cheung teaches wherein: the output generated by the generative neural network model is an image portion; the editing window is an image editing window; and the generative neural network model is one of a generative adversarial network, a variational autoencoder, a diffusion model, or a text-to-image model (FIG. 9G and Col. 22, lines 33-63 and Col. 36, line 50 to Col. 37, line 3: generative model aided generation includes an image generation model for generating novel images based on the web resource, a user input, and/or the generated prompt. As seen in image 956 of FIG. 9G, the editing window is an image editing window. The generative neural network model may be a text-to-image generation model).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Peleg in view of De Ridder by incorporating the teachings of Cheung so as to include wherein: the output generated by the generative neural network model is an image portion; the editing window is an image editing window; and the generative neural network model is one of a generative adversarial network, a variational autoencoder, a diffusion model, or a text-to-image model. Doing so would allow output of an image which may more quickly and effectively convey meaning to certain viewers, especially those who are visually oriented. In this way, viewers do not necessarily have to read text to comprehend a message. Rather, they may glean information from an image, which, under certain scenarios, like manual instructions or product advertisements, may offer greater context.
Response to Arguments
Applicant’s argument with respect to the 112(b) rejection of claim 6 has been fully considered and is persuasive. The 112(b) rejection of claim 6 has been withdrawn.
Applicant’s amendments have changed the scopes of the claims, necessitating the new ground(s) of rejection. Applicant’s arguments with respect to the amended independent claims have been considered but are moot because the new ground(s) of rejection does/do not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure, including:
US 2024/0086651 A1: presenting edits and suggestions to users as the document is edited
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to KENNY NGUYEN whose telephone number is (571)272-4980. The examiner can normally be reached M-Th 7AM to 5PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, KIEU D VU can be reached at (571)272-4057. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/KENNY NGUYEN/Primary Examiner, Art Unit 2171
Prosecution Insights