DETAILED ACTION
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This office action is in response to the application filed September 24, 2024.
Clams 1-17 are pending.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on November 24, 2024 was filed after the mailing date of the application on September 24, 2024. The submission is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1,3-10, 13-14 is/are rejected under 35 U.S.C. 103 as being unpatentable over “Zeng” (US PG Pub 2025/0068398) in view of Gorrono (US PG Pub 2023/0333827).
Regarding Claim 1, Zeng teaches:
1. A scenario management apparatus that manages a scenario for automating an operation on another application program, the scenario management apparatus comprising: an acceptance unit that accepts a user instruction in a natural language format by a user; (Zeng e.g. 406, Fig.4 ¶78 teaches receiving a user query for performing a task in the automatic workflow generation module)
a rule information acquisition unit that acquires rule information for describing the scenario in a general-purpose programming language from a storage unit;(Zeng 402-404 Fig. 4, ¶¶76-77 teaches assembling workflow API and information into a Lecture describing code to be used to generate response in combination with the user prompt text )
an input unit that inputs the user instruction accepted by the acceptance unit and the rule information acquired by the rule information acquisition unit to a machine learning model that has learned natural language processing;( Zeng 406 Fig 4 ¶¶78-79 teaches combining the user query and lecture and submitting them to the llm for automatic workflow generation)
an output information acquisition unit that acquires output information including a source code that describes the scenario, the source code being generated by the machine learning model in the general-purpose programming language according to the user instruction and the rule information;(Zeng S408¶¶79-80 teaches receiving automatically generated workflow back from the LLM, generated in e.g. the python code specific in lecture fig 6 according to the user prompt and lecture)
Zeng does not teach, but Gorrono teaches:
and a display generation unit that converts the source code included in the output information into display information to be displayed on a GUI. (Gorrono e.g. 918, Fig. 9 teaches converting workflow output of llm into workflow JSON and outputting it to the GUI with pre-populated input fields (500, Fig. 5 through Fig. 8B))
In addition, it would have been obvious to one of ordinary skill prior to the effective filing date of the application to combine the teachings of Zeng and Gorrono as each is directed to LLM-based workflow generation and output and Gorrono recognized the need for “ a display of the natural language from user input 406 and input field 514, as well as an overview 702 of the steps of the workflow from the automation exemplarily noted above.” (¶77).
Regarding Claim 3, Zeng teaches:
3. The scenario management apparatus according to claim 1, wherein the rule information includes language designation information designating the general-purpose programming language describing the scenario included in the output information. (Zeng Fig. 6 teaches the lecture specifying a programming language for coding the workflow tasks, in this example python code)
Regarding Claim 4, Zeng teaches:
4. The scenario management apparatus according to claim 3, wherein the rule information includes available function information for designating a function available to the description of the scenario among functions in a programming interface in the general-purpose programming language designated by the language designation information. (Zeng ¶88 teaches the lecture designation the structure API function calls for the workflow)
Regarding Claim 5, Zeng teaches:
5. The scenario management apparatus according to claim 4, wherein the rule information includes a sample code indicating a method for reading the available function indicated by the available function information. (Zeng ¶¶86-89 Fig. 6 includes the lecture indicating function code used to code the workflow tasks)
Regarding Claim 6, Zeng teaches:
6. The scenario management apparatus according to claim 1, wherein the rule information includes information designating a method for describing the scenario in the general-purpose programming language. ((Zeng ¶¶86-89 Fig. 6 includes the lecture indicating function code used to code the workflow tasks)
Regarding Claim 7, Zeng teaches:
7. The scenario management apparatus according to claim 1, wherein the output information acquisition unit acquires the output information including an explanatory sentence explaining the scenario in a natural language, and (Zeng Fig. 7, ¶92 teaches a NL description of generated code) the display generation unit distinguishes the source code and the explanatory sentence of the scenario from the output information and generates the display information including explanatory sentence display based on the explanatory sentence. (Zeng Fig. 7, ¶92 teaches a NL description of generated code, sepearated as seen in Fig. 7)
Regarding Claim 8, Zeng does not teach, but Gorrono teaches:
8. The scenario management apparatus according to claim 1, wherein the display generation unit generates the display information including input sentence display based on the user instruction accepted by the acceptance unit, the input sentence display and the scenario being associated with each other in the display information. (Gorrono ¶63, 406, Fig. 5 teaches an input sentence input into the system, then displayed above the generated information in the display) In addition, it would have been obvious to one of ordinary skill prior to the effective filing date of the application to combine the teachings of Zeng and Gorrono as each is directed to LLM-based workflow generation and output and Gorrono recognized the need for “ a display of the natural language from user input 406 and input field 514, as well as an overview 702 of the steps of the workflow from the automation exemplarily noted above.” (¶77).
Regarding Claim 9, Zeng further teaches:
9. The scenario management apparatus according to claim 8, wherein the scenario includes a plurality of activities that execute predetermined processing defined in the general-purpose programming language, (Zeng Fig. 6 teaches the lecture specifying a programming language for coding the workflow tasks, in this example python code)
, and when the acceptance unit accepts a second user instruction as the user instruction after the output information acquisition unit acquires the first output information, (Zeng Fig. 7 teaches accepting a second input, user feedback, related to the outputted workflow code)
Zeng does not teach, but Gorrono teaches:
the acceptance unit retains a first user instruction as the user instruction in the storage unit, (Gorrono 406, Fig 5 teaches retaining the first input in and displaying it in the display as seen in fig. 5)
the output information acquisition unit retains first output information as the output information in the storage unit, (Gorrono 502/506 Fig. 5 teaches retaining and displaying the first output)
the display generation unit generates the display information in which history display based on the first user instruction and the first output information retained in the storage unit, the input sentence display, and an activity block indicating the activity are associated with each other. (Gorrono ¶78, 702-704 Fig. 7 teaches displaying the first user prompt, followed by the output including blocks for the workflow tasks related to the output and editable fields to be configured as needed)
In addition, it would have been obvious to one of ordinary skill prior to the effective filing date of the application to combine the teachings of Zeng and Gorrono as each is directed to LLM-based workflow generation and output and Gorrono recognized the need for “ a display of the natural language from user input 406 and input field 514, as well as an overview 702 of the steps of the workflow from the automation exemplarily noted above.” (¶77).
Regarding Claim 10, Zeng does not teach, but Gorrono teaches:
10. The scenario management apparatus according to claim 9, wherein the display generation unit generates the display information not including at least a part of the history display in accordance with an operation of the user accepted by the acceptance unit. (Gorrono teaches e.g. partial history in Fig. 7 display without some of the previously displayed information of Figs. 5 and 6)
In addition, it would have been obvious to one of ordinary skill prior to the effective filing date of the application to combine the teachings of Zeng and Gorrono as each is directed to LLM-based workflow generation and output and Gorrono recognized the need for “ a display of the natural language from user input 406 and input field 514, as well as an overview 702 of the steps of the workflow from the automation exemplarily noted above.” (¶77).
Regarding Claim 13, Zeng does not teach, but Gorrono teaches:
13. The scenario management apparatus according to claim 1, wherein the display generation unit displays, on the GUI, a first window that accepts an input of the user instruction by the user, and a second window different from the first window, the second window displaying the display information converted from the source code by the display generation unit. (Gorrono Fig. 4-7,¶66-78 teaches GUI windows which accept and display the input and additional displays which display graphical representations of the generated code which included editable fields where needed) In addition, it would have been obvious to one of ordinary skill prior to the effective filing date of the application to combine the teachings of Zeng and Gorrono as each is directed to LLM-based workflow generation and output and Gorrono recognized the need for “ a display of the natural language from user input 406 and input field 514, as well as an overview 702 of the steps of the workflow from the automation exemplarily noted above.” (¶77).
Claim(s) 2,14, 16, and 17 is/are rejected under 35 U.S.C. 103 as being unpatentable over “Zeng” (US PG Pub 2025/0068398) in view of “Ross” (US Patent 11,747,970).
Regarding Claim 2, Zeng teaches:
2. A scenario management apparatus that manages a scenario for automating an operation on another application program, the scenario management apparatus comprising: an acceptance unit that accepts a user instruction in a natural language format by a user;( Zeng e.g. 406, Fig.4 ¶78 teaches receiving a user query for performing a task in the automatic workflow generation module)
a rule information acquisition unit that acquires rule information for describing the scenario in a general-purpose programming language from a storage unit;(Zeng 402-404 Fig. 4, ¶¶76-77 teaches assembling workflow API and information into a Lecture describing code to be used to generate response in combination with the user prompt text )
an input unit that inputs the user instruction accepted by the acceptance unit and the rule information acquired by the rule information acquisition unit to a machine learning model that has learned natural language processing;( Zeng 406 Fig 4 ¶¶78-79 teaches combining the user query and lecture and submitting them to the llm for automatic workflow generation)
an output information acquisition unit that acquires output information including a source code that describes the scenario, the source code being generated by the machine learning model in the general-purpose programming language according to the user instruction and the rule information(Zeng S408¶¶79-80 teaches receiving automatically generated workflow back from the LLM, generated in e.g. the python code specific in lecture fig 6 according to the user prompt and lecture)
Zeng does not teach, but Ross teaches:
and an editing operation unit that retains the source code included in the output information in a mode editable by the user in the storage unit. (Ross Figs. 7-9, Col. 11, Ln 1 to Col 12, Ln 2 teaches an interactive editable source code editor including a panel with a history of AI code translation suggestions including multiple versions to be selected from by the user, Col. 7, Ln 5-43 teaches the ml model outputting code into an editor interface)
In addition, it would have been obvious to one of ordinary skill prior to the effective filing date of the application to combine the teachings of Zeng and Ross as each is directed to output of automatic code generation systems and Ross recognized the need for “proposed graphical user interfaces can help manage multiple different versions/hypotheses simultaneously and provide display of an understandable and navigable presentation, according to one or more embodiments of the invention.” (Ross Col. 4, Ln 4-8)
Regarding Claim 16, Zeng teaches:
16. A computer-readable recording medium storing a scenario management program causing a computer to function as a scenario management apparatus that manages a scenario for automating an operation on another application program, the scenario management program causing the computer to execute steps of: accepting a user instruction in a natural language format by a user( Zeng e.g. 406, Fig.4 ¶78 teaches receiving a user query for performing a task in the automatic workflow generation module)
acquiring rule information for describing the scenario in a general-purpose programming language from a storage unit;(Zeng 402-404 Fig. 4, ¶¶76-77 teaches assembling workflow API and information into a Lecture describing code to be used to generate response in combination with the user prompt text )
inputting the user instruction and the rule information to a machine learning model that has learned natural language processing;( Zeng 406 Fig 4 ¶¶78-79 teaches combining the user query and lecture and submitting them to the llm for automatic workflow generation)
acquiring output information including a source code describes the scenario, the source code being generated by the machine learning model in the general-purpose programming language according to the user instruction and the rule information;(Zeng S408¶¶79-80 teaches receiving automatically generated workflow back from the LLM, generated in e.g. the python code specific in lecture fig 6 according to the user prompt and lecture)
Zeng does not teach, but Ross teaches:
and retaining the source code included in the output information in a mode editable by the user in the storage unit. (Ross Figs. 7-9, Col. 11, Ln 1 to Col 12, Ln 2 teaches an interactive editable source code editor including a panel with a history of AI code translation suggestions including multiple versions to be selected from by the user)
In addition, it would have been obvious to one of ordinary skill prior to the effective filing date of the application to combine the teachings of Zeng and Ross as each is directed to output of automatic code generation systems and Ross recognized the need for “proposed graphical user interfaces can help manage multiple different versions/hypotheses simultaneously and provide display of an understandable and navigable presentation, according to one or more embodiments of the invention.” (Ross Col. 4, Ln 4-8)
Regarding Claim 14, Zeng does not teach, but Ross teaches:
14. The scenario management apparatus according to claim 2, wherein the editing operation unit retains only a part of the scenario described by the source code in an editable mode. (Ross Figs. 7-9, Col. 11, Ln 1 to Col 12, Ln 2 teaches an interactive editable source code editor including a panel with a history of AI code translation suggestions including multiple versions to be selected from by the user, and retaining a selected portion from the GUI)
In addition, it would have been obvious to one of ordinary skill prior to the effective filing date of the application to combine the teachings of Zeng and Ross as each is directed to output of automatic code generation systems and Ross recognized the need for “proposed graphical user interfaces can help manage multiple different versions/hypotheses simultaneously and provide display of an understandable and navigable presentation, according to one or more embodiments of the invention.” (Ross Col. 4, Ln 4-8)
Regarding Claim 17, Zeng does not teach, but Ross teaches:
17. The recording medium according to claim 16, further comprising converting the source code included in the output information into display information to be displayed on a GUI. (Ross Figs. 7-9, Col. 11, Ln 1 to Col 12, Ln 2 teaches an interactive editable source code editor including a panel with a history of AI code translation suggestions including multiple versions to be selected from by the user, and retaining a selected portion from the GUI)
Claim(s) 11 is/are rejected under 35 U.S.C. 103 as being unpatentable over “Zeng” (US PG Pub 2025/0068398) in view of Gorrono (US PG Pub 2023/0333827) as applied above and further in view of “Ross” (US Patent 11,747,970).
Regarding Claim 11, Zeng et al do not further teach, but Ross teaches:
11. The scenario management apparatus according to claim 9, further comprising an editing operation unit that retains the source code included in the output information in an editable mode, (Ross Figs. 7-9, Col. 11, Ln 1 to Col 12, Ln 2 teaches an interactive editable source code editor including a panel with a history of AI code translation suggestions including multiple versions to be selected from by the user)
wherein the history display includes a first source code that is the source code included in the first output information and a second source code that is the source code included in the output information acquired by the output information acquisition unit after the first output information, and (Ross Figs. 7-9, Col. 11, Ln 1 to Col 12, Ln 2 teaches an interactive editable source code editor including a panel with a history of AI code translation suggestions including multiple versions to be selected from by the user)when the acceptance unit accepts an operation of selecting one of the first source code and the second source code, the editing operation unit retains the scenario of the selected one source code in an editable manner. (Ross Figs. 7-9, Col. 11, Ln 1 to Col 12, Ln 2 teaches an interactive editable source code editor including a panel with a history of AI code translation suggestions including multiple versions to be selected from by the user)
In addition, it would have been obvious to one of ordinary skill prior to the effective filing date of the application to combine the teachings of Zeng and Ross as each is directed to output of automatic code generation systems and Ross recognized the need for “proposed graphical user interfaces can help manage multiple different versions/hypotheses simultaneously and provide display of an understandable and navigable presentation, according to one or more embodiments of the invention.” (Ross Col. 4, Ln 4-8)
Claim(s) 12 and 15 is/are rejected under 35 U.S.C. 103 as being unpatentable over “Zeng” (US PG Pub 2025/0068398) in view of “Gorrono” (US PG Pub 2023/0333827) as applied above and further in view of “Clement” (US PG Pub 2024/0419917).
Regarding Claim 12, Zeng does not further teach, but Clement teaches:
12. The scenario management apparatus according to claim 9, wherein the storage unit retains a correction instruction template that instructs to correct the source code, (Clement e.g. Fig. 8, ¶63 teaches a code repair template which provides a template for a promopt to make a repair to the code)
when the acceptance unit accepts an operation input for selecting the correction instruction template in association with the activity block, (Clement e.g Fig. 7 teaches a process for receiving input to use and submit the code repair template of Fig. 8 to receive a fix)
the rule information acquisition unit acquires the correction instruction template from the storage unit, ((Clement e.g Fig. 7 teaches a process for receiving input to use and submit the code repair template of Fig. 8 to receive a fix)
and the input unit inputs the source code included in the output information, the correction instruction template, and the rule information to the machine learning model. ((Clement e.g Fig. 7 teaches a process for receiving input to use and submit the code repair template of Fig. 8 to receive a fix, including 712, send it to the LLM for a fix)
In addition, it would have been obvious to one of ordinary skill prior to the effective filing date of the application to combine the teachings of Zeng and Clement as each is directed to output of automatic code generation systems and Clement recognized “Prompt templates are associated with each software engineering task that include the requisite information needed for the large language model to perform the target task and generate a relevant output.” (¶3)
Regarding Claim 15, Zeng does not further teach, but Clement teaches:
15. The scenario management apparatus according to claim 1, further comprising a violation check unit that checks whether or not the source code included in the output information acquired by the output information acquisition unit violates the rule information, wherein, when the violation check unit determines that the source code violates the rule information, the display generation unit generates information for displaying, on the GUI, that the source code violates the rule information. ((Clement e.g Fig. 5 teaches a process for receiving input to use and submit the code review prompt template of Fig. 4, and see further Figs. 6-10 implements processes for templates for other software engineering tasks including code search, repair and unit test to determine and fix code bugs)
In addition, it would have been obvious to one of ordinary skill prior to the effective filing date of the application to combine the teachings of Zeng and Clement as each is directed to output of automatic code generation systems and Clement recognized “Prompt templates are associated with each software engineering task that include the requisite information needed for the large language model to perform the target task and generate a relevant output.” (¶3)
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. The prior art cited in the attached pto-892 form includes prior art relevant to the disclosures related to LLM usage in workflow code generation systems.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MATTHEW J BROPHY whose telephone number is (571)270-1642. The examiner can normally be reached Monday-Friday, 9am-4:30pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Wei Zhen can be reached at 571-272-3708. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
MJB
9/28/2026
/MATTHEW J BROPHY/Primary Examiner, Art Unit 2191
