Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This action is in response to the communication filed on April 22, 2026.
Response to Amendment
Applicant’s amendment filed on April 22, 2026 has been received, entered into the record and considered.
As a result of the amendment filed on April 22, 2026, no claims have been amended, cancelled or newly added.
Claims 1-20 are examined and are pending.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-20 are rejected under 35 U.S.C. 103 as being unpatentable over Du et al (US 2005/0094137 A1), in view of Xu et al (US 12,332,896 B1).
As per claim 1, Du discloses:
- a computer-implemented method performed by one or more computing devices, comprising (a computer implemented method, Para [0007], line 1-10),
- receiving a natural language description of a requested data pipeline (processing natural language description by requesting a data pipeline, Para [0006], [0030], [0082] – [0083]),
- generating a model prompt comprising the received natural language description and a prompt template, wherein (model prompt with received natural language description, Fig. 2, item 2002, 2004, Para [0080],
- the prompt template comprises a set of action labels and a processing example (node in a graph to process set of action, such as upload an image, drop here, rotate, scale, etc., Fig. 4A-4D, Para [0251]),
- each action label in the set of action labels indicates a respective data processing action (processing data in each action, Para [0007], [0080], [0103], [0281], Fig. 4A-4D),
- the processing example includes a sample query comprising a sample natural language description of a sample data pipeline and a sample answer comprising one or more sample action labels associated with the sample natural language description of the sample data pipeline, each of the one or more sample action labels being included in the set of action labels (examples with questions and answer set (i.e., sample query and sample answer) with sample natural language description and action labels, Para [0116] – [0166], [0174] – [0183],
- generating a project template from the one or more executable nodes, wherein the project template comprises the one or more executable nodes and one or more connections associated with the one or more executable nodes (generating software code (i.e., project template), for pipeline node, Para [0030], [0037] – [0038], [0060]),
- and generating the requested data pipeline from the project template (generating a data pipeline, Para [0040], [0043]),
Du does not explicitly disclose querying a multimodal model (MM) with the model prompt; receiving an MM response from the MM, wherein the MM response comprises one or more action labels corresponding to the natural language description of the requested data pipeline in a format guided by the prompt template; identifying one or more executable nodes in an executable node library, each executable node corresponding to a respective action label in the MM response and configured to perform the data processing action associated with the respective action label. However, in the same field of endeavor Xu in an analogous art disclose querying a multimodal model (MM) with the model prompt (querying a large language model (i.e., multimodal model), Fig. 1A, column 9, line 40-50, receiving an MM response from the MM, wherein the MM response comprises one or more action labels corresponding to the natural language description of the requested data pipeline in a format guided by the prompt template (response to the question or query or input, column 4, line 60-67, column 6, line 25-35, Fig. 1A), identifying one or more executable nodes in an executable node library, each executable node corresponding to a respective action label in the MM response and configured to perform the data processing action associated with the respective action label (identifying node to response to user input query or question and perform respective action, Fig. 1A, 1C-1D, item 134, 122, column 11, line 15-30, column 15, line 40-60).
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to incorporate multimodal query with the model prompt and identifying node to execute the response to the query input as taught by Xu as the means to process natural language query with description in a data pipeline in Du, (Du, Para [0006], [0030], [0082] – [0083], Xu, Fig. 1A, column 9, line 40-50). Du and Xu are analogous prior art since they both deal with processing natural language query with description in a multimodal model or generative model. A person of the ordinary skill in the art would have been motivated to make aforementioned modification to develop code for a particular action in a natural language question description in an efficient manner. This is because one aspect of Du develops software code using graphical representations of functionalities instead of writing traditional text-based code in an accurate and efficient way, as described in at least Para [0029]. Querying a multimodal data model in part of this process. However, Du doesn’t specify any particular manner in which query multimodal model and identify node to answer the natural language description query. This would have lead one of the ordinary skill in the art to seek and recognize the Querying a multimodal data model with user prompt as taught by Xu. Xu describes querying large language multimodal model using a graph construction in an efficient manner as described at least in column 7, line 25-30, as desired by Du.
As per claim 2, rejection of claim 1 is incorporated, and further Du discloses:
- wherein the generating the requested data pipeline comprises generating software code that, when executed, performs the requested data pipeline by combining, as provided in the project template, the one or more executable nodes configured to perform respective data processing actions (software code to generate data pipeline and execute in a node to perform a task (i.e., action), Abstract, line 12-18, Para [0030], [0037] – [0038], [0060], Fig. 1, 4A-4D).
As per claim 3, rejection of claim 1 is incorporated, and further Du discloses:
- the sample answer of the prompt template further comprises, for each of the one or more action labels associated with the sample natural language description, a data source label identifying a data source on which the data processing action of the action label will operate (uploading images (i.e., data) from different sources (i.e., identifying data sources), Para [0257], [0264], [0296], [0090]),
- the MM response further comprises one or more data source labels, each data source label associated with a respective one of the one or more action labels in the MM response (multimodal response with respective action, Para [0116] – [0166]).
As per claim 4, rejection of claim 3 is incorporated, and further Du discloses:
- wherein two or more action labels in the MM response are associated with identical data source labels (drop here, item 145 and 150 (two or more action labels) are associated with input image from a camera (i.e., identical data source), Fig. 4A, item 105-125, 140-165).
As per claim 5, rejection of claim 1 is incorporated, and further Du discloses:
- wherein the prompt template comprises additional processing examples, each additional processing example including an additional sample query and an additional sample answer (more than one question and answer (i.e. additional processing example with sample query and sample answer), Para [0116] – [0166], [0174] – [0183]).
As per claim 6, rejection of claim 1 is incorporated, and further Xu discloses:
- prior to receiving the natural language description: receiving, from a user, an initial natural language request (initial request prior to description, Fig. 1C, item 168, 172, Fig. 1, column 17, line 40-55),
- querying the MM with the initial natural language request to generate the natural language description of the requested data pipeline (Fig. 1, item 106, 134, searching large language model (i.e., multimodal model), column 7, line 20-30, column 9, line 55-65).
As per claim 7, rejection of claim 1 is incorporated, and further Du discloses:
- generating the set of action labels on-the-fly based on a listing of available executable nodes in the executable node library (quickly initialize a pipeline for an action available in the node library, Para [0046], [0073] – [0074], Fig. 1, item 40, Fig. 4-5).
As per claim 8, rejection of claim 1 is incorporated, and further Du discloses:
- generating a user interface to visualize the project template, wherein the user interface provides a preview mode comprising the project template, wherein the preview mode visualizes the one or more executable nodes and the one or more connections associated with the one or more executable nodes (visualizing project template, Fig. 5, item 565, preview to visualize connection of nodes, Fig. 5, Fig 4A-4D, Para [0056] – [0057], Para [0240]).
As per claim 9, rejection of claim 1 is incorporated, and further Xu discloses:
- wherein at least one of the set of action labels, one or more processing examples, or the one or more executable nodes are pre- defined in a database (nodes are predefined in a data base, Fig. 4, item 414, 428, column 25, line 35-45).
As per claim 10, rejection of claim 9 is incorporated, and further Xu discloses:
- updating the database by adding, to the database, at least one of a new action label, a new processing example, or a new executable node (modifying graph by adding or removing node or edge, column 16, line 45-58, Fig. 4).
As per claim 11, rejection of claim 9 is incorporated, and further Xu discloses:
- updating the database by deleting, from the database, at least one of the one or more action labels, the one or more processing examples, or the one or more executable nodes (modifying graph by adding or removing node or edge, column 16, line 45-58, Fig. 4).
As per claim 12, rejection of claim 1 is incorporated, and further Xu discloses:
- parsing the MM response to extract the one or more action labels (parsing unstructured data (i.e., parsing response), column 4, line 5-10, column 7, line 1-10, Fig. 1, 4-5).
- determining, for each extracted action label, a correlation between the extracted action label and the one or more executable nodes (relationship between dispersed pieces (i.e., correlation), column 2, line 55-67, column 6, line 1-15),
- selecting, from the executable node library for each extracted action label, a respective one of the one or more executable nodes correlating to the extracted action label or a default executable node based on the correlation between the extracted action label and the one or more executable nodes being low (selecting node to perm a task, column 14, line 60-67, column 16, line 45-55).
As per claim 13, rejection of claim 1 is incorporated, and further Xu discloses:
- wherein the one or more executable nodes collectively identify at least two types of data processing actions of the data pipeline (matching data with desired type of response (i.e., at lest two types of action), column 4, line 10-20, column 14, line 25-35, column 24, line 45-55, Fig. 4, 1).
As per claims 13-17 and 19,
Claims 13-17 and 19 are system claims corresponding to method claims 1, 3, 6, 8 and 12 respectively and rejected under the same reason set forth to the rejection of claims 1, 3, 6, 8 and 12 above.
As per claim 18, rejection of claim 14 is incorporated, and further Du discloses:
- updating, using a user interface, the project template based on additional information received from the user, wherein the additional information comprises one or more edits associated with at least one parameter of the one or more executable nodes or the one or more connections associated with the one or more executable nodes (edit to modify the code, Para [0005], [0008], [0032], [0061]).
As per claim 20,
Claim 20 is a computer readable medium claim corresponding to method claim 1 respectively and rejected under the same reason set forth to the rejection of claim 1 above.
Response to Arguments
Applicant's arguments filed on filed on April 22, 2026, with respect to claims 120 have been fully considered but they are not persuasive.
In response to applicant’s argument in page 10, line 27-33, applicants argued that, references fail to disclose at least the claimed limitations of "generating a model prompt comprising ... a prompt template ... the prompt template comprises a set of action labels," "receiving an MM response ... wherein the MM response comprises action labels," and "identifying one or more executable nodes." Examiner disagree and respectfully response that, Du teaches generating a model prompt comprising the received natural language description and a prompt template, in Fig. 2, item 202, 2004 Para [0080], [0251] and also in Para [0029], [0169]. Examiner broadest reasonable interpretation: according to applicant’s specification Para [0041], line 5-12 model prompt provide useful information to the MM, including identification of different types of nodes that the platform may support and which may be selected for use in the data pipeline. Based on the user input along with the model prompt, the system may query the MM using a one-shot or few-shot process, or an iterative process where the MM may be queried multiple times. Accordingly, Du in in Fig. 2, item 2008, Para [0080], [0169] teaches system prompt which can prompt the model 204 with description of each node and filtering out unrelated nodes for the target pipeline (i.e., identification of different types of node for the use in the data pipeline, see Para [0080] – [0082]), Applicant’s specification Para [041], also describes that the system may query the MM using a one-shot or few-shot process, Accordingly, Du in in Fig. 2, item 2008, Para [0169], prompt the pseudocode drafting model to generate the set of pseudocode to be supplied with a "few shot" prompt that includes a few examples of the correct output pseudocode based on example natural language descriptions.
Xu also reasonably teaches model prompt with received natural language description and a prompt template at least in column 25, line 50-65, Table 5-9. … How can one replicate the issue where <issue description>? Answer: Reproduction steps … Question: How can the ' can't login ' error be reproduced? Answer: Steps … to Reproduce (i.e., prompt with natural language description in a prompt template).
Du also teaches the prompt template comprises a set of action labels and a processing example in Fig. 4A-4D, Para [0251], also in Para [0029], [0081] – [0166]. Para [0029] Du teaches a real-time visual feedback, drag-and-drop components, and customizable templates where a user can provide an interactive and user-friendly interface for the graphical development of software code. Para [0081] – [0166], Du teaches prompt with set of action and processing such as You are an assistant tasked with aiding the user in constructing an AI pipeline (i.e., natural language description), [0083] For this assignment, select a small set of nodes to fulfill the user's pipeline request, guidelines and different question with description and answer (i.e., action label) as described in Para [0082] – [0166].
Xu also teaches prompt template comprises a set of action labels and a processing example in column 25, line 50-65, Table 5-9.
Du also teaches the processing example includes a sample query comprising a sample natural language description of a sample data pipeline and a sample answer comprising one or more sample action labels associated with the sample natural language description of the sample data pipeline, each of the one or more sample action labels being included in the set of action labels in Para [0116] – [0166], [0174] – [0183], examples with questions and answer set (i.e., sample query and sample answer) with sample natural language description and action labels,
Xu also reasonably teaches this limitation in column 25, line 50-65, Table 5-9.
Du also teaches generating a project template from the one or more executable nodes, wherein the project template comprises the one or more executable nodes and one or more connections associated with the one or more executable nodes in Para [0030], [0037] – [0038], [0060]), generating software code (i.e., project template), for pipeline node,
Xu also teaches generating the requested data pipeline from the project template, in Para [0040], [0043], generating a data pipeline for the project, see also in Para [0030].
In response to applicant’s remarks in Page 11, line 1-6. Applicant argued that, Xu fails to cure the deficiency regarding "identifying one or more executable nodes ... corresponding to a respective action label." For example, the cited portions of Xu describe a graph, where nodes of the graph represent portions of a document. Examiner disagree and respectfully response that, Xu teaches querying a multimodal model (MM) with the model prompt in Fig. 1A, column 9, line 40-50, querying a large language model (i.e., multimodal model), Xu also teaches receiving an MM response from the MM, wherein the MM response comprises one or more action labels corresponding to the natural language description of the requested data pipeline in a format guided by the prompt template in column 4, line 60-67, column 6, line 25-35, Fig. 1A, response to the question or query or input. Xu also teaches identifying one or more executable nodes in an executable node library, each executable node corresponding to a respective action label in the MM response and configured to perform the data processing action associated with the respective action label in Fig. 1A, 1C-1D, item 134, 122, column 11, line 15-30, column 15, line 40-60, (identifying node to response to user input query or question and perform respective action.
Examiner broadest reasonable interpretation:
Xu in Fig. 1A, item 106, receive a user natural language description query, “How to reproduce an issue where the admin sees several errors when attempting update of user emails on dashboard”. An intent understanding component 110 in Xu interprets the first query portion 108 into a query intent 116. The query intent 116 understand the query STEPS_TO_REPRODUCE and pass/communicate with large language model along with a graph template to extract subgraph of node 122 from node library 120 to take responsive action label 134, “to reproduce the issue where admin seeing several errors when attempting updates on dashboard, “follow these steps: 1. Refer to CSV file provided in the context. 2. Open the dashboard ID 123456. 3. Click on instance > profile. 4. Search for updates from csv file, See Fig. 1, item 110, 116, 118, 128, 122, 134, column 9, line 25-50, column 10, line 30-55, column 11, line 5-7. Examiner broadly interpret the graph 120 as a node library which includes N documents each with plurality of nodes including nodes 121. Each of these nodes includes natural language query description and respective action label using embedding based retrieval component 124. Embedding-based retrieval component 124 matches an embedding of the first node 121 with the embedding of the entity 118 admin sees several errors when attempting update of user emails on dashboard – issue description, and perform processing action associated with respective action level, “to reproduce the issue where admin seeing several errors when attempting updates on dashboard, “follow these steps: 1. Refer to CSV file provided in the context. 2. Open the dashboard ID 123456. 3. Click on instance > profile. 4. Search for updates from csv file. This concept is also taught in Fig. 1C and 1D.
Beside Xu, Du also teaches node library in Fig. 1, item 40, Para [0046], [0049], [0053], [0077], this node library can be a central repository of pre-made components or code sets that users can incorporate into their applications for user query description and respective action label, as claimed.
Therefore, examiner firmly believe that, Du and Xu alone or in combination reasonably teaches the argued limitation and claim 1, 14 and 20 as claimed.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Contact Information
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MOHAMMED R UDDIN whose telephone number is (571)270-3138. The examiner can normally be reached M-F: 9:00 AM-5:00 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Beausoliel Robert can be reached at 571-272-3645. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MOHAMMED R UDDIN/Primary Examiner, Art Unit 2167
Prosecution Insights