Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
Claims 1-18 and 20 are pending. Claims 1, 9, and 17 are independent and have different scopes in a mix and match of the limitation format and have been amended. Claim 19 is canceled and included in the independent Claim 17 as a part of the amendments.
This Application was published as U.S. 20250156460.
Apparent priority: 10 November 2023 (provisional).
Applicant’s amendments and arguments are considered but are either unpersuasive or moot in view of the new grounds of rejection that, if presented, were necessitated by the amendments to the Claims.
This action is Final.
Response to Amendments and Arguments
112(d)
Rejection of Claim 18 under 35 U.S.C. 112(d) is withdrawn in view of the amendments to this Claim and to Claim 17.
102/103
Applicant’s arguments are directed to the amended language and are moot in view of the modified grounds of rejection.
Claims 1, 9 and 17 are amended as follows.
Claims 1 and 9 are amended similarly with material discussed during the interview.
Claim 17 is amended to additionally included the canceled Claim 19 material.
1. A computing system for generating a workflow data structure, comprising:
one or more processors; and
one or more non-transitory computer-readable media storing instructions that are executable to cause the one or more processors to perform operations, the operations comprising:
receiving input data comprising a user query and query context data;
classifying, using a large language model (LLM), the user query to at least one content cluster of a plurality of content clusters based on the query context data;
constructing, using the LLM, a workflow data structure as a function of the classifying,
wherein constructing the workflow data structure comprises:
labeling at least a portion of the plurality of content clusters with metadata tags; and
defining one or more relationship mappings between at least a portion of the plurality of content clusters based on the metadata tags; and
generating, using the LLM, a query response as a function of the user query, the query context data, and the workflow data structure.
9. A method for generating a workflow data structure, comprising:
receiving, by one or more processors, input data comprising a corpus of documents, a user query, and query context data;
processing, by the one or more processors, the corpus of documents to generate training data;
training a large language model (LLM) using the training data;
classifying, using the LLM operating on the one or more processors, the user query to at least one content cluster of a plurality of content clusters based on the query context data;
constructing, using the LLM, a workflow data structure as a function of the classifying,
wherein constructing the workflow data structure comprises:
labeling at least a portion of the plurality of content clusters with metadata tags;and
defining one or more relationship mappings between at least a portion of the plurality of content clusters based on the metadata tags; and
generating, using the LLM, a query response as a function of the user query, the query context data, and the workflow data structure.
17. A computing system for generating a workflow data structure, comprising:
one or more processors; and
one or more transitory or non-transitory computer-readable media storing instructions that are executable to cause the one or more processors to perform operations, the operations comprising:
receiving input data comprising a corpus of documents;
processing the corpus of documents to generate training data,
wherein processing the corpus of documents comprises:
segmenting the corpus of documents into a plurality of segments based on a semantic pattern;
producing one or more embeddings for each segment of the plurality of segments; and
generating the training data based on the one or more embeddings; and
training a large language model (LLM) using the training data,
receiving a user query and query context data;
classifying, using the LLM, the user query to at least one content cluster of a plurality of content clusters based on the query context data;
constructing, using the LLM, a workflow data structure as a function of the classifying, wherein constructing the workflow data structure comprises:
labeling at least a portion of the plurality of content clusters with metadata tags; and
defining one or more relationship mappings between at least a portion of the plurality of content clusters based on the metadata tags; and
generating, using the LLM, a query response as a function of the user query, the query context data, and the workflow data structure.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 4, and 6-8 are rejected under 35 U.S.C. 103 as being unpatentable over Zangrilli (U.S. 20250131235) in view of Lowin (U.S. 20230019354).
Regarding Claim 1, Zangrilli teaches:
1. A computing system for generating a workflow data structure, [Zangrilli, Figure 1, “computing device 12.”] comprising:
one or more processors; and [Zangrilli, Figure 1, “processing circuitry 14.”]
one or more non-transitory computer-readable media storing instructions that are executable to cause the one or more processors to perform operations, the operations comprising: [Zangrilli, Figure 1, “memory 18” including “instructions 18.”]
receiving input data comprising a user query [Zangrilli, Figure 1, “GUI 26” including “Chat Interface 24” which receives “Query 22.” Figure 2 shows the “Query 22” as “How do I add a taxpayer?”] and query context data; [Zangrilli, Figure 1, “user info response 68” including “user value 66B” shown in Figure 2 and “My business and the cod is … 68” which provides context for the user query. “[0018] Upon executing the tax application virtual assistant program 20 in an inference phase, the processing circuitry 14 is configured to receive a user query 22, which may be a tax-related user query when interacting with a tax application or another type of query directed to a computer application in another field. The user query 22 may be input during a turn-based dialog session via a chat interface 24 in a graphical user interface (GUI) 26 of the computing device 12. The user query 22 may be a question about configuring the tax application software, a command perform an action using the tax application, a question about the tax application product and/or features, or a request for customer support, for example….”] [Figure 5, 502.]
classifying, using a large language model (LLM), the user query to at least one content cluster of a plurality of content clusters based on the query context data; [Zangrilli, Figure 1, “intent processor 28” including “intent 30” classifies the query into a particular category / “content cluster” based on the context / user identity/ “user value 66B.” “[0018] Upon executing the tax application virtual assistant program 20 in an inference phase,… In response to the query 22, an intent processor 28 may be implemented to identify at least one intent 30 in the user query 22 based on information in the query. The intent processor 28 is trained to classify the at least one intent 30 based on a corresponding intent command 32. The corresponding intent command may be selected from a plurality of intent commands stored in a command library 34, and in accordance with predefined rules for classifying intents 30.” The “inference phase” means the LLM is working on it: “[0017] The tax application virtual assistant program 20 (i.e., virtual assistant) is implemented to interface with a generative model, such as trained large language model 52. The large language model can be a generative pre-trained transformer model, such as Chat-GPT 40, LLaMA, etc….”] [Figure 5, 504.]
constructing, using the LLM, a workflow data structure as a function of the classifying; and [Zangrilli, Figure 1, “workflow selector 36” and “selected workflow 40.” As a function of the “intent 30” / “classifying” of the Claim. “[0028] Returning to FIG. 1, after the intent 30 has been identified and classified to a corresponding intent command 32, a workflow selector 36 is executed to select a workflow from a plurality of workflows….”] [Figure 5, 506.]
wherein constructing the workflow data structure comprises:
labeling at least a portion of the plurality of content clusters with metadata tags; and
defining one or more relationship mappings between at least a portion of the plurality of content clusters based on the metadata tags; and
generating, using the LLM, a query response as a function of the user query, the query context data, and the workflow data structure. [Zangrilli, Figure 1, “response 62” going through a number of phases and output as “query response 62B” to the “chat interface 24.” “… The processing circuitry is further configured to receive a response to the information augmentation prompt, generate and input a user query response prompt to the large language model, receive the user query response, and display the user query response in a chat interface.” Abstract. “[0033] When each operation in the selected workflow 40 is complete, the workflow orchestrator 42 may trigger the LLM prompt generator 48 to generate a user query response prompt 50B that instructs the LLM 52 to generate a user query response 62B to the user query 22. The user query response prompt 50B is input to the LLM 52, and the user query response 62B is received as output from the LLM 52. Natural language text 64B corresponding to the user query response 62B is then generated and displayed in the chat interface 24.”] [Figure 5, 524 and 526.]
Zangrilli does not teach metadata tags.
Lowin teaches:
wherein constructing the workflow data structure comprises: [Lowin, Figure2, teaches that a work flow data structure may be changed by including new tasks according to their associated metadata: “[0022] One feature of embodiments of the present invention is that there is the elimination of any requirement to access actual code or data, either for a given workflow or that may be operated upon by a given workflow. As such, these systems only need access to metadata that describes a given workflow itself, which allows the creation of a process that allows end users to utilize Core to design workflows on local machines. Once designed, these workflows are “built”, i.e., deployed to infrastructure owned and/or controlled by the end-user.”]
labeling at least a portion of the plurality of content clusters with metadata tags; and [Lowin the tasks are labeled with metadata tags that show the relationships between the tasks in the workflow. The “task definitions” of Levidow teach the “content clusters” of the Claim. “[0023] … Metadata describing a given workflow and/or its state may include, but is not limited to, information such as the name of one or more tasks comprising a given workflow, dependencies between various tasks comprising a given workflow, which may comprise various combinations of temporal and data dependencies, etc. A workflow administrator may use this information to reconstruct “dummy” versions of a workflow, which it may then use to orchestrate the real workflows without ever knowing any substantive information regarding any given workflow….” The “content clusters” in the context of the instant Application are instructions on how to perform a particular part of a task and can be taught by “tasks” that are steps that when performed to accomplish a workflow.]
defining one or more relationship mappings between at least a portion of the plurality of content clusters based on the metadata tags; and [Lowin, the tasks are represented with metadata tags that show the dependencies/relationships between the tasks in the workflow. Figure 2 and “[0035] … A workflow in accordance with embodiments of the present invention allows for the end-user to arbitrarily define dependencies between tasks in a workflow when conducting extract, transform, and load operations, which includes the use of input parameters to further define the scope of or input to a given task. …” Other types of information about the workflow is also coded in metadata. See [0031]-[0033] including: “[0031] In certain embodiments, a workflow contains additional items of information as metadata, including storage (where the workflow program code and required data is stored) and execution environment (a description of how to run the workflow). The workflow administrator agent 104 uses those pieces of metadata to retrieve the workflow program code from the workflow registry 106 and instruct the workflow engine 108 execute it on the end-user computing infrastructure.”]
PNG
media_image1.png
514
869
media_image1.png
Greyscale
PNG
media_image2.png
738
415
media_image2.png
Greyscale
Zangrilli and Lowin pertain workflow structures and it would have been obvious to add the use of metadata for establishing various components/tasks/ “content clusters” of a work flow and their dependencies as using metadata is a convenient method of labeling components and showing their relationship. This is combining known elements according to known methods for obtaining a known outcome.
Regarding Claim 4, Zangrilli teaches:
4. The computing system of claim 1, wherein generating the query response comprises generating a workflow report based on the workflow data structure. [Zangrilli, Figure 5, 506, 508, 510, 516, 518, 520, 522, 524. The query causes a sequence of operations to be performed and the “user query response” includes responses generated when the operations are complete which teaches the “workflow report” of the Claim: “[0046] Advancing from step 506 to step 508, the method 500 may include implementing a workflow orchestrator to schedule a sequence of the plurality of operations…” “0052] Advancing from step 518 to step 520, the method 500 may include generating a user query response prompt for the large language model. When each operation in the selected workflow is complete, the workflow orchestrator may trigger the LLM prompt generator to generate a user query response prompt to generate a user query response to the tax-related user query.”]
Regarding Claim 6, Zangrilli teaches:
6. The computing system of claim 1, wherein the operations further comprise processing the user query and the query context data to generate a smart prompt. [Zangrilli, Figure 1, “LLM Prompt Generator 48” and “Prompt 50.” Figure 5, 512. This is a smart prompt. “[0030] In determining the dependencies between the operations needed to process the user query, the dependency resolver 44 may identify that further information is needed to complete a target operation of the plurality of operations. To resolve the dependency, the workflow orchestrator 42 may trigger a large language model (LLM) prompt generator 48 to generate an information augmentation prompt 50 for an LLM 52. The information augmentation prompt 50 may be a system information prompt 50A that instructs the LLM 52 to retrieve the information needed to complete the target operation. The system information prompt 50A may be input to the LLM 52 via an application programming interface (API) 54.” “[0048] Proceeding from step 510 to step 512, the method 500 may include generating an information augmentation prompt for a large language model. To retrieve the information needed to complete the target operation and resolve the dependency, the workflow orchestrator may trigger a large language model (LLM) prompt generator to generate a prompt that instructs the LLM to retrieve the necessary information.”] (Smart Prompt is defined by the instant Application as one that includes context: “[0081] In an embodiment, the operations can further include processing the user query 112 and the query context data 114 to generate a smart prompt. As used in the current disclosure, the smart prompt is a specialized query designed to interact with the LLM 110. The smart prompt is generated to maximize the performance of the LLM 110 by using specific wording, structure, and contextual information. The smart prompt includes key details and context that can be used to guide the LLM 110 to provide a more refined answer. The smart prompt might provide additional information to the LLM 110 regarding which portion of the prompt to focus on, the length and format of the response, the key attributes, and the like.” Instant Application.)
Regarding Claim 7, Zangrilli teaches:
7. The computing system of claim 1, wherein the query context data comprises a user profile. [Zangrilli, Figure 1, “User Info Response 68” including “User Value 66B” includes information that teaches the user profile for example NY which the state where the user is from: “[0036] Continuing with the example in FIG. 2, natural language text 64C corresponding to the user information request 62C for a business name and tax code is displayed in the chat interface 24. A user information response 68 including a user value 66B for the information needed to complete the target operation is received in the chat interface 34, and the user value 66B is input to the dependency resolver 44 to complete the target operation. …. When additional information is needed from the user, natural language text 64C corresponding to the user information request 62C for a state is displayed in the chat interface 24. The user information response 68 including the user value 66B for the information needed to complete the target operation is received in the chat interface 24. As shown in FIG. 2, the example user value 66B is “NY,” which is input to the dependency resolver 44 to complete the selected workflow 40.”]
Regarding Claim 8, Zangrilli teaches:
8. The computing system of claim 1, wherein the operations further comprise:
generating one or more contextual inquiries based on the user query and the query context data; [Zangrilli, Figure 1, the “user info request 62C” is coming to the user from the LLM and is the “contextual inquiry” of the Claim. Figure 2, 64C and 64B include questions by the LLM from the user based on user’s initial query.]
receiving a second user query from a user based on the one or more contextual inquiries; and [Zangrilli, Figure 1, “user info response 68.” Figure 2, the responses by the user to LLM questions shown at various dialog portions marked 68 are responses by the user that teach the “second user query” of the Claim.]
updating the query context data based on the second user query. [Zangrilli, the inputs by the user in response to the questions by the system / “user info requests 62C” add to context/ “user info 68.” “[0036] Continuing with the example in FIG. 2, natural language text 64C corresponding to the user information request 62C for a business name and tax code is displayed in the chat interface 24. A user information response 68 including a user value 66B for the information needed to complete the target operation is received in the chat interface 34, and the user value 66B is input to the dependency resolver 44 to complete the target operation. Natural language text 64B corresponding to the user query response 64B is displayed in the chat interface 24, including a question with regard to whether a further subsequent task (add a registration) should be performed. In the illustrated example, the user again responds affirmatively, and the tax application virtual assistant program 20 classifies the effective user query 22 to intent command 32C, which commands the system to perform an add registration task to respond to the user query 22. When additional information is needed from the user, natural language text 64C corresponding to the user information request 62C for a state is displayed in the chat interface 24. The user information response 68 including the user value 66B for the information needed to complete the target operation is received in the chat interface 24. As shown in FIG. 2, the example user value 66B is “NY,” which is input to the dependency resolver 44 to complete the selected workflow 40.”]
Claims 2-3 are rejected under 35 U.S.C. 103 as being unpatentable over Zangrilli and Lowin in view of Jha (U.S. 20240346086) and Marwah (U.S. 20250036878).
Regarding Claim 2, Zangrilli teaches “[0017] The tax application virtual assistant program 20 (i.e., virtual assistant) is implemented to interface with a generative model, such as trained large language model 52….” But does not teach the steps of training. Neither does Lowin which was cited for use of metadata.
Jha teaches:
2. The computing system of claim 1, wherein the operations further comprise:
receiving a corpus of documents; [Jha, Figure 1, input of the “unlabeled documents 120”. Figure 2, input of “Unlabeled documents 205.”]
processing the corpus of documents to generate training data, wherein processing the corpus of documents comprises: [Jha teaches that is method is in service of generating training data for LLMs. “[0005] Also, recent developments in artificial intelligences (Als), machine learnings (MLs), large language models (LLMs) such as ChatGPT, and Bidirectional Encoder Representations from Transformers (BERT) and the likes require well-organized training data, which includes documents and corresponding labels. The labels are one of the most important factors that play a major role in training models for AIs, MLs, LLMs, etc.”]
segmenting the corpus of documents into a plurality of segments based on a semantic pattern; and [Jha, Figure 2, “vectorizer 215” generates embeddings from a whole document or portions like a sentence which implies segmentation and sentences are semantic patterns: “[0047] … For example, a document, a sentence, phrase, word, or character may be vectorized so as to be represented by embeddings or numerical vectors, of which each includes an ordered set of numbers, such as (0.1, 0.2, 0.3, 0.4, . . . , 0.9), so that mathematical and statistical operations can be performed to analyze and compare the vectors.”]
producing one or more embeddings for each segment of the plurality of segments; and [Jha, Figure 2, “Vectorizer 215”: “[0047] After the unlabeled documents 205 have been preprocessed by the preprocessor 210, the vectorizer 215 performs vectorization over the preprocessed documents or, in other words, converts the preprocessed documents into numerical vector (embedding) format….”]
generating the training data based on the one or more embeddings; and [Jha, the goal of document category discovery of Jah is to generate training data for other models. See [0005] above. “[0074] Classification accuracy metric 236 and classification accuracy normalized metric 238 may be used to evaluate the quality of a cluster-set of documents. Two or more approaches may be utilized to calculate the classification accuracy. For example, one approach may use the clustered documents as a training dataset for document-classification, where cluster-IDs are the categories or classes. Then, a weak classifier (e.g., a classifier using LogisticRegression or Decision Tree algorithm) may be built from a portion of this training dataset, and the weak classifier is evaluated over the remaining training dataset.”]
training the LLM using the training data. [Jha, the goal of document category discovery of Jah is to generate training data for other models. See [0005] above.]
Zangrilli/Lowin and Jha pertain to query/response by use of LLMs and it would have been obvious to combine the LLM training of Jha with the use of LLM by the combination as two parts of a process that work in tandem. This combination falls under combining prior art elements according to known methods to yield predictable results or use of known technique to improve similar devices (methods, or products) in the same way. See MPEP 2141, KSR, 550 U.S. at 418, 82 USPQ2d at 1396.
While Jha impliedly teaches the segmentation of the Claim, a more express reference is cited.
Marwah teaches:
receiving a corpus of documents; [Marwah, Figure 2, input of “Document Corpora 112.”]
processing the corpus of documents to generate training data, wherein processing the corpus of documents comprises:
segmenting the corpus of documents into a plurality of segments based on a semantic pattern; and [Marwah, Figure 2, “Segment Generation 202” receiving the documents and generating “Document Segments 204.]
producing one or more embeddings for each segment of the plurality of segments; and [Marwah, Figure 2, “Embeddings Computation 206” in communication with an “LLM based encoder 210.”]
generating the training data based on the one or more embeddings; and [Marwah, Figure 2, “Semantic Embedding Space 208” that can be used as training data.]
training the LLM using the training data.
PNG
media_image3.png
628
996
media_image3.png
Greyscale
PNG
media_image4.png
650
882
media_image4.png
Greyscale
Zangrilli/Lowin/Jha and Marwah pertain to query/response by use of LLMs and it would have been obvious to combine the document corpus ingestion of Marwah which is used in service of and LLM based query/response system with the system of Zangrilli/Jha which does not discuss the details of the LLM that generates the responses as two parts of a process that work in tandem. This combination falls under combining prior art elements according to known methods to yield predictable results or use of known technique to improve similar devices (methods, or products) in the same way. See MPEP 2141, KSR, 550 U.S. at 418, 82 USPQ2d at 1396.
Regarding Claim 3, Zangrilli does not teach the steps of training. Neither does Lowin which was cited for use of metadata.
Jha teaches:
3. The computing system of claim 2, wherein processing the corpus of documents further comprises classifying the one or more embeddings to at least one content cluster of the plurality of content clusters. [Jha is directed to self-organizing digital data by clustering the compressed document vectors (embeddings) into clusters of different categories / types of content. See Figures 1 and 2. 205, 225 below. Jha teaches that its method is to be used to generate training data for LLMs. See [0004]-[0005]. “[0010] One aspect illustrated herein includes a method that may be practiced. The method is for self-organizing modeling for digital data. The method includes acts for receiving a set of digital documents, documents, generating hyperparameter-optimization (HPO)-driven configurations for vectorizer, compressor, and cluster, vectorizing the set of digital documents to generate document vectors (also known as embeddings), compressing the document vectors, clustering the compressed document vectors into a cluster-set, calculating a utility score of the cluster-set of the compressed document vectors, iterating over this procedure to optimize over such cluster sets to find cluster-sets with “peak” utility scores.”]
PNG
media_image5.png
494
406
media_image5.png
Greyscale
PNG
media_image6.png
654
494
media_image6.png
Greyscale
Rationale for combination as provided for Claim 2.
Claim 5 rejected under 35 U.S.C. 103 as being unpatentable over Zangrilli and Lowin in view of Marwah.
Regarding Claim 5, Zangrilli does not teach the steps of training. Neither does Lowin which was cited for use of metadata.
Marwah teaches:
5. The computing system of claim 1, wherein the operations further comprise grounding the query response as a function of a corpus of documents by cross-referencing the query response against the corpus of documents, [Marwah, Figure 3 shows that the “embeddings 206” generated from the input “user query 302” are compared against the embeddings in the “semantic embedding space 208” which was generated in Figure 2 from the corpus of documents in order to generate the “response 312” and the response would be grounded in the truth represented by the corpus of documents. “[0102] FIG. 3 depicts data flow 300 in accordance with embodiments of the present disclosure. Data flow 300 is executed after at least one execution of data flow 200 (see FIG. 2) and for each query initiated by user 102. User 102 executes user query 302, such as via inputs to computer 104, which are then received by server 108 having access to latent semantic embedding space 208 maintained on data storage 110. Computation 206 is performed by server 108 using embedding model 304. Computation 206 maps the user query onto latent semantic embedding space 208 to find a result, such as by using the k-nearest neighbor in order to find nearest segment 310 and, as a result, mapping to the document segments and to the originating document.” ](Marwah matches the definition provided in the instant Application for the “grounding process” appears to fit the method of Marwah. “[0107] The grounding process can include a data-driven methodology where the generated query responses 120 is cross-referenced with the corpus of documents 106. This can include identifying and extracting real-world information associated with the relevant content clusters 116 or segment of document that was referenced in the query response 120….” Instant Application.)
wherein grounding the query response further comprises removing ungrounded information from the query response. [Marwah, Figure 5, 506, 508, 510. Only the responses that match the document segments in the corpus of documents are provided as response at 510. The N-best is determined and a certain number of most matching ones are considered grounded. The rest are discarded. “[0110] … tep 508 submits the prompt to LLM 308 and receives a response therefrom. The response comprises indicia of a document segment of a document wherein the answer provided may be attributed. If no reasonably correct answer is determined, a response indicating an absence of an answer (e.g., “I don't know.”) may be provided. The response is then provided 510 back to the user via computer 104 such as in response dialog 408 of display dialog 400.”]
Zangrilli/Lowin and Marwah pertain to query/response by use of LLMs and it would have been obvious to combine the document corpus ingestion of Marwah which is used in service of and LLM based query/response system with the system of Zangrilli which does not discuss the details of the LLM that generates the responses as two parts of a process that work in tandem. This combination falls under combining prior art elements according to known methods to yield predictable results or use of known technique to improve similar devices (methods, or products) in the same way. See MPEP 2141, KSR, 550 U.S. at 418, 82 USPQ2d at 1396.
Claims 9, 12, and 14-16 are rejected under 35 U.S.C. 103 as being unpatentable over Zangrilli in view of Jha and Lowin.
Regarding Claim 9, Zangrilli teaches:
9. A method for generating a workflow data structure, comprising:
receiving, by one or more processors, input data comprising a corpus of documents, a user query, and query context data; [Claim 1 and Claim 2] [Zangrilli, Figure 1, “GUI 26” including “Chat Interface 24” which receives “Query 22.” For context: “user info response 68” including “user value 66B” shown in Figure 2 and “My business and the cod is … 68” which provides context for the user query. Figure 5, 502.]
processing, by the one or more processors, the corpus of documents to generate training data; [Claim 2]
training a large language model (LLM) using the training data; [Claim 2]
classifying, using the LLM operating on the one or more processors, the user query to at least one content cluster of a plurality of content clusters based on the query context data; [Claim 1] [Zangrilli, Figure 1, “intent processor 28” including “intent 30” classifies the query into a particular category / “content cluster” based on the context / user identity/ “user value 66B.” Figure 5, 504.]
constructing, using the LLM, a workflow data structure as a function of the classifying,[Claim 1] [Zangrilli, Figure 1, “workflow selector 36” and “selected workflow 40.” As a function of the “intent 30” / “classifying” of the Claim. Figure 5, 506.]
wherein constructing the workflow data structure comprises:
labeling at least a portion of the plurality of content clusters with metadata tags;and
defining one or more relationship mappings between at least a portion of the plurality of content clusters based on the metadata tags; and
generating, using the LLM, a query response as a function of the user query, the query context data, and the workflow data structure. [Claim 1] [Zangrilli, Figure 1, “response 62” going through a number of phases and output as “query response 62B” to the “chat interface 24.” Figure 5, 524 and 526.]
Zangrilli does not teach training of the LLM by a corpus of training documents.
Jha teaches:
receiving, by one or more processors, input data comprising a corpus of documents, a user query, and query context data; [Claim 1 and Claim 2] [Jha, Figure 1, input of the “unlabeled documents 120”. Figure 2, input of “Unlabeled documents 205.”]
processing, by the one or more processors, the corpus of documents to generate training data; [Claim 2] [Jha teaches that is method is in service of generating training data for LLMs. “[0005] Also, recent developments in artificial intelligences (Als), machine learnings (MLs), large language models (LLMs) … require well-organized training data, which includes documents and corresponding labels. The labels are one of the most important factors that play a major role in training models for AIs, MLs, LLMs, etc.”]
training a large language model (LLM) using the training data; [Claim 2] [Jha, the goal of document category discovery of Jah is to generate training data for other models. See [0005] above.]
Rationale as provided for Claim 2.
Zangrilli and Jha do not teach the use of metadata.
Lowin teaches
wherein constructing the workflow data structure comprises:
labeling at least a portion of the plurality of content clusters with metadata tags; and [Lowin: see mapping in Claim 1.]
defining one or more relationship mappings between at least a portion of the plurality of content clusters based on the metadata tags; and [Lowin: see mapping in Claim 1.]
Zangrilli/Jha and Lowin are combined under a rationale similar to the rationale presented for Claim 1.
Claim 12 is a method claim with limitations corresponding to the limitations of Claim 4 and is rejected under similar rationale. (Limitations mapped to Zangrilli.)
Claim 14 is a method claim with limitations corresponding to the limitations of Claim 6 and is rejected under similar rationale. (Limitations mapped to Zangrilli.)
Claim 15 is a method claim with limitations corresponding to the limitations of Claim 7 and is rejected under similar rationale. (Limitations mapped to Zangrilli.)
Claim 16 is a method claim with limitations corresponding to the limitations of Claim 8 and is rejected under similar rationale. (Limitations mapped to Zangrilli.)
Claims 10-11 and 13 are rejected under 35 U.S.C. 103 as being unpatentable over Zangrilli in view of Jha and Lowin and Marwah.
Claim 10 is a method claim with limitations corresponding to the limitations of Claim 2 and is rejected under similar rationale. (Limitations mapped to both Jha and Marwah.)
Claim 11 is a method claim with limitations corresponding to the limitations of Claim 3 and is rejected under similar rationale. (Limitations mapped to Jha only but Claim 11 depends from 10.)
Claim 13 is a method claim with limitations corresponding to the limitations of Claim 5 and is rejected under similar rationale. (Limitations mapped to Marwah.)
Zangrilli/Jha/Lowin and Marwah pertain to query/response by use of LLMs and it would have been obvious to combine the document corpus ingestion of Marwah which is used in service of and LLM based query/response system and in order to generate responses that are grounded in the corpus with the system of Zangrilli/Jha which does not discuss the details of the LLM that generates the responses as two parts of a process that work in tandem. This combination falls under combining prior art elements according to known methods to yield predictable results or use of known technique to improve similar devices (methods, or products) in the same way. See MPEP 2141, KSR, 550 U.S. at 418, 82 USPQ2d at 1396.
Claims 17 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Jha and Marwah and Zangrilli and Lowin.
Regarding Claim 17, Jha and Marwah teach (See Rejection of Claim 2 for more detail):
17. A computing system for generating a workflow data structure, comprising:
one or more processors; and [Marwah, Figure 6, “processor 604.”]
one or more transitory or non-transitory computer-readable media storing instructions that are executable to cause the one or more processors to perform operations, the operations comprising: [Marwah, Figure 6, “memory 606.”]
receiving input data comprising a corpus of documents; [Claim 2] [Jha, Figure 1, input of the “unlabeled documents 120”. Figure 2, input of “Unlabeled documents 205.”] [Marwah, Figure 2, input of “Document Corpora 112.”]
processing the corpus of documents to generate training data, [Claim 2] [Jha teaches that is method is in service of generating training data for LLMs. [0005].]
wherein processing the corpus of documents comprises:
segmenting the corpus of documents into a plurality of segments based on a semantic pattern; [Claim 2] [Jha, Figure 2, “vectorizer 215” generates embeddings from a whole document or portions like a sentence which implies segmentation and sentences are semantic patterns: [0047].] [Marwah, Figure 2, “Segment Generation 202” receiving the documents and generating “Document Segments 204.]
producing one or more embeddings for each segment of the plurality of segments; and [Claim 2] [Jha, Figure 2, “Vectorizer 215”: “[0047] After the unlabeled documents 205 have been preprocessed by the preprocessor 210, the vectorizer 215 performs vectorization over the preprocessed documents or, in other words, converts the preprocessed documents into numerical vector (embedding) format….”] [Marwah, Figure 2, “Embeddings Computation 206” in communication with an “LLM based encoder 210.”]
generating the training data based on the one or more embeddings; and [Claim 2] Jha, the goal of document category discovery of Jah is to generate training data for other models. See [0005] and [0074].] [Marwah, Figure 2, “Semantic Embedding Space 208” that can be used as training data.]
training a large language model (LLM) using the training data; [Claim 2] [Jha, the goal of document category discovery of Jah is to generate training data for other models. See [0005] above.]
receiving a user query and query context data; [Marwah, Figure 3, “user query 302.”]
classifying, using the LLM, the user query to at least one content cluster of a plurality of content clusters based on the query context data; [Marwah, Figure 3, the Query is matched/classified to a document 310 in the ingested semantic embedding space 208 of the corpus of documents/content.]
constructing, using the LLM, a workflow data structure as a function of the classifying,
wherein constructing the workflow data structure comprises:
labeling at least a portion of the plurality of content clusters with metadata tags; and
defining one or more relationship mappings between at least a portion of the plurality of content clusters based on the metadata tags; and
generating, using the LLM, a query response as a function of the user query, the query context data, and the workflow data structure. [Marwah, Figure 3, “Response 312.”]
Jha and Marwah pertain to the use of LLMs and it would have been obvious to combine the document corpus ingestion of Marwah which is more detailed with the system of Jha which does not discuss some parts of document corpus digestion in as much detail to provide the whole method. This combination falls under simple substitution of one known element for another to obtain predictable results or use of known technique to improve similar devices (methods, or products) in the same way. See MPEP 2141, KSR, 550 U.S. at 418, 82 USPQ2d at 1396.
Jha and Marwah do not teach the workflow features of the Claim.
Zangrilli teaches (all limitations map to limitations of Claim 1 and more detailed mapping is shown with respect to Claim 1):
receiving a user query and query context data; [Claim 1] [Zangrilli, Figure 1, “GUI 26” including “Chat Interface 24” which receives “Query 22.” For context: “user info response 68” including “user value 66B” shown in Figure 2 and “My business and the cod is … 68” which provides context for the user query. Figure 5, 502.]
classifying, using the LLM, the user query to at least one content cluster of a plurality of content clusters based on the query context data; [Claim 1] [Zangrilli, Figure 1, “intent processor 28” including “intent 30” classifies the query into a particular category / “content cluster” based on the context / user identity/ “user value 66B.” Figure 5, 504.]
constructing, using the LLM, a workflow data structure as a function of the classifying,[Claim 1] [Zangrilli, Figure 1, “workflow selector 36” and “selected workflow 40.” As a function of the “intent 30” / “classifying” of the Claim. Figure 5, 506.]
wherein constructing the workflow data structure comprises:
labeling at least a portion of the plurality of content clusters with metadata tags; and
defining one or more relationship mappings between at least a portion of the plurality of content clusters based on the metadata tags; and
generating, using the LLM, a query response as a function of the user query, the query context data, and the workflow data structure. [Claim 1] [Zangrilli, Figure 1, “response 62” going through a number of phases and output as “query response 62B” to the “chat interface 24.” Figure 5, 524 and 526.]
Jha/Marwah and Zangrilli pertain to query/response by use of LLMs and it would have been obvious to combine the workflow generation features of Zanglrilli which is a type of response with the system of combination as a substitute for the simpler response generation of the combination. This combination falls under simple substitution of one known element for another to obtain predictable results or use of known technique to improve similar devices (methods, or products) in the same way. See MPEP 2141, KSR, 550 U.S. at 418, 82 USPQ2d at 1396.
Lowin teaches
wherein constructing the workflow data structure comprises:
labeling at least a portion of the plurality of content clusters with metadata tags; and [Lowin: see mapping in Claim 1.]
defining one or more relationship mappings between at least a portion of the plurality of content clusters based on the metadata tags; and [Lowin: see mapping in Claim 1.]
Jha/Marwah/ Zangrilli and Lowin are combined under a rationale similar to the rationale presented for Claim 1.
Claim 20 is a method claim with limitations corresponding to the limitations of Claim 8 and the limitations are mapped under similar rationale. (Mapped to Zangrilli.) (Claim 20 depends from 17.)
Claim 18 is rejected under 35 U.S.C. 103 as being unpatentable over Jha and Marwah and Zangrilli and Lowin and further in view of Nair (U.S. 20240303496).
Regarding Claim 18, Jha and Marwah and Zangrilli either teach or rely on training and the classification of Jha can be used for domain-specific training. But, this point is not express.
Nair teaches:
18.The computing system of claim 17, wherein the training data comprises task-specific training data; and training the LLM further comprises specifically training the LLM using the task-specific training data. (This Claim is interpreted as training using domain-specific data and needs to be corrected to overcome the 112(b and d) rejections; depending on the amendments used to overcome the 112(b and d) the art rejection may be modified.) [Nair teaches use of domain specific training data to train a domain specific LLM for responding to queries. Figure 2, 240.]
Jha/Marwah/Zangrilli/Lowin and Nair pertain to query/response by use of LLMs and it would have been obvious to add the domain-specific training of Nair to the general training of the combination for more accuracy. This combination falls under simple substitution of one known element for another to obtain predictable results or use of known technique to improve similar devices (methods, or products) in the same way. See MPEP 2141, KSR, 550 U.S. at 418, 82 USPQ2d at 1396.
PNG
media_image7.png
722
534
media_image7.png
Greyscale
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Levidow (US 20080244565) teaches:
wherein constructing the workflow data structure comprises: [Levidow teaches that a work flow data structure may be changed by including new tasks: “[0027] During the execution of a workflow, the device 104 may perform a query to a remote update server 120 through the Internet 118 or other network connection. A query may include a request for updates to the workflow or new tasks.”]
labeling at least a portion of the plurality of content clusters with metadata tags; and [Levidow, the tasks are labeled with metadata tags that show the relationships between the tasks in the workflow. The “task definitions” of Levidow teach the “content clusters” of the Claim. “[0005] The workflow definition may be adaptable to modification during execution by adding, removing, or changing a task definition dynamically. A query to a remote server may return updated setup task metadata that defines changes to the workflow structure. The remote server may also return executable or other definitions for a new setup task module that is performed to accomplish the new task.” The “content clusters” in the context of the instant Application are instructions on how to perform a particular part of a task and can be taught by “task definitions” that are steps that when performed to accomplish a task. “[0007] Further, the term "workflow" is used in a generic sense to refer to any type of definition of a sequence or method. …” ]
defining one or more relationship mappings between at least a portion of the plurality of content clusters based on the metadata tags; and [Levidow, the tasks are labeled with metadata tags that show the relationships between the tasks in the workflow: “[0028] The update server 120 may have updated setup media 122 that contains new task metadata 124 with dependencies 126 for the new task. Along with the metadata 124, new task modules 128 may also be available.” “[0029] The update server 120 may provide new task metadata 124 and dependencies 126 that may be incorporated into the workflow 114 that may be executing. In some instances, the update server 120 may also have an updated workflow 130 that may be used in conjunction with or to replace the task workflow 114. In such an instance, the device 104 may update the entire workflow 114 to include any new tasks by requesting the updated workflow 130 from the update server 120 prior to executing the workflow 114.”]
PNG
media_image8.png
698
509
media_image8.png
Greyscale
PNG
media_image9.png
756
495
media_image9.png
Greyscale
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to FARIBA SIRJANI whose telephone number is (571)270-1499. The examiner can normally be reached 9 to 5, M-F.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Pierre Desir can be reached at 571-272-7799. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Fariba Sirjani/
Primary Examiner, Art Unit 2659
Prosecution Insights