Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
This communication is in response to the amendment filed on 3/12/2026. The Examiner acknowledges amended claims 1-6, 9-16, and 19. Claims 7-8, 17-18, and 20 have been cancelled. Claims 1-6, 9-16, and 19 are pending and claims 1-6, 9-16, and 19 are rejected. Claims 1 and 11 is/are independent.
The objections to informalities in the claims are withdrawn in view of Applicant's amendments.
The rejection(s) of claims under 35 U.S.C. § 102/103 have been updated based on new grounds of rejection as indicated below.
Response to Arguments
Applicant's arguments filed 3/12/2026 have been fully considered. Applicant argues (see Remarks, page 8, first paragraph through page 10, bottom paragraph) that the references cited in the previous rejection fail to disclose the newly amended claim features. This argument is persuasive. Therefore, the rejections are withdrawn. However, upon further consideration, a new ground of rejection is made in view of Lancioni et al. U.S. Publication 20250117593 (hereinafter “Lancioni”) in view of McAuliffe et al. U.S. Publication 20220179761 (hereinafter “McAuliffe”), further in view of Clement et al. U.S. Publication 20240386103 (hereinafter “Clement”). Lancioni teaches receiving a prompt from a user, submitting the prompt to the LLM and examining for compliance the response to the prompt received from the LLM (para. 35, 37, 52). McAuliffe teaches an LSM tree data structure which can hold key value pairs storing information such as undesirable terminology (para. 209, 216). Clement teaches generating a prompt injection signal indicating a prompt injection attack responsive to a response from an LLM failing to comply with response requirements. The detection is facilitated by submitting a secret to the LLM and detecting that the secret is not included in the response from LLM. Such secret can be stored in the LSM tree of McAuliffe (figure 42, element 4210, 85:25-26). The LSM tree can also hold key value pairs storing information such as a secret used to detect prompt injection attacks as taught by Clement. The combination of references discloses the newly amended limitations of claim 1.
Independent claim 11 recites limitations analogous to the limitations of claim 1 and is also rejected for similar reasons. Regarding applicant’s arguments with respect to the dependent claims 2-6, 9-10, 12-16, and 19, applicant’s amendments to the independent claims have necessitated a new ground of rejection with respect to the independent claims from which the dependent claims depend, thereby requiring new grounds of rejection for the dependent claims also.
Accordingly, Applicant's argument is persuasive, the rejection is withdrawn, and new ground(s) of rejection are presented herein. Note that this action is made FINAL. See MPEP § 706.07(a).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1, 6, 10-11, and 16 is/are rejected under 35 U.S.C. 103 as being unpatentable over Lancioni et al. U.S. Publication 20250117593 (hereinafter “Lancioni”) (filing date December 22, 2023, and provisional application priority date October 6, 2023) in view of McAuliffe et al. U.S. Publication 20220179761 (hereinafter “McAuliffe”), further in view of Clement et al. U.S. Publication 20240386103 (hereinafter “Clement”).
As per claim 1, Lancioni discloses
A method comprising:
[0002] This disclosure relates generally to machine learning and, more particularly, to methods and apparatus to self-guardrail large language model responses.
receiving, at a server [guardrail platform, para 35; server, para 52] from a first user device, a first user prompt segment [receive prompt from user, para. 35; client hardware device, para. 52] to a large language model (LLM);[ large language model, para. 37]
[0035] In some examples, the guardrail platform 110 includes means for accessing. For example, the means for accessing may be implemented by request accessor circuitry 130. The example request accessor circuitry 130 of the illustrated example of FIG. 1 receives a prompt from a user1. In some examples, the prompt may be received via an application programming interface (API). In some other examples, the prompt may be received via a web interface. In some examples, the prompt may originate from an application (e.g., an email application) on behalf of the user.
Lancioni Para. 52 The machine readable instructions may be distributed across multiple hardware devices and/or executed by two or more hardware devices (e.g., a server and a client hardware device).
[0037] In some examples, the guardrail platform 110 includes means for interfacing with a large language model. For example, the means for interfacing may be implemented by large language model interface circuitry 150. The example large language model interface circuitry 150 of the illustrated example of FIG. 1 receives the prompt accessed by the request accessor circuitry 130 and provides the prompt to the large language model circuitry 155. In some examples, the large language model interface circuitry 150 receives a prompt that has been modified by the prompt modifier circuitry 175, and provides the modified prompt to the large language model circuitry 155. In some examples, the large language model interface circuitry 150 operates on a prompt from the message validator 170 to assist in validation of response message. Once the prompt is provided to the large language model circuitry 155, the large language model circuitry 155 provides a response to the prompt (e.g., an output of the execution of the large language model) to large language model interface circuitry 150. The example large language model interface circuitry 150 receives this response from the large language model circuitry 155. This response may then be parsed and/or utilized by other components of the guardrail platform 110. In some examples, when communicating with the large language model circuitry 155, the example large language model interface circuitry 150 identifies a model (e.g., model 155A, model 155B, etc.) that is to be used by the large language model circuitry 155 to process the prompt.
generating a first LLM prompt from the first user prompt segment;
[prompt that has been modified by the prompt modifier circuitry 175, para. 37]
sending the first LLM prompt to the LLM;
[provides the modified prompt to the large language model circuitry 155, para. 37]
Lancioni [0059] The example large language model interface circuitry 150 provides the prompt to the LLM circuitry 155 for generation of a response message. (Block 230). In examples disclosed herein, a first model is executed by the LLM circuitry 155 to generate the response message. The response message is provided to the message validator circuitry 170 for validation.
receiving a first response from the LLM;
[the large language model circuitry 155 provides a response to the prompt, para. 37]
comparing [review to determine if acceptable to send to user, para. 43] the first response to filtering data [terms, phrases, sayings, offensive language, slang, that are inappropriate, para. 43] for the first response to generate a comparison result[determines that the message is not appropriate, para 46; determine if message is acceptable, para. 43] ; and[response message is provided to the message validator circuitry 170 for validation, para 59]
[0043] In some examples, the guardrail platform 110 includes means for validating a message. For example, the means for validating may be implemented by message validator circuitry 170. The example message validator circuitry 170 of the illustrated example of FIG. 1 reviews the message returned to the large language model interface circuitry 150 to determine if the message is acceptable for sending to the user 102. In some examples, the message validator circuitry 170 augments the message by placing the message received from the large language model circuitry 155 into a prompt template that is designed to request the large language model circuitry 155 to determine whether any terms, phrases, sayings, etc. are included in the message that indicate that the message is inappropriate. Such phrases may include, for example, offensive language, slang or unprofessionally written language, an answer that does not relate to a topic at hand (e.g., the answer does not relate to cybersecurity, when it should), the answer contains harmful suggestions or advice, the answer judges a person, etc.
Lancioni [0046] if the message validator circuitry 170 determines that the message is not appropriate because the answer contains offensive language, the example prompt modifier circuitry 175 may modify the prompt to request that offensive language not be used. The modified prompt is then provided to the LLM circuitry 155 via the large language model interface circuitry 150 for generation of a subsequent response message.
detecting, based on the comparison result, that the first response fails [determines that the message is not appropriate because the answer contains offensive language, para 46;] to comply with the offensive language policy; and
However, Lancioni does not expressly disclose
comparing the first response to a structured data schema for the first response to validate the first response;
detecting, based on the comparison result, that the first response fails to comply with the structured data schema; and
generating a prompt injection signal indicating a prompt injection attack in the first LLM prompt responsive to the first response failing to comply with the structured data schema.
McAuliffe discloses storing data in a hierarchy [LSM tree, para 209] of key value pairs [LSM trees maintain key value pairs, para. 209] and searching [look for item, para. 216] for values in the hierarchy
[Para. 39 of this specification describes structured data schema as follows:
]
specification [0039] The response schema (144) is a structured data schema. The structured data schema specifies a hierarchy of key value pairs. In the key value pairs, keys are the name that defines what the value represents.
Citations from McAuliffe:
McAuliffe [0209] The storage systems described above may be used to support applications that store data in a log-structured merge tree. A log-structured merge-tree (hereafter, ‘LSM tree’) is a data structure with performance characteristics that make it attractive for providing indexed access to files with high insert volume, such as transactional log data. LSM trees, like other search trees, maintain key-value pairs. LSM trees maintain data in one or more separate structures, each of which is optimized for its respective underlying storage medium; data is synchronized between the structures efficiently, in batches.
McAuliffe [0216] In other embodiments of the present disclosure, accessing LSM trees may also be improved. Readers will appreciate that if you have a LSM tree, conceptually you have a bunch of data stores that are stacked on top of each other. If you want to look for a data item within this stack of data stores, you have to work your way through this structure by searching each data store in the stack until the data item is found. Furthermore, if there is a cursor that the user (the application) has, it's actually backed by multiple cursors (one for each layer in the stack) and performing operations such as ‘next’ requires walking the entire stack until the appropriate entry is found. If the comparisons are expensive operations, it would be beneficial to organize the cursors into a tree to optimize searching. As such, in embodiments of the present disclosure, the cursors for each layer may be sorted into a tree structure for efficient searching. For example, in a LSM tree that has four layers, the cursors that point to each layer could be organized into a tree where each layer is a leaf of the tree. Through the use of a tree structure and caching of previous comparisons, operations such as ‘next’ can become far more efficient as only a portion of the tree must be scanned. For example, if a particular range of values is being sought, areas of the tree that are known to not include any portion of the range can be ignored. As such, embodiments described here enable us to go from a linear number of compares per next (N/2) to a logarithmic number of compares per next (log N) to a logarithmic number of compares per stop (log N). This may be particularly valuable where we see clustering and every next doesn't hit a new leaf.
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have modified Lancioni with the technique for storing data in a hierarchy of key value pairs of McAuliffe to include
comparing the first response to a structured data schema for the first response to validate the first response;
detecting, based on the comparison result, that the first response fails to comply with the structured data schema; and
One of ordinary skill in the art would have made this modification to improve the ability of the system to store undesirable terminology in a data structure that is efficiently accessible so that the system may be able to quickly determine whether a response from the LLM is valid. The system of the primary reference can be modified to utilize a hierarchy comprising a LSM tree to store the undesirable terminology data to facilitate the efficient validation of LLM responses. The LSM tree discloses structured data schema (specification para. 39 and McAuliffe at para. 209 LSM trees contain key value pairs). When the LSM tree is populated with the undesirable language and it is detected that the LLM response contains some of such language, that discloses the first response fails to comply with the structured data schema.
However, the combination of Lancioni and McAuliffe does not expressly disclose
generating a prompt injection signal indicating a prompt injection attack in the first LLM prompt responsive to the first response failing to comply with the structured data schema.
Clement discloses
generating a prompt injection signal indicating a prompt injection attack in the first LLM prompt responsive to the first response failing to comply with the LLM response requirements
[0062] A computer-implemented method is disclosed, comprising: providing instructions to a large language model (LLM) for answering a conversational query, wherein the instructions indicate a scope of a response to the conversational query, wherein the conversational query is associated with a single user session; receiving a first user prompt of the conversational query for the LLM to return a first response, wherein the first user prompt is associated with a user identifier and a session identifier of the single user session; constructing a secret for the user identifier and the session identifier; creating a LLM prompt comprising the secret and the first user prompt; receiving the first response from the large language model given the LLM prompt; detecting a prompt injection attack when the first response fails to include the secret in the first response; and returning an error message upon detection of the prompt injection attack.
[0007] The secret is a random sequence of natural language words. Natural language words require fewer tokens to represent the secret and which are easier for the model to repeat since the model is trained on sequences of natural language words.
[0008] In addition, the secret may be associated with a turn count which is a limit on how often a same secret is used in a conversation which may span several user prompts and responses within a single session. Alternatively, the turn count may limit how often a same secret is used for a particular user identifier. When the turn count limit is exceeded, a new secret is used in the conversation.
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have modified the combination of Lancioni and McAuliffe with the technique for generating a prompt injection signal indicating a prompt injection attack of Clement to include
generating a prompt injection signal indicating a prompt injection attack in the first LLM prompt responsive to the first response failing to comply with the structured data schema.
One of ordinary skill in the art would have made this modification to improve the ability of the system to detect and respond to a prompt injection attack. The system of the primary reference can be modified to generate the secret as taught by Clement and include the secret in the prompt, and upon detecting that the secret is not included in the LLM response, returning indication that there has been a prompt rejection attack. The structured data schema is disclosed by McAuliffe, and the structured data schema can store information such as the Clement secret and the undesirable terminology of McAuliffe. When the LLM response fails to include the secret, then that is failing to comply with the structured data schema which contains the secret and the system may then generate the prompt injection signal indicating the prompt injection attack.
As per claim 6, the rejection of claim 1 is incorporated herein.
Lancioni discloses generating a second LLM prompt [“direct the modification of prompts” [note the plural form that means there are multiple prompts being modified], para. 45; prompt that has been modified by the prompt modifier circuitry 175, para. 37] comprising a second user prompt from a second user device;[the claim does not require that the second user device be different from the first user device of claim 1, so the same user device can disclose second user device]
sending the second LLM prompt to the LLM;
[provides the modified prompt to the large language model circuitry 155, para. 37]
Lancioni [0059] The example large language model interface circuitry 150 provides the prompt to the LLM circuitry 155 for generation of a response message. (Block 230). In examples disclosed herein, a first model is executed by the LLM circuitry 155 to generate the response message. The response message is provided to the message validator circuitry 170 for validation.
receiving a second response to the second LLM prompt from the LLM;
[the large language model circuitry 155 provides a response to the prompt, para. 37]
comparing [review to determine if acceptable to send to user, para. 43] the second response to the filtering data [terms, phrases, sayings, offensive language, slang, that are inappropriate, para. 43] for the second response to obtain a comparison result; and
[response message is provided to the message validator circuitry 170 for validation, para 59]
[0043] In some examples, the guardrail platform 110 includes means for validating a message. For example, the means for validating may be implemented by message validator circuitry 170. The example message validator circuitry 170 of the illustrated example of FIG. 1 reviews the message returned to the large language model interface circuitry 150 to determine if the message is acceptable for sending to the user 102. In some examples, the message validator circuitry 170 augments the message by placing the message received from the large language model circuitry 155 into a prompt template that is designed to request the large language model circuitry 155 to determine whether any terms, phrases, sayings, etc. are included in the message that indicate that the message is inappropriate. Such phrases may include, for example, offensive language, slang or unprofessionally written language, an answer that does not relate to a topic at hand (e.g., the answer does not relate to cybersecurity, when it should), the answer contains harmful suggestions or advice, the answer judges a person, etc.
Lancioni [0046] if the message validator circuitry 170 determines that the message is not appropriate because the answer contains offensive language, the example prompt modifier circuitry 175 may modify the prompt to request that offensive language not be used. The modified prompt is then provided to the LLM circuitry 155 via the large language model interface circuitry 150 for generation of a subsequent response message.
sending, responsive to the second comparison result, the second response to the first user device.
[determine if the message is acceptable for sending to the user 102, para 43; para 63
]
[0063] The example message validator circuitry 170 evaluates the result(s) of the premises to determine whether the response message is acceptable. (Block 256). If the message is acceptable (e.g., block 256 returns a result of YES, indicating that none of the premises tested at block 240 returned a result indicating non-acceptability of the response message), the example message provider circuitry 180 provides the response message to the user. (Block 257).
However, Lancioni does not expressly disclose
comparing the second response to the structured data schema for the second response to obtain a comparison result;
detecting, based on the comparison result, that the second response fails to comply with the structured data schema; and
generating a prompt injection signal responsive to the second response failing to comply with the structured data schema.
McAuliffe discloses storing data in a hierarchy [LSM tree, para 209] of key value pairs [LSM trees maintain key value pairs, para. 209] and searching [look for item, para. 216] for values in the hierarchy
[0209] The storage systems described above may be used to support applications that store data in a log-structured merge tree. A log-structured merge-tree (hereafter, ‘LSM tree’) is a data structure with performance characteristics that make it attractive for providing indexed access to files with high insert volume, such as transactional log data. LSM trees, like other search trees, maintain key-value pairs. LSM trees maintain data in one or more separate structures, each of which is optimized for its respective underlying storage medium; data is synchronized between the structures efficiently, in batches.
McAuliffe [0216] In other embodiments of the present disclosure, accessing LSM trees may also be improved. Readers will appreciate that if you have a LSM tree, conceptually you have a bunch of data stores that are stacked on top of each other. If you want to look for a data item within this stack of data stores, you have to work your way through this structure by searching each data store in the stack until the data item is found. Furthermore, if there is a cursor that the user (the application) has, it's actually backed by multiple cursors (one for each layer in the stack) and performing operations such as ‘next’ requires walking the entire stack until the appropriate entry is found. If the comparisons are expensive operations, it would be beneficial to organize the cursors into a tree to optimize searching. As such, in embodiments of the present disclosure, the cursors for each layer may be sorted into a tree structure for efficient searching. For example, in a LSM tree that has four layers, the cursors that point to each layer could be organized into a tree where each layer is a leaf of the tree. Through the use of a tree structure and caching of previous comparisons, operations such as ‘next’ can become far more efficient as only a portion of the tree must be scanned. For example, if a particular range of values is being sought, areas of the tree that are known to not include any portion of the range can be ignored. As such, embodiments described here enable us to go from a linear number of compares per next (N/2) to a logarithmic number of compares per next (log N) to a logarithmic number of compares per stop (log N). This may be particularly valuable where we see clustering and every next doesn't hit a new leaf.
[see rejection of claim 1; see description of structured data schema in specification at para.39]
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have modified Lancioni with the technique for storing data in a hierarchy of key value pairs and searching for values in the hierarchy of McAuliffe to include
comparing the second response to the structured data schema for the second response to obtain a comparison result;
detecting, based on the comparison result, that the second response fails to comply with the structured data schema; and
generating a prompt injection signal responsive to the second response failing to comply with the structured data schema.
One of ordinary skill in the art would have made this modification to improve the ability of the system to store undesirable terminology in a data structure that is efficiently accessible so that the system may be able to quickly determine whether a response from the LLM is valid. The system of the primary reference can be modified to utilize a hierarchy of a LSM tree to store the undesirable terminology data to facilitate the efficient validation of LLM responses.
As per claim 10, the rejection of claim 1 is incorporated herein.
Lancioni discloses obtaining a first additional prompt segment from a first prompt data source; and [hardware logic circuitry, of the prompt modifier circuitry, that generates additional directives, para. 46-47]
generating the first LLM prompt comprising the first additional prompt segment and the first user prompt segment.
[example prompt modifier circuitry 175 inserts an additional directive to generate the modified prompt, para. 46]
[0046] In some examples, the guardrail platform 110 includes means for modifying a prompt. For example, the means for modifying may be implemented by prompt modifier circuitry 175. The example prompt modifier circuitry 175 of the illustrated example of FIG. 1 modifies a prompt at the direction of the message validator circuitry 170. An example of a modified prompt is described below in connection with FIG. 5. In general, the example prompt modifier circuitry 175 inserts an additional directive that causes the LLM circuitry 155, when processing the modified prompt to generate a response that is more likely to be appropriate. For example, if the message validator circuitry 170 determines that the message is not appropriate because the answer contains offensive language, the example prompt modifier circuitry 175 may modify the prompt to request that offensive language not be used. The modified prompt is then provided to the LLM circuitry 155 via the large language model interface circuitry 150 for generation of a subsequent response message.
Lancioni [0047] In some examples, the prompt modifier circuitry 175 may be instantiated by programmable circuitry such as the example programmable circuitry 612 of FIG. 6. For instance, the prompt modifier circuitry 175 may be instantiated by the example microprocessor 700 of FIG. 7 executing machine executable instructions such as those implemented by at least block 280 of FIG. 2A. In some examples, the prompt modifier circuitry 175 may be instantiated by hardware logic circuitry, which may be implemented by an ASIC, XPU, or the FPGA circuitry 800 of FIG. 8 configured and/or structured to perform operations corresponding to the machine readable instructions.
As per claim 11, the claim(s) is/are directed to a system with limitations which correspond to limitations of claim 1, and is/are rejected for the reasons detailed with respect to claim 1. Claim 11 also recites A system comprising:
at least one computer processor; and
a large language model (LLM) prompt manager executing on the at least one computer processor and configured to:
Lancioni discloses A system comprising: [ example systems]
at least one computer processor; and [processor, para. 30]
a large language model (LLM) prompt manager [guardrail platform 110, para 30] executing on the at least one computer processor and configured to:
[0124] From the foregoing, it will be appreciated that example systems, apparatus, articles of manufacture, and methods have been disclosed that enable large language models to be utilized in various contexts while providing guardrails for the responses that are provided the LLM.
[0030] FIG. 1 is a block diagram of an example guardrail platform constructed in accordance with teaching of this disclosure and shown in an example environment of use. The guardrail platform 110 of FIG. 1 may be instantiated (e.g., creating an instance of, bring into being for any length of time, materialize, implement, etc.) by programmable circuitry such as a Central Processor Unit (CPU) executing first instructions.
As per claim 16, the claim(s) is/are directed to a system with limitations which correspond to limitations of claim 6, and is/are rejected for the reasons detailed with respect to claim 6.
Claims 2, and 12 is/are rejected under 35 U.S.C. 103 as being unpatentable over Lancioni in view of McAuliffe, in view of Clement, further in view of Shah et al. U.S. Publication 20190268423 (hereinafter “Shah”).
As per claim 2, the rejection of claim 1 is incorporated herein.
However, the combination of Lancioni, McAuliffe, and Clement does not expressly disclose
wherein comparing the first response to the structured data schema comprises:
for each key value pair of a plurality of key value pairs in the structured data schema:
obtaining, from the first response, the key value pair comprising a first key and a corresponding value in the first response,
matching the first key to a second key in the structured data schema, and
determining whether the corresponding value in the first response matches
a set of value properties in the structured data schema.
Shah discloses receiving a key pair value [look for key/value pair 702 para. 132] for searching in a storage of key pairs [message with multiple key pairs, para. 131] and confirming that a value from the received key pair matches required properties[the proxy server confirms that the receiving value corresponds to properties of the key, e.g., corresponds to “tenant,” which is in turn a nested property of “channelData.”para. 132;]
Shah [0131] FIG. 7A, for example, shows a message sent from a Slack® chat application. A template for Slack® chat application messages may direct proxy server application 604 to look for key/value pair 700 (“team_id”: “TXXXXXXX”), where “team_id” is the key, and “TXXXXXXX” is the value and identifier.
Shah [0132] FIG. 7B shows a message sent from a MICROSOFT® Teams chat application. A template for MICROSOFT® Teams chat application messages may direct proxy server application 604 to look for key/value pair 702 (“id”: “72f988bf-86f1-41af-91ab-2d7cd011db47”), where “id” is the key, and “72P988bf-86f1-41af-91ab-2d7cd011db47” is the value and identifier.” In particular, because other “id” keys appear in the message, the template may indicate that the “id” key in key/value pair 702 that proxy server application 604 should look for is a property of “tenant,” which is in turn a nested property of “channelData.”
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have modified the combination of Lancioni, McAuliffe, and Clement with the technique for receiving a key pair value for searching in a storage of key pairs and confirming that a value from the received key pair matches required properties of Shah to include
wherein comparing the first response to the structured data schema comprises:
for each key value pair of a plurality of key value pairs in the structured data schema:
obtaining, from the first response, the key value pair comprising a first key and a corresponding value in the first response,
matching the first key to a second key in the structured data schema, and
determining whether the corresponding value in the first response matches
a set of value properties in the structured data schema.
One of ordinary skill in the art would have made this modification to improve the ability of the system to search for a key pair to validate the key pair. The system of the primary reference can be modified to receive a key pair value for searching in a storage of key pairs and confirm that a value from the received key pair matches required properties.
As per claim 12, the claim(s) is/are directed to a system with limitations which correspond to limitations of claim 2, and is/are rejected for the reasons detailed with respect to claim 2.
Claims 3-4 and 13-14 is/are rejected under 35 U.S.C. 103 as being unpatentable over Lancioni in view of McAuliffe, in view of Clement, further in view of Shah, further in view of Liu et al. U.S. Publication 20230128180 (hereinafter “Liu”). As per claim 3, the rejection of claim 2 is incorporated herein.
However, the combination of Lancioni, McAuliffe, Clement, and Shah does not expressly disclose wherein the set of value properties comprises a data type and a value format.
Liu discloses data type [attribute data, edges, or adjacent nodes, para 36 ] and value format [attribute data, edges, or adjacent nodes, each of these would have a corresponding format para 36; for example, node data would not include edge data] for properties associated with a key
[0036] With the key-value search and the key-value invert search format, the nodes and/or edges of a graph may be stored as key-value pairs so that key-value search may be performed efficiently on the graph. For example, for a given node identifier, the key-value search format may perform an efficient search for the node identifier’s attribute data, edges, or adjacent nodes. Moreover, an inverse search may also be performed in an efficient manner. For example for a given attribute data, an edge, or an adjacent node, the key-value inverse search format may perform an efficient search for the corresponding node identifier.
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have modified the combination of Lancioni, McAuliffe, Clement, and Shah with the teaching of data type and value format as properties of key value pairs of Liu to include wherein the set of value properties comprises a data type and a value format.
One of ordinary skill in the art would have made this modification to improve the ability of the system to specify a data type and value format for the keys. The system of the primary reference can be modified to include a data type and value format properties associated with the keys.
As per claim 4, the rejection of claim 2 is incorporated herein.
However, the combination of Lancioni, McAuliffe, Clement, and Shah does not expressly disclose wherein the set of value properties comprises a range of permitted values.
Liu discloses a key value may include a subset of available values [adjacent nodes, para. 36]. In other words, the value can only be adjacent nodes, which is a range of permitted values.
[0036] With the key-value search and the key-value invert search format, the nodes and/or edges of a graph may be stored as key-value pairs so that key-value search may be performed efficiently on the graph. For example, for a given node identifier, the key-value search format may perform an efficient search for the node identifier’s attribute data, edges, or adjacent nodes. Moreover, an inverse search may also be performed in an efficient manner. For example for a given attribute data, an edge, or an adjacent node, the key-value inverse search format may perform an efficient search for the corresponding node identifier.
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have modified the combination of Lancioni, McAuliffe, Clement, and Shah with the teaching of data type and value format as properties of key value pairs of Liu to include wherein the set of value properties comprises a range of permitted values.
One of ordinary skill in the art would have made this modification to improve the ability of the system to specify a range of allowed values associated with key value pairs. The system of the primary reference can be modified to so that the values associated with key value pairs can be specific subset of values.
As per claim 13, the claim(s) is/are directed to a system with limitations which correspond to limitations of claim 3, and is/are rejected for the reasons detailed with respect to claim 3.
As per claim 14, the claim(s) is/are directed to a system with limitations which correspond to limitations of claim 4, and is/are rejected for the reasons detailed with respect to claim 4.
Claims 5 and 15 is/are rejected under 35 U.S.C. 103 as being unpatentable over Lancioni in view of McAuliffe, in view of Clement, further in view of Coleman et al. U.S. Publication 20180210905 (hereinafter “Coleman”).
As per claim 5, the rejection of claim 1 is incorporated herein.
However, the combination of Lancioni, McAuliffe, and Clement does not expressly disclose wherein comparing the first response to the structured data schema comprises:
validating that a hierarchy of keys in the first response matches a hierarchy of keys in the structured data schema.
[The claim does not specify how many keys need to match and/or how many values need to match for the hierarchies to match
]
Coleman discloses determining that multiple keys in a first hierarchy of keys matches respective multiple keys from a second hierarchy of keys
[0022] In some embodiments, the generated Soundex keys for each data element may be paired with a value describing the data element's name used to generate the Soundex key. Each of the key and value pairs may be organized and inserted into a source data structure or target data structure, depending on whether the key and value pairs derive from a source element or target element. For example, the key and value pairs may be inserted into a hash table, hash map, tree map or an array such as an associative array. Subsequently, auto mapping between the source data structure and the target data structure may be performed by executing a lookup function on each key in the source table and comparing the key with the keys stored in the target data structure. In some embodiments, when a single matching key is found, the auto mapper may proceed to match the source data element to the target data element. However, in some instances, where multiple matching keys are identified during the lookup function, the auto mapper may rely on applying the Soundex algorithm to each of the parent elements in the hierarchy of the source and target data elements. Similar to the source and target data elements, each of the corresponding parent elements in the hierarchy may have the Soundex algorithm applied recursively to each of the parent elements. The Soundex keys for each parent elements in the respective source or target hierarchy may be compared in a manner similar to the source and target elements. Comparing parent elements of the source and target elements may increase the accuracy of the data mapping and ensure that the data elements having matching keys are in fact corresponding data elements.
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have modified the combination of Lancioni, McAuliffe, and Clement with the technique for determining that multiple keys in a first hierarchy of keys matches respective multiple keys from a second hierarchy of keys of Coleman to include wherein comparing the first response to the structured data schema comprises:
validating that a hierarchy of keys in the first response matches a hierarchy of keys in the structured data schema.
One of ordinary skill in the art would have made this modification to improve the ability of the system to determine that there is sufficient similarity between two hierarchies of keys, so that the system may determine that for example, data comprising key value pairs is sufficiently similar to another set of data. The system of the primary reference can be modified to determine that two hierarchies of key value pairs are sufficiently similar.
As per claim 15, the claim(s) is/are directed to a system with limitations which correspond to limitations of claim 5, and is/are rejected for the reasons detailed with respect to claim 5.
Claims 9 and 19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Lancioni in view of McAuliffe, in view of Clement, in view of Shah, further in view of Halder et al. U.S. Patent No. 12229174 (hereinafter “Halder”).
As per claim 9, the rejection of claim 2 is incorporated herein.
However, the combination of Lancioni, McAuliffe, Clement, and Shah does not expressly disclose obtaining a plurality of historical responses from the LLM;
extracting a plurality of keys from the plurality of historical responses; and
for each key of the plurality of keys:
determining a set of value properties based on a set of values related to the key in the plurality of historical responses; and
associating the set of value properties with the key at a position of the key in the structured data schema.
Halder discloses extracting queries from historical responses and associating metadata with the historical queries, which discloses limitations:
obtaining a plurality of historical responses from the LLM;[query responses are received from the LLM and stored in historical database, 9:30-41; 2:6-19; ]
extracting a plurality of keys [determining queries from historical responses, 8:12-23;” user query may be compared with the each of the historical queries in the historical database”; this may happen separately, earlier in a process, from the associating metadata process described in Halder 16:25-46] from the plurality of historical responses; and [historical database may include a plurality of historical query-outputs corresponding to each of a plurality of historical queries, 2:6-19]
for each key [a query, such as a primitive query, user query, or optimized query 16:25-46] of the plurality of keys:[ user query, primitive query and optimized query 16:25-46]
determining a set of value properties [determine metadata 16:25-46] based on a set of values [based on keywords of the queries 9:24-29; store the metadata of the user query 12:42-58; the Halder metadata is determined based on the keywords of the queries and used to associate the metadata with historical queries 16:25-46] related to the key in the plurality of historical responses; and
associating [associating the metadata with corresponding query in the historical database 16:25-46] the set of value properties with the key at a position [updating historical database that stores the user query/optimized query 16:25-46, 12:42-58] of the key in the structured data schema.
Halder 2:6-19 The method may further include optimizing, by the processor, the set of primitive queries to determine an optimized query based on the query type and the metadata using the NLP model and a historical database. In an embodiment, the historical database may include a plurality of historical query-outputs corresponding to each of a plurality of historical queries and corresponding metadata of the plurality of historical query-outputs and the plurality of historical queries. The method may further include determining, by the processor, the set of code corresponding to the user-defined coding language based on the query type and the optimized query by querying the LLM.
9:30-41 (45) historical database may include a plurality of historical query-outputs corresponding to each of a plurality of historical queries and corresponding metadata of the plurality of historical query-outputs and the plurality of historical queries. In an embodiment, the historical query-output may be output generated by the LLM in response to a corresponding historical query
Halder 12:42-58 (56) Further, the historical database updating module 226 may update the historical database 114 based on the set of code output by the LLM in case the rating may be above a predefined threshold such as, but not limited to, 4 and 5, or the updated set of code input by the user in case the set of code generated by LLM is rated 3 or below 3. Further, the historical database updating module 228 may update the historical database 114 to also store the metadata of the user query and the optimized query and the set of code generated by the LLM or the set of updated code input by the user.
Halder 8:12-23 (38) The historical database search module 204 may determine if the preprocessed user query matches with any of the historical queries saved as historical data in the historical database 114. In an embodiment, the preprocessed user query may be compared with the each of the historical queries in the historical database 114. Further, a matching percentage may be determined between the preprocessed user query and each of the historical queries. In case the matching percentage for a particular historical query is determined to be above predefined threshold such as, but not limited to, 95% above, then historical code generated by the LLM for that particular historical query may be output to the user.
Halder 16:25-46 (79) the historical database 404 may be updated based on the set of code output by the LLM 408 in case the rating is above a predefined threshold or the updated set of code is input by the user. In an embodiment, the metadata database 406 may also store the metadata of the user query and the optimized query and the set of code generated by the LLM 408 or the set of updated code input by the user. In an embodiment, the historical database 404 may store the user query and the optimized query and the set of code generated by the LLM 408 or the set of updated code input by the user. In an embodiment, the metadata stored in the metadata database 406 corresponding to the user query and the optimized query and the set of code generated by the LLM 408 or the set of updated code input by the user may be associated to the corresponding data in the historical database 404. Further, the LLM 408 may be tuned based on the rating and/or the updated set of code corresponding to the set of code output by the LLM 408 and the optimized query.
9:24-29 (44) Further, the metadata determination module 216 may determine metadata from the set of primitive queries. In an embodiment, the metadata may include, but is not limited to, a number of input keywords in the set of input keywords, a number of conditional keywords in the set of conditional keywords, etc.
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have modified the combination of Lancioni, McAuliffe, Clement, and Shah with the technique for extracting queries from historical responses and associating metadata with the historical queries of Halder to include
obtaining a plurality of historical responses from the LLM;
extracting a plurality of keys from the plurality of historical responses; and
for each key of the plurality of keys:
determining a set of value properties based on a set of values related to the key in the plurality of historical responses; and
associating the set of value properties with the key at a position of the key in the structured data schema.
One of ordinary skill in the art would have made this modification to improve the ability of the system to associate metadata with historical key values, to facilitate data processing functionality enabled by the association with the metadata. The system of the primary reference can be modified to determine and associate metadata with historical key values, to thereby discover patterns or facilitate new applications of the data based on data processing of historical key values.
As per claim 19, the claim(s) is/are directed to a system with limitations which correspond to limitations of claim 9, and is/are rejected for the reasons detailed with respect to claim 9.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to HOWARD H LOUIE whose telephone number is (571)272-0036. The examiner can normally be reached on Monday-Friday 9 AM-5 PM EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jung W. Kim can be reached on 571-272-3804. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/HOWARD H. LOUIE/Examiner, Art Unit 2494
/THEODORE C PARSONS/Primary Examiner, Art Unit 2494
1 Emphasis is additional throughout.
Prosecution Insights