DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 4 recites the limitation "the normalized request". There is insufficient antecedent basis for this limitation in the claim.
Claim 11 recites the limitation "the normalized request". There is insufficient antecedent basis for this limitation in the claim.
Claim 18 recites the limitation "the normalized request". There is insufficient antecedent basis for this limitation in the claim.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Step 1
According to the first part of the analysis, in the instant case, claims 1-7 are directed to a apparatus, claims 8-14 are directed to a non-transitory computer-readable storage medium and claims 15-21 are directed to a method. Each of these claims fall within one of the four statutory categories (i.e., process, machine, manufacture, or composition of matter).
For claim 1,
Step 2A Prong One
detect a similar prior request based on the array of tokens;
(This step for detecting is a mental process)
Step 2A Prong Two
to access a request to execute a generative artificial intelligence model;
(This step for accessing data is insignificant extra-solution activity. See MPEP § 2106.05(g))
An apparatus comprising: interface circuitry to … computer readable instructions; and programmable circuitry to at least one of execute or instantiate the instructions to:
(This step for implementing on a generic computing device is mere-instructions to apply an exception. See MPEP § 2106.05(f))
replace a named entity within the request with a generic tag to generate a modified request; tokenize the modified request to create an array of tokens;
(This step for data pre-processing is insignificant extra-solution activity. See MPEP § 2106.05(g)) and cause output of a cached response to the request without execution of the generative artificial intelligence model.
(This step for outputting data is insignificant extra-solution activity. See MPEP § 2106.05(g))
Step 2B
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception because, when considered individually and in combination, they do not add significantly more (also known as an inventive concept) to the exception. The claim recites mental processes while the additional elements are insignificant extra-solution activity and mere-instructions to apply an exception recited at a high level of generality.
For claim 2,
Step 2A Prong One
(Claim 2 depends on claim 1, which has been determined to recite abstract ideas including mental processes. Therefore, claim 2 also recites an abstract idea.)
Step 2A Prong Two
after a failure to detect the similar prior request, trigger execution of the generative artificial intelligence model based on the request; and provide a generated response to the request.
(These steps for providing input to an AI model to generate output after detecting is insignificant extra-solution activity. See MPEP § 2106.05(g))
Step 2B
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception because, when considered individually and in combination, they do not add significantly more (also known as an inventive concept) to the exception. The claim recites mental processes while the additional elements are insignificant extra-solution activity recited at a high level of generality.
For claim 3,
Step 2A Prong One
(Claim 3 depends on claim 1, which has been determined to recite abstract ideas including mental processes. Therefore, claim 3 also recites an abstract idea.)
Step 2A Prong Two
wherein the programmable circuitry is to store the generated response in the cache.
(This step for storing data in a cache is insignificant extra-solution activity. See MPEP § 2106.05(g))
Step 2B
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception because, when considered individually and in combination, they do not add significantly more (also known as an inventive concept) to the exception. The claim recites mental processes while the additional elements are insignificant extra-solution activity recited at a high level of generality.
For claim 4,
Step 2A Prong One
(Claim 4 depends on claim 1, which has been determined to recite abstract ideas including mental processes. Therefore, claim 4 also recites an abstract idea.)
Step 2A Prong Two
wherein the programmable circuitry is to store the normalized request in association with the generated response in the cache.
(This step for storing data in a cache is insignificant extra-solution activity. See MPEP § 2106.05(g))
Step 2B
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception because, when considered individually and in combination, they do not add significantly more (also known as an inventive concept) to the exception. The claim recites mental processes while the additional elements are insignificant extra-solution activity recited at a high level of generality.
For claim 5,
Step 2A Prong One
(Claim 5 depends on claim 1, which has been determined to recite abstract ideas including mental processes. Therefore, claim 5 also recites an abstract idea.)
Step 2A Prong Two
wherein the programmable circuitry is to store the array of tokens in association with the generated response in the cache.
(This step for storing data in a cache is insignificant extra-solution activity. See MPEP § 2106.05(g))
Step 2B
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception because, when considered individually and in combination, they do not add significantly more (also known as an inventive concept) to the exception. The claim recites mental processes while the additional elements are insignificant extra-solution activity recited at a high level of generality.
For claim 6,
Step 2A Prong One
(Claim 6 depends on claim 1, which has been determined to recite abstract ideas including mental processes. Therefore, claim 6 also recites an abstract idea.)
Step 2A Prong Two
wherein the programmable circuitry is to normalize the modified request prior to tokenization of the modified request.
(This step for pre-processing data is insignificant extra-solution activity. See MPEP § 2106.05(g))
Step 2B
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception because, when considered individually and in combination, they do not add significantly more (also known as an inventive concept) to the exception. The claim recites mental processes while the additional elements are insignificant extra-solution activity recited at a high level of generality.
For claim 7,
Step 2A Prong One
(Claim 7 depends on claim 1, which has been determined to recite abstract ideas including mental processes. Therefore, claim 7 also recites an abstract idea.)
Step 2A Prong Two
wherein the programmable circuitry, to detect the similar request, is to: create a MinHash value based on the array of tokens; and query the cache using the MinHash value.
(This step for using a MinHash to perform the ‘detecting’ is generally linking the use of the judicial exception to a particular technological environment without improvement. See MPEP § 2106.05(e))
Step 2B
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception because, when considered individually and in combination, they do not add significantly more (also known as an inventive concept) to the exception. The claim recites mental processes while the additional elements do not impose a meaningful limit on the abstract idea, or provide an implementation that improves the technology.
For claim 8,
Step 2A Prong One
detect a similar prior request based on the array of tokens;
(This step for detecting is a mental process)
Step 2A Prong Two
At least one non-transitory computer-readable storage medium comprising instructions that cause one or more of at least one processor circuitry to at least:
(This step for implementing on a generic computing device is mere-instructions to apply an exception. See MPEP § 2106.05(f))
access a request for execution of a generative artificial intelligence model; replace a named entity within a request for generation of a response using a generative artificial intelligence model, the named entity to be replaced with a generic tag to modify the request; tokenize the modified request to create an array of tokens … and cause a cached response to the request to be provided without providing the request or the modified request to the generative artificial intelligence model.
(These steps for accessing data, executing a machine learning model, data pre-processing [modifying the request], and producing output [from the cache or the model] are insignificant extra-solution activity. See MPEP § 2106.05(g))
Step 2B
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception because, when considered individually and in combination, they do not add significantly more (also known as an inventive concept) to the exception. The claim recites mental processes while the additional elements are insignificant extra-solution activity and mere-instructions to apply an exception recited at a high level of generality.
For claims 9-14,
Claims 9-14 are non-transitory computer-readable storage medium claims that are substantially similar to apparatus claims 2-7, and are rejected using similar reasoning.
For claim 15,
Step 2A Prong One
detecting a similar prior request using the array of tokens;
(This step for detecting is a mental process)
Step 2A Prong Two
An method for use of cached artificial intelligence responses, the method comprising: accessing a request for execution of a generative artificial intelligence model; replacing a named entity within the request with a generic tag to form a modified request; tokenizing the modified request to create an array of tokens; … and after the detection of the similar prior request, providing a cached response to the request without execution of the generative artificial intelligence model in response to the request.
(These steps for accessing request data, executing a machine learning model, data pre-processing [modifying the request, tokenizing], and producing output [from the cache or the model] are insignificant extra-solution activity. See MPEP § 2106.05(g))
Step 2B
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception because, when considered individually and in combination, they do not add significantly more (also known as an inventive concept) to the exception. The claim recites mental processes while the additional elements are insignificant extra-solution activity recited at a high level of generality.
For claims 16-21
Claims 16-21 are method claims that are substantially similar to apparatus claims 2-7, and are rejected using similar reasoning.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-3, 7-10, 14-17, 21 is/are rejected under 35 U.S.C. 103 as being unpatentable over Sixing Lu et al. (hereinafter Lu) (US 12579974 B1, 2026-03-17), in view of Pranav Singh et al. (hereinafter Singh) (US 20240144921 A1, 2024-05-02) further in view of Andrei Gudkov (hereinafter Gudkov) (“Efficient implementation of MinHash, part 1”, 2019-06-24).
Regarding claim 1, Lu teaches;
An apparatus comprising: interface circuitry to access a request to execute a generative artificial intelligence model;
([col 45, ln 64-66] input to the system may be in form of text data 1013, for example as a result of a user typing an input into a user interface of user device … [col 60, ln 16-17] Each of these devices (110/120/1025) may include one or more controllers/processors … [col 4 ln 17-18] the system processes the user input and the context data using the LLM.)
NOTE: The user device, implemented using circuitry / processors, accesses text input from the user which is used to execute the large language model (LLM).
Thus, Lu teaches interface circuitry (the user device) to access a request (the text input accessed from the user, which is a prompt / request for the LLM) to execute a generative artificial intelligence model (the input is used for processing the LLM).
computer readable instructions; and programmable circuitry to at least one of execute or instantiate the instructions to:
([col 60, ln 35-42] Computer instructions for operating each device (110/120/1025) and its various components may be executed by the respective device's controller(s)/processor(s) (1304/1404), using the memory (1306/1406) as temporary “working” storage at runtime. A device's computer instructions may be stored in a non-transitory manner in non-volatile memory (1306/1406), storage (1308/1408), or an external device(s).)
NOTE: Lu teaches computer readable instructions, and programmable circuitry (processors) to at least one of execute or instantiate the instructions to perform each of the methods disclosed by Lu.
detect a similar prior request based on the
([col 17 ln 18-20] In some instances, the user input data 127 may correspond to a text or tokenized representation of a user input … [col 15 ln 5-8] keys for the cache may be generated using the user input data 127 … and the cache may store outputs that can be used to respond to similar/same user input data 127)
NOTE: Lu teaches text / tokenized input data, where the input data can be used to generate keys to detect similar user input from the cache.
Thus, Lu teaches detecting a similar prior request (user input prompting the LLM) based on the tokens.
and cause output of a cached response to the request without execution of the generative artificial intelligence model.
([col 8 ln 24-28] there may be a cache hit, that is the cache 145 includes the key 132, in which case, the data (e.g., the final LLM output 146 or the partial LLM output 148) associated with the key 132 is returned to the cache lookup component 140… [Abstract] For a cache hit, the stored output is used to respond to the user input.)
NOTE: Lu teaches a cache which stores past outputs of the LLM. Lu teaches detecting a similar request / input using the aforementioned key, and on a cache hit, returning the cached output associated with the key, rather than executing the LLM using the request.
Thus, Lu teaches causing output of a cached response to the request / input without execution of the generative artificial intelligence model (LLM).
Lu fails to explicitly teach but Singh teaches;
replace a named entity within the request with a generic tag to generate a modified request; tokenize the modified request ...
([0064] In text cases, training system 246 may further comprise performing named entity recognition on the text data and replacing the text data for tagged named entities with a named entity type tag. Performing named entity recognition and replacing surface token values (e.g., “cheese burger”) with their corresponding general named entity type tag (e.g., <DISHES>) can help map multiple different query data samples, e.g. collected from different restaurants, to a common or shared representation that may facilitate the clustering.)
NOTE: Singh teaches replacing a named entity within the request / query with a generic tag (replaces named entities with a generic ‘type tag’) to generate a modified request / query.
Singh further teaches that the modified query / request is represented using tokens, indicating that the modified request / query is tokenized.
OBVIOUSNESS TO COMBINE SINGH WITH LU: Singh and Lu are analogous art to the present disclosure because they both pertain to processing tokenized data using a generative AI model.
Lu already teaches an LLM cache which uses tokenized input data to detect similar prior requests from the cache, and returning associated cached LLM outputs. Additionally, Lu already teaches a method for detecting entities in the input / request data;
([col 29, ln 54-56] In other embodiments, the ER component may be configured to process the action data 647a-n to determine the one or more entities included in the user input)
Singh provides a method for replacing entities in a request (query) using generic tags.
Singh further states;
([0064] Performing named entity recognition and replacing surface token values (e.g., “cheese burger”) with their corresponding general named entity type tag (e.g., <DISHES>) can help map multiple different query data samples, e.g. collected from different restaurants, to a common or shared representation that may facilitate the clustering.)
NOTE: Singh teaches that replacing token values with generic / type tags reduces variation caused by entity-specific values, which improves matching of requests having the same structure or intent but different specific entity values.
Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to use the method of Singh to replace entities within the requests of Lu with generic / type tags, to decrease variance between requests with similar intent, thereby increasing cache hit rate and further reducing latency by avoiding unnecessary model execution.
Lu and Singh fail to teach but Gudkov teaches;
create an array of tokens
([pg. 3] First, we split text into an array of separate words. Next we are iterating over every sequence of K=3 adjacent words, compute and emit hash.)
[pg. 4]
PNG
media_image1.png
208
150
media_image1.png
Greyscale
NOTE: Gudkov teaches creating an array of tokens representing words of a text string.
OBVIOUSNESS TO COMBINE GUDKOV WITH LU AND SINGH:
Gudkov is analogous art to the present disclosure as it pertains to using a minhash function based on an array of tokens.
Lu provides the LLM cache framework, including a hashing model to generate a key to look-up similar inputs / requests in the cache.
Singh supplies the modified request / generic tag concept, replacing token values with generic type tags to improve clustering of similar data.
Gudkov supplies the array-of-tokens and MinHash mechanics;
([pg. 1] The high-level workflow is the following: for every text you have, split it into sequence of words, then create a set of every K-tuple of adjacent words (I will use K=3 below; every such triplet is called a shingle), then hash these shingles into 32-bit hashes, and finally select smallest unique N shingles. These N shingles become the fingerprint (MinHash) of the text. N is chosen typically from about 50 to a couple of hundreds depending on the median size of the text of the collection … Once fingerprints are generated for every text, we can compute similarity of any pair of texts by computing their Jaccard score of their fingerprints … The good thing about MinHash is that it is robust to minor modifications to the text.)
NOTE: Gudkov teaches that the aforementioned array of tokens is utilized in the MinHash workflow, and that MinHash is a good choice for determining similarity between texts because it is robust to minor modifications in the text.
Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to alter the system taught by Lu to tokenize the modified requests of Singh to create an array of tokens, and to use the array of tokens in the MinHash workflow of Gudkov when detecting similar requests, to improve the robustness to small changes between requests when querying the cache.
Regarding claim 2, Lu teaches;
The apparatus of claim 1, wherein the programmable circuitry is to: after a failure to detect the similar prior request, trigger execution of the generative artificial intelligence model based on the request; and provide a generated response to the request.
([col 4, ln 2-3] For a cache miss, the LLM may process the context data and the user input to generate an output)
NOTE: Lu teaches that the previously taught programmable circuitry is to; after a failure to detect the similar prior request (a cache miss), trigger execution of the generative artificial intelligence model (LLM) based on the request (user input prompt to the LLM); and provide a generated response (output) to the request (on a cache miss, the LLM is executed using the user input to generate an output).
Regarding claim 3, Lu teaches;
The apparatus of claim 2, wherein the programmable circuitry is to store the generated response in the cache.
([col 7 ln 54-55] The cache 145 may store data associated with unique keys, where the data represents an LLM output.)
NOTE: Lu teaches that the aforementioned programmable circuitry is to store the generated response (the LLM output) in the cache (the cache stores the LLM output).
Regarding claim 7, Lu teaches;
wherein the programmable circuitry, to detect the similar request, is to: … query the cache using the
([col 7, ln 19-21] The processed input may be provided to the signal hashing model 130, output of which may be used as a key for the cache 145… [col 8 ln 21-22] The cache lookup component 140 may receive results from the cache 145 for the key 132.)
NOTE: Lu teaches the aforementioned programmable circuitry, to detect a similar request (as previously taught), is to query the cache using a hash value (the key generated by the hashing model can be considered a hash value, and is used to query the cache).
Lu and Singh fail to teach but Gudkov teaches;
create a MinHash value based on the array of tokens;
(The high-level workflow is the following: for every text you have, split it into sequence of words, then create a set of every K-tuple of adjacent words (I will use K=3 below; every such triplet is called a shingle), then hash these shingles into 32-bit hashes, and finally select smallest unique N shingles. These N shingles become the fingerprint (MinHash) of the text... Once fingerprints are generated for every text, we can compute similarity of any pair of texts by computing their Jaccard score of their fingerprints)
NOTE: Gudkov teaches creating a MinHash value (fingerprint) based on the array of tokens (the aforementioned array of words).
OBVIOUSNESS TO COMBINE:
Using the same reasoning from claim 1, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to alter the system taught by Lu to tokenize the modified requests of Singh to create an array of tokens, and to use the array of tokens in the MinHash workflow of Gudkov when detecting similar requests, to improve the robustness to small changes between requests when querying the cache.
Regarding claim 8, Lu teaches;
At least one non-transitory computer-readable storage medium comprising instructions that cause one or more of at least one processor circuitry to at least:
([col 60, ln 35-42] Computer instructions for operating each device (110/120/1025) and its various components may be executed by the respective device's controller(s)/processor(s) (1304/1404), using the memory (1306/1406) as temporary “working” storage at runtime. A device's computer instructions may be stored in a non-transitory manner in non-volatile memory (1306/1406), storage (1308/1408), or an external device(s).)
NOTE: Lu teaches at least one non-transitory computer-readable storage medium comprising instructions that cause one or more of at least one processor circuitry to perform the methods of the disclosure.
access a request for execution of a generative artificial intelligence model;
([col 45, ln 64-66] input to the system may be in form of text data 1013, for example as a result of a user typing an input into a user interface of user device)
NOTE: The user device accesses text input from the user which is used to execute the large language model (LLM).
Thus, Lu teaches accessing a request (the text input accessed from the user, which is a prompt / request for the LLM) to execute a generative artificial intelligence model (the input is used for processing the LLM).
detect a similar prior request using the
([col 17 ln 18-20] In some instances, the user input data 127 may correspond to a text or tokenized representation of a user input … [col 15 ln 5-8] keys for the cache may be generated using the user input data 127 … and the cache may store outputs that can be used to respond to similar/same user input data 127)
NOTE: Lu teaches text / tokenized input data, where the input data can be used to generate keys to detect similar user input from the cache.
Thus, Lu teaches detecting a similar prior request (user input prompting the LLM) based on the tokens.
and cause a cached response to the request to be provided without providing the request or the modified request to the generative artificial intelligence model.
([col 8 ln 24-28] there may be a cache hit, that is the cache 145 includes the key 132, in which case, the data (e.g., the final LLM output 146 or the partial LLM output 148) associated with the key 132 is returned to the cache lookup component 140… [Abstract] For a cache hit, the stored output is used to respond to the user input.)
NOTE: Lu teaches causing a cached response to the request to be provided without providing the request or the modified request to the generative AI model (for a cache hit, output the LLM output / response stored in the cache in response to the input / request associated with the key, instead of executing the LLM)
Lu fails to explicitly teach but Singh teaches;
replace a named entity within a request for generation of a response
NOTE: Singh teaches replacing a named entity within the request / query with a generic tag (replaces named entities with a generic ‘type tag’) to generate a modified request / query, where the query is used downstream to generate a response.
Singh further teaches that the modified query / request is represented using tokens, indicating that the modified request / query is tokenized.
Lu already teaches using a generative AI model to generate a response for a request, and it would be obvious to modify the requests of Lu using the process of Singh, explained below.
OBVIOUSNESS:
Using the same reasoning from claim 1, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to use the method of Singh to replace entities within the requests of Lu with generic / type tags, to decrease variance between requests with similar intent, thereby increasing cache hit rate and further reducing latency by avoiding unnecessary model execution.
Lu and Singh fail to teach but Gudkov teaches;
create an array of tokens
([pg. 3] First, we split text into an array of separate words. Next we are iterating over every sequence of K=3 adjacent words, compute and emit hash.)
[pg. 4]
PNG
media_image1.png
208
150
media_image1.png
Greyscale
NOTE: Gudkov teaches creating an array of tokens representing words of a text string.
OBVIOUSNESS:
Using the same reasoning from claim 1, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to alter the system taught by Lu to tokenize the modified requests of Singh to create an array of tokens, and to use the array of tokens in the MinHash workflow of Gudkov when detecting similar requests, to improve the robustness to small changes between requests when querying the cache.
Regarding claim 9, Lu teaches;
The at least one non-transitory computer-readable storage medium of claim 8, wherein the instructions cause one or more of the at least one processor circuitry to: after a failure to detect the similar prior request, provide at least one of the request or the modified request to the generative artificial intelligence model for execution.
([col 4, ln 2-3] For a cache miss, the LLM may process the context data and the user input to generate an output)
NOTE: Lu teaches that the previously taught instructions cause one or more of the at least one processor circuitry to: after a failure to detect the similar prior request (a cache miss), provide at least one of the request (user input prompt to the LLM) or the modified request to the generative artificial intelligence model for execution (on a cache miss, the LLM is executed using the user input to generate an output).
Regarding claim 10,
Claim 10 is a non-transitory computer-readable storage medium claim that is substantially similar to apparatus claim 3, and is rejected using the same reasoning.
Regarding claim 14,
Claim 14 is a non-transitory computer-readable storage medium claim that is substantially similar to apparatus claim 7, and is rejected using the same reasoning.
Regarding claim 15, Lu teaches;
An method for use of cached artificial intelligence responses, the method comprising: accessing a request for execution of a generative artificial intelligence model;
([Abstract] Techniques for cache management for LLM processing are described… For a cache hit, the stored output is used to respond to the user input… [col 45, ln 64-66] input to the system may be in form of text data 1013, for example as a result of a user typing an input into a user interface of user device)
NOTE: Lu teaches a method for use of cached artificial intelligence responses, comprising accessing a request for execution of a generative artificial intelligence model (user input prompting execution of the LLM).
detecting a similar prior request using the
([col 17 ln 18-20] In some instances, the user input data 127 may correspond to a text or tokenized representation of a user input … [col 15 ln 5-8] keys for the cache may be generated using the user input data 127 … and the cache may store outputs that can be used to respond to similar/same user input data 127)
NOTE: Lu teaches text / tokenized input data, where the input data can be used to generate keys to detect similar user input from the cache.
Thus, Lu teaches detecting a similar prior request (user input prompting the LLM) based on the tokens.
and after the detection of the similar prior request, providing a cached response to the request without execution of the generative artificial intelligence model in response to the request.
([col 8 ln 24-28] there may be a cache hit, that is the cache 145 includes the key 132, in which case, the data (e.g., the final LLM output 146 or the partial LLM output 148) associated with the key 132 is returned to the cache lookup component 140… [Abstract] For a cache hit, the stored output is used to respond to the user input.)
NOTE: Lu teaches a cache which stores past outputs of the LLM. Lu teaches detecting a similar request / input using the aforementioned key, and on a cache hit, returning the cached output associated with the key, rather than executing the LLM in response to the request / user input.
Thus, Lu teaches after the detection of the similar prior request (cache lookup using the key), providing a cached response to the request (providing stored LLM output) without execution of the generative artificial intelligence model in response to the request.
Lu fails to explicitly teach but Singh teaches;
replacing a named entity within the request with a generic tag to form a modified request; tokenizing the modified request…
([0064] In text cases, training system 246 may further comprise performing named entity recognition on the text data and replacing the text data for tagged named entities with a named entity type tag. Performing named entity recognition and replacing surface token values (e.g., “cheese burger”) with their corresponding general named entity type tag (e.g., <DISHES>) can help map multiple different query data samples, e.g. collected from different restaurants, to a common or shared representation that may facilitate the clustering.)
NOTE: Singh teaches replacing a named entity within the request / query with a generic tag (replaces named entities with a generic ‘type tag’) to generate a modified request / query.
Singh further teaches that the modified query / request is represented using tokens, indicating that the modified request is tokenized.
OBVIOUSNESS TO COMBINE SINGH WITH LU:
Using the same reasoning from claim 1, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to use the method of Singh to replace entities within the requests of Lu with generic / type tags, to decrease variance between requests with similar intent, thereby increasing cache hit rate and further reducing latency by avoiding unnecessary model execution.
Lu and Singh fail to teach but Gudkov teaches;
create an array of tokens;
([pg. 3] First, we split text into an array of separate words. Next we are iterating over every sequence of K=3 adjacent words, compute and emit hash.)
[pg. 4]
PNG
media_image1.png
208
150
media_image1.png
Greyscale
NOTE: Gudkov teaches creating an array of tokens representing words of a text string.
OBVIOUSNESS TO COMBINE GUDKOV WITH LU AND SINGH:
Using the same reasoning from claim 1, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to alter the system taught by Lu to tokenize the modified requests of Singh to create an array of tokens, and to use the array of tokens in the MinHash workflow of Gudkov when detecting similar requests, to improve the robustness to small changes between requests when querying the cache.
Regarding claim 16, Lu teaches;
The method of claim 15, further including: after a failure to detect the similar prior request, providing the received request to the generative artificial intelligence model for execution; and providing a generated response to the received request.
([col 4, ln 2-3] For a cache miss, the LLM may process the context data and the user input to generate an output)
NOTE: Lu teaches after a failure to detect the similar prior request (a cache miss), providing the received request to the generative artificial intelligence model for execution (user input / request is processed / provided to the LLM), and providing a generated response to the received request (the LLM processes the input / request to generate a response / output).
Regarding claim 17, Lu teaches;
The method of claim 16, further including storing the generated response in the cache.
([col 7 ln 54-55] The cache 145 may store data associated with unique keys, where the data represents an LLM output.)
NOTE: Lu teaches storing the generated response (LLM output) in the cache.
Regarding claim 21, Lu teaches;
wherein the detection of the similar request includes: … querying the cache using the
([col 7, ln 19-21] The processed input may be provided to the signal hashing model 130, output of which may be used as a key for the cache 145… [col 8 ln 21-22] The cache lookup component 140 may receive results from the cache 145 for the key 132.)
NOTE: Lu teaches querying the cache using a hash value (the key generated by the hashing model can be considered a hash value, and is used to query the cache) to identify the similar prior request.
Lu and Singh fail to teach but Gudkov teaches;
creating a MinHash value based on the array of tokens;
(The high-level workflow is the following: for every text you have, split it into sequence of words, then create a set of every K-tuple of adjacent words (I will use K=3 below; every such triplet is called a shingle), then hash these shingles into 32-bit hashes, and finally select smallest unique N shingles. These N shingles become the fingerprint (MinHash) of the text... Once fingerprints are generated for every text, we can compute similarity of any pair of texts by computing their Jaccard score of their fingerprints)
NOTE: Gudkov teaches creating a MinHash value (fingerprint) based on the array of tokens (the aforementioned array of words).
OBVIOUSNESS TO COMBINE:
Using the same reasoning from claim 1, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to alter the system taught by Lu to tokenize the modified requests of Singh to create an array of tokens, and to use the array of tokens in the MinHash workflow of Gudkov when detecting similar requests, to improve the robustness to small changes between requests when querying the cache.
Claim(s) 4-6, 11-13, 18-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Lu (US 12579974 B1, 2026-03-17), in view of Singh (US 20240144921 A1, 2024-05-02) further in view of Gudkov (“Efficient implementation of MinHash, part 1”, 2019-06-24) as applied to claims 1, 3, 8, 10, 15, 17 above, and further in view of OpenNMT (“Tokenization – OpenNMT”, 2018-07-18).
Regarding claim 4, Lu teaches;
wherein the programmable circuitry is to store the
([col 17, ln 19-20] In some instances, the user input data 127 may correspond to a text or tokenized representation of a user input … keys for the cache may be generated using the user input data 127 … [col 6 ln 64 - col 7 ln 1] the cache 145 may store key that is a combination of the key 132 and the user input data 127 (e.g., the key may be “turn off my lamp+lam light on.” … [col 7 ln 54-55] The cache 145 may store data associated with unique keys, where the data represents an LLM output.)
NOTE: Lu teaches storing the keys and their associated generated responses (LLM outputs) in the cache, where the keys can include the input data, i.e. the request to the LLM.
Thus, Lu teaches the aforementioned programmable circuitry storing the request (the key comprising the input data / request) in association with the generated response (the LLM output) in the cache (the cache stores the key and the LLM output).
Lu, Singh, and Gudkov fail to teach but OpenNMT teaches;
normalized data
([pg. 1] normalization - which applies some uniform transformation on the source sequences to identify and protect some specific sequences (for instance url), normalize characters (for instance all types of quotes, unicode variants) or even to normalize some variants (like dates) into unique representation simpler for the translation process)
OBVIOUSNESS TO COMBINE OPENNMT WITH LU, SINGH, AND GUDKOV: OpenNMT is analogous art to the present disclosure as it pertains to normalizing and tokenizing data.
Lu provides the base response and generative model output caching framework, Singh provides a method for modifying requests to include generic tags to decrease variance between similar data, Gudkov provides a more efficient means of detecting similar data using token arrays and MinHash values, while OpenNMT provides a means to normalize data before subsequent processing.
OpenNMT states;
([pg. 1] The goal of the tokenization is to convert raw sentences into sequences of tokens. In that process two main operations are performed in sequence: normalization - which applies some uniform transformation on the source sequences to identify and protect some specific sequences (for instance url), normalize characters (for instance all types of quotes, unicode variants) or even to normalize some variants (like dates) into unique representation simpler for the translation process … the tokenization itself - )
NOTE: OpenNMT details that normalization standardizes different textual variants into a common representation before tokenization, making subsequent processing simpler and more consistent.
Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to use the methods of OpenNMT to normalize the responses of Lu before they are stored in the cache, to standardize the data into a common representation, to make subsequent processing simpler and more consistent.
Regarding claim 5, Lu teaches;
wherein the programmable circuitry is to store the
([col 17, ln 19-20] In some instances, the user input data 127 may correspond to a text or tokenized representation of a user input … keys for the cache may be generated using the user input data 127 … [col 6 ln 64 - col 7 ln 1] the cache 145 may store key that is a combination of the key 132 and the user input data 127 (e.g., the key may be “turn off my lamp+lam light on.” … [col 7 ln 54-55] The cache 145 may store data associated with unique keys, where the data represents an LLM output.)
NOTE: Lu teaches storing the keys and their associated generated responses (LLM outputs) in the cache, where the keys can include the input data, which can be represented using tokens.
Thus, Lu teaches the aforementioned programmable circuitry storing the tokens (the key comprising the input tokens) in association with the generated response in the cache.
Lu and Singh fail to teach but Gudkov teaches;
the array of tokens
([pg. 3] First, we split text into an array of separate words. Next we are iterating over every sequence of K=3 adjacent words, compute and emit hash.)
[pg. 4]
PNG
media_image1.png
208
150
media_image1.png
Greyscale
OBVIOUSNESS TO COMBINE:
Using the same reasoning from claim 1, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to alter the system taught by Lu to tokenize the modified requests of Singh to create an array of tokens, and to use the array of tokens in the MinHash workflow of Gudkov when detecting similar requests, to improve the robustness to small changes between requests when querying the cache (where the cache would store other generated arrays of tokens).
Regarding claim 6, Lu teaches;
wherein the programmable circuitry is to
([col 60 ln 35-43] Computer instructions for operating each device (110/120/1025) and its various components may be executed by the respective device's controller(s)/processor(s) (1304/1404), using the memory (1306/1406) as temporary “working” storage at runtime. A device's computer instructions may be stored in a non-transitory manner in non-volatile memory (1306/1406), storage (1308/1408), or an external device(s).)
NOTE: Lu teaches programmable circuitry to perform the methods of their disclosure.
Lu fails to teach but Singh teaches;
modified request
([0064] In text cases, training system 246 may further comprise performing named entity recognition on the text data and replacing the text data for tagged named entities with a named entity type tag.)
NOTE: Singh teaches replacing a named entity within the request / query with a generic tag (replaces named entities with a generic ‘type tag’) to generate a modified request / query.
OBVIOUSNESS:
Using the same reasoning from claim 1, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to use the method of Singh to replace entities within the requests of Lu with generic / type tags, to decrease variance between requests with similar intent, thereby increasing cache hit rate and further reducing latency by avoiding unnecessary model execution.
Lu, Singh, Gudkov fail to teach but OpenNMT teaches;
normalize the
([pg. 1] The goal of the tokenization is to convert raw sentences into sequences of tokens. In that process two main operations are performed in sequence: normalization … the tokenization itself)
NOTE: Gudkov teaches normalizing data prior to tokenization of the data. Reasoning as to why it would have been obvious to perform this process using the modified requests of Singh will be explained below.
OBVIOUSNESS:
Using the same reasoning from claim 4, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to use the methods of OpenNMT to normalize the modified responses of Lu, to standardize the data into a common representation, to make subsequent processing (such as tokenization) simpler and more consistent.
Regarding claim 11-13,
Claims 11-13 are non-transitory computer-readable storage medium claims that are substantially similar to apparatus claims 7, and are rejected using the same reasoning.
Regarding claim 18, Lu teaches;
further including storing the
([col 17, ln 19-20] In some instances, the user input data 127 may correspond to a text or tokenized representation of a user input … keys for the cache may be generated using the user input data 127 … [col 6 ln 64 - col 7 ln 1] the cache 145 may store key that is a combination of the key 132 and the user input data 127 (e.g., the key may be “turn off my lamp+lam light on.” … [col 7 ln 54-55] The cache 145 may store data associated with unique keys, where the data represents an LLM output.)
NOTE: Lu teaches storing the keys and their associated generated responses (LLM outputs) in the cache, where the keys can include the input data, i.e. the request to the LLM.
Thus, Lu teaches storing the request (the key comprising the input data / request) in association with the generated response (the LLM output) in the cache (the cache stores the key and the LLM output).
Lu, Singh, and Gudkov fail to teach but OpenNMT teaches;
normalized data
([pg. 1] normalization - which applies some uniform transformation on the source sequences to identify and protect some specific sequences (for instance url), normalize characters (for instance all types of quotes, unicode variants) or even to normalize some variants (like dates) into unique representation simpler for the translation process)
OBVIOUSNESS:
Using the same reasoning from claim 4, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to use the methods of OpenNMT to normalize the responses of Lu before they are stored in the cache, to standardize the data into a common representation, to make subsequent processing simpler and more consistent.
Regarding claim 19, Lu teaches;
further including storing the ([col 17, ln 19-20] In some instances, the user input data 127 may correspond to a text or tokenized representation of a user input … keys for the cache may be generated using the user input data 127 … [col 6 ln 64 - col 7 ln 1] the cache 145 may store key that is a combination of the key 132 and the user input data 127 (e.g., the key may be “turn off my lamp+lam light on.” … [col 7 ln 54-55] The cache 145 may store data associated with unique keys, where the data represents an LLM output.)
NOTE: Lu teaches storing the keys and their associated generated responses (LLM outputs) in the cache, where the keys can include the input data, which can be represented using tokens.
Thus, Lu teaches storing the tokens (the key comprising the input tokens) in association with the generated response in the cache.
Lu and Singh fail to teach but Gudkov teaches;
the array of tokens
([pg. 3] First, we split text into an array of separate words. Next we are iterating over every sequence of K=3 adjacent words, compute and emit hash.)
[pg. 4]
PNG
media_image1.png
208
150
media_image1.png
Greyscale
OBVIOUSNESS TO COMBINE:
Using the same reasoning from claim 1, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to alter the system taught by Lu to tokenize the modified requests of Singh to create an array of tokens, and to use the array of tokens in the MinHash workflow of Gudkov when detecting similar requests, to improve the robustness to small changes between requests when querying the cache (where the cache would store other generated arrays of tokens).
Regarding claim 20, Lu fails to teach but Singh teaches;
modified request
([0064] In text cases, training system 246 may further comprise performing named entity recognition on the text data and replacing the text data for tagged named entities with a named entity type tag.)
NOTE: Singh teaches replacing a named entity within the request / query with a generic tag (replaces named entities with a generic ‘type tag’) to generate a modified request / query.
OBVIOUSNESS:
Using the same reasoning from claim 1, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to use the method of Singh to replace entities within the requests of Lu with generic / type tags, to decrease variance between requests with similar intent, thereby increasing cache hit rate and further reducing latency by avoiding unnecessary model execution.
Lu, Singh, Gudkov fail to teach but OpenNMT teaches;
normalizing the
([pg. 1] The goal of the tokenization is to convert raw sentences into sequences of tokens. In that process two main operations are performed in sequence: normalization … the tokenization itself)
NOTE: Gudkov teaches normalizing data prior to tokenization of the data. Reasoning as to why it would have been obvious to perform this process using the modified requests of Singh will be explained below.
OBVIOUSNESS:
Using the same reasoning from claim 4, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to use the methods of OpenNMT to normalize the modified responses of Lu, to standardize the data into a common representation, to make subsequent processing (such as tokenization) simpler and more consistent.
CONCLUSION
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Matthew Alan Cady whose telephone number is (571) 272-7229. The examiner can normally be reached Monday - Friday, 7:30 am - 5:00 pm ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Cesar Paula can be reached on (571)272-4128. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC)
at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MATTHEW ALAN CADY/ Examiner, Art Unit 2145
/CESAR B PAULA/ Supervisory Patent Examiner, Art Unit 2145
Prosecution Insights