Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
2. This communication is responsive to the amendment filed on 06/11/2026.
3. Claims 1-20 are currently pending in this Office action. This action is made Final.
Claim Rejections - 35 USC § 112
4. The 35 U.S.C. 112 rejection made in the prior Office action is withdrawn.
Claim Rejections - 35 USC § 101
5. The 35 U.S.C. 101 rejections made in the prior Office action are withdrawn.
Claim Rejections - 35 USC § 103
6. In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
7. The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
8. Claims 1-20 are rejected under 35 U.S.C. 103 as being unpatentable over
U.S. 2024/0265041 (hereinafter Rennie) in view of non-patent literature “Privacy-Aware Semantic Cache for Large Language Models”, arXiv:2403.02695v1 [cs.LG], 05 Mar 2024, 12 pages (hereinafter Gill).
Claims 1 and 7, Rennie discloses a method comprising:
receiving, at a device, a query for input to a language model ([0177]; “Another possible process to perform question modification (e.g., represented by the block 340) to produce further augmented questions is one based on a generative large language model (LLM), which may be implemented locally…”);
making, by the device and using the [dynamically variable similarity] threshold, a determination as to whether the query matches a cached query [for which the language model previously issued a response]; providing, by the device and based on the determination, the response associated with the cached query in lieu of inputting the query to the language model ([0147-0148]; “…Thus, in some embodiments, the query stack is configured to determine whether received query data matches (semantically, as may be determined by a machine learning answer cache), and to generate the output data based on one or more answer data records (possibly stored within the answer cache) in response to determining that the received query data matches one of the pre-determined questions. In some embodiments, the matching of query data to the past questions and associated answers stored in cache is performed by computing a score that is based on the combination of the questions and their answers, and ranking the computed scores to identify one or more likely matching candidates”).
Rennie does not explicitly disclose the features of selecting, by the device, a value for a dynamically variable similarity threshold based on information associated with the query, wherein selecting the value comprises selecting a lower semantic similarity threshold value to increase a cache hit rate when the information associated with the query indicates at least one of: (i) a latency for satisfying the query by the language model satisfies a latency threshold, (ii) a resource cost for satisfying the query by the language model satisfies a cost threshold, or (iii) network connectivity with the language model satisfies a network connectivity threshold; making, by the device and using the dynamically variable similarity threshold, a determination as to whether the query matches a cached query for which the language model previously issued a response. However, Gill discloses that “…This paper introduces MeanCache, a semantic cache for LLMs that identifies semantically similar queries to determine cache hit or miss. Using MeanCache, the response to a user’s semantically similar query can be retrieved from a local cache rather than re querying the LLM, thus reducing costs, service provider load, and environmental impact…” (pg 1, [Abstract]). In addition, Gill discloses that “…. An effective semantic search cache must be efficient, compact, and privacy-preserving. MeanCache provides a privacy-preserving caching system that can return the response to repeated queries directly from the local cache of the user, bypassing the need to query the LLMbasedwebservice…” (pg. 2, [Key Insights and Contributions of MeanCache]). Gill additionally discloses that “…. The system design of MeanCache is briefly described as follows: (1) MeanCache employs a small embedding model, which has a lower computational overhead compared to large Language Learning Model (LLM) based embedding models, such as Llama 2. (2) MeanCache uses federated learning for training to fine-tune the embedding model. This approach generates high-quality embeddings and improves the accuracy of embedding matching for retrieving similar queries. Notably, federated learning does not necessitate the storage of user queries in a central location, as it can train a high-quality model using the users’ distributed data. (3) To reduce storage and memory overhead, as well as to expedite the search time for finding similar queries in the cache, MeanCache compresses the embeddings using Principal Component Analysis (PCA). (4) MeanCache autonomously determines the optimal threshold for cosine similarity. It does this by varying the threshold and selecting the one that optimizes the F-score of the cache…” (pg. 4, [3.2 MeanCache’s System Design]). Moreover, Gill discloses that “In addition to privacy preserving training of the embedding model, identifying the optimal threshold (𝜏) for cosine similarity is equally important. The range of 𝜏 is between 0 and 1. This threshold (𝜏) dictates the level of similarity above which a cached query is considered relevant to the current user query. Setting the threshold too low could result in numerous false hits, leading to the retrieval of irrelevant queries from the cache. Conversely, a threshold set too high might cause many false miss, where relevant queries are not retrieved from the cache. Hence, finding the optimal threshold (𝜏) for cosine similarity is vital…” (pg. 4, [Finding the Optimal Threshold for Cosine Similarity]; pg. 7, [4.2 Cosine Similarity Threshold Impact on Semantic Matching]) and it would have been obvious for one with ordinary skill in the art to utilize the teachings of Gill in the system of Rennie in view of the desire to enhance the data process system by utilizing the autonomously determined threshold resulting in improving the efficiency of outputting the relevant result. Additionally, Rennie discloses one or more network interfaces; a processor coupled to the one or more network interfaces and configured to execute one or more processes; and a memory configured to store a process that is executable by the processor ([0041]; fig. 1).
Regarding claim 2, Rennie in view of Gill discloses the method further comprising: selecting the dynamically variable similarity threshold further based on a threshold parameter received from a user interface (Rennie: [0148]) and (Gill: pg. 4, [3.2 MeanCache’s System Design]). Therefore, the limitations of claim 2 are also rejected in the analysis of claim 1, and the claim is rejected on that basis.
Regarding claims 3 and 13, Rennie in view of Gill discloses the method wherein the information associated with the query indicates an application via which the query was generated (Rennie: [0177]).
Regarding claims 4 and 14, Rennie in view of Gill discloses disclose the method wherein the application generated the query automatically (Rennie: [0127]).
Regarding claims 5 and 15, Rennie in view of Gill discloses the method wherein making the determination as to whether the query matches the cached query comprises:
determining whether a semantic distance between the query and the cached query exceeds the dynamically variable similarity threshold (Rennie: [0178]), (Beller: [0034]) and (Gill: pg. 4, [Client Training]; pg. 7, [4.2 Cosine Similarity Threshold Impact on Semantic Matching]). Therefore, the limitations of claims 5 and 15 are rejected in the analysis of claims 1 or 7, and the claims are rejected on that basis.
Regarding claim 6, Rennie in view of Gill discloses the method wherein the language model is a large language model (LLM) that the device accesses via an application programming interface (API) (Rennie: [0276]).
Regarding claim 8, Rennie in view of Gill discloses the apparatus wherein the information associated with the query indicates a query type associated with the query (Rennie: [0039]).
Regarding claim 9, Rennie in view of Gill discloses the apparatus wherein the information associated with the query indicates a latency associated with sending the query to the language model to produce an output (Rennie: [0111]).
Regarding claim 10, Rennie in view of Gill discloses the apparatus wherein the information associated with the query indicates a level of performance associated with a computer network via which the apparatus accesses the language model (Rennie: [0109 and 0112]).
Regarding claim 11, Rennie in view of Gill discloses the apparatus wherein the information associated with the query indicates a threshold parameter received from a user interface (Rennie: [0177]).
Regarding claim 12, Rennie in view of Gill discloses the apparatus wherein the information associated with the query indicates a resource cost associated with sending the query to the language model to produce an output (Rennie: [0111 and 0177]) and (Gill: pgs. 1-2, [1 INTRODUCTION]). Therefore, the limitations of claim 12 are rejected in the analysis of claim 7, and the claim is rejected on that basis.
Regarding claim 16, Rennie discloses a method for improving response times to a query answering service, wherein the query answering service provides natural language responses to queries, the method comprising steps of:
storing the queries made to the query answering service and corresponding responses from the query answering service in a cache ([0148]; “In some embodiments, the interface 130 and/or the query processing module may be coupled toa query cache 135 and a question generation unit (which may be part of the cache 135 or of the query processing module 136, or may be a separate unit). The query cache 135 stores, among other things, answers/contents corresponding to frequently asked questions…”);
receiving a query ql at a device ([0177]; “Another possible process to perform question modification (e.g., represented by the block 340) to produce further augmented questions is one based on a generative large language model (LLM), which may be implemented locally…”);
determining a [value for a dynamically variable semantic similarity] threshold using information associated with the query q1, returning the response rl stored in the cache corresponding to the at least one query q2 ([0147-0148 and 0263]; “The query processing module 136…to the one or more candidate portions identified based on their coarse transformed vectors, at least one fine-detail transformed content record matching, according to a second criterion (e.g., some other closeness or similarity metric, or the same criterion applied…”; and “…Thus, in some embodiments, the query stack is configured to determine whether received query data matches (semantically, as may be determined by a machine learning answer cache), and to generate the output data based on one or more answer data records (possibly stored within the answer cache) in response to determining that the received query data matches one of the pre-determined questions…”).
Rennie does not explicitly disclose the features of determining a value for a dynamically variable semantic similarity threshold using information associated with the query q1, wherein determining the value comprises selecting a lower semantic similarity threshold value to increase a cache hit rate when the information associated with the query indicates at least one of: (i) a latency for satisfying the query by the language model satisfies a latency threshold, (ii) a resource cost for satisfying the query by the language model satisfies a cost threshold, or (iii) network connectivity with the language model satisfies a network connectivity threshold; wherein the dynamically variable similarity threshold is correlated with a level of semantic similarity between two natural language texts; determining a semantic similarity between the query q1 and at least one query q2 stored in the cache for which the query answering service previously issued a response r1; and In response to determining the at least one query q2 stored in the cache for which the semantic similarity between the query q1 and the at least one query q2 is greater than or equal to the dynamically variable semantic similarity threshold. However, Gill discloses that “…This paper introduces MeanCache, a semantic cache for LLMs that identifies semantically similar queries to determine cache hit or miss. Using MeanCache, the response to a user’s semantically similar query can be retrieved from a local cache rather than re querying the LLM, thus reducing costs, service provider load, and environmental impact…” (pg 1, [Abstract]). In addition, Gill discloses that “…. An effective semantic search cache must be efficient, compact, and privacy-preserving. MeanCache provides a privacy-preserving caching system that can return the response to repeated queries directly from the local cache of the user, bypassing the need to query the LLMbasedwebservice…” (pg. 2, [Key Insights and Contributions of MeanCache]). Gill additionally discloses that “…. The system design of MeanCache is briefly described as follows: (1) MeanCache employs a small embedding model, which has a lower computational overhead compared to large Language Learning Model (LLM) based embedding models, such as Llama 2. (2) MeanCache uses federated learning for training to fine-tune the embedding model. This approach generates high-quality embeddings and improves the accuracy of embedding matching for retrieving similar queries. Notably, federated learning does not necessitate the storage of user queries in a central location, as it can train a high-quality model using the users’ distributed data. (3) To reduce storage and memory overhead, as well as to expedite the search time for finding similar queries in the cache, MeanCache compresses the embeddings using Principal Component Analysis (PCA). (4) MeanCache autonomously determines the optimal threshold for cosine similarity. It does this by varying the threshold and selecting the one that optimizes the F-score of the cache…” (pg. 4, [3.2 MeanCache’s System Design]). Moreover, Gill discloses that “…For instance, some users may generate more repetitive queries compared to others, while certain users may not produce any repetitive queries at all. Interestingly, MeanCache’s multitask learning objective can benefit from learning even from a user with no repetitive queries. This is because MeanCache’s global embedding model (𝑊𝑡+1 𝑔𝑙𝑜𝑏𝑎𝑙 ) will learn to widen the distance between unique queries, thereby effectively learning true misses of the non-duplicate queries and consequently minimizing the false hits during the search process.” (pg. 4, [Client Training]). Furthermore, Gill discloses that “In addition to privacy preserving training of the embedding model, identifying the optimal threshold (𝜏) for cosine similarity is equally important. The range of 𝜏 is between 0 and 1. This threshold (𝜏) dictates the level of similarity above which a cached query is considered relevant to the current user query. Setting the threshold too low could result in numerous false hits, leading to the retrieval of irrelevant queries from the cache. Conversely, a threshold set too high might cause many false miss, where relevant queries are not retrieved from the cache. Hence, finding the optimal threshold (𝜏) for cosine similarity is vital…” (pg. 4, [Finding the Optimal Threshold for Cosine Similarity]; pg. 7, [4.2 Cosine Similarity Threshold Impact on Semantic Matching]). and it would have been obvious for one with ordinary skill in the art to utilize the teachings of Gill in the system of Rennie in view of the desire to enhance the data process system by utilizing the autonomously determined threshold resulting in improving the efficiency of outputting the relevant result.
Regarding claim 17, Rennie in view of Gill discloses the method further comprising:
in response to failing to determine the at least one query q2 stored in the cache for which the semantic similarity between the query q1 and the at least one query q2 is greater than or equal to the semantic similarity threshold, returning a response r2 obtained by sending the query q1 to the query answering service (Rennie: [0147]) and (Gill: pg. 4, [Client Training]; pg. 7, [4.2 Cosine Similarity Threshold Impact on Semantic Matching]). Therefore, the limitations of claim 17 are rejected in the analysis of claim 16, and the claim is rejected on that basis.
Regarding claim 18, Rennie in view of Gill discloses the method wherein the semantic similarity threshold is dynamically modified based on at least one of the user preference, the query type, the latency for receiving at least one response from the query answering service, the cost to make the query to the query answering service, and the level of network connectivity with the query answering service (Rennie: [0020, 0109, 0112 and 0157]) and (Gill: pg. 1, Abstract). Therefore, the limitations of claim 18 are rejected in the analysis of claim 16, and the claim is rejected on that basis.
Regarding claim 19, Rennie in view of Gill discloses the method wherein determining the semantic similarity between the query ql and the at least one query q2 comprises:
computing a vector corresponding to each of the query q1 and the at least one query q2 being compared; and determining the semantic similarity by comparing vectors (Rennie: [0143 and 0147]).
Regarding claim 20, Rennie in view of Gill discloses the method further comprising:
in response to determining that the at least one query q2 stored in the cache for which the semantic similarity between the query q1 and the at least one query q2 is greater than or equal to the semantic similarity threshold, returning the response rl stored in the cache, wherein the semantic similarity between the query q1 and the at least one query q2 stored in the cache corresponding to the response rl is a maximum value for all cached queries compared with the query ql (Rennie: [0147 and 0178]) and (Gill: pgs. 1-2, [1 INTRODUCTION]; pg. 2, [Key Insights and Contributions of MeanCache]). Therefore, the limitations of claim 20 are rejected in the analysis of claim 16, and the claim is rejected on that basis.
Response to Arguments
9. Applicant’s arguments have been considered but are deemed to be moot in view of new grounds of rejection presented in this Office action.
Conclusion
10. Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MONICA M PYO whose telephone number is (571)272-8192. The examiner can normally be reached Monday-Friday 8am-4pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, APU MOFIZ can be reached at 571-272-4080. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MONICA M PYO/ Primary Examiner, Art Unit 2161
Prosecution Insights