Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
Introduction
This office action is in response to Applicant’s submission filed on 12/19/2024. As such, claims 1-20 have been examined.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 3, 8-10, 12, and 17-18 are rejected under 35 U.S.C. 103 as being unpatentable over Sivulka (US 20250173365) in view of He (US 20200272845).
Regarding claim 1, Sivulka discloses: 1. A method for executing a large matching task by a language model, the large matching task comprising a request to the language model to match a first dataset to a second dataset in which the request exceeds a maximum token constraint of the language model, ([0079] the data analytics system may generate a single LLM prompt based on multiple sources and parse the response to that single prompt for each of the cells corresponding to each of the sources. The data analytics system may identify sources that are sufficiently similar to each other to be analyzed together in a single prompt. For example, the data analytics system may compare metadata for each source or embeddings generated based on each source to identify similar sources and thereby identify sources to include in the same prompt to the LLM. In some embodiments, the data analytics system computes similarity scores for sources by comparing embeddings for the sources and selects sources to include together in a prompt if the similarity scores exceed a threshold. The data analytics system may also include some rules or restrictions on which sources can be combined. For example, the data analytics system may limit the combined length of the sources to ensure that all sources can fit within a context window for an LLM.) [The reference addresses the scenario where the request exceeds the maximum token constraint.]
the method comprising: receiving the large matching task for the language model, executing a ([0079] The data analytics system may identify sources that are sufficiently similar to each other to be analyzed together in a single prompt. For example, the data analytics system may compare metadata for each source or embeddings generated based on each source to identify similar sources and thereby identify sources to include in the same prompt to the LLM. In some embodiments, the data analytics system computes similarity scores for sources by comparing embeddings for the sources and selects sources to include together in a prompt if the similarity scores exceed a threshold.)
selecting a plurality of candidate matches between the first entry and a subset of the plurality of second entries, wherein the matches have selected matching scores among the plurality of matching scores, and wherein the selected matching scores exceed a threshold value; ([0079] The data analytics system may identify sources that are sufficiently similar to each other to be analyzed together in a single prompt. For example, the data analytics system may compare metadata for each source or embeddings generated based on each source to identify similar sources and thereby identify sources to include in the same prompt to the LLM. In some embodiments, the data analytics system computes similarity scores for sources by comparing embeddings for the sources and selects sources to include together in a prompt if the similarity scores exceed a threshold.)
generating a prompt for the language model to identify a matching dataset set from among the plurality of candidate matches; ([0079] The data analytics system may identify sources that are sufficiently similar to each other to be analyzed together in a single prompt. For example, the data analytics system may compare metadata for each source or embeddings generated based on each source to identify similar sources and thereby identify sources to include in the same prompt to the LLM.)
executing the language model with the prompt to output the matching dataset; and returning the matching dataset. ([0079] The data analytics system may identify sources that are sufficiently similar to each other to be analyzed together in a single prompt. For example, the data analytics system may compare metadata for each source or embeddings generated based on each source to identify similar sources and thereby identify sources to include in the same prompt to the LLM. In some embodiments, the data analytics system computes similarity scores for sources by comparing embeddings for the sources and selects sources to include together in a prompt if the similarity scores exceed a threshold.)
Sivulka does not appear to discloses a separate matching model to output probability of a match between dataset entries.
He in the related art discloses: a matching model on the first dataset and the second dataset to generate a plurality of matching scores, wherein each of the plurality of matching scores represents a probability of match between a first entry in the first dataset and one of a plurality of second entries in the second dataset; ([0035] The table-level entity matcher 145 may evaluate the set of weighted scores using a table-level entity matching model to identify a common entity included in the first input and the second input. In an example, a probability of a match between an entity in the first input and an entity in the second input may be calculated using the set of weighted scores and the common entity may be identified based on the probability of a match being outside a threshold. For example, the weighted scores may be used to generate an aggregate score for entity pairs in the first input and the second input. The aggregate score may be compared to a threshold value (e.g., a probability determined to be high enough to indicate the entities are a match, etc.) to determine if the entity pairs should be linked (e.g., identified as the same real-world entity, etc.). For example, based on the attribute-level weights and scores for John H. Smith in the first input and Jonathan H. Smith in the second input, the two entities may be linked as referring to the same real entity (e.g., the human Jonathan Hector Smith).)
Sivulka and He are considered analogous art. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Sivulka to combine the teaching of He, because identifying and combining data eliminates redundant work and helps with resource constraint mitigation (He, [0035]).
Regarding Claim 3, Sivulka and He disclose all the element of claim 1,
He further discloses: wherein the matching model comprises a supervised machine learning model, and wherein the method further comprises training the supervised machine learning model on training data comprising a first sample dataset, a second sample dataset, and a plurality of known matches between first entries in the first sample dataset and second entries in the second sample dataset. ([0041] The attribute-level entity matching engine 220 may take as input two entity values (e.g., “Dave M. Smith” and “David Smith” in Table 1, etc.) and may produce a score indicating a likelihood of a match between the two entity values. The attribute-level entity matching engine 220 may use two types of attribute-level EM models that may be pre-trained offline: (1) Type-specific models: For each known KB type T (e.g. person, etc.), a separate model may be pre-trained to predict match/non-match for values in T. For example, the pre-trained type-specific model 225 may be pre-trained to predict matches for a name attribute type. Synonymous entity names of type T in the KB (e.g., “Bill Gates” is also known as “William Gates”, “William Henry Gates” and “William H. Gates”, etc.) may be used as training data and to develop hierarchical deep models to learn name variations specific to each type T for accurate match/non-match decisions. (2) Unified model: This may be a single model (or multiple models) that predict match/non-match for attribute values for attribute types not in known in the KB. While the model architecture may be the same as the type-specific models, synonymous entity names taken from the union of many KB types may be used to pre-train a unified attribute-level EM (e.g., pre-trained unified model 230) that captures common name variations across different types (e.g., spelling variations, etc.). The pre-trained unified model 230 may be reasonably accurate and may be fine-tuned using limited training data to quickly fit a new type not known a priori.) [explicitly mentions using known KB type T and synonymous entity names of type T in the KB as training data to learn name variations. The reference uses synonymous entity names from within a Knowledge Base (KB) to train the model, treating variations of names (e.g., "Bill Gates" and "William Gates") as positive match training data. The reference also explicitly mentions using hierarchical deep models and states that synonymous entity names... may be used as training data – this reads on supervised training.]
Where the rationale for the combination would be similar to the one already provided.
Regarding Claim 8, Sivulka and He disclose all the element of claim 1,
He further discloses: wherein returning the matching dataset comprises passing the matching dataset to a processing algorithm programmed to use the matching dataset to output a result. ([0035] The table-level entity matcher 145 may evaluate the set of weighted scores using a table-level entity matching model to identify a common entity included in the first input and the second input. In an example, a probability of a match between an entity in the first input and an entity in the second input may be calculated using the set of weighted scores and the common entity may be identified based on the probability of a match being outside a threshold. For example, the weighted scores may be used to generate an aggregate score for entity pairs in the first input and the second input. The aggregate score may be compared to a threshold value (e.g., a probability determined to be high enough to indicate the entities are a match, etc.) to determine if the entity pairs should be linked (e.g., identified as the same real-world entity, etc.). For example, based on the attribute-level weights and scores for John H. Smith in the first input and Jonathan H. Smith in the second input, the two entities may be linked as referring to the same real entity (e.g., the human Jonathan Hector Smith).)
Where the rationale for the combination would be similar to the one already provided.
Regarding Claim 9, Sivulka and He disclose all the element of claim 1,
Sivulka further discloses: wherein returning the matching dataset comprises storing the matching dataset in a data repository. ([0080] In some embodiments, the data analytics system caches cell output values and uses the cached values when the same column prompt is applied to the same source. For example, the data analytics system may store cell output values in association with the column prompt and the source. In some embodiments, the data analytics system hashes the column prompt and the source (or source identification information) to associate the cell output value with the prompt and source in a database. Before prompting an LLM using a column prompt and a source, the data analytics system may check whether the column prompt has already been applied to that source. If so, the data analytics system may use the cached cell value for the grid-based data structure.)
Regarding Claim 10, Sivulka discloses: 10. A system comprising: a computer processor; ([0132] processor) a data repository in communication with the computer processor and storing ([0006] database that stores):
As for rest of the claim, they recite elements that corresponds to the method of claim 1, therefore rejection applied to claim 1 is also applicable.
Claim 12 recites limitations similar to the limitations of Claim 3 and is rejected under similar rationale.
Claim 17 recites limitations similar to the limitations of Claim 8 and is rejected under similar rationale.
Claim 18 recites limitations similar to the limitations of Claim 9 and is rejected under similar rationale.
Claims 2 and 11 are rejected under 35 U.S.C. 103 as being unpatentable over Sivulka (US 20250173365) in view of He (US 20200272845), and further in view of Ramarao (US 20250173556).
Regarding Claim 2, Sivulka and He disclose all the element of claim 1,
Sivulka and He does not disclose repeating, for each additional entry in the first dataset, executing the matching model, selecting, generating, executing the language model, and returning, wherein repeating generates a plurality of matching datasets including the matching dataset; and returning the plurality of matching datasets.
Ramarao in the related art discloses: further comprising: repeating, for each additional entry in the first dataset, executing the matching model, selecting, generating, executing the language model, and returning, wherein repeating generates a plurality of matching datasets including the matching dataset; and returning the plurality of matching datasets. ([0123] Similarly, the maximal matching engine 315 provides the sentence included in the fifth output segment (“′Qualifications: Ph.D. in a scientific discipline, such as biology, chemistry, or pharmacology, with a focus on clinical research At least 5 years of experience designing and conducting clinical research studies Strong data analysis and statistical skills Ability to work independently and as part of a team”) to the GPT LLM 312 with a prompt to generate a new sentence based on the previous sentence, with an increased relevance to the content “clinical research scientist.” The text content generation platform 310 may iteratively (a) generate job description segments, (b) encode the segments as vectors, (c) perform maximal matching based on relevancy scores, and (d) provide sentences to the GPT LLM 312 for re-writing until the relevancy score for each output segment among the input segment/output segment pairs corresponding to the maximal matching sets of pairs meets the threshold of 0.5.)
Sivulka/He/Ramarao are considered analogous art. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Sivulka/He to combine the teaching of Ramarao, because the method provides quality control and enhance content accuracy (Ramarao, [0123]).
Claim 11 recites limitations similar to the limitations of Claim 2 and is rejected under similar rationale.
Claims 4 and 13 are rejected under 35 U.S.C. 103 as being unpatentable over Sivulka (US 20250173365) in view of He (US 20200272845), and further in view of Wagner (US 20240119284).
Regarding Claim 4, Sivulka and He disclose all the element of claim 3,
Sivulka and He does not disclose wherein the training data further comprises differences in time between the first dataset and the second dataset.
Wagner in the related art discloses: wherein the training data further comprises differences in time between the first dataset and the second dataset. ([0005] According to various embodiments of the present invention, a method for training a machine learning model is provided, comprising: determining a plurality of training sequences of training-input data elements, wherein for each training sequence each training-input data element contains sensor data for a time point from a time period assigned to the training sequence in which a prespecified event takes place at least once at one or more respective event time points; determining, for each training-input data element, the temporal distance between the time point for which the training-input data element contains sensor data and one of the one or more respective event time points; and training the machine learning model on the basis of the determined temporal distances.)
Sivulka/He/Wagner are considered analogous art. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Sivulka/He to combine the teaching of Wagner, because performance (e.g. prediction accuracy) of the trained machine learning model and the data efficiency of training can be increased (Wagner, [0006]).
Claim 13 recites limitations similar to the limitations of Claim 4 and is rejected under similar rationale.
Claims 5 and 14 are rejected under 35 U.S.C. 103 as being unpatentable over Sivulka (US 20250173365) in view of He (US 20200272845), and further in view of Potharaju (US 20250124316).
Regarding Claim 5, Sivulka and He disclose all the element of claim 3,
Sivulka and He does not disclose wherein the training data further comprises wherein: the matching model comprises a slim matching model, the slim matching model comprises a matching accuracy less than a predetermined matching accuracy specified for the large matching task, and the language model comprises at least the predetermined matching accuracy.
Potharaju in the related art discloses: wherein: the matching model comprises a slim matching model, the slim matching model comprises a matching accuracy less than a predetermined matching accuracy specified for the large matching task, and the language model comprises at least the predetermined matching accuracy. ([0005] Notably, the first LM can be a smaller language model (SLM), whereas the second LM can be a larger language model (LLM). The SLM can be considered “smaller” relative to the LLM in that it includes fewer parameters and/or is more computationally efficient. Accordingly, the system can generate the initial response using the SLM in an attempt to generate a suitable response and without having to invoke the LLM. However, if the initial response generated using the SLM is determined to be insufficient (e.g., as indicated by the at least one verifier and the meta-verifier), the system can then invoke the LLM without any user intervention from the user of the client device or other users. Accordingly, the present disclosure automatically mixed usage of at least the SLM and the LLM, in processing the NL based input, to balance the latency and computational resource consumption benefits of the SLM with the robustness and accuracy benefits of the LLM.)
Sivulka/He/Potharaju are considered analogous art. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Sivulka/He to combine the teaching of Potharaju, because there is a need in the art to automatically mix usage of smaller generative models and larger generative models, in processing given NL content and/or other input(s), to balance the latency and computational resource consumption benefits of these smaller generative models with the robustness and accuracy benefits of these larger generative models (Potharaju, [0003]).
Claim 14 recites limitations similar to the limitations of Claim 5 and is rejected under similar rationale.
Claims 6 and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Sivulka (US 20250173365) in view of He (US 20200272845), and further in view of Zhang (US 20230044508).
Regarding Claim 6, Sivulka and He disclose all the element of claim 3,
Sivulka and He does not disclose wherein the matching model comprises a gradient boosting machine classifier.
Zhang in the related art discloses: wherein the matching model comprises a gradient boosting machine classifier. ([0093] The parameter of the matching model, for example, parameters for the gradient boosted decision tree classifier and the logic regression model, may be adjusted based on a difference between the predicted classification result and the actual classification result using any appropriate loss function and model training method.)
Sivulka/He/Zhang are considered analogous art. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Sivulka/He to combine the teaching of Zhang, because Gradient Boosted Classifier can capture complex, nonlinear relationship in data (Zhang, [0093]).
Claim 15 recites limitations similar to the limitations of Claim 6 and is rejected under similar rationale.
Claims 7 and 16 are rejected under 35 U.S.C. 103 as being unpatentable over Sivulka (US 20250173365) in view of He (US 20200272845), and further in view of Dang (US 20260050583).
Regarding Claim 7, Sivulka and He disclose all the element of claim 1,
Sivulka and He does not disclose wherein generating the prompt comprises: retrieving a prompt template comprising prompt instructions to match a first data subset and a second data subset, adding the first entry to the prompt as the first data subset, and adding the plurality of second entries to the prompt as the second data subset.
Dang in the related art discloses: wherein generating the prompt comprises: retrieving a prompt template comprising prompt instructions to match a first data subset and a second data subset, adding the first entry to the prompt as the first data subset, and adding the plurality of second entries to the prompt as the second data subset. ([0113] Then, the method proceeds to step 730 where a second prompt is constructed (e.g., by the prompt assembly proxy 536 or 136), e.g., by inserting the filtered sample data of the source table and the matching target tables into a second prompt template. Optionally, additional task context 702 including domain-specific information about the source and target databases can be provided by the user and inserted into the second prompt template.) [The reference disclose inserting the filtered sample data of the source table and the matching target tables into a second prompt template. The filtered sample data read on first data subset, and the matching target tables read on second entries.] Also see para 0098.
Sivulka/He/Dang are considered analogous art. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Sivulka/He to combine the teaching of Dang, because the technologies described herein address the challenges of schema matching by using a two-stage schema matching approach, leveraging the power of generative artificial intelligence (AI). As described more fully below, this approach provides a comprehensive and adaptive solution that combines the strengths of preliminary filtering and advanced data modeling, thereby achieving accurate, efficient, and scalable schema matching for diverse database applications (Dang, [0022]).
Claim 16 recites limitations similar to the limitations of Claim 7 and is rejected under similar rationale.
Allowable Subject Matter
Claims 19-20 contains allowable subject matter, the reason are as follows.
Regarding claim 19, the claim discloses features contains in claim 1 plus features from combination of claims 2, 4, 5, 6, and 7. Although the claim can be rejected using the combination of Sivulka (US 20250173365), He (US 20200272845), Ramarao (US 20250173556), Wagner (US 20240119284), Potharaju (US 20250124316), Zhang (US 20230044508), and Dang (US 20260050583). However, the number of references involved and the combination references involved does not appear to be reasonable combination to disclose the claimed as currently recited. Furthermore, it would not have been obvious to one of ordinary skill in the art to modify the prior art in order to arrive at the claimed invention. Therefore claim 19 is allowed. Dependent claim 20 contains similar allowable subject and also further limits claim 19, therefore it is also allowed.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure: Yadav (US 20260161643) – discloses natural language to SQL query system, it finds the most relevant templates/examples in the database that matches the meaning of the query using similarity scores. See Abstract, para 0157, 0212, 0240, 0467, 0469, 0475, 0477, 0487, 0494, 0504, and 0511, and figs. 2 and 6 for additional details.
Mei (US 20230359651)- discloses providing category label to datasets based on matching. See para 0056 for additional details.
Bergström, V. (2024). Breaking the Token Barrier: Leveraging Retrieval Methods to Facilitate Text-to-SQL on Extensive Tabular Data. – discloses text to SQL system to query massive database by retrieval only specific, relevant columns rather than sending the entire schema. See Abstract for details.
Li, H., Li, S., Hao, F., Zhang, C. J., Song, Y., & Chen, L. (2024, May). Booster: leveraging large language models for enhancing entity resolution. In Companion Proceedings of the ACM Web Conference 2024 (pp. 1043-1046). -discloses creating set of possible matches, probability distribution in booster framework as way to reduce token input limitation. See Abstract and sections 2 and 3, and fig. 2 for additional details.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Philip H Lam whose telephone number is (571)272-1721. The examiner can normally be reached 9 AM-2 PM Pacific time.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Bhavesh Mehta can be reached on 571-272-7453. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/PHILIP H LAM/Examiner, Art Unit 2656
Prosecution Insights