Prosecution Insights
Last updated: July 17, 2026
Application No. 19/030,684

SYSTEMS AND METHODS FOR OPTIMAL LARGE LANGUAGE MODEL ENSEMBLE ATTRIBUTE EXTRACTION

Final Rejection §103
Filed
Jan 17, 2025
Priority
Jan 31, 2024 — provisional 63/627,370
Examiner
GEBRESENBET, DINKU W
Art Unit
2164
Tech Center
2100 — Computer Architecture & Software
Assignee
Walmart Apollo LLC
OA Round
2 (Final)
71%
Grant Probability
Favorable
3-4
OA Rounds
1y 11m
Est. Remaining
99%
With Interview

Examiner Intelligence

Grants 71% — above average
71%
Career Allowance Rate
431 granted / 608 resolved
+15.9% vs TC avg
Strong +35% interview lift
Without
With
+34.9%
Interview Lift
resolved cases with interview
Typical timeline
3y 5m
Avg Prosecution
11 currently pending
Career history
619
Total Applications
across all art units

Statute-Specific Performance

§101
1.6%
-38.4% vs TC avg
§103
84.7%
+44.7% vs TC avg
§102
11.4%
-28.6% vs TC avg
§112
0.3%
-39.7% vs TC avg
Black line = Tech Center average estimate • Based on career data from 608 resolved cases

Office Action

§103
DETAILED ACTION Notice of Pre-AIA or AIA Status The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA . Response to amendment This office action is in response to an amendment filed on February 13, 2026 in response to PTO office action dated November 25, 2025. The amendment has been entered and considered. Claims 1-4, 8-11 and 15-17 have been amended. Claims 1-20 are pending in this office action. The objection of the Abstract has been withdrawn. Applicant’s arguments with respect to the rejection of claims under 35 U.S.C. § 103(a) have been fully considered bur are moot in view of the new grounds of rejection. This action is FINAL. Claims rejection 35 U.S.C. 103 The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action: A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made. Claims 1-20 are rejected under 35 U.S.C. 103 as being unpatentable over Paulraj et al. (US 20250238308 A1) in view of Schlaich et al. (US 20230222561 A1) further in view of Pisner (US 20240161017 A1). Regarding claims 1, 8 and 15 Paulraj discloses a system, comprising: a processor (see Paulraj paragraph [0200] system 400 includes processor 401, memory 403); and a non-transitory memory, storing instructions that, when executed, cause the processor to (see Paulraj paragraph [0200] system 400 includes processor 401, memory 403): receive an input dataset (see Paulraj paragraph [0048], data obtained from various data sources (not shown) may be used as training data (e.g., used to train the inference models to perform the computer-implemented services), and/or as ingest data (e.g., used as input to the trained inference models in order to perform the computer-implemented services); see Paulraj paragraph [0029], generating a structured knowledge labeled dataset using one or more structured knowledge attributes stored in a structured knowledge repository and one or more failure prediction prompts stored in a failure prediction prompt library); generate a plurality of preliminary attribute labels for at least a first attribute of a first element in the input dataset, wherein each preliminary attribute label in the plurality of preliminary attribute labels is generated by one of a plurality of large language models (LLM) (see Paulraj paragraph [0076], Training data preparation process 206 may include verifying and/or performing data labeling (e.g., associating two or more data samples from the collected training data). For example, a full log file (e.g., input) may be associated with a past failure type (e.g., output). However, labeled training data may not always be reliable (e.g., a data sample may be improperly labeled by a user) and, if incorrectly labeled training data is used to train an inference model, the trained inference model may generate inaccurate inferences; see Paulraj paragraph [0133], the structured knowledge labeled dataset 272 is used (as part of inference model fine-tuning process 274) to train and/or fine-tune an untrained inference model D 280 (e.g., a large language model (LLM))…see Paulraj paragraph [0134] Once untrained inference model D 280 is trained and/or fine-tuned using structured knowledge labeled dataset 272, a trained inference model D 282 may be obtained. In embodiments, the training and/or fine-tuning of untrained inference model D 280 may be implemented using any type of inference model training and/or fine-tuning approaches and/or techniques); Schlaich expressly discloses generate a final attribute label for the first attribute based on a weighted combination of the plurality of preliminary attribute labels for the first attribute (see Schlaich paragraph [0053], the tagging model can retroactively and dynamically update the tags assigned to a product in a product data structure based on the system updating the tagging model to include a new tag or modify an existing tag; see Schlaich paragraph [0098], At block 608, the inventory management system 202 can assign, for each product listing, product tags 314 to the product based on the one or more images and text strings. Each product tag 314 can include in a tag universe 214 maintained by the inventory management system 202. The tag universe 214 can be configured to transmit the plurality of product tags 314 to the product data structure 204 in response to the inventory management system 202 assigning the plurality of product tags 314 to the product); and update a data structure representative of the first element to include the final attribute label for the first attribute (see Schlaich paragraph [0099], At block 610, the inventory management system 202 can generate, for each product listing, an entry 306 in the product data structure 204, the entry 306 including the one or more images and the one or more text strings, the product tags 314 assigned to the product, a first identifier identifying the third party content server 242 from which the product listing was imported, and a second identifier identifying the first entity. The one or more entries 306 can be stored in the product data structure 204). It would have been obvious to a person of ordinary skill in art before the effective filing date of the claimed invention to incorporate the teaching of Schlaich into the method of Verma to have update a data structure representative of the first element to include the final attribute label for the first attribute. Here, combining Schlaich with Verma, which are both related to data processing, improves Verma by providing methods and system to effectively identify unique products for which there may be a relatively low supply in response to an electronic query for such products (see Schlaich paragraph [0002]). Pisner expressly discloses apply an iterative refinement process to a plurality of initial weights and the plurality of preliminary attribute labels that converges the plurality of initial weights to generate a plurality of refined weights, wherein each of the plurality of initial weights corresponds to one of the plurality of large language models (see Pisner paragraph [0006], Ensemble learning is a machine learning technique that combines multiple models or feature representations to improve overall predictive performance, often by reducing overfitting, increasing generalization, and providing more accurate predictions…Boosting is an iterative technique that adjusts the weights of training instances based on the performance of previously trained models. Initially, all instances have equal weights. In each iteration, a new model is trained on the weighted instances, and the weights of incorrectly classified instances are increased so that the new model focuses more on those challenging instances. …); apply each of the plurality of refined weights to a corresponding one of the preliminary attribute labels to generate weighted plurality of preliminary attribute labels (see Pisner paragraph [0006],…The final prediction is obtained by a weighted combination of the base models' predictions. Boosting can improve the accuracy of weak learners by focusing on the most difficult instances in the training data. [0009] Stacking (Stacked Generalization): Stacking involves training multiple base models on the same training data, and then using another model, called the meta-model or meta-learner, to combine the base models' predictions. The base models are trained on the original training data, and their predictions are used as input features for training the meta-model. The meta-model learns to make a final prediction based on the base models' predictions, effectively leveraging their strengths and mitigating their weaknesses). It would have been obvious to a person of ordinary skill in art before the effective filing date of the claimed invention to incorporate the teaching of Pisner into the method of Verma to have applying an iterative refinement process to a plurality of initial weights. Here, combining Pisner Schlaich with Verma, which are both related to data processing, improves Verma by providing a machine learning technique that combines multiple models or feature representations to improve overall predictive performance, often by reducing overfitting, increasing generalization, and providing more accurate predictions (see Pisner paragraph [0004]). Regarding claims 2, 9 and 16 Schlaich expressly discloses wherein, prior to generating the final attribute label, the instructions cause the processor to (see Schlaich paragraph [0053], utilizes a complex tagging model that enables the systems described herein to categorize unlimited images and products automatically through image recognition, automated field completion, and predictive product categorization. The tagging model is dynamically updated as more and more data is ingested by the model): apply a first set of weights to the plurality of preliminary attribute labels, wherein the first set of weights includes at least one LLM specific weight for each preliminary attribute label of the plurality of preliminary attribute labels (see Schlaich paragraph [0072], The tag matcher 422 can be configured to match the plurality of the first tags to the one or more product tags 314 assigned to one or more candidate products. In some implementations, the tag matcher 422 is configured to search the product tags 314 of the tag universe 214 for the one or more product tags 314 that match with the plurality of first tags. In some implementations, the tag matcher 422 can compute one or more match scores based on the comparisons between the product tags 314 and one first tag of the plurality of first tags); receive an updated set of weights; and apply the updated set of weights to the plurality of preliminary attribute labels during the combination (see Schlaich paragraph [0072], As a result of a first-pass matching, the tag matcher 422 can determine a number of the one or more product tags 314 that matches the first tag and compare the number to a lower threshold count and a higher threshold count. If the number that matches is higher than the higher threshold count, then the tag matcher 422 can re-compute the matching scores and compare to a threshold score that is lower than the previous threshold score. If the number that matches is lower than the lower threshold count, then the tag matcher 422 can re-compute the matching scores and compare to a threshold score that is higher than the previous threshold score. The tag matcher 422 can repeat iterations of comparing the match scores to the updated threshold score until the number of one or more products tag 314 that matches the first tag is in between the lower threshold count and the higher threshold count). It would have been obvious to a person of ordinary skill in art before the effective filing date of the claimed invention to incorporate the teaching of Schlaich into the method of Verma to have apply a first set of weights to the plurality of preliminary attribute labels. Here, combining Schlaich with Verma, which are both related to data processing, improves Verma by providing methods and system to effectively identify unique products for which there may be a relatively low supply in response to an electronic query for such products (see Schlaich paragraph [0002]). Expressly discloses the weighted plurality of preliminary attributes label (see Pisner paragraph [0006],…The final prediction is obtained by a weighted combination of the base models' predictions. Boosting can improve the accuracy of weak learners by focusing on the most difficult instances in the training data. [0009] Stacking (Stacked Generalization): Stacking involves training multiple base models on the same training data, and then using another model, called the meta-model or meta-learner, to combine the base models' predictions. The base models are trained on the original training data, and their predictions are used as input features for training the meta-model. The meta-model learns to make a final prediction based on the base models' predictions, effectively leveraging their strengths and mitigating their weaknesses). It would have been obvious to a person of ordinary skill in art before the effective filing date of the claimed invention to incorporate the teaching of Pisner into the method of Verma to have applying an iterative refinement process to a plurality of initial weights. Here, combining Pisner Schlaich with Verma, which are both related to data processing, improves Verma by providing a machine learning technique that combines multiple models or feature representations to improve overall predictive performance, often by reducing overfitting, increasing generalization, and providing more accurate predictions (see Pisner paragraph [0004]). Regarding claims 3, 10 and 17 Schlaich expressly discloses wherein the weighted combination of the plurality of preliminary attribute labels comprises: assigning a first weight to a first preliminary attribute label of the plurality of preliminary attribute labels generated by a first LLM (see Schlaich paragraph [0053], The tagging model is dynamically updated as more and more data is ingested by the model. In some embodiments, the tagging model is trained using data that is obtained from web resources, databases, or other publicly and privately available data. This data can already include tags or labels assigned by the source of the data); and assigning a second weight to a second preliminary attribute label of the plurality of preliminary attribute labels generated by a second LLM (see Schlaich paragraph [0053], these tags or labels are incorrect or do not match the tags or labels that form part of the vernacular of the tagging model. In some embodiments, the system can compare the tags included in the data obtained from various sources to the tags in the tagging model and update the tagging model to include these tags according to a tag policy. In some embodiments, the system can compare the tags included in the data obtained from various sources to the tags in the tagging model and update the tagging model to modify existing tags based on the tags included in the data according to the tag policy. As a result, the tagging model can retroactively and dynamically update the tags assigned to a product in a product data structure based on the system updating the tagging model to include a new tag or modify an existing tag). It would have been obvious to a person of ordinary skill in art before the effective filing date of the claimed invention to incorporate the teaching of Schlaich into the method of Paulraj to have apply a first set of weights to the plurality of preliminary attribute labels. Here, combining Schlaich with Paulraj, which are both related to data processing, improves Paulraj by providing methods and system to effectively identify unique products for which there may be a relatively low supply in response to an electronic query for such products (see Schlaich paragraph [0002]). Regarding claims 4, 11 and 18 Paulraj discloses one of a plurality of large language models (LLM) (see Paulraj paragraph [0032] The inference model is a large language model (LLM) and fine-tuning the inference model using the structured knowledge labeled dataset may include implementation of a private infrastructure fine-tuning method). Schlaich expressly discloses wherein determining the weighted combination is an iterative process that optimizes the combination of weights … (see Schlaich paragraph [0052], the present solution assigns weights to tags generated from the query (or the object detection algorithm) based on the tags assigned to products in the products data structure. In this way, the search process is optimized by relying on tags of the query that can be matched to tags of the products in the product data structure; see Schlaich paragraph [0072],If the number that matches is lower than the lower threshold count, then the tag matcher 422 can re-compute the matching scores and compare to a threshold score that is higher than the previous threshold score. The tag matcher 422 can repeat iterations of comparing the match scores to the updated threshold score until the number of one or more products tag 314 that matches the first tag is in between the lower threshold count and the higher threshold count). It would have been obvious to a person of ordinary skill in art before the effective filing date of the claimed invention to incorporate the teaching of Schlaich into the method of Paulraj to have apply a first set of weights to the plurality of preliminary attribute labels. Here, combining Schlaich with Paulraj, which are both related to data processing, improves Paulraj by providing methods and system to effectively identify unique products for which there may be a relatively low supply in response to an electronic query for such products (see Schlaich paragraph [0002]). Regarding claims 5, 12 and 19 Schlaich expressly discloses wherein the instructions further cause the processor to generate at least one element to be displayed at a user interface, wherein the at least one element is determined based on the final attribute label stored in the data structure (see Schlaich paragraph [0052], the present solution assigns weights to tags generated from the query (or the object detection algorithm) based on the tags assigned to products in the products data structure. In this way, the search process is optimized by relying on tags of the query that can be matched to tags of the products in the product data structure; see Schlaich paragraph [0074] The presentation engine 226 can be configured to send the results 254 to a buyer via the one or more client devices 232. The results 254 can include a name of the product, the product ID 308, or the seller ID 310 associated with the candidate products). It would have been obvious to a person of ordinary skill in art before the effective filing date of the claimed invention to incorporate the teaching of Schlaich into the method of Verma to have apply a first set of weights to the plurality of preliminary attribute labels. Here, combining Schlaich with Paulraj, which are both related to data processing, improves Paulraj by providing methods and system to effectively identify unique products for which there may be a relatively low supply in response to an electronic query for such products (see Schlaich paragraph [0002]). Regarding claims 6, 13 and 20 Paulraj discloses wherein each respective preliminary attribute label is identified by a received prompt (see Paulraj paragraph [0027] may include an inference model (e.g., a large language model) configured to generate failure prediction prompts using the hidden knowledge. Such failure prediction prompts advantageously make the hidden knowledge friendlier for human consumption, interpretation, and use (e.g., by converting the hidden knowledge into human readable descriptive natural language; see Paulraj paragraph [0024], hidden knowledge may include structured knowledge attributes that describe relationships between objects (e.g., between input features of ingest data and/or inferences generated by the model that are associated with the ingest data), and/or rules, policies, or procedures for generating inferences (e.g., based on the ingest data). The hidden knowledge extracted from inference models may provide for interpretability of the outcomes (e.g., predictions) of the inference models, which may allow for the evaluation of the trustworthiness of the predictions (e.g., failure predictions)). Regarding claims 7 and 14 Paulraj discloses wherein each respective preliminary attribute label is predefined during generation each LLM of the plurality of LLMs (see Paulraj paragraph [0032] The inference model is a large language model (LLM) and fine-tuning the inference model using the structured knowledge labeled dataset may include implementation of a private infrastructure fine-tuning method). Remarks The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Carbune (US 12332074 B1) discloses a model trainer 660 that trains the machine-learned models 620 and/or 640 stored at the user computing device 602 and/or the server computing system 630 using various training or learning techniques, such as, for example, backwards propagation of errors. For example, a loss function can be backpropagated through the model(s) to update one or more parameters of the model(s) (e.g., based on a gradient of the loss function). Various loss functions can be used such as mean squared error, likelihood loss, cross entropy loss, hinge loss, and/or various other loss functions. Gradient descent techniques can be used to iteratively update the parameters over a number of training iterations. Conclusion Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a). A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the date of this final action. Any inquiry concerning this communication or earlier communications from the examiner should be directed to DINKU W GEBRESENBET whose telephone number is (571)270-1636. The examiner can normally be reached between 8:00AM-5:00PM. Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Amy Ng can be reached on 571- 270-1698. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300. Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. /DINKU W GEBRESENBET/Primary Examiner, Art Unit 2164
Read full office action

Prosecution Timeline

Jan 17, 2025
Application Filed
Nov 25, 2025
Non-Final Rejection mailed — §103
Jan 28, 2026
Applicant Interview (Telephonic)
Feb 05, 2026
Examiner Interview Summary
Feb 13, 2026
Response Filed
Jun 03, 2026
Final Rejection mailed — §103 (current)

Precedent Cases

Applications granted by this same examiner with similar technology

Patent 12670158
UNDERSTANDING STORAGE UTILIZATION OF A CACHING SYSTEM THROUGH INCOMING TRAFFIC
2y 1m to grant Granted Jun 30, 2026
Patent 12664152
METHOD AND SYSTEM FOR DATA STORAGE AND RETRIEVAL
1y 7m to grant Granted Jun 23, 2026
Patent 12625866
SYSTEM TO DETERMINE TUPLE VALUES
2y 3m to grant Granted May 12, 2026
Patent 12619627
CENTRAL USER INTERFACE FOR ACCESSING AND UPGRADING OF DATASET INTEGRATIONS
1y 4m to grant Granted May 05, 2026
Patent 12613868
Bifurcated System for Querying Multiple Data Types
2y 1m to grant Granted Apr 28, 2026
Study what changed to get past this examiner. Based on 5 most recent grants.

Strategy Recommendation AI-generated — please review before filing

Get a prosecution strategy drawn from examiner precedents, rejection analysis, and claim mapping.
Typically takes 5-10 seconds — AI-generated, attorney review required before filing

Prosecution Projections

3-4
Expected OA Rounds
71%
Grant Probability
99%
With Interview (+34.9%)
3y 5m (~1y 11m remaining)
Median Time to Grant
Moderate
PTA Risk
Based on 608 resolved cases by this examiner. Grant probability derived from career allowance rate.

Sign in with your work email

Enter your email to receive a magic link. No password needed.

Personal email addresses (Gmail, Yahoo, etc.) are not accepted.

Free tier: 3 strategy analyses per month