Prosecution Insights
Last updated: April 19, 2026
Application No. 18/421,888

SIMULATING A/B TESTING THROUGH LARGE LANGUAGE MACHINE-LEARNED MODELS

Non-Final OA §101
Filed
Jan 24, 2024
Examiner
WAESCO, JOSEPH M
Art Unit
3625
Tech Center
3600 — Transportation & Electronic Commerce
Assignee
Maplebear Inc.
OA Round
3 (Non-Final)
47%
Grant Probability
Moderate
3-4
OA Rounds
3y 1m
To Grant
90%
With Interview

Examiner Intelligence

Grants 47% of resolved cases
47%
Career Allow Rate
213 granted / 452 resolved
-4.9% vs TC avg
Strong +42% interview lift
Without
With
+42.4%
Interview Lift
resolved cases with interview
Typical timeline
3y 1m
Avg Prosecution
51 currently pending
Career history
503
Total Applications
across all art units

Statute-Specific Performance

§101
47.0%
+7.0% vs TC avg
§103
32.6%
-7.4% vs TC avg
§102
3.1%
-36.9% vs TC avg
§112
12.2%
-27.8% vs TC avg
Black line = Tech Center average estimate • Based on career data from 452 resolved cases

Office Action

§101
DETAILED ACTION Continued Examination Under 37 CFR 1.114 A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 2/9/2026 has been entered. In response to Final Communications received 11/7/2025, Applicant, on 2/9/2026, amended Claims 1, 5, 8, 12, 15, and 19. Claims 1-20 are pending in this case, are considered in this application, and have been rejected below. Response to Arguments Arguments regarding 35 USC §101 Alice – Applicant states there claims cannot be performed in the human mind and thus are eligible under 101. Examiner disagrees as although the Claims do not recite a Mental Process, they do, however, recite a Commercial Interaction, as the Claims recite limitations for outputting analyzed historical product data, which is a Certain Method of Organizing Human Activity. The treatment here is a display/presentation of information, at best an additional element which is not improved by the system. Applicant asserts the amended limitations improve the technical field of A/B testing for UI, stating that the simulated A/B testing is performed with at least two treatments that include a first treatment and a second treatment that vary a layout or design of a UI of an application, or presentation of an interface element associated with the application, and thus is necessarily rooted in computer technology rather than a mental process, similar to that of Example 37. Examiner disagrees as first, these limitations have nothing to do with Example 37, as there is no arranging icons on an interface, and the limitations are not practically integrated as the claims merely utilize the additional elements to perform the abstract limitations of the Claims, and the “vary a layout or design of a user interface (UI) of an application, or presentation of an interface element associated with the application” in Applicant’s Specification is: “[0036] In one embodiment, the online concierge system 140 described herein simulates A/B testing experiments in conjunction with one or more large-scale machine-learned models deployed by the online concierge system140. Companies utilize A/B testing to compare variations of an element to improve customer’s experiences and increase engagement. A/B testing evaluates variations in color schemes, button placement, navigation menus, or even entire layouts. For example, such companies might run an A/B test to evaluate the click-through-rates of different email campaigns or website designs. Companies may also run A/B tests to improve site navigation and user ability by enhancing the sequence of interfaces presented to the user during the user’s experience with the online concierge system 140. For example, the A/B tests can test several iterations of the interface elements presented to the user during a single experience with the online concierge system 140” Which states that these interface elements are presented on a system, are just elements displayed on a computer, with no description on how this is performed, and from this interpretation, one would reasonably deduce the aforementioned steps are all functions that are generic displays of information. There is no improvement any additional element, alone or in combination, nor to any technology, or technological process, and thus this is “Applying It” similar to Alice and not eligible by the MPEP. Therefore, the arguments are non-persuasive, the Claims are ineligible as there is no inventive concept, and the rejection of the Claims and their dependents are maintained under 35 USC 101. Claim Rejections - 35 USC § 101 35 U.S.C. 101 reads as follows: Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title. Alice – Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. Claims 1, 8, and 15 are directed at limitations for obtaining a user history database, wherein the user history database describes at least a user’s past behavior (a Commercial Interaction, i.e. analyzing product information ; a Certain Method of Organizing Human Activity), obtaining at least two treatments, wherein the treatment comprises a candidate interaction the user engages with, wherein the at least two treatments include a first treatment and a second treatment that vary (a Commercial Interaction, i.e. analyzing product information ; a Certain Method of Organizing Human Activity), generating a prompt for a machine learning language model including the at least two treatments, the user history database, and a request to generate predictions of an output of a desired output type, wherein the output type indicates how the user will interact with the treatment (a Commercial Interaction, i.e. analyzing product information ; a Certain Method of Organizing Human Activity), providing the prompt to a model serving system, hosting the machine learned language model, for execution, wherein the machine learned language model is configured as a transformer architecture including an attention operation, the attention operation coupled to receive input data and generate queries, keys, and values, and generate an attention output from the queries, the keys, and the values, (a Commercial Interaction, i.e. analyzing product information ; a Certain Method of Organizing Human Activity), receiving a response from the model serving system, wherein the response includes an output, and wherein the output indicates a prediction of the desired output type on how the user will respond to each received treatment (a Commercial Interaction, i.e. analyzing product information ; a Certain Method of Organizing Human Activity), selecting, a treatment based on the received prediction of the user (a Commercial Interaction, i.e. analyzing product information ; a Certain Method of Organizing Human Activity), and responsive to selecting the treatment based for the user, presenting the selected treatment to a user device associated with the user (a Commercial Interaction, i.e. analyzing product information ; a Certain Method of Organizing Human Activity), which under their broadest reasonable interpretation, covers performance of the limitation in the mind for the purposes of determining an output of analyzed product information, a Commercial Interaction, but for the recitation of generic computer components. That is, other than reciting a user history database, model serving system, user device, computer-readable medium, computer processor, computer system, and varying a layout or design of a user interface of an application or presentation of an interface element associated with the application, nothing in the claim element precludes the step from practically being performed for the purposes of a Commercial Interaction. The claims recite limitations for a Commercial Interaction, a “Certain Method of Organizing Human Activity”. Accordingly, the claim recites an abstract idea. This judicial exception is not integrated into a practical application. In particular, the claim recites the above stated additional elements to perform the abstract limitations as above. The system, user history database, computer processor, computer-readable medium, user device, and model serving system are recited at a high-level of generality (i.e., as a generic software/module performing a generic computer function of storing, retrieving, sending, and processing data) such that they amount to no more than mere instructions to apply the exception using generic computer components. Even if taken as an additional element, the receiving and transmission steps above are insignificant extra-solution activity as these are receiving, storing, and transmitting data as per the MPEP 2106.05(d). Accordingly, these additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea. The claim is directed to an abstract idea. The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception, when considered both individually and as an ordered combination. As discussed above with respect to integration of the abstract idea into a practical application, the additional element being used to perform the abstract limitations stated above amount to no more than mere instructions to apply the exception using generic computer components. Mere instructions to apply an exception using generic computer components cannot provide an inventive concept. The claim is not patent eligible. Applicant’s Specification states: “[0016] The picker client device 110 is a client device through which a picker may interact with the customer client device 100, the retailer computing system 120, or the online concierge system 140. The picker client device110 can be a personal or mobile computing device, such as a smartphone, a tablet, a laptop computer, or desktop computer. In some embodiments, the picker client device110 executes a client application that uses an application programming interface (API) to communicate with the online concierge system 140.” Which states that any type of device or computer may be used, such as any personal computer, laptop, mobile phone, tablet, etc., to perform the abstract limitations, and from this interpretation, one would reasonably deduce the aforementioned steps are all functions that can be done on generic components, and thus application of an abstract idea on a generic computer, as per the Alice decision and not requiring further analysis under Berkheimer, but for edification the Applicant’s specification has been used as above satisfying any such requirement. This is “Applying It” by utilizing current technologies. Further, the “vary a layout or design of a user interface (UI) of an application, or presentation of an interface element associated with the application” in Applicant’s Specification is: “[0036]In one embodiment, the online concierge system 140 described herein simulates A/B testing experiments in conjunction with one or more large-scale machine-learned models deployed by the online concierge system140. Companies utilize A/B testing to compare variations of an element to improve customer’s experiences and increase engagement. A/B testing evaluates variations in color schemes, button placement, navigation menus, or even entire layouts. For example, such companies might run an A/B test to evaluate the click-through-rates of different email campaigns or website designs. Companies may also run A/B tests to improve site navigation and user ability by enhancing the sequence of interfaces presented to the user during the user’s experience with the online concierge system 140. For example, the A/B tests can test several iterations of the interface elements presented to the user during a single experience with the online concierge system 140” Which states that these interface elements are presented on a system, but has nothing to do with the treatments and are broadly connected to them, but rather are just elements displayed on a computer, with no description on how this is performed, and from this interpretation, one would reasonably deduce the aforementioned steps are all functions that are generic, and thus application of an abstract idea on a generic computer, as per the Alice decision and not requiring further analysis under Berkheimer, but for edification the Applicant’s specification has been used as above satisfying any such requirement. This is “Applying It” by utilizing current technologies. For the receiving and transmission steps that were considered extra-solution activity in Step 2A above, if they were to be considered additional elements, they have been re-evaluated in Step 2B and determined to be well-understood, routine, conventional, activity in the field. The background does not provide any indication that the additional elements, such as the system, user device, processors, etc., nor the receiving and transmission steps as above, are anything other than a generic, and the MPEP Section 2106.05(d) indicates that mere collection or receipt, storing, or transmission of data is a well‐understood, routine, and conventional function when it is claimed in a merely generic manner (as it is here). For these reasons, there is no inventive concept. The claim is not patent eligible. Claims 2-7, 9-14, and 16-20 contain the identified abstract ideas, further narrowing them, with no additional elements to be considered as part of a practical application or under prong 2 of the 2019 PEG, thus not integrated into a practical application, nor are they significantly more for the same reasons and rationale as above. After considering all claim elements, both individually and in combination, Examiner has determined that the claims are directed to the above abstract ideas and do not amount to significantly more. Therefore, the claims and dependent claims are rejected under 35 U.S.C. 101 as being directed to non-statutory subject matter. See Alice Corporation Pty. Ltd. v. CLS Bank International, No. 13–298. Allowable Subject Matter Claims 1-20 are objected to as being dependent upon a rejected base claim, but would be allowable if the independent claims were amended in such a way as to overcome the 35 USC 101 rejection. The closest prior art of record are Neumann (U.S. Publication No. 2020/032,1116), Wexler (U.S. Publication No. 2021/038,3925), and Mikkulainen (U.S. Publication No. 2017/019,3366}. Neumann, a method and system for generating an alimentary instruction set identifying an individual prognostic mitigation plan, teaches obtaining a user history database, wherein the user history database data describes at least a user’s past behavior, receiving at least two treatments, wherein the treatment comprises a candidate interaction the user engages with, receiving a response from the model serving system, wherein the response includes an output, and wherein the output indicates a prediction of the desired output type on how the user will respond to each received treatment, responsive to selecting the treatment based for the user, presenting the selected treatment to a user device associated with the user, generating a prompt for use with a machine learning language model including the at least two treatments, the user history database where the history information and treatments are used with a prompt and machine learning to come up with a best predicted output, it does not explicitly state this is the desired output type or how the user will interact with the treatment, nor does it teach selecting, a treatment based on the received prediction of the user, varying of a layout or design based on the treatments, nor the machine learning language model configured as transformer architecture in the manner claimed. Wexler, a system and method for adaptive healthcare support and behavioral intervention, teaches a request to generate predictions of an output of a desired output type, wherein the output type indicates how the user will interact with the treatment and prediction of treatments, but it does not teach the varying of a layout or design based on the treatments, nor the machine learning language model configured as transformer architecture in the manner claimed. Mikkulainen, a system and method for webinterface generation and testing using artificial neural networks, teaches using targeted user behaviors to convert end users, changing the property of interface elements based on clusters stemming from a machine learning model which customizes the interface, and use a machine learning for multivariate, A/B testing, but not the specific varying of a layout or design based on the treatments, nor the machine learning language model configured as transformer architecture in the manner claimed. None of the prior art explicitly teaches this varying of a layout or design based on the treatments, nor the machine learning language model configured as transformer architecture in the manner claimed, and these are the reasons which adequately reflect the Examiner's opinion as to why Claims 1-20 are allowable over the prior art of record, and are objected to as provided above. Conclusion The prior art made of record is considered pertinent to applicant's disclosure. US 20210383925 A1 Wexler; Ydo et al. SYSTEMS FOR ADAPTIVE HEALTHCARE SUPPORT, BEHAVIORAL INTERVENTION, AND ASSOCIATED METHODS US 20200321116 A1 Neumann; Kenneth METHODS AND SYSTEMS FOR GENERATING AN ALIMENTARY INSTRUCTION SET IDENTIFYING AN INDIVIDUAL PROGNOSTIC MITIGATION PLAN US 20240007546 A1 Morris; Heather et al. TRANSMISSION OF MESSAGES IN COMPUTER NETWORKED ENVIRONMENTS US 20220300787 A1 WALL; Dennis et al. MODEL OPTIMIZATION AND DATA ANALYSIS USING MACHINE LEARNING TECHNIQUES US 20220188671 A1 Jain; Praduman et al. SYSTEMS AND METHODS FOR USING MACHINE LEARNING TO IMPROVE PROCESSES FOR ACHIEVING READINESS US 20210133509 A1 WALL; Dennis et al. MODEL OPTIMIZATION AND DATA ANALYSIS USING MACHINE LEARNING TECHNIQUES US 20200336450 A1 Gao; Adam et al. MESSAGING SELECTION SYSTEMS IN NETWORKED ENVIRONMENTS US 20200133966 A1 Canim; Mustafa et al. Predicting Intent of a User from Anomalous Profile Data US 20190279767 A1 Bates; James Stewart SYSTEMS AND METHODS FOR CREATING AN EXPERT-TRAINED DATA MODEL US 20250182185 A1 Putrevu; Jagannath et al. ALLOCATING SHOPPERS AND ORDERS FOR FULFILLMENT BY AN ONLINE CONCIERGE SYSTEM TO ACCOUNT FOR VARIABLE NUMBERS OF SHOPPERS ACROSS DIFFERENT TIME WINDOWS Any inquiry concerning this communication or earlier communications from the examiner should be directed to JOSEPH M WAESCO whose telephone number is (571)272-9913. The examiner can normally be reached on 8 AM - 5 PM M-F. Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, BETH BOSWELL can be reached on (571) 272-6737. The fax phone number for the organization where this application or proceeding is assigned is 571-273-1348. Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. /JOSEPH M WAESCO/Primary Examiner, Art Unit 3625B 3/8/2026
Read full office action

Prosecution Timeline

Jan 24, 2024
Application Filed
Jun 07, 2025
Non-Final Rejection — §101
Sep 03, 2025
Examiner Interview (Telephonic)
Sep 03, 2025
Examiner Interview Summary
Sep 10, 2025
Response Filed
Nov 05, 2025
Final Rejection — §101
Feb 03, 2026
Applicant Interview (Telephonic)
Feb 03, 2026
Examiner Interview Summary
Feb 09, 2026
Request for Continued Examination
Feb 28, 2026
Response after Non-Final Action
Mar 08, 2026
Non-Final Rejection — §101 (current)

Precedent Cases

Applications granted by this same examiner with similar technology

Patent 12602702
METHODS AND APPARATUS TO ESTIMATE CARDINALITY ACROSS MULTIPLE DATASETS REPRESENTED USING BLOOM FILTER ARRAYS
2y 5m to grant Granted Apr 14, 2026
Patent 12596348
SOURCE TO TARGET TRANSLATION FOR MANUFACTURING
2y 5m to grant Granted Apr 07, 2026
Patent 12591921
Optimize Shopping Route Using Purchase Embeddings
2y 5m to grant Granted Mar 31, 2026
Patent 12579519
GENERATING DIGITAL ASSOCIATIONS BETWEEN DOCUMENTS AND DIGITAL CALENDAR EVENTS BASED ON CONTENT CONNECTIONS
2y 5m to grant Granted Mar 17, 2026
Patent 12561659
Machine-Learned Robot Fleet Management for Value Chain Networks
2y 5m to grant Granted Feb 24, 2026
Study what changed to get past this examiner. Based on 5 most recent grants.

AI Strategy Recommendation

Get an AI-powered prosecution strategy using examiner precedents, rejection analysis, and claim mapping.
Powered by AI — typically takes 5-10 seconds

Prosecution Projections

3-4
Expected OA Rounds
47%
Grant Probability
90%
With Interview (+42.4%)
3y 1m
Median Time to Grant
High
PTA Risk
Based on 452 resolved cases by this examiner. Grant probability derived from career allow rate.

Sign in with your work email

Enter your email to receive a magic link. No password needed.

Personal email addresses (Gmail, Yahoo, etc.) are not accepted.

Free tier: 3 strategy analyses per month