DETAILED ACTION
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 2/26/26 has been entered.
Status of Claims
The following is a non-final office action in response to the communication received 2/26/26.
Claims 1, 13, 18 and 20 have been amended.
Claims 9 and 19 have been cancelled.
Claims 1-8, 10-18 and 20 are currently pending and have been examined.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant’s amendments and associated arguments, filed 2/26/26, with respect to the objection to the claims have been considered and are persuasive. The objection to the claims has been withdrawn.
Applicant’s amendments and associated arguments, filed 2/26/26, with respect to the rejection of the claims under 35 U.S.C. §101 have been considered but they are not persuasive.
Applicant argues that the limitation of "applying a contextual bandit model to the one or more query features and the one or more contextual features to generate a weight vector for ranking parameters used for ranking query results,... , wherein the contextual bandit model is a machine- learning model trained by reinforcement learning,” cannot be characterized as directed to any abstract idea. Examiner initially notes that, as currently recited, the model is merely characterized as a machine learning model that has been trained. The training process itself is not recited or improved upon. Subsequently, the application of the contextual bandit model merely amounts to applying the judicial exception, and the characterization of the model as being trained by reinforcement learning acts merely as a classifier as the training function itself is not actively recited. The application of a trained model, as identified in example 47, still amounts to apply it.
Applicant further argues that the limitations do not recite an abstract idea (specifically excluded methods of organizing human activity, mathematical concepts, or mental processes). Examiner initially notes that the application of the machine-learning model is identified as an additional element, specifically an additional element that applies the abstract idea. However, the claims recite multiple forms of abstract concepts, including mental processes (e.g., receiving a query, obtaining contextual features of said query, applying a model to a query, identify/score/rank/share results), mathematical concepts (e.g., generating vectors, generating scores based on weighted sums, etc.), and managing relationships or interactions between people (sending rules or instructions of a shopper to a picker). Per MPEP 2106, when claims may recite multiple abstract ideas, all should be clearly identified in the written record.
Applicant further argues that the limitations represent an integration into a practical application because they embody an improvement to the field of machine learning. Examiner respectfully disagrees. When recited at this level of breadth, the claims merely recite the application of a machine learning model, but does not recite model structure or active training steps the embody an improvement to the field of machine learning. The improvement identified (session-to-session adaptation of query processing based on a session’s context) is an abstract one, rather than an improvement to the field of machine learning.
Applicant’s amendments and associated arguments, filed 2/26/2026, with respect to the rejection of the under 35 U.S.C. §103 have been considered but are not persuasive.
Applicant states that the previously applied prior art does not teach the amended limitations of the applying limitation of claim 1. Examiner respectfully disagrees. With respect to the specific argument that the weight vector is not a scalar value or a relevance score, Examiner notes that the claims do not require the score to be a scalar value, and applied prior art Sirotkovic discloses a DNN model with vectors, or trainable parameters, that are used to obtain relevance scores for items that are used in the ranking of said items (see Figs 4/5, [0027]-[0033], [0043], [0047], as mapped in the rejection below). With respect to the specific arguments directed to Szvaras, Examiner first notes that one cannot show nonobviousness by attacking references individually where the rejections are based on combinations of references (See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981); In re Merck & Co., 800 F.2d 1091, 231 USPQ 375 (Fed. Cir. 1986)), and additionally notes that primary reference, Sirotkovic teaches the limitations directed to the weighted vector for scoring and ranking items via the application of a contextual model, and Szvaras discloses that contextual models may be trained contextual bandit models.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-8, 10-18 and 20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 of the Subject Matter Eligibility Test entails considering whether the claimed subject matter falls within the four statutory categories of patentable subject matter identified by 35 U.S.C. 101: Process, machine, manufacture, or composition of matter.
Claims 1-8, 10-18 and 20 are directed to a method (process), a system (machine or manufacture), and a non-transitory medium (manufacture), respectively. As such, the claims are directed to statutory categories of invention.
If the claim recites a statutory category of invention, the claim requires further analysis in Step 2A. Step 2A of the Subject Matter Eligibility Test is a two-prong inquiry. In Prong One, examiners evaluate whether the claim recites a judicial exception.
Claim 1 recites abstract limitations, including: those identified in bold below.
1. A computer-implemented method comprising:
receiving, from a client device, a user query for identifying one or more items by an online concierge system, the user query described by one or more query features;
obtaining one or more contextual features describing a context of the user query;
applying a query processing model to the user query to generate a relevance score for each query result of a plurality of query results;
applying a contextual bandit model to the one or more query features and the one or more contextual features to generate a weight vector for ranking parameters used for ranking query results, wherein the weight vector is applied uniformly to the query results responsive to the user query, wherein the ranking parameters include relevance of a query result to the user query and dependability of the query result representing a likelihood that the query result is available at a particular location, wherein the contextual bandit model is a machine-learning model trained by reinforcement learning;
generating, for each query result and based on the weight vector and ranking parameter values of the query result including the relevance score generated by the query processing model and a dependability score of the query result, a ranking score as a weighted sum of the weight vector and the ranking parameter values of the query result; and
transmitting the query results ranked according to the ranking scores for display on the client device; and
receiving, from the client device, user input adding an item from the query results to an order for fulfillment at the particular location and
transmitting the order to a second client device associated with a picker, wherein the order is presented on the second client device for fulfillment by the picker.
Independent claims 13 and 20 recite analogous abstract limitations as claim 1.
These limitations, as drafted, are a process that, under its broadest reasonable interpretation, cover performance of the limitations in the mind, or by a human using pen and paper, (e.g., receiving a query, obtaining contextual features of said query, applying a model to a query, identify/score/rank/share results) and therefore recite mental processes. The mere recitation of a generic computer does not take the claim out of the mental process grouping. Thus, the claim recites an abstract idea.
These limitations, as drafted, are a process that, under its broadest reasonable interpretation, represent mathematical relationships and are therefore mathematical concepts (e.g., models with weighted vectors, scoring, etc.). The mere recitation of a generic computing elements does not take the claim out of the mathematical concepts grouping. Thus, the claim recites an abstract idea.
These limitations, as drafted, also recite a process that, under its broadest reasonable interpretation, represents a commercial or legal interaction (order fulfillment) and a process for managing relationships or interactions between people (sending rules or instructions of a shopper to a picker) and are therefore a method of organizing human activity. More specifically, other than reciting that the process is performed using generic device, nothing in the claim element precludes the abstract steps recited above from practically being performed by a human(s). Thus, the claim recites an abstract idea.
If the claim recites a judicial exception in step 2A Prong One , the claim requires further analysis in step 2A Prong Two. In step 2A Prong Two, examiners evaluate whether the claim recites additional elements that integrate the exception into a practical application of that exception.
The claim recites additional elements, which are highlighted below
1. A computer-implemented method comprising:
receiving, from a client device, a user query for identifying one or more items by an online concierge system, the user query described by one or more query features;
obtaining one or more contextual features describing a context of the user query;
applying a query processing model to the user query to generate a relevance score for each query result of a plurality of query results;
applying a contextual bandit model to the one or more query features and the one or more contextual features to generate a weight vector for ranking parameters used for ranking query results, wherein the weight vector is applied uniformly to the query results responsive to the user query, wherein the ranking parameters include relevance of a query result to the user query and dependability of the query result representing a likelihood that the query result is available at a particular location, wherein the contextual bandit model is a machine-learning model trained by reinforcement learning;
generating, for each query result and based on the weight vector and ranking parameter values of the query result including the relevance score generated by the query processing model and a dependability score of the query result, a ranking score as a weighted sum of the weight vector and the ranking parameter values of the query result; and
transmitting the query results ranked according to the ranking scores for display on the client device; and
receiving, from the client device, user input adding an item from the query results to an order for fulfillment at the particular location and
transmitting the order to a second client device associated with a picker, wherein the order is presented on the second client device for fulfillment by the picker.
Claim 13 recites further additional elements including: A non-transitory computer-readable storage medium storing instructions that, when executed by a computer processor, cause the computer processor to perform operations.
Claim 20 recites further additional elements including: A system comprising: a computer processor; and a non-transitory computer-readable storage medium storing instructions that, when executed by the computer processor, cause the computer processor to perform operations
With respect to the sending/receiving of information between an online concierge system, a client device, and a second client device, the function itself is recited at a high level of generality and amounts to extra-solution activity.
With respect to the display of information on the client device, and the second client device, the function itself is recited at a high level of generality (i.e. as a general means of displaying the result of the abstract process), and amounts to extra-solution activity.
The functions of the online concierge system, the trained contextual bandit model, the client device, the second client device and the generic computing elements of claims 13 and 20 (sending/receiving/display data) are recited at a high-level of generality such that it amounts no more than mere instructions to apply the exception using a generic computer component.
Accordingly, in combination, these additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea.
If the additional elements do not integrate the exception into a practical application in step 2A Prong Two, then the claim is directed to the recited judicial exception, and requires further analysis under Step 2B to determine whether they provide an inventive concept (i.e., whether the additional elements amount to significantly more than the exception itself).
With respect to the sending/receiving of information between an online concierge system, a client device, and a second client device, the Symantec, TLI, OIP Techs. and buySAFE court decisions cited in MPEP 2106.05(d)(II) indicate that mere receiving or transmitting data over a network is a well‐understood, routine, conventional function when it is claimed in a merely generic manner (as it is here).
With respect to the display of information on the client device and the second client device, the Symantec, Internet Patent Corp. court decision cited in MPEP 2106.05(d)(II) indicate that a web browser’s back and forward functionality is a well‐understood, routine, conventional function when it is claimed in a merely generic manner (as it is here). Additionally, MPEP 2106.05(d)(II), and the cases cited therein, including in Trading Techs. Int’l v. IBG LLC, 921 F.3d 1084, 1093 (Fed. Cir. 2019), and Intellectual Ventures I LLC v. Erie Indemnity Co., 850 F.3d 1315, 1331 (Fed. Cir. 2017), for example, indicated that the mere displaying of data is a well understood, routine, and conventional function.
The specification also demonstrates the well-understood, routine, conventional nature of additional elements as it describes the additional elements as well-understood or routine or conventional (or an equivalent term), as a commercially available product, or in a manner that indicates that the additional elements are sufficiently well-known that the specification does not need to describe the particulars of such additional elements to satisfy 35 U.S.C. §112(a).
The functions of the online concierge system, the trained contextual bandit model, the client device, the second client device and the generic computing elements of claims 13 and 20 (sending/receiving/display data) are recited at a high-level of generality such that it amounts no more than mere instructions to apply the exception using a generic computer component. Use of a computer or other machinery in its ordinary capacity for economic or other tasks (e.g., to receive, store, or transmit data) or simply adding a general purpose computer or computer components after the fact to an abstract idea does not provide significantly more. See Affinity Labs v. DirecTV, 838 F.3d 1253, 1262, 120 USPQ2d 1201, 1207 (Fed. Cir. 2016) (cellular telephone); TLI Communications LLC v. AV Auto, LLC, 823 F.3d 607, 613, 118 USPQ2d 1744, 1748 (Fed. Cir. 2016) (computer server and telephone unit).
Thus, even when viewed as an ordered combination, nothing in the claims add significantly more (i.e. an inventive concept) to the abstract idea.
Claim 2 acts to further characterize the user query as being specific signal types, which amounts to a field of use and cannot integrate the judicial exception into a practical application and does not amount to significantly more than the exception itself (see MPEP 2106.05(h)).
Claims 3-5, and 7-8 act to narrow the previously recited abstract idea limitations (further characterizing extraction via models, the contextual features, computations of scores). For the reasons described above with respect to claim 1, this judicial exception is not meaningfully integrated into a practical application, or significantly more than the abstract idea. Examiner notes that the natural language processing model, a speech recognition model and image recognition model are not specifically recited to be machine learning models, but if intended to be, when recited at this level of breadth, would merely acts to generic computing components to apply the abstract idea.
Claim 6 acts to narrow the previously recited abstract idea limitations (further characterizing computations of scores and predicting likelihood of availability). The recitation of the trained dependency model, when recited at this level of breadth, merely acts as a generic computing component for applying the abstract idea. For the reasons described above with respect to claim 1, this judicial exception is not meaningfully integrated into a practical application, or significantly more than the abstract idea.
Claims 10-11 act to narrow the previously recited abstract idea limitations (further characterizing user selection of results). The characterization of the user interaction being received from the client data merely amounts to extra-solution activity. The Symantec, TLI, OIP Techs. and buySAFE court decisions cited in MPEP 2106.05(d)(II) indicate that mere receiving or transmitting data over a network is a well‐understood, routine, conventional function when it is claimed in a merely generic manner (as it is here). For the reasons described above with respect to claim 1, this judicial exception is not meaningfully integrated into a practical application, or significantly more than the abstract idea.
Claim 12 acts to narrow the previously recited abstract idea limitations (training the models and scoring selections). The use of the trained contextual bandit model, when recited at this level of breadth, merely acts as a generic computing component for applying the abstract idea. For the reasons described above with respect to claim 1, this judicial exception is not meaningfully integrated into a practical application, or significantly more than the abstract idea.
Dependent claims 14-18 recite limitations analogous to claims 2-8 and 10-12 and are thereby rejected for at least the same rationale.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1-8, 10-18 and 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Sirotkovic et al. (US 20190188295) in view of Sravas et al. (US 11392751) in view of McAllister et al. (US 10332181) and further in view of Deng et al. (US 20210224848).
With respect to claim 1, Sirotkovic (US 20190188295) discloses:
1. A computer-implemented method comprising:
receiving, from a client device, a user query for identifying one or more items by an online concierge system, the user query described by one or more query features; (Sirotkovic [0027] FIG. 4 shows an exemplary data flow 400 for processing electronic searches using the trained DNN model 235. In particular, a user may submit a search query 406 and a user ID 408 via a user device 112 (having a user interface for contextual and adaptive searching) to the adaptive search server 401. The adaptive search server 401 may pass the search query to the search engine 104.)
obtaining one or more contextual features describing a context of the user query; (Sirotkovic Fig. 4-5, e.g., search query with user ID)
applying a query processing model to the user query to generate a relevance score for each query result of a plurality of query results; (Sirotkovic [0027] The adaptive search server 401 (i.e. query processing model as claimed) may pass the search query to the search engine 104. The search engine 104 runs a search in response to the search query 406 against the information database 402 using, for example, normal keyword matching, and returns, from the information database 402, a set of search results in the form a set of information items. Each information item may be provided with a rank and a score according to the degree of keyword matching (i.e. relevance score as claimed). )
applying a contextual…model (Fig. 4/5, 235 DNN Model) to the one or more query features and the one or more contextual features (Fig. 4/5, 406/408) to generate a weight vector for ranking parameters used for ranking query results (Fig. 5, 570), wherein the weight vector is applied uniformly to the query results responsive to the user query (Fig. 5 [0028]- [0029] The process for training of the DNN model 235 and the process for using the trained DNN model to obtain adaptive and contextual relevance scores for information items may both be illustrated in view of the data and logic flow 500. [0044] The weight vectors of the fully connected MLP network 570 may be collectively referred to as trainable parameters T.sub.MLP.; [0047] Alternatively, the DNN model parameters including T.sub.word, T.sub.char, T.sub.QIR, T.sub.up, and T.sub.MLP may be trained jointly in a single training process via forward and backward propagations through the networks 510, 520, 540, 560, and 570 of FIG. 5 with joint targets of the query-information item relevance measure 561 and the relevance score 580 – Examiner note: application of a trained DNN model represents the “uniform” application of the model vectors); wherein the ranking parameters include relevance of a query result to the user query (518; [0043] a relevance score 580 for the information item with respect to … user profile 408) and dependability of the query result ( 560/561 based on 529/5050/552/554; see also Sirotkovic [0027], [0029] – [0033], [0043] The weight vectors of the fully connected MLP network 570 may be collectively referred to as trainable parameters T.sub.MLP.; see also [0056] for an example of the combination/analysis of each feature of element 570 )
generating, for each query result and based on the weight vector and ranking parameter values of the query result including the relevance score generated by the query processing model and a dependability score of the query result, a ranking score (Sirtkovic Fig. 4/5, [0043] Finally for FIG. 5, as shown by the data and logic flow 500, the user profile context vector 518, the query context vector 529, the title context vector 552, the description context vector 554, the query-information item relevance measure 561 and the data 503 are input into the MLP network 570 to obtain a relevance score 580 for the information item with respect to the search query 406 and user profile 408. In one implementation, these input vectors to the MLP network 570 may be concatenated into a single vector by a concatenation layer… The weight vectors of the fully connected MLP network 570 may be collectively referred to as trainable parameters T.sub.MLP.) as a weighted sum of the weight vector and the ranking parameter values of the query result (Sirotkovic [0043]-[0044] Finally for FIG. 5, as shown by the data and logic flow 500, the user profile context vector 518, the query context vector 529, the title context vector 552, the description context vector 554, the query-information item relevance measure 561 and the data 503 are input into the MLP network 570 to obtain a relevance score 580 for the information item with respect to the search query 406 and user profile 408. In one implementation, these input vectors to the MLP network 570 may be concatenated into a single vector by a concatenation layer… The weight vectors of the fully connected MLP network 570 may be collectively referred to as trainable parameters T.sub.MLP.); and
transmitting the query results ranked according to the ranking scores for display on the client device. (Sirotkovic [0028] Continuing with FIG. 4 and in one implementation, the relevance scores 420 for a plurality of information items returned by the search engine may be generated by the DNN model 235 and returned to the adaptive search server 401, as shown by the arrow 430. The adaptive search server 430 may order and/or filter the information items according to the relevance scores and control a presentation of the information item to the user device 112, as illustrated by arrow 440. For example, the adaptive search server 430 may present a predetermined number of information items in descending order of the relevance scores to the user.);
Sirtkovic discloses the application of a contextual model, but does not explicitly disclose that the model is a bandit model.
In a similar field of endeavor, Sravas discloses the application of a conceptual bandit mode, wherein the contextual bandit model is a machine-learning model trained by reinforcement learning (Sravas 11392751: (16) col. 3, l. 45 – col. 4, l.5; (79) col. 17, l. 55 – col. 18, l. 10; col. 11, ll. 40-50)
One of ordinary skill in the art at the time of filing would have recognized that applying bandit model of Sravas to the contextual model of Sirtkovic would have yielded predictable results and resulted in improved accuracy of future recommendations via optimized iterations (col. 8, ll. 25-45; col. 17, l. 55 – col. 18, l. 10).
Sravas also further discloses the ranking of search results based on location-based availability considerations, wherein ranking parameters include a likelihood that the query result is available at a particular location (Sravas (83) col. 18, ll. 30-55 Using machine learning-based optimization algorithms such as bandit algorithms as described, it may be possible to identify the right set of informational content elements to help achieve item consumption goals of the inventory owners at desired levels of optimization granularity—e.g., for different demographic or geographic groups of potential item consumers.; Sravas 11392751: (80) col. 18, ll. 5-25 An optimization goal or termination criterion may be identified in some embodiments, indicating the conditions under which the optimization iterations are to be discontinued. Example criteria may include, among others, when the net relative improvements in effectiveness achieved via one or more previous iterations fails to exceed some threshold, or when the absolute number of sales/consumption events for the item in a given timeframe falls below a threshold; see also Sravas (44) col. 10, l. 55- col. 11, l. 5) and shopper selection of a query result for insertion into a shopping cart (e.g, Szarvas col. 13, l. 55 – col. 14, l. 5),
One of ordinary skill in the art at the time of filing would have recognized that geographic availability considerations in the query optimization process of Sravas to the contextual model of Sirtkovic would have yielded predictable results and resulted in the identification of optimized content for presentation the user/shopper.
The combination of Sravas and Sirtovic disclose the ranking of search results based on location-based availability considerations (see above) and shopper selection of a query result for insertion into a shopping cart (e.g, Szarvas col. 13, l. 55 – col. 14, l. 5), but the concept of the cart being associated with a particular location is more explicitly disclosed by McCallister.
McAllister is directed to a system and method for ranking search results and recommendations. Similar to Sravas and Sirtovic, McCallister also discloses a query result ranking process:
wherein the ranking parameters include relevance of a query result to the user query and dependability of the query result representing a likelihood that the query result is available at a particular location (McAllister col. 3 ll. 25 – 35 (15) As is shown in FIG. 1C, upon receiving the search query 102B, the online marketplace may return a list of search results 135B, 125B-2, 125B-1, 125B-3 identifying items which pertain to the search query 102B that are available at either the fulfillment center 130 or accessible to one or more of the merchants 120-1, 120-2, 120-3, and the list of the search results 135B, 125B-2, 125B-1, 125B-3 may be displayed on the smartphone 172. … As is shown in FIG. 1C, the search results 135B, 125B-2, 125B-1, 125B-3 are typically ranked based on their respective relevance to the search query 102B, on a preference of the customer 170 or one or more like customers, e.g., a preferred source, type or kind of item preferred by the customer 170 or the one or more like customers, or on a combination of the relevance of the search results 135B, 125B-2, 125B-1, 125B-3 to the search query 102B and one or more preferences of the customer 170 or like customers.)
McAllister further discloses:
receiving, from the client device, user input adding an item from the query results to an order for fulfillment at the particular location (McAllister col. 3 ll. 25 – 35 (15) As is shown in FIG. 1C, upon receiving the search query 102B, the online marketplace may return a list of search results 135B, 125B-2, 125B-1, 125B-3 identifying items which pertain to the search query 102B that are available at either the fulfillment center 130 or accessible to one or more of the merchants 120-1, 120-2, 120-3, and the list of the search results 135B, 125B-2, 125B-1, 125B-3 may be displayed on the smartphone 172.. The search results 135B, 125B-2, 125B-1, 125B-3 further include features for adding any of the items associated with the respective search results to a virtual shopping cart, or to otherwise execute a purchase for one or more of the items.) ; and
transmitting the order to a second client device associated with a picker, wherein the order is presented on the second client device for fulfillment by the picker (McAllister col. 6, ll. 2-30 When a customer places an order for one of the items through one or more detail pages at the online marketplace, a transaction may be executed between the online marketplace and the customer, and the ordered item may be retrieved from storage, prepared for delivery, and shipped to the customer.; McAllister col. 13, ll. 50 – col. 14, l. 10 (52) The worker 250 may also handle or transport items associated with one or more of the merchants 220-1, 220-2, 220-3. For example, the worker 250 may retrieve one or more items from the storage area 233 or the distribution station 235 and deliver such items to an intended destination, such as the customer 270. Additionally, the worker 250 may travel to one or more of the merchants 220-1, 220-2, 220-3 to retrieve one or more items therefrom, and travel with such items to an intended destination, such as the customer 270. Those of ordinary skill in the pertinent arts will recognize that the worker 250 may retrieve items from multiple sources and deliver such items to an intended destination, such as the customer 270. For example, referring again to FIG. 1A, the worker 150 may retrieve items from one or more of the fulfillment center 130 or the merchants 120-1, 120-2, 120-3 and deliver such items to the customer 170. Those of ordinary skill in the pertinent arts will further recognize that the worker 250 may travel by any means in accordance with his or her duties. For example, the worker 250 may travel not only on foot but also by bicycle, car, truck, boat or aircraft, or any other type or form of vehicle (e.g., a personal transporter).
It would have been obvious to one of ordinary skill in the art at the time of filing to include a location-based product acquisition and delivery system, as disclosed by McAllister, to the product search and selection process as taught by the combination of Sravas and Sirtovic, since the claimed invention is merely a combination of old elements, and in the combination each element merely would have performed the same function as it did separately, and one of ordinary skill in the art would have recognized that the results of the combination were predictable.
Additionally and/or alternatively, it would have been obvious to one of ordinary skill in the art at the time of filing to include a location-based product acquisition and delivery system, as disclosed by McAllister, to the product search and selection process as taught by the combination of Sravas and Sirtovic for the purpose of providing enhanced delivery services on orders for specific items in selected areas (McAllister, col. 1 ll. 25-45).
With respect to the limitation , Sravas discloses the application of a conceptual bandit mode, and very strongly suggests that the model is trained by reinforcement learning. This relationship (wherein the contextual bandit model is a machine-learning model trained by reinforcement learning) is more explicitly disclosed by Deng, which recites that “the model training system 28 is configured to implement a machine learning process using a reinforcement learning mechanism, such as, for example, contextual bandit approach” (see Deng [0035]).
One of ordinary skill in the art at the time of filing would have recognized that applying reinforcement learning to a contextual bandit model, such as the contextual bandit model of Sravas and Sirtovic would have yielded predictable results and resulted in the generation of policies for optimal decision making. In addition, Deng explicitly establishes that contextual bandit models utilize reinforcement learning mechanisms.
With respect to claim 2, the combination of Sirotkovic (US 20190188295), Sravas (US 11392751), McAllister et al. (US 10332181) and Deng et al. (US 20210224848) disclose the limitations of claim 1, and further disclose: wherein the user query comprises text, audio signals, visual signals, or some combination thereof. (Sirotkovic [0027] The search engine 104 runs a search in response to the search query 406 against the information database 402 using, for example, normal keyword matching, and returns, from the information database 402, a set of search results in the form a set of information items; see also Sravas (51) col. 12, ll. 60-68, A number of different types of search tools may be used in different embodiments to submit queries about items in an inventory—e.g., text-based search (with or without auto-fill), image-base search and/or voice-based search may be supported, and each such tool may have a corresponding results interface for which ICE presentation may be optimized. .)
With respect to claim 3, the combination of Sirotkovic (US 20190188295), Sravas (US 11392751), McAllister et al. (US 10332181) and Deng et al. (US 20210224848) disclose the limitations of claim 2, and further disclose: extracting the one or more query features from the user query with a natural language processing model, a speech recognition model, an image recognition model, or some combination thereof. (Sirotkovic [0032] Each search query 406, each information title 502, and each information item description 504 may include a series of words, each containing at least one character, delineated by spaces, punctuations, or other delineating symbols. In the context of a language not based on alphabets, the delineation of words and characters may be predefined in any appropriate manner. For example, Chinese are based on calligraphy comprising collection of symbols each containing strokes. In one implementation for searches based on Chinese, each individual symbol may be equated with a character and common combinations of symbols may be equated with a word. In another alternative implementation, each stroke may be equated with a character whereas each symbol may be equated with a word. For some symbol-based languages, the symbols may not be separated by spaces.; see also Sirotkovic [0037]-[0039]; [0060]; see also Sravas col. 10, ll. 16-60; see also McAllister col. 14, ll. 25-45)
With respect to claim 4, the combination of Sirotkovic (US 20190188295), Sravas (US 11392751), McAllister et al. (US 10332181) and Deng et al. (US 20210224848) disclose the limitations of claim 1, and further disclose: wherein the contextual features comprise: one or more user features describing a user associated with the client device providing the user query; one or more retailer features describing one or more retailers hosted by the online concierge system; one or more item features describing one or more items listed on the online concierge system; or some combination thereof. (Sirotkovic [0027] FIG. 4 shows an exemplary data flow 400 for processing electronic searches using the trained DNN model 235. In particular, a user may submit a search query 406 and a user ID 408 via a user device 112 (having a user interface for contextual and adaptive searching) to the adaptive search server 401. The adaptive search server 401 may pass the search query to the search engine 104. The search engine 104 runs a search in response to the search query 406 against the information database 402 using, for example, normal keyword matching, and returns, from the information database 402, a set of search results in the form a set of information items. Each information item may be provided with a rank and a score according to the degree of keyword matching. The search engine 104 may be further responsible for updating the historical search log 301 with new user-click data. The user ID 408 associated with the search query 406 may be used by the adaptive search server 401 to obtain a user profile 416 from the user profile database 404. The user profile 416, and each information item 410, search engine score 412 and rank 414 for the information item may be input into the trained DNN model 235 to obtain an adaptive and contextual relevance score 420 for the information item 410 with respect to the contextual information in the query and the user profile 416.; see also Sravas Fig. 1, col. 6, ll. 5-40; see also McAllister col. 3, ll. 25-60)
With respect to claim 5, the combination of Sirotkovic (US 20190188295), Sravas (US 11392751), McAllister et al. (US 10332181) and Deng et al. (US 20210224848) disclose the limitations of claim 1, and further disclose: wherein the query processing model generates the relevance score for a query result based on a comparison of item features of the query result and query features of the user query. (Sirotkovic [0027] The adaptive search server 401 may pass the search query to the search engine 104. The search engine 104 runs a search in response to the search query 406 against the information database 402 using, for example, normal keyword matching, and returns, from the information database 402, a set of search results in the form a set of information items. Each information item may be provided with a rank and a score according to the degree of keyword matching.; Sirotkovic [0010] An electronic search engine may be configured to compare keywords in a user-generated search query with information sources or indexes of information sources to extract matching information items.)
With respect to claim 6, the combination of Sirotkovic (US 20190188295), Sravas (US 11392751), McAllister et al. (US 10332181) and Deng et al. (US 20210224848) disclose the limitations of claim 1, and further disclose: wherein the dependability score of the query result is generated by a dependability model (Sirotkovic 560/561 based on 529/5050/552/554; see also Sirotkovic [0027], [0029] – [0033], [0043]).
While Sirotkovic discloses a dependability score, which draws on inventory information, Sravas more explicitly discloses that the dependability model can be trained to predict likelihood that the query result is available at a particular location (Sravas (83) col. 18, ll. 30-55 Using machine learning-based optimization algorithms such as bandit algorithms as described, it may be possible to identify the right set of informational content elements to help achieve item consumption goals of the inventory owners at desired levels of optimization granularity—e.g., for different demographic or geographic groups of potential item consumers.; Sravas 11392751: (80) col. 18, ll. 5-25 An optimization goal or termination criterion may be identified in some embodiments, indicating the conditions under which the optimization iterations are to be discontinued. Example criteria may include, among others, when the net relative improvements in effectiveness achieved via one or more previous iterations fails to exceed some threshold, or when the absolute number of sales/consumption events for the item in a given timeframe falls below a threshold; see also Sravas (44) col. 10, l. 55- col. 11, l. 5).
One of ordinary skill in the art would have recognized that applying training for the location availability of Sravas to the contextual model of Sirtkovic would have yielded predictable results and resulted in improved accuracy of future recommendations via optimized iterations that consider consumption events in a particular timeframe (col. 8, ll. 25-45; col. 17, l. 55 – col. 18, l. 5; col. 10, l. 55 - col. 11, l. 5)
In addition, see the McAllister disclosure in claim 1 regarding the consideration of availability at a location (McAllister col. 3 ll. 25 – 35 (15) As is shown in FIG. 1C, upon receiving the search query 102B, the online marketplace may return a list of search results 135B, 125B-2, 125B-1, 125B-3 identifying items which pertain to the search query 102B that are available at either the fulfillment center 130 or accessible to one or more of the merchants 120-1, 120-2, 120-3, and the list of the search results 135B, 125B-2, 125B-1, 125B-3 may be displayed on the smartphone 172. … As is shown in FIG. 1C, the search results 135B, 125B-2, 125B-1, 125B-3 are typically ranked based on their respective relevance to the search query 102B, on a preference of the customer 170 or one or more like customers, e.g., a preferred source, type or kind of item preferred by the customer 170 or the one or more like customers, or on a combination of the relevance of the search results 135B, 125B-2, 125B-1, 125B-3 to the search query 102B and one or more preferences of the customer 170 or like customers.)
With respect to claim 7, the combination of Sirotkovic (US 20190188295), Sravas (US 11392751), McAllister et al. (US 10332181) and Deng et al. (US 20210224848) disclose the limitations of claim 1, and further disclose: wherein the ranking parameters further include: popularity of the query result; or rating of the query result. (Sirotkovic [0023]-[0024] The input data or data corpus 330 to the DNN model optimizer 238 may include historical search queries 302, historical search results 304 returned from the search engine in response to the historical search queries, historical search engine scores 306 and ranks 308 for the historical search results, user profiles 310 for users issuing the historical queries to the search engine, and historical user-click data 312 by users with respect to the search results.; [0031] The input to the logic and data flow 500 includes user profile 408, search query 406, and information item represented by information item title 502, information item description 504, and data 503 containing information item search engine rank/score/positive click data/negative click data).
With respect to claim 8, the combination of Sirotkovic (US 20190188295), Sravas (US 11392751), McAllister et al. (US 10332181) and Deng et al. (US 20210224848) disclose the limitations of claim 7, and further disclose: wherein the popularity of the query result or the rating of the query result are based on past user interactions with the query result. (Sirotkovic [0023]-[0024] The input data or data corpus 330 to the DNN model optimizer 238 may include historical search queries 302, historical search results 304 returned from the search engine in response to the historical search queries, historical search engine scores 306 and ranks 308 for the historical search results, user profiles 310 for users issuing the historical queries to the search engine, and historical user-click data 312 by users with respect to the search results.; [0031] The input to the logic and data flow 500 includes user profile 408, search query 406, and information item represented by information item title 502, information item description 504, and data 503 containing information item search engine rank/score/positive click data/negative click data.)
With respect to claim 10, the combination of Sirotkovic (US 20190188295), Sravas (US 11392751), McAllister et al. (US 10332181) and Deng et al. (US 20210224848) disclose the limitations of claim 1, and further disclose: receiving, from the client device, additional user input interacting with an item from the query results. (Sirotkovic [0023]-[0024] The input data or data corpus 330 to the DNN model optimizer 238 may include historical search queries 302, historical search results 304 returned from the search engine in response to the historical search queries, historical search engine scores 306 and ranks 308 for the historical search results, user profiles 310 for users issuing the historical queries to the search engine, and historical user-click data 312 by users with respect to the search results.; [0031] The input to the logic and data flow 500 includes user profile 408, search query 406, and information item represented by information item title 502, information item description 504, and data 503 containing information item search engine rank/score/positive click data/negative click data.; see also McAllister col. 3 ll. 25 – 35 (15) As is shown in FIG. 1C, upon receiving the search query 102B, the online marketplace may return a list of search results 135B, 125B-2, 125B-1, 125B-3 identifying items which pertain to the search query 102B that are available at either the fulfillment center 130 or accessible to one or more of the merchants 120-1, 120-2, 120-3, and the list of the search results 135B, 125B-2, 125B-1, 125B-3 may be displayed on the smartphone 172.. The search results 135B, 125B-2, 125B-1, 125B-3 further include features for adding any of the items associated with the respective search results to a virtual shopping cart, or to otherwise execute a purchase for one or more of the items)
With respect to claim 11, the combination of Sirotkovic (US 20190188295), Sravas (US 11392751), McAllister et al. (US 10332181) and Deng et al. (US 20210224848) disclose the limitations of claim 10, and further disclose: wherein the user input comprises at least one of: viewing an item from the query results; adding the item to a shopping cart; favoriting the item; or ordering the item. (Sirotkovic [0023]-[0024] The input data or data corpus 330 to the DNN model optimizer 238 may include historical search queries 302, historical search results 304 returned from the search engine in response to the historical search queries, historical search engine scores 306 and ranks 308 for the historical search results, user profiles 310 for users issuing the historical queries to the search engine, and historical user-click data 312 by users with respect to the search results.; [0031] The input to the logic and data flow 500 includes user profile 408, search query 406, and information item represented by information item title 502, information item description 504, and data 503 containing information item search engine rank/score/positive click data/negative click data. ; see also Sravas 11392751: (22) col. 5, ll. 45-55, In various embodiments, different types of effectiveness or utility metrics may be generated for ICEs at the machine learning models used (e.g., either during the iterative optimization phase, or during the identification of the baseline sets of ICEs). Such metrics may include, for example, web link click count metrics, sales metrics, shopping cart insertion metrics, wish list insertion metrics, and/or session engagement length metrics.).
Sirotkovic discloses an interaction with a query result, which strongly suggests viewing an item from the query results. This concept is more explicitly disclosed by Sravas, as shown above.
One of ordinary skill in the art would have recognized that applying additional interaction types of Sravas to user interactions of Sirtkovic would have yielded predictable results and resulted in improved accuracy of future recommendations identification of prioritized contexts (col 5, ll. 40-50)
With respect to claim 12, the combination of Sirotkovic (US 20190188295), Sravas (US 11392751), McAllister et al. (US 10332181) and Deng et al. (US 20210224848) disclose the limitations of claim 10, and further disclose: scoring a reward based on the user input; and training the contextual bandit model based on the reward. (Sravas 11392751: (16) col. 3, l. 45 – col., 4, l. 5, The exploration of the presentation space and the associated optimization may be performed in some embodiments using so-called “bandit” machine learning models and algorithms, such as a contextual bandit algorithm or other similar state-dependent adaptive trial algorithms. In a contextual bandit algorithm which may be employed in one embodiment, a multi-dimensional feature vector called a context vector may be generated from the available ICE choices. When making a particular choice for ICE recommendations in a given optimization iteration, the context vector and the rewards/losses corresponding to past choices may be analyzed, with the tradeoffs between exploration (the extent of changes made to the previously-presented sets of ICEs) and exploitation (maximization of rewards) being taken into account in such an embodiment. Over time, the bandit algorithm may examine enough information obtained from the ongoing collection of records of interactions with the ICEs to learn relationships among the rewards and the context vector elements, and may therefore be able to select more effective ICEs quickly and efficiently in various embodiments. Other optimization algorithms, including for example neural network-based reinforcement learning algorithms/models, may be employed)
With respect to claims 13-18, the limitations are analogous to the limitations of claims 1-8 and 10-12. See the rejections of claims 1-8 and 10-12 above. In addition, Sirotkovic discloses a non-transitory computer-readable storage medium storing instructions that, when executed by a computer processor, cause the computer processor to perform operations (Sirotkovic Fig. 1, 2, [0072]-[0073])
With respect to claim 20, the limitations are analogous to the limitations of claim 1. See the rejection of claims 1 above. In addition, Sirotkovic discloses a computer processor; and a non-transitory computer-readable storage medium storing instructions that, when executed by the computer processor, cause the computer processor to perform operations (Sirotkovic Fig. 1, 2, [0072]-[0073]).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure: Garner et al. (US 20220277382).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ABBY J FLYNN whose telephone number is (571)272-9855. The examiner can normally be reached Monday - Friday 8:30-5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, James Trammel can be reached at 571-272-6763. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ABBY J FLYNN/ Examiner, Art Unit 3663
Prosecution Insights