CTNF 19/029,686 CTNF 99430 DETAILED ACTION 12-151 AIA 26-51 12-51 Status of Claims Claims 1-20 submitted on 01/17/2025 are pending and have been examined. Notice of Pre-AIA or AIA Status 07-03-aia AIA 15-10-aia The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA. Priority Acknowledgment is made of applicant’s provisional application. The certified copy has been filed in parent Application No. 63/627,524, filed on 01/31/2024. Information Disclosure Statement The information disclosure statement (IDS) submitted on 06/04/2025 has been considered by the examiner. Claim Rejections - 35 USC § 101 07-04-01 AIA 07-04 35 U.S.C. 101 reads as follows: Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title. Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to a judicial exception without significantly more. The claims recite an abstract idea. This judicial exception is not integrated into a practical application. The claim(s) do not include additional elements that are sufficient to amount to significantly more than the judicial exception . Step 1 Claims 1-9 are directed to a machine, claims 10-16 are directed to a process, and claims 17-20 are directed to an article of manufacture (see MPEP 2106.03). Step 2A, Prong 1 Claim 1, taken as representative, recites at least the following limitations that recite an abstract idea: a system, comprising: receive a recommendation request for recommending items to a customer, determine an anchor image based on the recommendation request, generate at least one query based on the anchor image, generate, using a language model, textual recommendation data based on the at least one query, generate, using at least one model, at least one ranked list of recommended items based on the textual recommendation data, and transmit the at least one ranked list of recommended items to be displayed to the customer. The above limitation, under its broadest reasonable interpretation, falls within the “Certain Methods of Organizing Human Activity” grouping of abstract ideas, enumerated in MPEP 2106.04(a)(2)(II), in that it recites a commercial interaction. Claims 10 and 17 recites similar limitations as claim 1. Thus, under Prong 1 of Step 2A, claims 10 and 17 recite an abstract idea. Step 2A, Prong 2 Claim 1 includes the following additional elements that are bolded: a system, comprising: a processor; and a non-transitory memory storing instructions, that when executed, cause the processor to : receive a recommendation request for recommending items to a customer, determine an anchor image based on the recommendation request, generate at least one query based on the anchor image, generate, using a language model, textual recommendation data based on the at least one query, generate, using at least one machine learning model, at least one ranked list of recommended items based on the textual recommendation data, and transmit to a computing device the at least one ranked list of recommended items to be displayed to the customer. Claims 10 and 17 include the same additional elements as claim 1. The additional elements recited in claims 1, 10, and 17 merely invoke such elements as a tool to perform the abstract idea and generally link the use of the abstract idea to a particular technological environment of machine learning models and computers (see MPEP 2106.05(f) and MPEP 2106.05(h). These additional elements are described at a high level in Applicant’s specification without any meaningful detail about their structure or configuration (see ¶0023, ¶¶0036-0037, and Fig. 1). As such, under Prong 2 of Step 2A, when considered both individually and as a whole, the additional elements do not integrate the judicial exception into a practical application and, thus, claims 1, 10, and 17 are directed to an abstract idea. Step 2B As noted above, while the recitation of the additional elements in independent claims 1, 10, and 17 are acknowledged, claims 1, 10, and 17 merely invoke such additional elements as a tool to perform the abstract idea and generally link the use of the abstract idea to a particular technological environment (see MPEP 2106.05(f) and MPEP 2106.05(h)). Even when considered as an ordered combination, the additional elements of claim 1, 10, and 17 do not add anything that is not already present when they are considered individually. Therefore, under Step 2B, there are no meaningful limitations in claims 1, 10, and 17 that transform the judicial exception into a patent eligible application such that the claims amount to significantly more than the judicial exception itself (see MPEP 2106.05). As such, independent claims 1, 10, and 17 are ineligible. Dependent claims 3, 4, 6, 9, 11, 12, 14, 18, and 19 when analyzed as a whole, are held to be patent ineligible under 35 U.S.C. 101 because they do not add “significantly more” to the abstract idea. More specifically, dependent claims 3, 4, 6, 9, 11, 12, 14, 18, and 19 merely further define the abstract limitations of claims 1, 10, and 17 or provide further embellishments of the limitations recited in independent claims 1, 10, and 17. Claims 3, 4, 6, 9, 11, 12, 14, 18, and 19 do not introduce any further additional elements. Thus, dependent claims 3, 4, 6, 9, 11, 12, 14, 18, and 19 are ineligible. Furthermore, it is noted that certain dependent claims recite additional elements supplemental to those recited in independent claims 1, 10, and 17: a webpage (claim 2), large language model (claims 5, 13, and 20), and database (claims 7, 8, 15, and 16). However, these elements do not integrate the abstract idea into a practical application because they merely amount to using a computer to apply the abstract idea to a particular technological environment or field of use and thus do not act to integrate the abstract idea into a practical application of the abstract idea. Additionally, the additional elements do not amount to significantly more because they merely amount to using a computer to apply the abstract idea and amount to no more than a general link of the use of the abstract idea to a particular technological environment. Thus, dependent claims 2, 5, 7, 8, 13, 15, 16, and 20 are ineligible. Claim Rejections - 35 USC § 103 07-06 AIA 15-10-15 In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status. 07-20-02-aia AIA This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention. 07-20-aia AIA The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action: A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made. 07-21-aia AIA Claim (s) 1-7, 9-15, and 17-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Grossman et al. (US 2018/0197223 A1) in view of Rothschild-Keita et al. (US 2024/0420209 A1) . Regarding Claim 1 , Grossman et al., hereinafter, Grossman, discloses a system, comprising: a processor; and a non-transitory memory storing instructions, that when executed, cause the processor to (Fig. 1; ¶¶0031-0032[The back-end components 104 may include one or more servers 140 communicatively connected to the network 130 by a link 135. Each server 140 may include one or more processors 162 adapted and configured to execute various software applications and components of the system… The controller 150 may include a program memory 160, a processor 162, a RAM 164, and an I/O circuit 166, all of which may be interconnected via an address/data bus 165]): receive a recommendation request for recommending items to a customer (Figs. 3 and 10; ¶0036[A user of the client computing device 110 may receive or access images from a web site 210, a media feed 220, or a message server 230… The user may select such an image or portion of such an image to be searched to identify objects and associated products or services.] in view of ¶¶0091-0092), determine an anchor image based on the recommendation request (Fig. 10; ¶0092[The method 1000 may begin with obtaining an electronic image to be processed (block 1002)]), generate at least one query based on the anchor image (Figs. 3 and 10; ¶0042[At block 304, in some embodiments, the client computing device 110 may receive a user indication of a location within the image associated with an object of interest within the image. If no user indication is received, the method 300 may continue by identifying one or more objects within the electronic image and providing information regarding each of the identified objects] in view of ¶¶0091-0100[FIG. 10 illustrates a flow diagram of an exemplary service identification method 1000 for identifying purchase options regarding services associated with objects in electronic images. The method 1000 may be implemented using the components of the object identification system 100, either separately or in conjunction with any of the other methods disclosed herein.]), generate, textual recommendation data based on the at least one query (Figs. 3 and 10; ¶¶0091-0100[At block 1002, the server 140 may obtain an electronic image indicated by a user of a client computing device 110. Upon receiving the image, the server 140 may further identify keywords associated with the image at block 1004. Identifying a set of keywords may include obtaining labels associated with the keywords from one or more image services 240, as discussed elsewhere herein.] in view of ¶¶0046-0049; Examiner interprets “keywords derived from image object labels” as “textual recommendation data”), generate, using at least one machine learning model, at least one ranked list of recommended items based on the textual recommendation data (Fig. 10; ¶0098[At block 1010, the server 140 may generate a list of service purchase options from the vendor site search results. The list may include a recommendation level or ranking of the services, which may reflect the projected relevance of the service based upon the identified keywords] in view of ¶0067[Keyword filtering may include the application of rules or algorithms identified by machine learning techniques to remove low-quality search terms or to identify high-quality search terms. Such rules or algorithms may be generated by training the machine learning algorithms using the results of previous implementations of the method 800.] and ¶0048[The keywords obtained by analysis of the image may be returned to the client computing device 110 or may be used by the server 140 to further search for related products or services. If the keywords are returned to the client computing device 110, some or all of the keywords may be presented to the user to select from alternative categories of products or to select between a plurality of objects identified within the image.]), and transmit to a computing device the at least one ranked list of recommended items to be displayed to the customer (¶0099[At block 1012, the server 140 may send one or more service purchase options from the list to the client computing device 110 for presentation to the user (e.g., via the display 112)]). Although Grossman discloses textual recommendation data, Grossman does not explicitly disclose using a language model . However, Rothschild-Keita et al., hereinafter, Rothschild-Keita, teaches using a language model (Fig. 1; ¶¶0031-0036[In one or more embodiments, the language models are large language models (LLMs) that are trained on a large corpus of training data to generate outputs for the NLP tasks… The LLM may be pre-trained by the online concierge system 140 or one or more entities different from the online concierge system 140… The online concierge system 140 prepares a first prompt for input to the model serving system 150] in view of ¶0069[The content presentation module 210 causes a device of the customer (e.g., the customer client device 100) to display a user interface with one or more list of items for presentation to the customer… The list feature module 250 may rank each list of items based on a likelihood of conversion of that list of items by the customer (e.g., determined by the computer model deployed by the list feature module 250). The content presentation module 210 may present the list of items to the customer based on the ranking of each list of items. For example, a highest ranked list of items would be presented on the top and a lowest ranked list of items would be presented on the bottom] and ¶0086[to display a user interface with the one or more lists of items.]). The system of Rothschild-Keita is applicable to the system of Grossman as they share characteristics and capabilities, namely, they are both targeted to item recommendation. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the textual recommendation data as disclosed by Grossman to include a large language model and displaying ranked lists of items as taught by Rothschild-Keita. One of ordinary skill in the art would have been motivated to expand the system of Grossman in order to create time and/or opportunity for the customers to discover new items (¶0001). Regarding Claim 2 , Grossman in view of Rothschild-Keita teaches the system of claim 1, Grossman further discloses wherein the anchor image is at least one of: an image uploaded by the customer (); or an image of a product item with which the customer interacted on a webpage (¶0036[The input data sources may provide electronic images including objects to be identified. A user of the client computing device 110 may receive or access images from a web site 210, a media feed 220, or a message server 230. For example, the user may select an image containing an object of interest from a web site 210, such as an image accompanying a news story. Likewise, the user may select an image in a media feed 220, such as a social media site feed associated with the user.]; Examiner notes that the limitations are recited in the alternate and thus only one need be taught or disclosed). Regarding Claim 3 , Grossman in view of Rothschild-Keita teaches the system of claim 1, Grossman further discloses wherein the at least one query is generated based on: converting the anchor image to textual data (Figs. 3 and 10; ¶¶0091-0100[At block 1002, the server 140 may obtain an electronic image indicated by a user of a client computing device 110. Upon receiving the image, the server 140 may further identify keywords associated with the image at block 1004. Identifying a set of keywords may include obtaining labels associated with the keywords from one or more image services 240, as discussed elsewhere herein.] in view of ¶¶0046-0049; Examiner interprets “keywords derived from image object labels” as “textual data”); generating a caption describing the anchor image (Figs. 3 and 10; ¶¶0091-0100[At block 1002, the server 140 may obtain an electronic image indicated by a user of a client computing device 110. Upon receiving the image, the server 140 may further identify keywords associated with the image at block 1004. Identifying a set of keywords may include obtaining labels associated with the keywords from one or more image services 240, as discussed elsewhere herein.] in view of ¶¶0046-0049; Examiner interprets “keywords derived from image object labels” as a “caption describing the anchor image”); obtaining an input question from the customer (Figs. 3 and 10; ¶0036[A user of the client computing device 110 may receive or access images from a web site 210, a media feed 220, or a message server 230… The user may select such an image or portion of such an image to be searched to identify objects and associated products or services.] in view of ¶0037[one or more search engines 250 may be queried to identify vendor servers 260, vendor web sites, vendor media streams, vendor databases or other commercial data sources providing information regarding products or services associated with an object in an image based upon keywords derived from analysis of the image]; Examiner interprets the input image as an “input question from the customer”); and generate the at least one query based on: the textual data, the caption and the input question (Figs. 3 and 10; ¶¶0091-0100[At block 1002, the server 140 may obtain an electronic image indicated by a user of a client computing device 110. Upon receiving the image, the server 140 may further identify keywords associated with the image at block 1004. Identifying a set of keywords may include obtaining labels associated with the keywords from one or more image services 240, as discussed elsewhere herein.] in view of ¶0037[one or more search engines 250 may be queried to identify vendor servers 260, vendor web sites, vendor media streams, vendor databases or other commercial data sources providing information regarding products or services associated with an object in an image based upon keywords derived from analysis of the image] in view of ¶¶0046-0049). Regarding Claim 4 , Grossman in view of Rothschild-Keita teaches the system of claim 1, Grossman further discloses wherein: the at least one query includes a plurality of questions for generating the textual recommendation data (Figs. 3 and 10; ¶¶0091-0100[At block 1002, the server 140 may obtain an electronic image indicated by a user of a client computing device 110. Upon receiving the image, the server 140 may further identify keywords associated with the image at block 1004. Identifying a set of keywords may include obtaining labels associated with the keywords from one or more image services 240, as discussed elsewhere herein.] in view of ¶0037[one or more search engines 250 may be queried to identify vendor servers 260, vendor web sites, vendor media streams, vendor databases or other commercial data sources providing information regarding products or services associated with an object in an image based upon keywords derived from analysis of the image] in view of ¶¶0046-0049); and the plurality of questions are generated according to different types of recommendations associated with the recommendation request (Figs. 3 and 10; ¶¶0091-0100[At block 1002, the server 140 may obtain an electronic image indicated by a user of a client computing device 110. Upon receiving the image, the server 140 may further identify keywords associated with the image at block 1004. Identifying a set of keywords may include obtaining labels associated with the keywords from one or more image services 240, as discussed elsewhere herein.] in view of ¶0037[one or more search engines 250 may be queried to identify vendor servers 260, vendor web sites, vendor media streams, vendor databases or other commercial data sources providing information regarding products or services associated with an object in an image based upon keywords derived from analysis of the image] in view of ¶¶0046-0049). Although Grossman discloses a query with questions for generating recommendations, Grossman does not explicitly disclose prompt questions to be input into the language model and plurality of prompt questions. However, Rothschild-Keita teaches prompts into a language model and a plurality of prompts (Fig. 1; ¶¶0031-0036[In one or more embodiments, the language models are large language models (LLMs) that are trained on a large corpus of training data to generate outputs for the NLP tasks… The LLM may be pre-trained by the online concierge system 140 or one or more entities different from the online concierge system 140… The online concierge system 140 prepares a first prompt for input to the model serving system 150] in view of ¶0069[The content presentation module 210 causes a device of the customer (e.g., the customer client device 100) to display a user interface with one or more list of items for presentation to the customer… The list feature module 250 may rank each list of items based on a likelihood of conversion of that list of items by the customer (e.g., determined by the computer model deployed by the list feature module 250). The content presentation module 210 may present the list of items to the customer based on the ranking of each list of items. For example, a highest ranked list of items would be presented on the top and a lowest ranked list of items would be presented on the bottom] and ¶0086[to display a user interface with the one or more lists of items.]). The system of Rothschild-Keita is applicable to the system of Grossman as they share characteristics and capabilities, namely, they are both targeted to item recommendation. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the textual recommendation data as disclosed by Grossman to include a large language model and displaying ranked lists of items as taught by Rothschild-Keita. One of ordinary skill in the art would have been motivated to expand the system of Grossman in order to create time and/or opportunity for the customers to discover new items (¶0001). Regarding Claim 5 , Grossman in view of Rothschild-Keita teaches the system of claim 4, Grossman further discloses wherein: the textual recommendation data is generated based on the at least one query and product textual data on a product item page (Figs. 3 and 10; ¶¶0091-0100[At block 1002, the server 140 may obtain an electronic image indicated by a user of a client computing device 110. Upon receiving the image, the server 140 may further identify keywords associated with the image at block 1004. Identifying a set of keywords may include obtaining labels associated with the keywords from one or more image services 240, as discussed elsewhere herein.] in view of ¶0037[one or more search engines 250 may be queried to identify vendor servers 260, vendor web sites, vendor media streams, vendor databases or other commercial data sources providing information regarding products or services associated with an object in an image based upon keywords derived from analysis of the image] in view of ¶¶0046-0049); and the textual recommendation data includes different data for different questions corresponding to the different types of recommendations (¶0042[For example, the user may be interested in only one or a few objects within an image showing a number of objects, such as a particular shirt within an image showing multiple people each wearing different types of clothing. As another example, the user may be interested only in one type of product from among a plurality of products on a shelf.] in view of ¶0047[each image may be analyzed by a plurality of algorithms associated with a plurality of image services 240 to evaluate different aspects of the image. For example, separate image services 240 may be used to analyze the image with respect to identifying locations or landmarks, text within the image, object types, faces within the image, or logos within the image.]). Although Grossman discloses textual recommendation data, Grossman does not explicitly disclose the language model is a large language model pre-trained based on volumes of natural language data and different prompt questions. However, Rothschild-Keita teaches a large language model pre-trained on natural language data and prompts (Fig. 1; ¶¶0031-0036[In one or more embodiments, the language models are large language models (LLMs) that are trained on a large corpus of training data to generate outputs for the NLP tasks… The LLM may be pre-trained by the online concierge system 140 or one or more entities different from the online concierge system 140… The online concierge system 140 prepares a first prompt for input to the model serving system 150] in view of ¶0069[The content presentation module 210 causes a device of the customer (e.g., the customer client device 100) to display a user interface with one or more list of items for presentation to the customer… The list feature module 250 may rank each list of items based on a likelihood of conversion of that list of items by the customer (e.g., determined by the computer model deployed by the list feature module 250). The content presentation module 210 may present the list of items to the customer based on the ranking of each list of items. For example, a highest ranked list of items would be presented on the top and a lowest ranked list of items would be presented on the bottom] and ¶0086[to display a user interface with the one or more lists of items.]). The system of Rothschild-Keita is applicable to the system of Grossman as they share characteristics and capabilities, namely, they are both targeted to item recommendation. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the textual recommendation data as disclosed by Grossman to include a large language model and displaying ranked lists of items as taught by Rothschild-Keita. One of ordinary skill in the art would have been motivated to expand the system of Grossman in order to create time and/or opportunity for the customers to discover new items (¶0001). Regarding Claim 6 , Grossman in view of Rothschild-Keita teaches the system of claim 4, Grossman further discloses wherein: the at least one ranked list of recommended items includes recommended items (Fig. 10; ¶0098[At block 1010, the server 140 may generate a list of service purchase options from the vendor site search results. The list may include a recommendation level or ranking of the services, which may reflect the projected relevance of the service based upon the identified keywords]); each ranked list of recommended items is generated for a prompt question corresponding to a respective type of the different types of recommendations (Fig. 10; ¶0098[At block 1010, the server 140 may generate a list of service purchase options from the vendor site search results. The list may include a recommendation level or ranking of the services, which may reflect the projected relevance of the service based upon the identified keywords] in view of ¶¶0042-0047[¶0042[For example, the user may be interested in only one or a few objects within an image showing a number of objects, such as a particular shirt within an image showing multiple people each wearing different types of clothing. As another example, the user may be interested only in one type of product from among a plurality of products on a shelf.]]); and each ranked list of recommended items is to be displayed in a respective carousel of a user interface showing the anchor image to the customer (¶¶0098-0099[At block 1010, the server 140 may generate a list of service purchase options from the vendor site search results. The list may include a recommendation level or ranking of the services, which may reflect the projected relevance of the service based upon the identified keywords… At block 1012, the server 140 may send one or more service purchase options from the list to the client computing device 110 for presentation to the user (e.g., via the display 112). The purchase options may be presented in an order based upon relevance or location or a combination thereof]). Although Grossman discloses recommending items, Grossman does not explicitly disclose a plurality of ranked lists of items displaying on the same user interface showing an item. However, Rothschild-Keita teaches ranked lists of items being displayed on the same user interface (Fig. 1; ¶¶0031-0036[In one or more embodiments, the language models are large language models (LLMs) that are trained on a large corpus of training data to generate outputs for the NLP tasks… The LLM may be pre-trained by the online concierge system 140 or one or more entities different from the online concierge system 140… The online concierge system 140 prepares a first prompt for input to the model serving system 150] in view of ¶0069[The content presentation module 210 causes a device of the customer (e.g., the customer client device 100) to display a user interface with one or more list of items for presentation to the customer… The list feature module 250 may rank each list of items based on a likelihood of conversion of that list of items by the customer (e.g., determined by the computer model deployed by the list feature module 250). The content presentation module 210 may present the list of items to the customer based on the ranking of each list of items. For example, a highest ranked list of items would be presented on the top and a lowest ranked list of items would be presented on the bottom] and ¶0086[to display a user interface with the one or more lists of items.]). The system of Rothschild-Keita is applicable to the system of Grossman as they share characteristics and capabilities, namely, they are both targeted to item recommendation. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the textual recommendation data as disclosed by Grossman to include a large language model and displaying ranked lists of items as taught by Rothschild-Keita. One of ordinary skill in the art would have been motivated to expand the system of Grossman in order to create time and/or opportunity for the customers to discover new items (¶0001). Regarding Claim 7 , Grossman in view of Rothschild-Keita teaches the system of claim 6, Grossman further discloses wherein each ranked list of recommended items is generated based on: searching an item database to retrieve a plurality of items by comparing the textual recommendation data to item texts in the item database, wherein (Fig. 1; ¶0027[The server 140 may store and retrieve data within a database 146.] in view of ¶¶0033-0040[The data sources 170 may include public or proprietary databases including information that may be associated with objects in images, such as labels identifying objects previously identified within images. For example, a data source 170 may include information for a reverse image search that may be used to identify text labels for an image or portion thereof. The one or more data sources 170 may further include databases or servers associated with vendors, providing information regarding products or services… comparison of the labels associated with the various images to determine the most salient keywords.] and ¶0093[Identifying a set of keywords may include obtaining labels associated with the keywords from one or more image services 240]): each of the plurality of items is associated with a relevance indicating a degree of relevance between the item and the textual recommendation data based on the respective type of recommendation (Fig. 10; ¶0098[At block 1010, the server 140 may generate a list of service purchase options from the vendor site search results. The list may include a recommendation level or ranking of the services, which may reflect the projected relevance of the service based upon the identified keywords] in view of ¶0048[The keywords obtained by analysis of the image may be returned to the client computing device 110 or may be used by the server 140 to further search for related products or services. If the keywords are returned to the client computing device 110, some or all of the keywords may be presented to the user to select from alternative categories of products or to select between a plurality of objects identified within the image.]), and the relevance associated with the plurality of items are higher than a predetermined threshold (¶0087[A match may be determined to exist when the probability exceeds a matching threshold (e.g., 60% or 80%). If no match is identified by the server 140, the method 900 may terminate]); and ranking the plurality of items according to their respective relevance to generate the ranked list of recommended items (Fig. 10; ¶0098[At block 1010, the server 140 may generate a list of service purchase options from the vendor site search results. The list may include a recommendation level or ranking of the services, which may reflect the projected relevance of the service based upon the identified keywords] in view of ¶0048[The keywords obtained by analysis of the image may be returned to the client computing device 110 or may be used by the server 140 to further search for related products or services. If the keywords are returned to the client computing device 110, some or all of the keywords may be presented to the user to select from alternative categories of products or to select between a plurality of objects identified within the image]). Although Grossman discloses a degree of relevance between an item and textual recommendation data, Grossman does not explicitly disclose a relevance score , the relevance scores , and the relevance scores generating recommendations. However, Rothschild-Keita teaches scoring items (Fig. 2; ¶¶0049-0052[For example, the content presentation module 210 may score items and rank the items based on their scores. The content presentation module 210 displays the items with scores that exceed some threshold (e.g., the top n items or the p percentile of items)]). The system of Rothschild-Keita is applicable to the system of Grossman as they share characteristics and capabilities, namely, they are both targeted to item recommendation. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the textual recommendation data as disclosed by Grossman to include scoring items as taught by Rothschild-Keita. One of ordinary skill in the art would have been motivated to expand the system of Grossman in order to create time and/or opportunity for the customers to discover new items (¶0001). Regarding Claim 9 , Grossman in view of Rothschild-Keita teaches the system of claim 1, Grossman further discloses wherein the instructions, when executed, further cause the processor to: generate a feedback signal based on the textual recommendation data (Fig. 8; ¶¶0076-0077[The dataset may be separately generated for each of a plurality of searches, or the dataset may be iteratively updated with additional purchase option data from subsequent searches. As noted above, the purchase option data identified from each search of the vendor sites may be used to compare alternative sets of keywords… When multiple subsets of keywords are identified, as discussed above, separate or cumulative datasets may be generated or updated during each iteration]); and update, based on the feedback signal, at least one of: the anchor image or the at least one query (Fig. 8; ¶¶0076-0077[The dataset may be separately generated for each of a plurality of searches, or the dataset may be iteratively updated with additional purchase option data from subsequent searches. As noted above, the purchase option data identified from each search of the vendor sites may be used to compare alternative sets of keywords… When multiple subsets of keywords are identified, as discussed above, separate or cumulative datasets may be generated or updated during each iteration]). Regarding Claim 10 , Grossman discloses a computer-implemented method, comprising : receiving a recommendation request for recommending items to a customer (Figs. 3 and 10; ¶0036[A user of the client computing device 110 may receive or access images from a web site 210, a media feed 220, or a message server 230… The user may select such an image or portion of such an image to be searched to identify objects and associated products or services.] in view of ¶¶0091-0092); determining an anchor image based on the recommendation request (Fig. 10; ¶0092[The method 1000 may begin with obtaining an electronic image to be processed (block 1002)]); generating at least one query based on the anchor image (Figs. 3 and 10; ¶0042[At block 304, in some embodiments, the client computing device 110 may receive a user indication of a location within the image associated with an object of interest within the image. If no user indication is received, the method 300 may continue by identifying one or more objects within the electronic image and providing information regarding each of the identified objects] in view of ¶¶0091-0100[FIG. 10 illustrates a flow diagram of an exemplary service identification method 1000 for identifying purchase options regarding services associated with objects in electronic images. The method 1000 may be implemented using the components of the object identification system 100, either separately or in conjunction with any of the other methods disclosed herein.]); generating, textual recommendation data based on the at least one query (Figs. 3 and 10; ¶¶0091-0100[At block 1002, the server 140 may obtain an electronic image indicated by a user of a client computing device 110. Upon receiving the image, the server 140 may further identify keywords associated with the image at block 1004. Identifying a set of keywords may include obtaining labels associated with the keywords from one or more image services 240, as discussed elsewhere herein.] in view of ¶¶0046-0049; Examiner interprets “keywords derived from image object labels” as “textual recommendation data”); generating, using at least one machine learning model, at least one ranked list of recommended items based on the textual recommendation data (Fig. 10; ¶0098[At block 1010, the server 140 may generate a list of service purchase options from the vendor site search results. The list may include a recommendation level or ranking of the services, which may reflect the projected relevance of the service based upon the identified keywords] in view of ¶0048[The keywords obtained by analysis of the image may be returned to the client computing device 110 or may be used by the server 140 to further search for related products or services. If the keywords are returned to the client computing device 110, some or all of the keywords may be presented to the user to select from alternative categories of products or to select between a plurality of objects identified within the image.]); and transmitting to a computing device the at least one ranked list of recommended items to be displayed to the customer (¶0099[At block 1012, the server 140 may send one or more service purchase options from the list to the client computing device 110 for presentation to the user (e.g., via the display 112)]). Although Grossman discloses textual recommendation data, Grossman does not explicitly disclose using a language model . However, Rothschild-Keita teaches using a language model (Fig. 1; ¶¶0031-0036[In one or more embodiments, the language models are large language models (LLMs) that are trained on a large corpus of training data to generate outputs for the NLP tasks… The LLM may be pre-trained by the online concierge system 140 or one or more entities different from the online concierge system 140… The online concierge system 140 prepares a first prompt for input to the model serving system 150] in view of ¶0069[The content presentation module 210 causes a device of the customer (e.g., the customer client device 100) to display a user interface with one or more list of items for presentation to the customer… The list feature module 250 may rank each list of items based on a likelihood of conversion of that list of items by the customer (e.g., determined by the computer model deployed by the list feature module 250). The content presentation module 210 may present the list of items to the customer based on the ranking of each list of items. For example, a highest ranked list of items would be presented on the top and a lowest ranked list of items would be presented on the bottom] and ¶0086[to display a user interface with the one or more lists of items.]). The method of Rothschild-Keita is applicable to the method of Grossman as they share characteristics and capabilities, namely, they are both targeted to item recommendation. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the textual recommendation data as disclosed by Grossman to include a large language model and displaying ranked lists of items as taught by Rothschild-Keita. One of ordinary skill in the art would have been motivated to expand the method of Grossman in order to create time and/or opportunity for the customers to discover new items (¶0001). Regarding Claim 11 , Grossman in view of Rothschild-Keita teaches the computer-implemented method of claim 10, Grossman further discloses wherein generating the at least one query comprises: converting the anchor image to textual data (Figs. 3 and 10; ¶¶0091-0100[At block 1002, the server 140 may obtain an electronic image indicated by a user of a client computing device 110. Upon receiving the image, the server 140 may further identify keywords associated with the image at block 1004. Identifying a set of keywords may include obtaining labels associated with the keywords from one or more image services 240, as discussed elsewhere herein.] in view of ¶¶0046-0049; Examiner interprets “keywords derived from image object labels” as “textual data”); generating a caption describing the anchor image (Figs. 3 and 10; ¶¶0091-0100[At block 1002, the server 140 may obtain an electronic image indicated by a user of a client computing device 110. Upon receiving the image, the server 140 may further identify keywords associated with the image at block 1004. Identifying a set of keywords may include obtaining labels associated with the keywords from one or more image services 240, as discussed elsewhere herein.] in view of ¶¶0046-0049; Examiner interprets “keywords derived from image object labels” as a “caption describing the anchor image”); obtaining an input question from the customer (Figs. 3 and 10; ¶0036[A user of the client computing device 110 may receive or access images from a web site 210, a media feed 220, or a message server 230… The user may select such an image or portion of such an image to be searched to identify objects and associated products or services.] in view of ¶0037[one or more search engines 250 may be queried to identify vendor servers 260, vendor web sites, vendor media streams, vendor databases or other commercial data sources providing information regarding products or services associated with an object in an image based upon keywords derived from analysis of the image]; Examiner interprets the input image as an “input question from the customer”); and generate the at least one query based on: the textual data, the caption and the input question (Figs. 3 and 10; ¶¶0091-0100[At block 1002, the server 140 may obtain an electronic image indicated by a user of a client computing device 110. Upon receiving the image, the server 140 may further identify keywords associated with the image at block 1004. Identifying a set of keywords may include obtaining labels associated with the keywords from one or more image services 240, as discussed elsewhere herein.] in view of ¶0037[one or more search engines 250 may be queried to identify vendor servers 260, vendor web sites, vendor media streams, vendor databases or other commercial data sources providing information regarding products or services associated with an object in an image based upon keywords derived from analysis of the image] in view of ¶¶0046-0049). Regarding Claim 12 , Grossman in view of Rothschild-Keita teaches the computer-implemented method of claim 10, Grossman further discloses wherein: the at least one query includes a plurality of questions for generating the textual recommendation data (Figs. 3 and 10; ¶¶0091-0100[At block 1002, the server 140 may obtain an electronic image indicated by a user of a client computing device 110. Upon receiving the image, the server 140 may further identify keywords associated with the image at block 1004. Identifying a set of keywords may include obtaining labels associated with the keywords from one or more image services 240, as discussed elsewhere herein.] in view of ¶0037[one or more search engines 250 may be queried to identify vendor servers 260, vendor web sites, vendor media streams, vendor databases or other commercial data sources providing information regarding products or services associated with an object in an image based upon keywords derived from analysis of the image] in view of ¶¶0046-0049); and the plurality of questions are generated according to different types of recommendations associated with the recommendation request (Figs. 3 and 10; ¶¶0091-0100[At block 1002, the server 140 may obtain an electronic image indicated by a user of a client computing device 110. Upon receiving the image, the server 140 may further identify keywords associated with the image at block 1004. Identifying a set of keywords may include obtaining labels associated with the keywords from one or more image services 240, as discussed elsewhere herein.] in view of ¶0037[one or more search engines 250 may be queried to identify vendor servers 260, vendor web sites, vendor media streams, vendor databases or other commercial data sources providing information regarding products or services associated with an object in an image based upon keywords derived from analysis of the image] in view of ¶¶0046-0049). Although Grossman discloses a query with questions for generating recommendations, Grossman does not explicitly disclose prompt questions to be input into the language model and plurality of prompt questions. However, Rothschild-Keita teaches prompts into a language model and a plurality of prompts (Fig. 1; ¶¶0031-0036[In one or more embodiments, the language models are large language models (LLMs) that are trained on a large corpus of training data to generate outputs for the NLP tasks… The LLM may be pre-trained by the online concierge system 140 or one or more entities different from the online concierge system 140… The online concierge system 140 prepares a first prompt for input to the model serving system 150] in view of ¶0069[The content presentation module 210 causes a device of the customer (e.g., the customer client device 100) to display a user interface with one or more list of items for presentation to the customer… The list feature module 250 may rank each list of items based on a likelihood of conversion of that list of items by the customer (e.g., determined by the computer model deployed by the list feature module 250). The content presentation module 210 may present the list of items to the customer based on the ranking of each list of items. For example, a highest ranked list of items would be presented on the top and a lowest ranked list of items would be presented on the bottom] and ¶0086[to display a user interface with the one or more lists of items.]). The method of Rothschild-Keita is applicable to the method of Grossman as they share characteristics and capabilities, namely, they are both targeted to item recommendation. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the textual recommendation data as disclosed by Grossman to include a large language model and displaying ranked lists of items as taught by Rothschild-Keita. One of ordinary skill in the art would have been motivated to expand the method of Grossman in order to create time and/or opportunity for the customers to discover new items (¶0001). Regarding Claim 13 , Grossman in view of Rothschild-Keita teaches the computer-implemented method of claim 12, Grossman further discloses wherein: the textual recommendation data is generated based on the at least one query and product textual data on a product item page (Figs. 3 and 10; ¶¶0091-0100[At block 1002, the server 140 may obtain an electronic image indicated by a user of a client computing device 110. Upon receiving the image, the server 140 may further identify keywords associated with the image at block 1004. Identifying a set of keywords may include obtaining labels associated with the keywords from one or more image services 240, as discussed elsewhere herein.] in view of ¶0037[one or more search engines 250 may be queried to identify vendor servers 260, vendor web sites, vendor media streams, vendor databases or other commercial data sources providing information regarding products or services associated with an object in an image based upon keywords derived from analysis of the image] in view of ¶¶0046-0049); and the textual recommendation data includes different data for different questions corresponding to the different types of recommendations (¶0042[For example, the user may be interested in only one or a few objects within an image showing a number of objects, such as a particular shirt within an image showing multiple people each wearing different types of clothing. As another example, the user may be interested only in one type of product from among a plurality of products on a shelf.] in view of ¶0047[each image may be analyzed by a plurality of algorithms associated with a plurality of image services 240 to evaluate different aspects of the image. For example, separate image services 240 may be used to analyze the image with respect to identifying locations or landmarks, text within the image, object types, faces within the image, or logos within the image.]). Although Grossman discloses textual recommendation data, Grossman does not explicitly disclose the language model is a large language model pre-trained based on volumes of natural language data and different prompt questions. However, Rothschild-Keita teaches a large language model pre-trained on natural language data and prompts (Fig. 1; ¶¶0031-0036[In one or more embodiments, the language models are large language models (LLMs) that are trained on a large corpus of training data to generate outputs for the NLP tasks… The LLM may be pre-trained by the online concierge system 140 or one or more entities different from the online concierge system 140… The online concierge system 140 prepares a first prompt for input to the model serving system 150] in view of ¶0069[The content presentation module 210 causes a device of the customer (e.g., the customer client device 100) to display a user interface with one or more list of items for presentation to the customer… The list feature module 250 may rank each list of items based on a likelihood of conversion of that list of items by the customer (e.g., determined by the computer model deployed by the list feature module 250). The content presentation module 210 may present the list of items to the customer based on the ranking of each list of items. For example, a highest ranked list of items would be presented on the top and a lowest ranked list of items would be presented on the bottom] and ¶0086[to display a user interface with the one or more lists of items.]). The method of Rothschild-Keita is applicable to the method of Grossman as they share characteristics and capabilities, namely, they are both targeted to item recommendation. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the textual recommendation data as disclosed by Grossman to include a large language model and displaying ranked lists of items as taught by Rothschild-Keita. One of ordinary skill in the art would have been motivated to expand the method of Grossman in order to create time and/or opportunity for the customers to discover new items (¶0001). Regarding Claim 14 , Grossman in view of Rothschild-Keita teaches the computer-implemented method of claim 12, Grossman further discloses wherein: the at least one ranked list of recommended items includes recommended items (Fig. 10; ¶0098[At block 1010, the server 140 may generate a list of service purchase options from the vendor site search results. The list may include a recommendation level or ranking of the services, which may reflect the projected relevance of the service based upon the identified keywords]); each ranked list of recommended items is generated for a prompt question corresponding to a respective type of the different types of recommendations (Fig. 10; ¶0098[At block 1010, the server 140 may generate a list of service purchase options from the vendor site search results. The list may include a recommendation level or ranking of the services, which may reflect the projected relevance of the service based upon the identified keywords] in view of ¶¶0042-0047[¶0042[For example, the user may be interested in only one or a few objects within an image showing a number of objects, such as a particular shirt within an image showing multiple people each wearing different types of clothing. As another example, the user may be interested only in one type of product from among a plurality of products on a shelf.]); and each ranked list of recommended items is to be displayed in a respective carousel of a user interface showing the anchor image to the customer (¶¶0098-0099[At block 1010, the server 140 may generate a list of service purchase options from the vendor site search results. The list may include a recommendation level or ranking of the services, which may reflect the projected relevance of the service based upon the identified keywords… At block 1012, the server 140 may send one or more service purchase options from the list to the client computing device 110 for presentation to the user (e.g., via the display 112). The purchase options may be presented in an order based upon relevance or location or a combination thereof]). Although Grossman discloses recommending items, Grossman does not explicitly disclose a plurality of ranked lists of items displaying on the same user interface showing an item. However, Rothschild-Keita teaches ranked lists of items being displayed on the same user interface (Fig. 1; ¶¶0031-0036[In one or more embodiments, the language models are large language models (LLMs) that are trained on a large corpus of training data to generate outputs for the NLP tasks… The LLM may be pre-trained by the online concierge system 140 or one or more entities different from the online concierge system 140… The online concierge system 140 prepares a first prompt for input to the model serving system 150] in view of ¶0069[The content presentation module 210 causes a device of the customer (e.g., the customer client device 100) to display a user interface with one or more list of items for presentation to the customer… The list feature module 250 may rank each list of items based on a likelihood of conversion of that list of items by the customer (e.g., determined by the computer model deployed by the list feature module 250). The content presentation module 210 may present the list of items to the customer based on the ranking of each list of items. For example, a highest ranked list of items would be presented on the top and a lowest ranked list of items would be presented on the bottom] and ¶0086[to display a user interface with the one or more lists of items.]). The method of Rothschild-Keita is applicable to the method of Grossman as they share characteristics and capabilities, namely, they are both targeted to item recommendation. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the textual recommendation data as disclosed by Grossman to include a large language model and displaying ranked lists of items as taught by Rothschild-Keita. One of ordinary skill in the art would have been motivated to expand the method of Grossman in order to create time and/or opportunity for the customers to discover new items (¶0001). Regarding Claim 15 , Grossman in view of Rothschild-Keita teaches the computer-implemented method of claim 14, Grossman further discloses wherein each ranked list of recommended items is generated based on: searching an item database to retrieve a plurality of items by comparing the textual recommendation data to item texts in the item database, wherein (Fig. 1; ¶0027[The server 140 may store and retrieve data within a database 146.] in view of ¶¶0033-0040[The data sources 170 may include public or proprietary databases including information that may be associated with objects in images, such as labels identifying objects previously identified within images. For example, a data source 170 may include information for a reverse image search that may be used to identify text labels for an image or portion thereof. The one or more data sources 170 may further include databases or servers associated with vendors, providing information regarding products or services… comparison of the labels associated with the various images to determine the most salient keywords.] and ¶0093[Identifying a set of keywords may include obtaining labels associated with the keywords from one or more image services 240]): each of the plurality of items is associated with a relevance indicating a degree of relevance between the item and the textual recommendation data based on the respective type of recommendation (Fig. 10; ¶0098[At block 1010, the server 140 may generate a list of service purchase options from the vendor site search results. The list may include a recommendation level or ranking of the services, which may reflect the projected relevance of the service based upon the identified keywords] in view of ¶0048[The keywords obtained by analysis of the image may be returned to the client computing device 110 or may be used by the server 140 to further search for related products or services. If the keywords are returned to the client computing device 110, some or all of the keywords may be presented to the user to select from alternative categories of products or to select between a plurality of objects identified within the image.]), and the relevance associated with the plurality of items are higher than a predetermined threshold (¶0087[A match may be determined to exist when the probability exceeds a matching threshold (e.g., 60% or 80%). If no match is identified by the server 140, the method 900 may terminate]); and ranking the plurality of items according to their respective relevance to generate the ranked list of recommended items (Fig. 10; ¶0098[At block 1010, the server 140 may generate a list of service purchase options from the vendor site search results. The list may include a recommendation level or ranking of the services, which may reflect the projected relevance of the service based upon the identified keywords] in view of ¶0048[The keywords obtained by analysis of the image may be returned to the client computing device 110 or may be used by the server 140 to further search for related products or services. If the keywords are returned to the client computing device 110, some or all of the keywords may be presented to the user to select from alternative categories of products or to select between a plurality of objects identified within the image]). Although Grossman discloses a degree of relevance between an item and textual recommendation data, Grossman does not explicitly disclose a relevance score , the relevance scores , and the relevance scores generating recommendations. However, Rothschild-Keita teaches scoring items (Fig. 2; ¶¶0049-0052[For example, the content presentation module 210 may score items and rank the items based on their scores. The content presentation module 210 displays the items with scores that exceed some threshold (e.g., the top n items or the p percentile of items)]). The method of Rothschild-Keita is applicable to the method of Grossman as they share characteristics and capabilities, namely, they are both targeted to item recommendation. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the textual recommendation data as disclosed by Grossman to include scoring items as taught by Rothschild-Keita. One of ordinary skill in the art would have been motivated to expand the method of Grossman in order to create time and/or opportunity for the customers to discover new items (¶0001). Regarding Claim 17 , Grossman discloses a non-transitory computer readable medium having instructions stored thereon, wherein the instructions, when executed by at least one processor, cause at least one device to perform operations comprising (Fig. 1; ¶¶0031-0032[The back-end components 104 may include one or more servers 140 communicatively connected to the network 130 by a link 135. Each server 140 may include one or more processors 162 adapted and configured to execute various software applications and components of the system… The controller 150 may include a program memory 160, a processor 162, a RAM 164, and an I/O circuit 166, all of which may be interconnected via an address/data bus 165]): receiving a recommendation request for recommending items to a customer (Figs. 3 and 10; ¶0036[A user of the client computing device 110 may receive or access images from a web site 210, a media feed 220, or a message server 230… The user may select such an image or portion of such an image to be searched to identify objects and associated products or services.] in view of ¶¶0091-0092); determining an anchor image based on the recommendation request (Fig. 10; ¶0092[The method 1000 may begin with obtaining an electronic image to be processed (block 1002)]); generating at least one query based on the anchor image (Figs. 3 and 10; ¶0042[At block 304, in some embodiments, the client computing device 110 may receive a user indication of a location within the image associated with an object of interest within the image. If no user indication is received, the method 300 may continue by identifying one or more objects within the electronic image and providing information regarding each of the identified objects] in view of ¶¶0091-0100[FIG. 10 illustrates a flow diagram of an exemplary service identification method 1000 for identifying purchase options regarding services associated with objects in electronic images. The method 1000 may be implemented using the components of the object identification system 100, either separately or in conjunction with any of the other methods disclosed herein.]); generating, textual recommendation data based on the at least one query (Figs. 3 and 10; ¶¶0091-0100[At block 1002, the server 140 may obtain an electronic image indicated by a user of a client computing device 110. Upon receiving the image, the server 140 may further identify keywords associated with the image at block 1004. Identifying a set of keywords may include obtaining labels associated with the keywords from one or more image services 240, as discussed elsewhere herein.] in view of ¶¶0046-0049; Examiner interprets “keywords derived from image object labels” as “textual recommendation data”); generating, using at least one machine learning model, at least one ranked list of recommended items based on the textual recommendation data (Fig. 10; ¶0098[At block 1010, the server 140 may generate a list of service purchase options from the vendor site search results. The list may include a recommendation level or ranking of the services, which may reflect the projected relevance of the service based upon the identified keywords] in view of ¶0048[The keywords obtained by analysis of the image may be returned to the client computing device 110 or may be used by the server 140 to further search for related products or services. If the keywords are returned to the client computing device 110, some or all of the keywords may be presented to the user to select from alternative categories of products or to select between a plurality of objects identified within the image.]); and transmitting to a computing device the at least one ranked list of recommended items to be displayed to the customer (¶0099[At block 1012, the server 140 may send one or more service purchase options from the list to the client computing device 110 for presentation to the user (e.g., via the display 112)]). Although Grossman discloses textual recommendation data, Grossman does not explicitly disclose using a language model . However, Rothschild-Keita teaches using a language model (Fig. 1; ¶¶0031-0036[In one or more embodiments, the language models are large language models (LLMs) that are trained on a large corpus of training data to generate outputs for the NLP tasks… The LLM may be pre-trained by the online concierge system 140 or one or more entities different from the online concierge system 140… The online concierge system 140 prepares a first prompt for input to the model serving system 150] in view of ¶0069[The content presentation module 210 causes a device of the customer (e.g., the customer client device 100) to display a user interface with one or more list of items for presentation to the customer… The list feature module 250 may rank each list of items based on a likelihood of conversion of that list of items by the customer (e.g., determined by the computer model deployed by the list feature module 250). The content presentation module 210 may present the list of items to the customer based on the ranking of each list of items. For example, a highest ranked list of items would be presented on the top and a lowest ranked list of items would be presented on the bottom] and ¶0086[to display a user interface with the one or more lists of items.]). The system of Rothschild-Keita is applicable to the system of Grossman as they share characteristics and capabilities, namely, they are both targeted to item recommendation. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the textual recommendation data as disclosed by Grossman to include a large language model and displaying ranked lists of items as taught by Rothschild-Keita. One of ordinary skill in the art would have been motivated to expand the system of Grossman in order to create time and/or opportunity for the customers to discover new items (¶0001). Regarding Claim 18 , Grossman in view of Rothschild-Keita teaches the non-transitory computer readable medium of claim 17, Grossman further discloses wherein generating the at least one query comprises: converting the anchor image to textual data (Figs. 3 and 10; ¶¶0091-0100[At block 1002, the server 140 may obtain an electronic image indicated by a user of a client computing device 110. Upon receiving the image, the server 140 may further identify keywords associated with the image at block 1004. Identifying a set of keywords may include obtaining labels associated with the keywords from one or more image services 240, as discussed elsewhere herein.] in view of ¶¶0046-0049; Examiner interprets “keywords derived from image object labels” as “textual data”); generating a caption describing the anchor image (Figs. 3 and 10; ¶¶0091-0100[At block 1002, the server 140 may obtain an electronic image indicated by a user of a client computing device 110. Upon receiving the image, the server 140 may further identify keywords associated with the image at block 1004. Identifying a set of keywords may include obtaining labels associated with the keywords from one or more image services 240, as discussed elsewhere herein.] in view of ¶¶0046-0049; Examiner interprets “keywords derived from image object labels” as a “caption describing the anchor image”); obtaining an input question from the customer (Figs. 3 and 10; ¶0036[A user of the client computing device 110 may receive or access images from a web site 210, a media feed 220, or a message server 230… The user may select such an image or portion of such an image to be searched to identify objects and associated products or services.] in view of ¶0037[one or more search engines 250 may be queried to identify vendor servers 260, vendor web sites, vendor media streams, vendor databases or other commercial data sources providing information regarding products or services associated with an object in an image based upon keywords derived from analysis of the image]; Examiner interprets the input image as an “input question from the customer”); and generate the at least one query based on: the textual data, the caption and the input question (Figs. 3 and 10; ¶¶0091-0100[At block 1002, the server 140 may obtain an electronic image indicated by a user of a client computing device 110. Upon receiving the image, the server 140 may further identify keywords associated with the image at block 1004. Identifying a set of keywords may include obtaining labels associated with the keywords from one or more image services 240, as discussed elsewhere herein.] in view of ¶0037[one or more search engines 250 may be queried to identify vendor servers 260, vendor web sites, vendor media streams, vendor databases or other commercial data sources providing information regarding products or services associated with an object in an image based upon keywords derived from analysis of the image] in view of ¶¶0046-0049). Regarding Claim 19 , Grossman in view of Rothschild-Keita teaches the non-transitory computer readable medium of claim 17, Grossman further discloses wherein: the at least one query includes a plurality of questions for generating the textual recommendation data (Figs. 3 and 10; ¶¶0091-0100[At block 1002, the server 140 may obtain an electronic image indicated by a user of a client computing device 110. Upon receiving the image, the server 140 may further identify keywords associated with the image at block 1004. Identifying a set of keywords may include obtaining labels associated with the keywords from one or more image services 240, as discussed elsewhere herein.] in view of ¶0037[one or more search engines 250 may be queried to identify vendor servers 260, vendor web sites, vendor media streams, vendor databases or other commercial data sources providing information regarding products or services associated with an object in an image based upon keywords derived from analysis of the image] in view of ¶¶0046-0049); and the plurality of questions are generated according to different types of recommendations associated with the recommendation request (Figs. 3 and 10; ¶¶0091-0100[At block 1002, the server 140 may obtain an electronic image indicated by a user of a client computing device 110. Upon receiving the image, the server 140 may further identify keywords associated with the image at block 1004. Identifying a set of keywords may include obtaining labels associated with the keywords from one or more image services 240, as discussed elsewhere herein.] in view of ¶0037[one or more search engines 250 may be queried to identify vendor servers 260, vendor web sites, vendor media streams, vendor databases or other commercial data sources providing information regarding products or services associated with an object in an image based upon keywords derived from analysis of the image] in view of ¶¶0046-0049). Although Grossman discloses a query with questions for generating recommendations, Grossman does not explicitly disclose prompt questions to be input into the language model and plurality of prompt questions. However, Rothschild-Keita teaches prompts into a language model and a plurality of prompts (Fig. 1; ¶¶0031-0036[In one or more embodiments, the language models are large language models (LLMs) that are trained on a large corpus of training data to generate outputs for the NLP tasks… The LLM may be pre-trained by the online concierge system 140 or one or more entities different from the online concierge system 140… The online concierge system 140 prepares a first prompt for input to the model serving system 150] in view of ¶0069[The content presentation module 210 causes a device of the customer (e.g., the customer client device 100) to display a user interface with one or more list of items for presentation to the customer… The list feature module 250 may rank each list of items based on a likelihood of conversion of that list of items by the customer (e.g., determined by the computer model deployed by the list feature module 250). The content presentation module 210 may present the list of items to the customer based on the ranking of each list of items. For example, a highest ranked list of items would be presented on the top and a lowest ranked list of items would be presented on the bottom] and ¶0086[to display a user interface with the one or more lists of items.]). The system of Rothschild-Keita is applicable to the system of Grossman as they share characteristics and capabilities, namely, they are both targeted to item recommendation. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the textual recommendation data as disclosed by Grossman to include a large language model and displaying ranked lists of items as taught by Rothschild-Keita. One of ordinary skill in the art would have been motivated to expand the system of Grossman in order to create time and/or opportunity for the customers to discover new items (¶0001). Regarding Claim 20 , Grossman in view of Rothschild-Keita teaches the non-transitory computer readable medium of claim 19, Grossman further discloses wherein: the textual recommendation data is generated based on the at least one query and product textual data on a product item page (Figs. 3 and 10; ¶¶0091-0100[At block 1002, the server 140 may obtain an electronic image indicated by a user of a client computing device 110. Upon receiving the image, the server 140 may further identify keywords associated with the image at block 1004. Identifying a set of keywords may include obtaining labels associated with the keywords from one or more image services 240, as discussed elsewhere herein.] in view of ¶0037[one or more search engines 250 may be queried to identify vendor servers 260, vendor web sites, vendor media streams, vendor databases or other commercial data sources providing information regarding products or services associated with an object in an image based upon keywords derived from analysis of the image] in view of ¶¶0046-0049); the textual recommendation data includes different data for different questions corresponding to the different types of recommendations (¶0042[For example, the user may be interested in only one or a few objects within an image showing a number of objects, such as a particular shirt within an image showing multiple people each wearing different types of clothing. As another example, the user may be interested only in one type of product from among a plurality of products on a shelf.] in view of ¶0047[each image may be analyzed by a plurality of algorithms associated with a plurality of image services 240 to evaluate different aspects of the image. For example, separate image services 240 may be used to analyze the image with respect to identifying locations or landmarks, text within the image, object types, faces within the image, or logos within the image.]); the at least one ranked list of recommended items includes recommended items (Fig. 10; ¶0098[At block 1010, the server 140 may generate a list of service purchase options from the vendor site search results. The list may include a recommendation level or ranking of the services, which may reflect the projected relevance of the service based upon the identified keywords]); each ranked list of recommended items is generated for a prompt question corresponding to a respective type of the different types of recommendations (Fig. 10; ¶0098[At block 1010, the server 140 may generate a list of service purchase options from the vendor site search results. The list may include a recommendation level or ranking of the services, which may reflect the projected relevance of the service based upon the identified keywords] in view of ¶¶0042-0047[¶0042[For example, the user may be interested in only one or a few objects within an image showing a number of objects, such as a particular shirt within an image showing multiple people each wearing different types of clothing. As another example, the user may be interested only in one type of product from among a plurality of products on a shelf.]); and each ranked list of recommended items is to be displayed in a respective carousel of a user interface showing the anchor image to the customer (¶¶0098-0099[At block 1010, the server 140 may generate a list of service purchase options from the vendor site search results. The list may include a recommendation level or ranking of the services, which may reflect the projected relevance of the service based upon the identified keywords… At block 1012, the server 140 may send one or more service purchase options from the list to the client computing device 110 for presentation to the user (e.g., via the display 112). The purchase options may be presented in an order based upon relevance or location or a combination thereof]). Although Grossman discloses textual recommendation data, Grossman does not explicitly disclose the language model is a large language model pre-trained based on volumes of natural language data , different prompt questions, a plurality of ranked lists of items displaying on the same user interface showing an item. However, Rothschild-Keita teaches a large language model pre-trained on natural language data, prompts, and a plurality of ranked lists of items displaying on the same user interface (Fig. 1; ¶¶0031-0036[In one or more embodiments, the language models are large language models (LLMs) that are trained on a large corpus of training data to generate outputs for the NLP tasks… The LLM may be pre-trained by the online concierge system 140 or one or more entities different from the online concierge system 140… The online concierge system 140 prepares a first prompt for input to the model serving system 150] in view of ¶0069[The content presentation module 210 causes a device of the customer (e.g., the customer client device 100) to display a user interface with one or more list of items for presentation to the customer… The list feature module 250 may rank each list of items based on a likelihood of conversion of that list of items by the customer (e.g., determined by the computer model deployed by the list feature module 250). The content presentation module 210 may present the list of items to the customer based on the ranking of each list of items. For example, a highest ranked list of items would be presented on the top and a lowest ranked list of items would be presented on the bottom] and ¶0086[to display a user interface with the one or more lists of items.]). The system of Rothschild-Keita is applicable to the system of Grossman as they share characteristics and capabilities, namely, they are both targeted to item recommendation. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the textual recommendation data as disclosed by Grossman to include a large language model and displaying ranked lists of items as taught by Rothschild-Keita. One of ordinary skill in the art would have been motivated to expand the system of Grossman in order to create time and/or opportunity for the customers to discover new items (¶0001) . 07-21-aia AIA Claim (s) 8 and 16 is/are rejected under 35 U.S.C. 103 as being unpatentable over Grossman in view of Rothschild-Keita in view of Zhou et al. (US 2023/0068103 A1) . Regarding Claim 8 , Grossman in view of Rothschild-Keita teaches the system of claim 6, Grossman further discloses wherein each ranked list of recommended items is generated based on: textual recommendation data to query images (Figs. 3 and 10; ¶¶0091-0100[At block 1002, the server 140 may obtain an electronic image indicated by a user of a client computing device 110. Upon receiving the image, the server 140 may further identify keywords associated with the image at block 1004. Identifying a set of keywords may include obtaining labels associated with the keywords from one or more image services 240, as discussed elsewhere herein.] in view of ¶¶0046-0049; Examiner interprets “keywords derived from image object labels” as “textual recommendation data”); searching an item database to retrieve a plurality of items by comparing the query images to item images in the item database, wherein (Fig. 1; ¶0027[The server 140 may store and retrieve data within a database 146.] in view of ¶¶0033-0040[The data sources 170 may include public or proprietary databases including information that may be associated with objects in images, such as labels identifying objects previously identified within images. For example, a data source 170 may include information for a reverse image search that may be used to identify text labels for an image or portion thereof. The one or more data sources 170 may further include databases or servers associated with vendors, providing information regarding products or services… comparison of the labels associated with the various images to determine the most salient keywords.] and ¶0093[Identifying a set of keywords may include obtaining labels associated with the keywords from one or more image services 240]): each of the plurality of items is associated with a relevance indicating a degree of relevance between the item and the query images based on the respective type of recommendation (Fig. 10; ¶0098[At block 1010, the server 140 may generate a list of service purchase options from the vendor site search results. The list may include a recommendation level or ranking of the services, which may reflect the projected relevance of the service based upon the identified keywords] in view of ¶0048[The keywords obtained by analysis of the image may be returned to the client computing device 110 or may be used by the server 140 to further search for related products or services. If the keywords are returned to the client computing device 110, some or all of the keywords may be presented to the user to select from alternative categories of products or to select between a plurality of objects identified within the image.]), and the relevance associated with the plurality of items are higher than a predetermined threshold (¶0087[A match may be determined to exist when the probability exceeds a matching threshold (e.g., 60% or 80%). If no match is identified by the server 140, the method 900 may terminate]); and ranking the plurality of items according to their respective relevance scores to generate the ranked list of recommended items (Fig. 10; ¶0098[At block 1010, the server 140 may generate a list of service purchase options from the vendor site search results. The list may include a recommendation level or ranking of the services, which may reflect the projected relevance of the service based upon the identified keywords] in view of ¶0048[The keywords obtained by analysis of the image may be returned to the client computing device 110 or may be used by the server 140 to further search for related products or services. If the keywords are returned to the client computing device 110, some or all of the keywords may be presented to the user to select from alternative categories of products or to select between a plurality of objects identified within the image]). Although Grossman discloses textual recommendation data, Grossman does not explicitly disclose a relevance score and relevance scores . However, Rothschild-Keita teaches scoring items (Fig. 2; ¶¶0049-0052[For example, the content presentation module 210 may score items and rank the items based on their scores. The content presentation module 210 displays the items with scores that exceed some threshold (e.g., the top n items or the p percentile of items)]). The system of Rothschild-Keita is applicable to the system of Grossman as they share characteristics and capabilities, namely, they are both targeted to item recommendation. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the textual recommendation data as disclosed by Grossman to include scoring items as taught by Rothschild-Keita. One of ordinary skill in the art would have been motivated to expand the system of Grossman in order to create time and/or opportunity for the customers to discover new items (¶0001). Although Grossman discloses textual recommendation data, Grossman in view of Rothschild-Keita does not explicitly teach converting the data. However, Zhou et al., hereinafter, Zhou, teaches converting data into images (Figs. 2A and 2B; ¶0023[The style description text may then be converted to product images by using a text prediction-based first image generation model]). The system of Zhou is applicable to the system of Grossman in view of Rothschild-Keita as they share characteristics and capabilities, namely, they are all targeted to item recommendation. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the textual recommendation data as taught by Grossman in view of Rothschild-Keita to include converting data as taught by Zhou. One of ordinary skill in the art would have been motivated to expand the system of Grossman in view of Rothschild-Keita in order to provide an automation product image generation capability for a manufacturing industry (¶0013). Regarding Claim 16 , Grossman in view of Rothschild-Keita teaches the computer-implemented method of claim 14, Grossman further discloses wherein each ranked list of recommended items is generated based on: textual recommendation data to query images (Figs. 3 and 10; ¶¶0091-0100[At block 1002, the server 140 may obtain an electronic image indicated by a user of a client computing device 110. Upon receiving the image, the server 140 may further identify keywords associated with the image at block 1004. Identifying a set of keywords may include obtaining labels associated with the keywords from one or more image services 240, as discussed elsewhere herein.] in view of ¶¶0046-0049; Examiner interprets “keywords derived from image object labels” as “textual recommendation data”); searching an item database to retrieve a plurality of items by comparing the query images to item images in the item database, wherein (Fig. 1; ¶0027[The server 140 may store and retrieve data within a database 146.] in view of ¶¶0033-0040[The data sources 170 may include public or proprietary databases including information that may be associated with objects in images, such as labels identifying objects previously identified within images. For example, a data source 170 may include information for a reverse image search that may be used to identify text labels for an image or portion thereof. The one or more data sources 170 may further include databases or servers associated with vendors, providing information regarding products or services… comparison of the labels associated with the various images to determine the most salient keywords.] and ¶0093[Identifying a set of keywords may include obtaining labels associated with the keywords from one or more image services 240]): each of the plurality of items is associated with a relevance indicating a degree of relevance between the item and the query images based on the respective type of recommendation (Fig. 10; ¶0098[At block 1010, the server 140 may generate a list of service purchase options from the vendor site search results. The list may include a recommendation level or ranking of the services, which may reflect the projected relevance of the service based upon the identified keywords] in view of ¶0048[The keywords obtained by analysis of the image may be returned to the client computing device 110 or may be used by the server 140 to further search for related products or services. If the keywords are returned to the client computing device 110, some or all of the keywords may be presented to the user to select from alternative categories of products or to select between a plurality of objects identified within the image.]), and the relevance associated with the plurality of items are higher than a predetermined threshold (¶0087[A match may be determined to exist when the probability exceeds a matching threshold (e.g., 60% or 80%). If no match is identified by the server 140, the method 900 may terminate]); and ranking the plurality of items according to their respective relevance scores to generate the ranked list of recommended items (Fig. 10; ¶0098[At block 1010, the server 140 may generate a list of service purchase options from the vendor site search results. The list may include a recommendation level or ranking of the services, which may reflect the projected relevance of the service based upon the identified keywords] in view of ¶0048[The keywords obtained by analysis of the image may be returned to the client computing device 110 or may be used by the server 140 to further search for related products or services. If the keywords are returned to the client computing device 110, some or all of the keywords may be presented to the user to select from alternative categories of products or to select between a plurality of objects identified within the image]). Although Grossman discloses textual recommendation data, Grossman does not explicitly disclose a relevance score and relevance scores . However, Rothschild-Keita teaches scoring items (Fig. 2; ¶¶0049-0052[For example, the content presentation module 210 may score items and rank the items based on their scores. The content presentation module 210 displays the items with scores that exceed some threshold (e.g., the top n items or the p percentile of items)]). The method of Rothschild-Keita is applicable to the method of Grossman as they share characteristics and capabilities, namely, they are both targeted to item recommendation. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the textual recommendation data as disclosed by Grossman to include scoring items as taught by Rothschild-Keita. One of ordinary skill in the art would have been motivated to expand the method of Grossman in order to create time and/or opportunity for the customers to discover new items (¶0001). Although Grossman discloses textual recommendation data, Grossman in view of Rothschild-Keita does not explicitly teach converting the data. However, Zhou teaches converting data into images (Figs. 2A and 2B; ¶0023[The style description text may then be converted to product images by using a text prediction-based first image generation model]). The method of Zhou is applicable to the method of Grossman in view of Rothschild-Keita as they share characteristics and capabilities, namely, they are all targeted to item recommendation. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the textual recommendation data as taught by Grossman in view of Rothschild-Keita to include converting data as taught by Zhou. One of ordinary skill in the art would have been motivated to expand the method of Grossman in view of Rothschild-Keita in order to provide an automation product image generation capability for a manufacturing industry (¶0013) . Conclusion 07-96 AIA The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Hu (US 2019/0318405 A1) discloses a selection of an image, and for analyzing the image to determine a location within the image of one or more products. “A mobile visual search application for content based image retrieval in the fashion domain” discloses content-Based Image Retrieval engine for online shopping in the fashion domain. Any inquiry concerning this communication or earlier communications from the examiner should be directed to AHOORA LADONI whose email is Ahoora.Ladoni@uspto.gov and telephone number is (703) 756-5617. The examiner can normally be reached M-F 0900–1700 ET. Examiner interviews are available via telephone, in-person and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice. Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. /AHOORA LADONI/Examiner, Art Unit 3689 /MARISSA THEIN/Supervisory Patent Examiner, Art Unit 3689 Application/Control Number: 19/029,686 Page 2 Art Unit: 3689 Application/Control Number: 19/029,686 Page 3 Art Unit: 3689 Application/Control Number: 19/029,686 Page 4 Art Unit: 3689 Application/Control Number: 19/029,686 Page 5 Art Unit: 3689 Application/Control Number: 19/029,686 Page 6 Art Unit: 3689 Application/Control Number: 19/029,686 Page 7 Art Unit: 3689 Application/Control Number: 19/029,686 Page 8 Art Unit: 3689 Application/Control Number: 19/029,686 Page 9 Art Unit: 3689 Application/Control Number: 19/029,686 Page 10 Art Unit: 3689 Application/Control Number: 19/029,686 Page 11 Art Unit: 3689 Application/Control Number: 19/029,686 Page 12 Art Unit: 3689 Application/Control Number: 19/029,686 Page 13 Art Unit: 3689 Application/Control Number: 19/029,686 Page 14 Art Unit: 3689 Application/Control Number: 19/029,686 Page 15 Art Unit: 3689 Application/Control Number: 19/029,686 Page 16 Art Unit: 3689 Application/Control Number: 19/029,686 Page 17 Art Unit: 3689 Application/Control Number: 19/029,686 Page 18 Art Unit: 3689 Application/Control Number: 19/029,686 Page 19 Art Unit: 3689 Application/Control Number: 19/029,686 Page 20 Art Unit: 3689 Application/Control Number: 19/029,686 Page 21 Art Unit: 3689 Application/Control Number: 19/029,686 Page 22 Art Unit: 3689 Application/Control Number: 19/029,686 Page 23 Art Unit: 3689 Application/Control Number: 19/029,686 Page 24 Art Unit: 3689 Application/Control Number: 19/029,686 Page 25 Art Unit: 3689 Application/Control Number: 19/029,686 Page 26 Art Unit: 3689 Application/Control Number: 19/029,686 Page 27 Art Unit: 3689 Application/Control Number: 19/029,686 Page 28 Art Unit: 3689 Application/Control Number: 19/029,686 Page 29 Art Unit: 3689 Application/Control Number: 19/029,686 Page 30 Art Unit: 3689 Application/Control Number: 19/029,686 Page 31 Art Unit: 3689 Application/Control Number: 19/029,686 Page 32 Art Unit: 3689 Application/Control Number: 19/029,686 Page 33 Art Unit: 3689 Application/Control Number: 19/029,686 Page 34 Art Unit: 3689 Application/Control Number: 19/029,686 Page 35 Art Unit: 3689 Application/Control Number: 19/029,686 Page 36 Art Unit: 3689 Application/Control Number: 19/029,686 Page 37 Art Unit: 3689 Application/Control Number: 19/029,686 Page 38 Art Unit: 3689 Application/Control Number: 19/029,686 Page 39 Art Unit: 3689 Application/Control Number: 19/029,686 Page 40 Art Unit: 3689 Application/Control Number: 19/029,686 Page 41 Art Unit: 3689 Application/Control Number: 19/029,686 Page 42 Art Unit: 3689 Application/Control Number: 19/029,686 Page 43 Art Unit: 3689
Prosecution Insights