DETAILED ACTION
Continued Examination under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 03/12/2026 has been entered.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Examiner’s Comment
This Action is in response to the Request for Continued Examination filed on 03/12/2026 with Amended Claims and Applicant's Remarks filed on 03/12/2026.
Applicant has amended claims 1, 14, and 20 according to Amendments filed on 03/12/2026. Claims 1-9, 11-16, and 18-20 are pending and currently under consideration for patentability.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-9, 11-16, and 18-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter. The claims are directed to a judicial exception (i.e., a law of nature, natural phenomenon, or abstract idea) without significantly more.
Step 1: In a test for patent subject matter eligibility, claims 1-9, 11-16, and 18-20 are found to be in accordance with Step 1 (see 2019 Revised Patent Subject Matter Eligibility), as they are related to a process, machine, manufacture, or composition of matter. Claims 1-9, 11-13 recite a method, claims 14-16, 18, 19 recite a non-transitory computer-readable medium, and claim 20 recites a system. When assessed under Step 2A, Prong I, they are found to be directed towards an abstract idea. The rationale for this finding is explained below:
Step 2A, Prong I: Under Step 2A, Prong I, claims 1-9, 11-16, and 18-20 are directed to an abstract idea without significantly more, as they all recite a judicial exception. Independent Claims 1, 14, and 20 recite limitations directed to the abstract idea including receiving a request to generate a digital flyer, wherein the request includes one or more design conditions for the digital flyer; accessing an item catalog storing item data; generating a query including a prompt to generate the digital flyer, the one or more design conditions, and the item data accessed from the item catalog; providing the query; receiving a batch of one or more digital flyers generated on the query; and causing presentation of the augmented first digital flyer. These further limitations are not seen as any more than the judicial exception. Claims 1, 14, and 20 recite additional limitations including “from a client device; for a machine-learned generative model; to a model serving system for execution by the machine-learned generative model; from the model serving system; by executing the machine-learned generative model; applying an image segmentation model to a first digital flyer to parse the first digital flyer into a plurality of image segments, wherein each image segment is classified into one of a plurality of segment categories including one segment category relating to images of items; applying an item matching model to each image segment classified into the segment category relating to images of items to match the image segment to an item in the item catalog; augmenting the first digital flyer by overlaying one or more user-interactable elements on one or more image segments, wherein each user-interactable element is encoded to, responsive to a user interaction with the image segment, perform one or more actions related to the item matched to the image segment; and on an electronic display of a client device.” Causing presentation of augmented first digital flyer based on a batch of digital flyers determined according to a query including design conditions and accessed item data is considered to be an abstract idea, specifically, certain methods of organizing human activity; such as managing personal behavior or relationships or interactions between people (including social activities, teaching, and following rules or instructions) because the claims are directed to managing a relationship between parties in order to cause presentation of an augmented first digital flyer to a user. Furthermore, causing presentation of augmented first digital flyer determined according to a query including design conditions and accessed item data is also considered to be fall under another grouping of abstract idea, specifically, Mental Processes; such as concepts performed in the human mind (including an observation, evaluation, judgment, opinion) because the claims are directed to receiving information (i.e. request to generate a digital flyer), accessing information (i.e. item catalog data), providing information (i.e. query to model), receiving information (i.e. a batch of digital flyers) and causing presentation of information (i.e. digital flyers in the batch for presentation) which can all be performed in the human mind. Therefore, under Step 2A, Prong I, Claims 1, 14, and 20 are directed towards an abstract idea.
Step 2A, Prong II: Step 2A, Prong II is to determine whether any claim recites any additional element that integrate the judicial exception (abstract idea) into a practical application. Claims 1, 14, and 20 recite additional limitations including “from a client device; for a machine-learned generative model; to a model serving system for execution by the machine-learned generative model; from the model serving system; by executing the machine-learned generative model; applying an image segmentation model to a first digital flyer to parse the first digital flyer into a plurality of image segments, wherein each image segment is classified into one of a plurality of segment categories including one segment category relating to images of items; applying an item matching model to each image segment classified into the segment category relating to images of items to match the image segment to an item in the item catalog; augmenting the first digital flyer by overlaying one or more user-interactable elements on one or more image segments, wherein each user-interactable element is encoded to, responsive to a user interaction with the image segment, perform one or more actions related to the item matched to the image segment; and on an electronic display of a client device.” The additional limitations reciting – “from a client device; for a machine-learned generative model; to a model serving system for execution by the machine-learned generative model; from the model serving system; by executing the machine-learned generative model; applying an image segmentation model to a first digital flyer to parse the first digital flyer into a plurality of image segments, wherein each image segment is classified into one of a plurality of segment categories including one segment category relating to images of items; applying an item matching model to each image segment classified into the segment category relating to images of items to match the image segment to an item in the item catalog; and on an electronic display of a client device” are not found to integrate the judicial exception into a practical application because they are seen as adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea - see MPEP 2106.05(f) and generally linking the use of the judicial exception to a particular technological environment or field of use – see MPEP 2106.05(h). Accordingly, alone, and in combination, these additional elements are seen as using a computer or tool to perform an abstract idea, adding insignificant-extra-solution activity to the judicial exception. They do no more than link the judicial exception to a particular technological environment or field of use, i.e. client device/model serving system/machine-learned generative model/image segmentation model/item matching model, and therefore do not integrate the abstract idea into a practical application. The courts decided that although the additional elements did limit the use of the abstract idea, the court explained that this type of limitation merely confines the use of the abstract idea to a particular technological environment and this fails to add an inventive concept to the claims (See Affinity Labs of Texas v. DirecTV, LLC,). Under Step 2A, Prong II, these claims remain directed towards an abstract idea.
Step 2B: Claims 1, 14, and 20 recite additional limitations including “from a client device; for a machine-learned generative model; to a model serving system for execution by the machine-learned generative model; from the model serving system; by executing the machine-learned generative model; applying an image segmentation model to a first digital flyer to parse the first digital flyer into a plurality of image segments, wherein each image segment is classified into one of a plurality of segment categories including one segment category relating to images of items; applying an item matching model to each image segment classified into the segment category relating to images of items to match the image segment to an item in the item catalog; augmenting the first digital flyer by overlaying one or more user-interactable elements on one or more image segments, wherein each user-interactable element is encoded to, responsive to a user interaction with the image segment, perform one or more actions related to the item matched to the image segment; and on an electronic display of a client device.” The additional limitations reciting – “from a client device; for a machine-learned generative model; to a model serving system for execution by the machine-learned generative model; from the model serving system; by executing the machine-learned generative model; applying an image segmentation model to a first digital flyer to parse the first digital flyer into a plurality of image segments, wherein each image segment is classified into one of a plurality of segment categories including one segment category relating to images of items; applying an item matching model to each image segment classified into the segment category relating to images of items to match the image segment to an item in the item catalog; and on an electronic display of a client device” do not integrate the judicial exception (abstract idea) into a practical application because of the analysis provided in Step 2A, Prong II. Claims 1, 14, and 20 also recite additional limitations including – “augmenting the first digital flyer by overlaying one or more user-interactable elements on one or more image segments, wherein each user-interactable element is encoded to, responsive to a user interaction with the image segment, perform one or more actions related to the item matched to the image segment.” However, merely adding UI elements to the content in order to receive input from users is seen as simply appending well-understood, routine, conventional activities previously known to the industry, specified at a high level of generality, to the judicial exception - see MPEP 2106.05(d) and Berkheimer Memo. For example, according to Col. 7 Lines 6-9 of U.S. Patent 10,863,230 to Pham; “In conventional overlay UI applications or user interface applications that utilize overlay elements, users may provide input or place the overlay UI elements 204, 206, and 208 in any area or portion of user interface 200. For example, the user (player 212) may provide input to move overlay UI element 204 to position 220 which would obscure the primary portion or area of content 218 from viewers who are consuming the content provided by user interface 200.” Independent claims 1, 14, and 20 do not include additional elements or a combination of elements that result in the claims amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional elements listed amount to no more than mere instructions to apply an exception using a generic computer component. In addition, the applicant’s specifications describe generic computer-based elements, ¶ [0132], for implementing “computer processor”, which do not amount to significantly more than the abstract idea of itself, which is not enough to transform an abstract idea into eligible subject matter. Furthermore, there is no improvement in the functioning of the computer or technological field, and there is no transformation of subject matter into a different state. Under Step 2B in a test for patent subject matter eligibility, these claims are not patent eligible.
Dependent claims 2-9, 11-13, and 15, 16, 18, and 19 further recite independent claims 1 and 14, respectively. Dependent claims 2-9, 11-13, 15, 16, 18, and 19 when analyzed as a whole are held to be patent ineligible under 35 U.S.C. 101 because the additional recited limitation fail to establish that the claims are not directed to an abstract idea:
Under Step 2A, Prong I, these additional claims only further narrow the abstract idea set forth in Claims 1, 14, and 20. For example, claims 2-9, 11-13, 15, 16, 18, and 19 describe the limitations for providing a digital flyer for presentation to a user based on a batch of digital flyers determined according to a query including design conditions and accessed item data – which is only further narrowing the scope of the abstract idea recited in the independent claims.
Under Step 2A, Prong II, for dependent claims 2-9, 11-13, 15, 16, 18, and 19, there are no additional elements introduced. For example, dependent claims 9, 11, 12, 18, and 19 recite applying a multimodal model / object recognition algorithm / natural language processing algorithm. However, merely applying a model/algorithm to data in order to determine data does not integrate the judicial exception into a practical application because they are seen as adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea - see MPEP 2106.05(f) and generally linking the use of the judicial exception to a particular technological environment or field of use – see MPEP 2106.05(h). Thus, they do not present integration into a practical application, or amount to significantly more.
Under Step 2B, the dependent claims 2-9, 11-13, 15, 16, 18, and 19 do not include any additional elements that are sufficient to amount to significantly more than the judicial exception. Additionally, there is no improvement in the functioning of the computer or technological field, and there is no transformation of subject matter into a different state. As discussed above with respect to integration of the abstract idea into a practical application, the additional claims do not provide any additional elements that would amount to significantly more than the judicial exception. Under Step 2B, these claims are not patent eligible.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-9, 11, 12, 14-16, and 18-20 are rejected under 35 U.S.C. 103 as being unpatentable over U.S. Publication 2024/0362427 to Gupta in view of U.S. Publication 2022/0377424 to Deng and in further view of U.S. Patent 10,949,907 to Jain.
Claims 1-13, 14-19, and 20 are method, computer-readable media, and system claims, respectively, with substantially indistinguishable features between each group. For purposes of compact prosecution, the Office has grouped the common method, system and non-transitory computer readable storage medium claims in applying applicable prior art.
With respect to Claim 1:
Gupta teaches:
A computer-implemented method comprising: receiving, from a client device, a request to generate a digital flyer, wherein the request includes one or more design conditions for the digital flyer (i.e. receiving input from user to generate digital content/pamphlet/flyer, wherein the input includes design conditions or objective or vector representation describing candidate layout) (Gupta: ¶¶ [0031] [0032] “The generation module 110 is illustrated as having, receiving, and/or transmitting input data 114 describing a characteristic 116 for digital content. For instance, the digital content is to be generated by the generation module 110 and the characteristic 116 indicates an objective for the digital content and/or how to generate the digital content. In the illustrated example, the characteristic 116 is a natural language statement of "Invitation to hot air balloon festival in Albuquerque." In this example, the characteristic 116 is an objective of the digital content (e.g., to invite pilots/passengers/observers of hot air balloons to a festival in Albuquerque)… In other examples, the generation module 110 generates the vector representation of the characteristic 116 using a hash function such as a locality-sensitive hash function. For example, the generation module 110 leverages the vector representation of the characteristic 116 to identify candidate layouts and/or candidate strategies for achieving the objective of the digital content that is to be generated.”);
generating a query for a machine-learned generative model including a prompt to generate the digital flyer, the one or more design conditions, and the item data [[accessed from the item catalog]] (i.e. generate query/input data to generate digital content/flyer, the input data including design conditions such as festival in Albuquerque and item data such as hot air balloon) (Gupta: ¶ [0035] “In an example, the generation module 110 generates input text for processing by a first machine learning model. In this example, the input text includes indications of the types of digital content components in the relative order. For instance, the input text also includes the characteristic 116 which is the objective of the digital content to be generated by the generation module 110 (e.g., to invite pilots and passengers of hot air balloons to a festival in Albuquerque).” Furthermore, as cited in ¶ [0031] “For instance, the digital content is to be generated by the generation module 110 and the characteristic 116 indicates an objective for the digital content and/or how to generate the digital content. In the illustrated example, the characteristic 116 is a natural language statement of "Invitation to hot air balloon festival in Albuquerque." In this example, the characteristic 116 is an objective of the digital content (e.g., to invite pilots/passengers/observers of hot air balloons to a festival in Albuquerque).”);
providing the query to a model serving system for execution by the machine-learned generative model (i.e. input is provided to machine learning model for execution) (Gupta: ¶ [0037] “The first machine learning model is included in or available to the generation module 110, and the generation module 110 implements the first machine learning model to process the input text. For example, the generation module 110 receives the output text generated by the first machine learning model based on processing the input text, and the output text is formatted in the JavaScript Object Notation. For instance, the output text includes descriptions of the types of digital content components in the relative order. The output text also includes alternative text generated for the types of digital content components that are digital images.”);
receiving, from the model serving system, a batch of one or more digital flyers generated by executing the machine-learned generative model on the query (i.e. receives digital content and alternative digital content corresponding to the input prompt executed by the machine learning model) (Gupta: ¶ [0054] “The output text 502 also includes alternative text 506 for a type of digital content component that is a second digital image; alternative text 508 for a type of digital content component that is a third digital image; and alternative text 510 for a type of digital content component that is a fourth digital image. The alternative text 506 is "desert canyons;" the alternative text 508 is "aerial view of city;" and the alternative text 510 is "balloon in sky." The language module 206 generates the text data 214 as describing the output text 502.” Furthermore, as cited in ¶ [0057] “The display module 208 generates digital content component 604 based on the alternative text 506 of "desert canyons," and the digital content component 604 is a digital image depicting a desert landscape with canyons. Similarly, the display module 208 generates digital content component 606 using the alternative text 508 of "aerial view of city." As shown in FIG. 6, the digital content component 606 is a digital image that depicts a portion of a town viewed from a high elevation. Finally, the display module 208 generates digital content component 608 based on the alternative text 510 of "balloon in sky," and the digital content component 608 is a digital image depicting a hot air balloon flying in the sky.”); and
causing presentation of the [[augmented]] first digital flyer on an electronic display of a client device of a user (i.e. providing the digital content/flyer/pamphlet for display) (Gupta: ¶ [0057] “The display module 208 generates digital content component 604 based on the alternative text 506 of "desert canyons," and the digital content component 604 is a digital image depicting a desert landscape with canyons. Similarly, the display module 208 generates digital content component 606 using the alternative text 508 of "aerial view of city." As shown in FIG. 6, the digital content component 606 is a digital image that depicts a portion of a town viewed from a high elevation. Finally, the display module 208 generates digital content component 608 based on the alternative text 510 of "balloon in sky," and the digital content component 608 is a digital image depicting a hot air balloon flying in the sky.”).
Gupta does not explicitly disclose accessing an item catalog storing item data; and augmenting the first digital flyer by overlaying one or more user-interactable elements on one or more image segments, wherein each user-interactable element is encoded to, responsive to a user interaction with the image segment, perform one or more actions related to the item matched to the image segment.
However, Deng further discloses:
accessing an item catalog storing item data (i.e. accessing product catalog storing product data) (Deng: ¶ [0063] “The method begins at 402 wherein store trend data comprising product browsing data and purchase transactions of a given merchant, such as a store are logged. In an example, the store may be an online store or a physical store and the product browsing data of the online store may be collected from user clicks, web feeds that may provide real-time data such as user comments, or data from any live online events featured by the store, etc… The trained deep learning (DL)s are used to extract the product features from the store trend data at 406. The data thus extracted may be stored as product knowledge at 408. The extracted data is aggregated at 410 into higher levels such as but not limited to product groups, product families, product lines, catalogs, etc., to distill the corresponding level of product knowledge.”); and
augmenting the first digital flyer by overlaying one or more user-interactable elements on one or more image segments, wherein each user-interactable element is encoded to, responsive to a user interaction with the image segment, perform one or more actions related to the item matched to the image segment (i.e. overlaying digital content, which may be an image, with user interactable elements responsive to user interaction related to image of digital content in order to perform actions such as already purchased item) (Deng: ¶¶ [0114] [0115] “The processor 702 may execute instructions 736 to provide a digital content item 722 enabled for collecting feedback data on the user device 792. The user 796 may provide feedback regarding whether the digital content item 722 may be added to the exclusions list 774 in the user profile 772 associated with the user 796. Particularly, the user 796 may be enabled to provide feedback regarding whether the user 796 wishes to receive similar digital content items or digital content items from the same source e.g., advertisements from the same advertiser…Various options are discussed below for an example wherein the digital content item 722 may include an advertisement. The digital content item 722 may include a link such as "Why I Am Seeing The Ad" (WAIST), an overlay, or other user interaction mechanism that allows the user navigation to a tool to manage digital content item preferences. A screen/tab may allow the user 796 to select from one or more options which may include 1) Action already taken/already purchased, 2) Repeated digital content, and 3) Don't like this digital content item. Another tab/screen of the tool may allow the user 796 to choose one of 1) Take me off from this ad 2) Take me off this and similar ads and 3) Take me off all ads from this advertiser. When the user 796 selects option (1) to "Take me off from this ad", it may translate to exclude the user 796 from that ad and all ads from the same ad set which may have been created from the permutations/combinations of contents of the digital content item 722 by varying creatives, contents, image/video formats, colors, durations, frame ratio, other degrees of freedom.”).
Therefore, it would have been obvious to one of ordinary skill in the art, at the time the invention was made, to add Deng’s accessing an item catalog storing item data; and augmenting the first digital flyer by overlaying one or more user-interactable elements on one or more image segments, wherein each user-interactable element is encoded to, responsive to a user interaction with the image segment, perform one or more actions related to the item matched to the image segment to Gupta’s causing presentation of a first digital flyer on an electronic display of a client device. One of ordinary skill in the art would have been motivated to do so because “With the proliferation of different types of digital content delivery mechanisms (e.g., mobile phone, portable computing devices, tablet devices, etc.), it has become crucial that content providers, such as vendors looking to advertise a good or service, engage users with content of interest. As a result, content providers are continuously looking for ways to deliver more appealing content.” (Deng: ¶ [0137]).
Gupta and Deng do not explicitly disclose applying an image segmentation model to a first digital flyer to parse the first digital flyer into a plurality of image segments, wherein each image segment is classified into one of a plurality of segment categories including one segment category relating to images of items; applying an item matching model to each image segment classified into the segment category relating to images of items to match the image segment to an item in the item catalog; and providing presentation of the augmented first digital flyer on an electronic display of a client device of a user.
However, Jain further discloses:
applying an image segmentation model to a first digital flyer to parse the first digital flyer into a plurality of image segments, wherein each image segment is classified into one of a plurality of segment categories including one segment category relating to images of items (i.e. extracting image data from content, wherein the image data is segmented into a plurality of objects according to image classification model) (Jain: Col. 9 Lines 10-23 “Each product listed is described by image data, different classes of textual data including numerical data (e.g. title, product description, price), and categorical data (e.g., color, condition). In generating these matches, the PRICE M4 system may first analyze image data and textual data associated with the reference product to extract categorical data (e.g., convertible crib, furniture, pine wood, DaVinci brand, etc.), then select a matching model based on available categorical data and/or other factors such as user preferences on shipping and delivery, before applying the selected matching model to compare the multi-modal attribute data of candidate products to those of the reference product.” Furthermore, as cited in Col. 18 Lines 42-53 “FIG. 10 is an exemplary architecture diagram 1000 showing a visual data analyzer 655 having an architecture similar to audio data analyzer 645, according to some embodiments of the present invention. Input image and/or video data 650 may be passed through an image recognition and classification NN 1030 and a categorical attribute extraction module 1040, while in parallel being pre-processed, and analyzed through a Siamese Convolutional NN 1050 to generate signatures 1052 and 1054. Imagine classification NN 1030 may comprise one or more neural networks to segment objects from an image, to extract textual information from the image, and to perform scene analysis when necessary.”);
applying an item matching model to each image segment classified into the segment category relating to images of items to match the image segment to an item in the item catalog (i.e. candidate products are matched to the reference product or the image segment classified into category of items) (Jain: Col. 9 Lines 14-61 “In generating these matches, the PRICE M4 system may first analyze image data and textual data associated with the reference product to extract categorical data (e.g., convertible crib, furniture, pine wood, DaVinci brand, etc.), then select a matching model based on available categorical data and/or other factors such as user preferences on shipping and delivery, before applying the selected matching model to compare the multi-modal attribute data of candidate products to those of the reference product…Candidate products that are compared to the reference product may be in various conditions, states, or types, including but not limited to, new, used, refurbished, renewed, open-box, generic, rental, offline, local resale, and auction…The reference product may be compared to a set of candidate products from database 235 to find a list of matching products, using a matching model selected based on the user query and retrieved from database 235.” Furthermore, as cited in Col. 17 Lines 58-67 “In a Siamese network for image mode data, standard CNN models may be used for identifying items of interest, such as color, text that can be fed into Optical Character Recognition (OCR) readers, and shape. Furthermore, training of individual Siamese NNs mentioned above may occur as part of the overall training for the system 600 shown in FIG. 6, and training data may be prepared manually based on a specific product class or category that the reference and/or candidate products fall into.”); and
causing presentation of the augmented first digital flyer on an electronic display of a client device of a user (i.e. candidate products are provided for presentation) (Jain: Col. 19 Lines 20-62 “The web browser extension may provide support for a dropdown toolbar which searches the PRICE database for matching product offers, some at cheaper prices than the product currently being viewed in the browser. In this particular example, the tablet product listed for $329.99 has been found at another site for $139.99. Clicking on the link in the PRICE extension toolbar leads to a product display page (PDP) on PRICE.COM showing many other sites where this product may be purchased, with a majority of the matching offers being at a lower price… FIGS. 13A and 13B are respective illustrative screencaps showing a use case for the PRICE product matching system on a social media platform, according to some embodiments of the present invention. Upon the user's permission, PRICE may assess user needs for specific products, user interests in particular products or discounts, and user preferences on particular product features through user information including IP address, saved address, browsing histories, shopping histories, and the like. The PRICE system may further take into account of products, discussion, and events current trending on the platform, for the purpose of product recommendation to the user, for example in the form of social media posts or individual alerts. In FIG. 13A, a social media post is presented to recommend disinfectants to a user who follows PRICE's social media account during a respiratory virus pandemic. Once the user clink on the provided link, an interface similar to the one shown in FIG. 4A may be displayed. Similarly, in FIG. 13B, not only a social media post is presented, but a reminder popup is generated to collect user inputs for product search and product match.”).
Therefore, it would have been obvious to one of ordinary skill in the art, at the time the invention was made, to add Jain’s applying an image segmentation model to a first digital flyer to parse the first digital flyer into a plurality of image segments, wherein each image segment is classified into one of a plurality of segment categories including one segment category relating to images of items; applying an item matching model to each image segment classified into the segment category relating to images of items to match the image segment to an item in the item catalog; and causing presentation of the augmented first digital flyer on an electronic display of a client device of a user to Gupta’s causing presentation of a first digital flyer on an electronic display of a client device. One of ordinary skill in the art would have been motivated to do so in order “to develop a high-performance product matching system that fully exploits all available descriptive data when matching commercial products.” (Jain: Col. 2 Lines 12-14).
With respect to Claims 14 and 20:
All limitations as recited have been analyzed and rejected to claim 1. Claim 14 recites “A non-transitory computer-readable storage medium storing instructions that, when executed by a computer processor, cause the computer processor to perform operations comprising:” (Gupta: ¶ [0073]) the steps of method claim 1. Claim 20 recites “A computing system comprising: a computer processor; and a non-transitory computer-readable storage medium storing instructions that, when executed by the computer processor, cause the computer processor to perform operations comprising: (Gupta: ¶ [0072]) the steps of method claim 1. Claims 14 and 20 do not teach or define any new limitations beyond claim 1. Therefore they are rejected under the same rationale.
With respect to Claim 2:
Gupta does not explicitly disclose the computer-implemented method of claim 1, wherein a first design condition includes one or more cornerstone items of a retailer in the digital flyer, and wherein generating the query comprises generating the query to include instructions to promote the cornerstone items indicated in the first design condition.
However, Deng further discloses wherein a first design condition includes one or more cornerstone items of a retailer in the digital flyer, and wherein generating the query comprises generating the query to include instructions to promote the cornerstone items indicated in the first design condition (i.e. customized content design includes trendy products or products that are popular to the user, and wherein instructions or query comprise trendy products or products that are popular to the user to be included in the customized content) (Deng: ¶ [0063] “The content customized to the trend data that matches the user preferences may be thus identified or generated. Ads creatives may be customized based on the trend preferences of the user. When creating an ad for a targeted audience segment, trend embeddings from users in the targeted segment may be aggregated to select the best products and creatives featured in the ad. The customized content may be transmitted to the user at 510. For example, ads, including images, audio, or text data customized to the matching trend data may be identified. The ad impressions that are delivered may include but are not limited to text, description, language, call to action, friend's endorsements/likes/purchases, etc. The most trending product may be identified and the most trending creatives may be dynamically generated to deliver this unique customized ad impression to yield optimal results, influenced by the user's social circle.” Furthermore, as cited in ¶ [0066] “When designing a customized storefront, the trendiest product catalog may be generated to showcase to say visitors from different places thereby allowing users to identify trends at various geographic locales. In an example, the trendiest product catalog may include a dynamically ordered listing of products that are being discussed/purchased most as identified from the store trend data. As new store trend data is collected, the trendy product catalog may be updated dynamically. Furthermore, the store trend data when matched with the user preference data enables a merchant to present a user-specific trendy product catalog. Different content items such as an image may be transmitted concurrently with an audio content item so that both the items which pertain to trend data matching user preferences may be provided to the user simultaneously. In an example, the store trend data may be provided to an ad platform wherein the AI bots of the ad platform select suitable ads to be presented in response to the trend data. Successful conversions such as purchases may be used as positive training labels, while abandoned shopping carts as negative training labels, to enhance the deep learning (DL) models gravitating towards ever-changing patterns between products and users. Accordingly, the data collected from the user browsing history, purchase data, abandoned cart data, etc., may be collected as feedback at 512. The data thus collected at 512 may be employed to refresh the artificial intelligence (AI)/deep learning (DL) models that are used to match the user embeddings with the product embeddings at 514.”).
Therefore, it would have been obvious to one of ordinary skill in the art, at the time the invention was made, to add Deng’s first design condition includes one or more cornerstone items of a retailer in the digital flyer, and wherein generating the query comprises generating the query to include instructions to promote the cornerstone items indicated in the first design condition to Gupta’s causing presentation of a first digital flyer on an electronic display of a client device. One of ordinary skill in the art would have been motivated to do so because “With the proliferation of different types of digital content delivery mechanisms (e.g., mobile phone, portable computing devices, tablet devices, etc.), it has become crucial that content providers, such as vendors looking to advertise a good or service, engage users with content of interest. As a result, content providers are continuously looking for ways to deliver more appealing content.” (Deng: ¶ [0137]).
With respect to Claim 3:
Gupta teaches:
The computer-implemented method of claim 1, wherein a first design condition includes a theme for the digital flyer indicating a season, a holiday, or an event, and wherein generating the query comprises generating the query to include instructions to generate the digital flyer according to the theme (i.e. layouts include theme or objective for digital content, wherein the objective indicates a hot air balloon festival in Albuquerque or a kite festival and is used as input to generate the digital content/flyer according to the theme) (Examiner takes official notice that a hot air balloon festival is an event, kite festivals indicate a holiday, and hot air balloon festival indicates a seasonal event) (Gupta: ¶¶ [0031] [0032] “The generation module 110 is illustrated as having, receiving, and/or transmitting input data 114 describing a characteristic 116 for digital content. For instance, the digital content is to be generated by the generation module 110 and the characteristic 116 indicates an objective for the digital content and/or how to generate the digital content. In the illustrated example, the characteristic 116 is a natural language statement of "Invitation to hot air balloon festival in Albuquerque." In this example, the characteristic 116 is an objective of the digital content (e.g., to invite pilots/passengers/observers of hot air balloons to a festival in Albuquerque).” Furthermore, as cited in ¶ [0051] “For types of the digital content components that are digital images, the input text 404 requests alternative text describing the digital images. The input text 404 includes portions of the objective 302 such as natural language text stating "Invite registrations for kite flying festival." As shown, the input text 404 also includes a discount of"15% off" based on the particular strategy for achieving the objective 302 of "Incentives."”).
With respect to Claim 4:
Gupta teaches:
The computer-implemented method of claim 1, wherein the request includes a template flyer, and wherein generating the query comprises generating the query to include instructions to use a layout of the template flyer (i.e. prompt includes request to generate digital content or flyer and the prompt includes layout of digital content) (Gupta: ¶ [0052] “For example, the prompt module 204 generates the prompt data 212 as describing the input text 404. The language module 206 receives and processes the prompt data 212 in order to generate text data 214. FIG. 5 illustrates a representation 500 of generating output text by processing input text. As shown, the representation 500 includes the input text 404.” Furthermore as cited in ¶ [0050] “The prompt module 204 receives and processes the match data 210 in order to generate prompt data 212. FIG. 4 illustrates a representation 400 of generating input text. As shown, the representation 400 includes the digital template 306 described by the match data 210, and the prompt module 204 parameterizes the particular layout with placeholders 402 which will be replaced by digital content components generated using the second machine learning model. The prompt module 204 classifies the placeholders 402 into types of digital content components such as digital images, slogans ( e.g., lines of text with less than 10 words rendered using a font having a weight greater than 700), paragraphs of text, headings/headers, footers, call-to-action buttons, etc. The prompt module 204 also classifies content blocks of the particular layout which include digital images and/or text.”).
With respect to Claim 5:
Gupta does not explicitly disclose the computer-implemented method of claim 1, wherein accessing the item catalog comprises accessing the item catalog to retrieve information on available promotions on a subset of the items represented in the item catalog.
However, Deng further discloses wherein accessing the item catalog comprises accessing the item catalog to retrieve information on available promotions on a subset of the items represented in the item catalog (i.e. accessing store data to retrieve information on products available for sale) (Deng: ¶¶ [0064] [0065] “The store data such as but not limited to ad statistics, shop traffic, shopping cart events such as completed transactions, abandoned carts, etc., transaction data, including purchases, exchanges, returns are also logged at 412. Again, deep learning (DL) models may be trained at 412 on the store data collected at 410 to capture users' preferences on different features such as colors, shapes, styles, etc. The deep learning (DL) models are therefore trained to identify trends for a combination of different factors. For example, when a user A is browsing for product B from a country C, etc., then the specific trend embedding of (A, B, C, ... ) may be read or refreshed as needed based on a received user request…At 506, the products available for sale in the store may be ranked based on the distances between the user profile data and the product embeddings or product data. The user preference data from the user profile may be matched with the store trend data. For example, user trends and product trend embeddings may be matched using a two-tower model calculating embedding distances for the best experience. At 508, the top-ranked products may be filtered for display to the user.” Furthermore, ¶¶ [0123]-[0130] illustrate incentives or other types of promotions.).
Therefore, it would have been obvious to one of ordinary skill in the art, at the time the invention was made, to add Deng’s accessing the item catalog comprises accessing the item catalog to retrieve information on available promotions on a subset of the items represented in the item catalog to Gupta’s causing presentation of a first digital flyer on an electronic display of a client device. One of ordinary skill in the art would have been motivated to do so because “With the proliferation of different types of digital content delivery mechanisms (e.g., mobile phone, portable computing devices, tablet devices, etc.), it has become crucial that content providers, such as vendors looking to advertise a good or service, engage users with content of interest. As a result, content providers are continuously looking for ways to deliver more appealing content.” (Deng: ¶ [0137]).
With respect to Claim 6:
Gupta does not explicitly disclose The computer-implemented method of claim 1, further comprising: obtaining historical order data describing past orders by users of an online system, each past order including one or more items in the item catalog, wherein generating the query comprises generating the query to include the historical order data.
However, Deng further discloses:
obtaining historical order data describing past orders by users of an online system, each past order including one or more items in the item catalog (i.e. accessing store data to retrieve historical information on transactions associated with products) (Deng: ¶ [0064] “The store data such as but not limited to ad statistics, shop traffic, shopping cart events such as completed transactions, abandoned carts, etc., transaction data, including purchases, exchanges, returns are also logged at 412. Again, deep learning (DL) models may be trained at 412 on the store data collected at 410 to capture users' preferences on different features such as colors, shapes, styles, etc. The deep learning (DL) models are therefore trained to identify trends for a combination of different factors. For example, when a user A is browsing for product B from a country C, etc., then the specific trend embedding of (A, B, C, ... ) may be read or refreshed as needed based on a received user request.”),
wherein generating the query comprises generating the query to include the historical order data (i.e. store data comprising historical information on transactions associated with products is used to as input to train machine learning model) (Deng: ¶ [0064] “The store data such as but not limited to ad statistics, shop traffic, shopping cart events such as completed transactions, abandoned carts, etc., transaction data, including purchases, exchanges, returns are also logged at 412. Again, deep learning (DL) models may be trained at 412 on the store data collected at 410 to capture users' preferences on different features such as colors, shapes, styles, etc. The deep learning (DL) models are therefore trained to identify trends for a combination of different factors. For example, when a user A is browsing for product B from a country C, etc., then the specific trend embedding of (A, B, C, ... ) may be read or refreshed as needed based on a received user request.”).
Therefore, it would have been obvious to one of ordinary skill in the art, at the time the invention was made, to add Deng’s obtaining historical order data describing past orders by users of the online system, each past order including one or more items in the item catalog, wherein generating the query comprises generating the query to include the historical order data to Gupta’s causing presentation of a first digital flyer on an electronic display of a client device. One of ordinary skill in the art would have been motivated to do so because “With the proliferation of different types of digital content delivery mechanisms (e.g., mobile phone, portable computing devices, tablet devices, etc.), it has become crucial that content providers, such as vendors looking to advertise a good or service, engage users with content of interest. As a result, content providers are continuously looking for ways to deliver more appealing content.” (Deng: ¶ [0137]).
With respect to Claim 7:
Gupta teaches:
The computer-implemented method of claim 1, further comprising: providing, to the client device, the batch of the one or more digital flyers (i.e. provide a second and third alternative digital content with corresponding alternative text) (Gupta: Fig. 6 and ¶ [0054] “The output text 502 also includes alternative text 506 for a type of digital content component that is a second digital image; alternative text 508 for a type of digital content component that is a third digital image; and alternative text 510 for a type of digital content component that is a fourth digital image. The alternative text 506 is "desert canyons;" the alternative text 508 is "aerial view of city;" and the alternative text 510 is "balloon in sky." The language module 206 generates the text data 214 as describing the output text 502.” Furthermore, as cited in ¶ [0057] “The display module 208 generates digital content component 604 based on the alternative text 506 of "desert canyons," and the digital content component 604 is a digital image depicting a desert landscape with canyons. Similarly, the display module 208 generates digital content component 606 using the alternative text 508 of "aerial view of city." As shown in FIG. 6, the digital content component 606 is a digital image that depicts a portion of a town viewed from a high elevation. Finally, the display module 208 generates digital content component 608 based on the alternative text 510 of "balloon in sky," and the digital content component 608 is a digital image depicting a hot air balloon flying in the sky.”); and
receiving, from the client device, a selection of the first digital flyer, wherein providing the first digital flyer for presentation is based on the selection (i.e. user defines order of digital content, wherein defining order reads on selecting a first digital flyer/content) (Gupta: element 610 in Fig. 6 and ¶ [0058] “In order to generate digital content 610 that includes the digital content components 602-608, the display module 208 leverages the relative order of the types of digital content components defined by the particular layout of the digital template 306 which is also encoded in the output text 502 in the JavaScript Object Notation format. Using this relative order, the display module 208 generates the digital content 610 as including the digital content component 602 as a hero image and including the digital content components 604-608 as section images.”).
With respect to Claim 15:
All limitations as recited have been analyzed and rejected to claim 7. Claim 15 does not teach or define any new limitations beyond claim 7. Therefore it is rejected under the same rationale.
With respect to Claim 8:
Gupta teaches:
The computer-implemented method of claim 7, further comprising: receiving, from the client device, one or more modifications to the first digital flyer (i.e. receiving alternative text for the digital content) (Gupta: ¶ [0053] “In an example, the language module 206 implements the first machine learning model to generate output text 502 in the JavaScript Object Notation format by processing the input text 404. For example, the output text 502 includes alternative text 504 for a type of digital content component that is a first digital image. In this example, the alternative text 504 is "hot air balloon." For instance, the alternative text 504 describes an object.”);
generating a subsequent query including a prompt to modify the first digital flyer according to the received one or more modifications (i.e. alternative text data is used as a prompt to generate modified digital content) (Gupta: ¶¶ [0055] [0056] “The display module 208 receives the text data 214 describing the output text 502. For example, the display module 208 includes or has access to the second machine learning model which includes a generative machine learning model. Examples of generative machine learning models included in the second machine learning model include a model trained on training data to generate digital images, a diffusion model, a Generative Pre-Trained Transformer 4 model (GPT-4), a Hierarchical Text-Conditional Image Generation with CLIP Latents model (DALLE 2), etc. In some examples, the second machine learning model includes systems of generative machine learning models…In an example, the display module 208 implements the second machine learning module to generate digital content components 602-608 by processing the output text 502. For instance, the display module 208 generates digital content component 602 using the second machine learning model based on the alternative text 504 of "hot air balloon." As shown in the representation 600, the digital content component 602 is a digital image that depicts a hot air balloon.”); and
providing the subsequent query to the model serving system for execution by the generative model (i.e. prompt with alternative text is provided to machine learning model in order to generate modified digital content) (Gupta: ¶ [0056] “In an example, the display module 208 implements the second machine learning module to generate digital content components 602-608 by processing the output text 502. For instance, the display module 208 generates digital content component 602 using the second machine learning model based on the alternative text 504 of "hot air balloon." As shown in the representation 600, the digital content component 602 is a digital image that depicts a hot air balloon.”); and
receiving, from the model serving system, a modified version of the first digital flyer generated by executing the generative model on the subsequent query, wherein providing the first digital flyer for presentation comprises providing the modified version of the first digital flyer for presentation (i.e. providing alternative digital content corresponding to alternative text) (Gupta: Fig. 6 and ¶ [0057] “The display module 208 generates digital content component 604 based on the alternative text 506 of "desert canyons," and the digital content component 604 is a digital image depicting a desert landscape with canyons. Similarly, the display module 208 generates digital content component 606 using the alternative text 508 of "aerial view of city." As shown in FIG. 6, the digital content component 606 is a digital image that depicts a portion of a town viewed from a high elevation. Finally, the display module 208 generates digital content component 608 based on the alternative text 510 of "balloon in sky," and the digital content component 608 is a digital image depicting a hot air balloon flying in the sky.”).
With respect to Claim 16:
All limitations as recited have been analyzed and rejected to claim 8. Claim 16 does not teach or define any new limitations beyond claim 8. Therefore it is rejected under the same rationale.
With respect to Claim 9:
Gupta and Deng do not explicitly disclose the computer-implemented method of claim 1, wherein the machine-learned generative model is a multimodal model.
However, Jain further discloses wherein the machine-learned generative model is a multimodal model (i.e. machine learning model is a multimodal model) (Jain: Col. 8 Lines 40-45 “Unlike conventional systems, PRICE is a novel, deep learning-based, multi-model, multi-modal (M4) product matching system that leverages both structured and unstructured data in finding identical, equivalent, and/or generic matches to a given reference product.” Furthermore, as cited in Col. 16 Lines 1-8 “FIG. 6 is an exemplary architecture diagram 600 for a neural-network based, multi-modal matching model, according to some embodiments of the present invention. In an M4 architecture's general usage, an object pair (X,Y) may be classified into mutually exclusive and exhaustive subsets U1, j=1, 2, ... , r that will have a generic M4 with architecture as in FIG. 6, but with different parameters and/or hyperparameters for each j.”).
Therefore, it would have been obvious to one of ordinary skill in the art, at the time the invention was made, to add Jain’s machine-learned generative model is a multimodal model to Gupta’s causing presentation of a first digital flyer on an electronic display of a client device. One of ordinary skill in the art would have been motivated to do so in order “to develop a high-performance product matching system that fully exploits all available descriptive data when matching commercial products.” (Jain: Col. 2 Lines 12-14).
With respect to Claim 11:
Gupta and Deng do not explicitly disclose the computer-implemented method of claim 1, wherein applying the item matching model comprises: applying an object recognition algorithm to the image segment and the image of each item in the item catalog to determine a similarity score between the image segment and the image of the item; and matching the image segment to the item in the item catalog with a highest similarity score.
However, Jain further discloses:
applying an object recognition algorithm to the image segment and the image of each item in the item catalog to determine a similarity score between the image segment and the image of the item (i.e. applying object recognition model to content in order to extract image segments into classified categories, wherein the model also determines a match score between the image segment associated with the reference product and candidate products) (Jain: Col. 18 Lines 42-53 “FIG. 10 is an exemplary architecture diagram 1000 showing a visual data analyzer 655 having an architecture similar to audio data analyzer 645, according to some embodiments of the present invention. Input image and/or video data 650 may be passed through an image recognition and classification NN 1030 and a categorical attribute extraction module 1040, while in parallel being pre-processed, and analyzed through a Siamese Convolutional NN 1050 to generate signatures 1052 and 1054. Imagine classification NN 1030 may comprise one or more neural networks to segment objects from an image, to extract textual information from the image, and to perform scene analysis when necessary.” Furthermore, as cited in Col. 13 Lines 1-10 “The match score m may refer to any quantitative distance, probability, correlation, resemblance, and likelihood measures that may directly or indirectly indicate whether two products resemble or are identical to each other. For example, in some embodiments, identical products that match exactly, and/or equivalent or generic products that match with a non-zero likelihood or probability may be returned from the matching process in step 360, where a computed match score may be compared to a predetermined or a dynamic threshold.”); and
matching the image segment to the item in the item catalog with a highest similarity score (i.e. candidate products with the highest match score or match score above a threshold are selected) (Jain: Col. 16 Lines 15-25 “Once a M4 selected, attribute data for the reference product and attribute for a candidate product are compared to generate a matching score or matching probability. FIG. 6 is an exemplary architecture diagram 600 for a multimodal product matching model, according to some embodiments of the present invention. Product information 610 for both the reference product and a candidate product from the PRICE database may be partitioned into different modes, including but not limited to, text data 620, categorical and engineered attribute data 630, audio and speech data 640, and image and video data 650.” Furthermore, as cited in Col. 13 Lines 11-42 “It would be understood by persons skilled in the art that the term "match score" above may refer to a similarity score, similarity measure, distance metric, distance measure, variations thereof, and the like, all of which may indicate quantitative, mathematical measures of closeness or resemblance of two objects. Herein, all such concepts are subsumed under the term "match score." In the case of a similarity score, high values may indicate closeness, and in the case of a distance measure, low values may indicate closeness. Having a match score meeting a threshold refers to having a similarity score above the threshold, or having a distance metric below the threshold. When such measures are normalized, for example, between O and 1, a distance metric may be computed from a similarity score by a subtracting the similarity score from 1. Furthermore, matching results may be further validated at a step 370 to remove identifiable errors, such as those with outlier prices, and the remaining match results 380 may be returned to the user. In some embodiments, post product validation process 370 may include steps to simply compare a price of each matching product to an average price to remove outliers. In some embodiments, post product validation process 370 may include steps to cluster matching products from step 360 based on an attribute data item such as price, shipping cost, or inventory information, to determine whether an outlier cluster exists, and to remove the outlier cluster correspondingly. While not shown in FIG. 3, the validated matching products 380 may be presented to the user, and upon a user selection of one of the matching products, an online purchase page for the selected product, or a link to the purchase page, may be provided to the user.”).
Therefore, it would have been obvious to one of ordinary skill in the art, at the time the invention was made, to add Jain’s applying an object recognition algorithm to the image segment and the image of each item in the item catalog to determine a similarity score between the image segment and the image of the item; and matching the image segment to the item in the item catalog with a highest similarity score to Gupta’s causing presentation of a first digital flyer on an electronic display of a client device. One of ordinary skill in the art would have been motivated to do so in order “to develop a high-performance product matching system that fully exploits all available descriptive data when matching commercial products.” (Jain: Col. 2 Lines 12-14).
With respect to Claim 18:
All limitations as recited have been analyzed and rejected to claim 11. Claim 18 does not teach or define any new limitations beyond claim 11. Therefore it is rejected under the same rationale.
With respect to Claim 12:
Gupta does not explicitly disclose the computer-implemented method of claim 11, wherein applying the item matching model further comprises: applying a natural language processing algorithm to an image segment classified into another segment category relating to item information in text form to identify one or more items in the item catalog described by the item information; and matching the image segment to the item in the item catalog based on the identified one or more items.
However, Deng further discloses:
applying a natural language processing algorithm to an image segment classified into another segment category relating to item information in text form to identify one or more items in the item catalog described by the item information (i.e. applying NLP to extracted image segments in order to extract text descriptions of products) (Deng: ¶ [0063] ‘For example, a deep learning (DL) may be trained on image data for extraction of features such as shape, whereas another deep learning (DL) may be trained to extract the color feature from video data, whereas yet another deep learning (DL) may be trained to extract descriptions from textual data using natural language processing (NLP) techniques. Therefore, a plurality of deep learning (DL)s are trained to extract specific product attributes from the trend data. The trained deep learning (DL)s are used to extract the product features from the store trend data at 406. The data thus extracted may be stored as product knowledge at 408. The extracted data is aggregated at 410 into higher levels such as but not limited to product groups, product families, product lines, catalogs, etc., to distill the corresponding level of product knowledge.”); and
matching the image segment to the item in the item catalog based on the identified one or more items (i.e. products are matched according to extracted product attributes or image segment) (Deng: ¶ [0065] “In an example wherein the user is individually identified, the user data may include the user's social circle data so that the users' contact preferences may be factored into the user profile data. At 506, the products available for sale in the store may be ranked based on the distances between the user profile data and the product embeddings or product data. The user preference data from the user profile may be matched with the store trend data. For example, user trends and product trend embeddings may be matched using a two-tower model calculating embedding distances for the best experience. At 508, the top-ranked products may be filtered for display to the user. The content customized to the trend data that matches the user preferences may be thus identified or generated.”).
Therefore, it would have been obvious to one of ordinary skill in the art, at the time the invention was made, to add Deng’s applying a natural language processing algorithm to an image segment classified into another segment category relating to item information in text form to identify one or more items in the item catalog described by the item information; and matching the image segment to the item in the item catalog based on the identified one or more items to Gupta’s causing presentation of a first digital flyer on an electronic display of a client device. One of ordinary skill in the art would have been motivated to do so because “With the proliferation of different types of digital content delivery mechanisms (e.g., mobile phone, portable computing devices, tablet devices, etc.), it has become crucial that content providers, such as vendors looking to advertise a good or service, engage users with content of interest. As a result, content providers are continuously looking for ways to deliver more appealing content.” (Deng: ¶ [0137]).
With respect to Claim 19:
All limitations as recited have been analyzed and rejected to claim 12. Claim 19 does not teach or define any new limitations beyond claim 12. Therefore it is rejected under the same rationale.
Claim(s) 13 is rejected under 35 U.S.C. 103 as being unpatentable over Gupta, Deng, and Jain in further view of U.S. Publication 2018/0189620 to Sheth.
With respect to Claim 13:
Gupta does not explicitly disclose the computer-implemented method of claim 11, wherein augmenting the first digital flyer comprises augmenting the first digital flyer with one or more user-interactable elements configured to, responsive to user interaction with the image segment: magnify the image segment; present additional details relating to the item matched to the image segment; provide an option to add the item matched to the image segment to an order; provide an option to view one or more similar items to the item matched to the image segment; provide an option to view one or more recipes using the item matched to the image segment; or present an available promotion for the item matched to the image segment.
However, Deng further discloses:
wherein augmenting the first digital flyer comprises augmenting the first digital flyer with one or more user-interactable elements configured to, responsive to user interaction with the image segment: […] present additional details relating to the item matched to the image segment (i.e. interaction includes user can purchase or browse which require additional details upon user interaction) (Deng: ¶ [0063] “The method begins at 402 wherein store trend data comprising product browsing data and purchase transactions of a given merchant, such as a store are logged. In an example, the store may be an online store or a physical store and the product browsing data of the online store may be collected from user clicks, web feeds that may provide real-time data such as user comments, or data from any live online events featured by the store, etc.”);
provide an option to add the item matched to the image segment to an order (i.e. interaction includes adding an item to the digital content or the shopping cart) (Deng: ¶ [0140] “An example of a creating user may be a vendor seeking to generate content items to sell a product or service. As used herein, a "supplemental effect" may include any specified aspect that may be added to a content item. In some examples, the supplemental effect may be added in real-time during generation of the content item by a publishing user.” Furthermore, as cited in ¶ [0086] “As a result, the content data store 605 may involve any digital content associated with online activity of an item such as but not limited to searching, purchasing, adding to cart/wish list, etc., mapping a geography, etc.”);
provide an option to view one or more similar items to the item matched to the image segment […] (i.e. interaction includes user selects option to view similar items in digital content) (Deng: ¶ [0116] “The information or feedback data that may be stored in the exclusions list 774 may include a triplet of the form (user ID, item ID, preferences), wherein the user ID may be explicitly provided by the user, the exact item ID identifiable as the user may have clicked on WAIST on receiving the digital content item 722 and the preference may include the user's choice related to the digital content item 722, similar digital content items and the digital content items from a specified content provider/digital content source.” Furthermore, as cited in ¶ [0120] “It is determined at 868 if further digital content items remain for similarity determination. If yes, the method returns to 860 to select the digital content item, else the method terminates on the end block.”); or
present an available promotion for the item matched to the image segment (i.e. interaction includes available promotions in which the user can click on) (Deng: ¶¶ [0064] [0065] “The store data such as but not limited to ad statistics, shop traffic, shopping cart events such as completed transactions, abandoned carts, etc., transaction data, including purchases, exchanges, returns are also logged at 412. Again, deep learning (DL) models may be trained at 412 on the store data collected at 410 to capture users' preferences on different features such as colors, shapes, styles, etc. The deep learning (DL) models are therefore trained to identify trends for a combination of different factors. For example, when a user A is browsing for product B from a country C, etc., then the specific trend embedding of (A, B, C, ... ) may be read or refreshed as needed based on a received user request…At 506, the products available for sale in the store may be ranked based on the distances between the user profile data and the product embeddings or product data. The user preference data from the user profile may be matched with the store trend data. For example, user trends and product trend embeddings may be matched using a two-tower model calculating embedding distances for the best experience. At 508, the top-ranked products may be filtered for display to the user.”).
Therefore, it would have been obvious to one of ordinary skill in the art, at the time the invention was made, to add Deng’s augmenting the first digital flyer comprises augmenting the first digital flyer with one or more user-interactable elements configured to, responsive to user interaction with the image segment: […] present additional details relating to the item matched to the image segment; provide an option to add the item matched to the image segment to an order; provide an option to view one or more similar items to the item matched to the image segment […]; or present an available promotion for the item matched to the image segment to Gupta’s causing presentation of a first digital flyer on an electronic display of a client device. One of ordinary skill in the art would have been motivated to do so because “With the proliferation of different types of digital content delivery mechanisms (e.g., mobile phone, portable computing devices, tablet devices, etc.), it has become crucial that content providers, such as vendors looking to advertise a good or service, engage users with content of interest. As a result, content providers are continuously looking for ways to deliver more appealing content.” (Deng: ¶ [0137]).
Gupta, Deng, and Jain do not explicitly disclose magnify the image segment; present additional details relating to the item matched to the image segment; provide an option to add the item matched to the image segment to an order; provide an option to view one or more similar items to the item matched to the image segment; provide an option to view one or more recipes using the item matched to the image segment; or present an available promotion for the item matched to the image segment.
However, Sheth further discloses:
magnify the image segment […] (i.e. interaction includes zoom in on image segment) (Sheth: ¶ [0188] “The product 110 image section may also provide user interface capabilities such as zoom, pan, and the like. In many aspects, the image section may be view-only, whereas the text section may support editing, deleting, and entering text.”); or
provide an option to view one or more recipes using the item matched to the image segment (i.e. interaction includes viewing product recipe) (Sheth: ¶ [0188] “In many aspects, when a user selects a text panel, such as ingredients, a corresponding enlarged packaging image of one of the products 110 that includes these may be responsively displayed in the image section. As the user navigates within the text section, relevant portions of the enlarged image in the image section may be highlighted. In many examples, when a user selects an ingredient description for editing, the corresponding ingredient text in the image section may be highlighted in its greater context. The text section may include a plurality of tabs, each tab associated with a panel of associated text entries. A user may select a tab, review, and edit contents of the tab, and select another tab, and perform other actions. In many aspects, when a user selects, for example, an ingredient in an ingredient tab of the text section, a context portion of the user interface may automatically update to reflect context for the selected ingredient. Context in this example, may include an indication of the active source of the description, such as a description directly from the product labels 100, description determined (e.g., transformed) from the product labels 100, a company-specific description, an edited ingredient description and the like. Other context may include whether there is corresponding information in other tabs, such as allergen information, claims, nutritional content, and other information to be reviewed if the description of the ingredient is changed. Yet other information may include product recipe information, such as an amount of the ingredient in a unit of the product 110, and other information.”).
Therefore, it would have been obvious to one of ordinary skill in the art, at the time the invention was made, to add Sheth’s magnify the image segment; or provide an option to view one or more recipes using the item matched to the image segment to Gupta’s causing presentation of a first digital flyer on an electronic display of a client device. One of ordinary skill in the art would have been motivated to do so in order “to understand how products should be positioned relative to third party products.” (Sheth: ¶ [0003]).
Response to Arguments
Applicant’s arguments see pages 1-6 of the Remarks disclosed, filed on 03/12/2026, with respect to the 35 U.S.C. § 101 rejection(s) of claim(s) 1-9, 11-16, and 18-20 have been considered but are not persuasive. The Applicant asserts “Such is the case with the present claims. The additional elements relate to steps for image analysis of the digital content item, and for augmentation of the digital content item to enable interface functionality. This augmented digital flyer provides for improvements over traditional interface technology. Traditionally, flyers were static and non-interactable. Flyers would have been physically distributed for viewing by a user to absorb the content presented. The user would then act on that information presented. With the additional elements, the claimed invention leverages the computer-based models to parse the digital flyer and to augment the flyer with interface elements. The encoded elements empower the digital content item to enable user interaction with the augmented flyer-transforming the static content item into a dynamic and interactable content item… The additional elements are not so widely prevalent. Wide prevalence requires a high degree of understanding among artisans in the field, such that it need not be described in detail. See id., further referencing 35 U.S.C. § 112(a). The additional elements are not so widely prevalent as to render them conventional, routing, or well understood. Critically, the prior art does not teach nor suggest the additional elements of sequential deployment of computer-based models to parse the digital flyer and to match image segments against items in the item catalog and further augmentation of the digital flyer with the user-interactable elements.” The Examiner respectfully disagrees. Claims 1, 14, and 20 recite additional limitations including “from a client device; for a machine-learned generative model; to a model serving system for execution by the machine-learned generative model; from the model serving system; by executing the machine-learned generative model; applying an image segmentation model to a first digital flyer to parse the first digital flyer into a plurality of image segments, wherein each image segment is classified into one of a plurality of segment categories including one segment category relating to images of items; applying an item matching model to each image segment classified into the segment category relating to images of items to match the image segment to an item in the item catalog; augmenting the first digital flyer by overlaying one or more user-interactable elements on one or more image segments, wherein each user-interactable element is encoded to, responsive to a user interaction with the image segment, perform one or more actions related to the item matched to the image segment; and on an electronic display of a client device.” The additional limitations reciting – “from a client device; for a machine-learned generative model; to a model serving system for execution by the machine-learned generative model; from the model serving system; by executing the machine-learned generative model; applying an image segmentation model to a first digital flyer to parse the first digital flyer into a plurality of image segments, wherein each image segment is classified into one of a plurality of segment categories including one segment category relating to images of items; applying an item matching model to each image segment classified into the segment category relating to images of items to match the image segment to an item in the item catalog; and on an electronic display of a client device” are not found to integrate the judicial exception into a practical application because they are seen as adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea - see MPEP 2106.05(f) and generally linking the use of the judicial exception to a particular technological environment or field of use – see MPEP 2106.05(h). Accordingly, alone, and in combination, these additional elements are seen as using a computer or tool to perform an abstract idea, adding insignificant-extra-solution activity to the judicial exception. They do no more than link the judicial exception to a particular technological environment or field of use, i.e. client device/model serving system/machine-learned generative model/image segmentation model/item matching model, and therefore do not integrate the abstract idea into a practical application. The courts decided that although the additional elements did limit the use of the abstract idea, the court explained that this type of limitation merely confines the use of the abstract idea to a particular technological environment and this fails to add an inventive concept to the claims (See Affinity Labs of Texas v. DirecTV, LLC,)… Claims 1, 14, and 20 also recite additional limitations including – “augmenting the first digital flyer by overlaying one or more user-interactable elements on one or more image segments, wherein each user-interactable element is encoded to, responsive to a user interaction with the image segment, perform one or more actions related to the item matched to the image segment.” However, merely adding UI elements to the content in order to receive input from users is seen as simply appending well-understood, routine, conventional activities previously known to the industry, specified at a high level of generality, to the judicial exception - see MPEP 2106.05(d) and Berkheimer Memo. For example, according to Col. 7 Lines 6-9 of U.S. Patent 10,863,230 to Pham; “In conventional overlay UI applications or user interface applications that utilize overlay elements, users may provide input or place the overlay UI elements 204, 206, and 208 in any area or portion of user interface 200. For example, the user (player 212) may provide input to move overlay UI element 204 to position 220 which would obscure the primary portion or area of content 218 from viewers who are consuming the content provided by user interface 200.” Independent claims 1, 14, and 20 do not include additional elements or a combination of elements that result in the claims amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional elements listed amount to no more than mere instructions to apply an exception using a generic computer component. In addition, the applicant’s specifications describe generic computer-based elements, ¶ [0132], for implementing “computer processor”, which do not amount to significantly more than the abstract idea of itself, which is not enough to transform an abstract idea into eligible subject matter. Furthermore, there is no improvement in the functioning of the computer or technological field, and there is no transformation of subject matter into a different state. Therefore, the rejection(s) of claim(s) 1-9, 11-16, and 18-20 under 35 U.S.C. § 101 is maintained above with an updated analysis.
Applicant’s arguments see pages 6-9 of the Remarks disclosed, filed on 03/12/2026, with respect to the 35 U.S.C. § 103 rejection(s) of claim(s) 1-9, 11, 12, 14-16, and 18-20 over Gupta in view of Deng and Jain with claim 13 being rejected in further view of Sheth have been considered but are not persuasive. The Applicant asserts “Here, Jain describes general image analysis to extract categorical data. There is no description on application of an "image segmentation model" to parse the digital flyer into distinct image segments… This description does not teach performing matching between image segments parsed from the digital flyer against items in the item catalog… Here, this description is provisioning of results from a user query. The user inputs a query or a search, and the platform identifies products relevant to the query and presents those query results in a list for the user. This is not augmentation of the digital flyer.” The Examiner respectfully disagrees. The Jain reference teaches “"image segmentation model" to parse the digital flyer into distinct image segments” (See pages 15-16 above and Jain: Col. 18 Lines 42-53 which teaches image data is segmented into a plurality of objects according to image classification model). The Jain reference also teaches “performing matching between image segments parsed from the digital flyer against items in the item catalog” (See pages 15-16 above and Jain: Col. 9 Lines 14-61 and Col. 17 Lines 58-67 which teaches candidate products are matched to the reference product or the image segment classified into category of items). Examiner would like to note that the Deng reference teaches “augmenting the first digital flyer by overlaying one or more user-interactable elements on one or more image segments, wherein each user-interactable element is encoded to, responsive to a user interaction with the image segment, perform one or more actions related to the item matched to the image segment” (See pages 13-14 and Deng ¶¶ [0114] [0115] which teaches overlaying digital content, which may be an image, with user interactable elements responsive to user interaction related to image of digital content in order to perform actions such as already purchased item). Therefore, the rejection(s) of claim(s) 1-9, 11-16, and 18-20 under 35 U.S.C. § 103 is provided above with updated citations.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. The following references are cited to further show the state of the art:
U.S. Publication 2023/0214899 to Martinez for disclosing processing an image using visual and textual information. An example apparatus includes at least one memory, instructions in the apparatus, and processor circuitry to execute the instructions to detect regions of interest corresponding to a product promotion of an input digital leaflet, extract textual features from the product promotion by applying an optical character recognition (OCR) algorithm to the product promotion and associating output text data with corresponding ones of the regions of interest, determine a search attribute corresponding to the product promotion, generate a first dataset of candidate products corresponding to the product in the product promotion by comparing the search attribute against a second dataset of products, and select a product from the first dataset of candidate products to associate with the product promotion, the product selected based on a match determination.
U.S. Publication 2024/0126997 to Bent for disclosing obtaining via a conversational campaign assistant interface, by a custom language model, natural language input. The method includes generating, by the custom language model, an output comprising a predicted user intent. The method includes determining actions to perform and determining a natural language response. The method includes transmitting, to an action component, the action data structure comprising executable instructions that cause the action component to automatically perform operations associated with completing the action. The method includes transmitting to the conversation campaign assistant interface, the response data structure comprising the natural language response to be provided for display to a user via the conversational campaign assistant interface. The method includes obtaining user input indicative of a validation of the action data structure or the response data structure and updating the custom language model based on the user input.
U.S. Patent 11,589,128 to Greiner for disclosing providing an interactive, program based product introduction experience are described. In one example, video content may be captured, enriched with product data, and provided on-demand to interactive user devices. Users viewing the content may be presented with supplemental information or resources allowing those users to obtain more information about objects appearing in the video, or purchase products that may be a sociopathic with those objects. The objects appearing in the video may be passively up hearing, rather than explicitly introduced or offered for sale as part of the original video content. However, the supplemental information allows such users to discover and/or purchase new products through interactivity with the video content.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Azam Ansari, whose telephone number is (571) 272-7047. The examiner can normally be reached from Monday to Friday between 8 AM and 4:30 PM.
If any attempt to reach the examiner by telephone is unsuccessful, the examiner's supervisor, Waseem Ashraf, can be reached at (571) 270-3948.
Another resource that is available to applicants is the Patent Application Information Retrieval (PAIR). Information regarding the status of an application can be obtained from the (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAX. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pairdirect.uspto.gov. Should you have questions on access to the Private PAIR system, please feel free to contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free).
Applicants are invited to contact the Office to schedule either an in-person or a telephonic interview to discuss and resolve the issues set forth in this Office Action. Although an interview is not required, the Office believes that an interview can be of use to resolve any issues related to a patent application in an efficient and prompt manner.
/AZAM A ANSARI/
Primary Examiner, Art Unit 3621
May 7, 2026
Prosecution Insights