DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of Claims
Applicant's “Amendment” filed on 3/30/2026 has been considered.
Rejection to Claims 1-20 under 35 USC 101 have been overcome.
Claims 1-2, 9-10, 17-18 are amended.
Claims 5-7, 13-15 are cancelled.
Claims 1-4, 8-12, 16-20 are currently pending and have been examined.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-4, 6, 8-12, 14, 16-18, 20 are rejected under 35 U.S.C. 103 as being unpatentable over U.S. Patent Application No. 2019/0080383 A1 to Garcia Duran in view of US 12020470 B1 to SARAEE in view of U.S. Patent Application No. 2019/0197307 A1 to YEH.
Regarding Claim 1, GARCIA DURAN discloses a method, performed at a computer system comprising a processor and a non-transitory computer readable medium, comprising:
retrieving linking data for the user, wherein the linking data describes historical interactions by the user with one or more of the plurality of the catalog products; ([0018] The recommender system uses the history of previous user ratings and text from the associated product reviews and relates the users, products, and reviews as a multi-relational graph.)
generating a set of candidate catalog products for each of the item descriptors in the list by applying an item association model to the linking data to generate a score for each of the plurality of catalog products, ([0038] The interface 210 sends recommendations 218 of products with the highest predicted ratings/scores to the active users. The interface 210 may also keep track of categories of products that each user is interested in while the users are navigating through the retailer webpage. [0023] The scenario in FIG. 1 is depicted as a bipartite graph, which has on one side the set of users 102, 104, 106 and on the other side the set of products 118, 120, 122, and corresponding ratings and reviews 108, 110, 112, 114, 116 link the users 102, 104, 106 to the products 118, 120, 122. Embodiments of the present invention can be effectively applied to model interactions between the users 102, 104, 106 and the products 118, 120, 122, via both ratings and reviews 108, 110, 112, 114, 116. According to embodiments of the invention, machine learning algorithms can be used to predict ratings for unseen (user, product) pairs, and based on these ratings predictions, make recommendations to the users 102, 104, 106.) wherein the item association model comprises a machine learning model that was trained by:
accessing a set of training examples including linking data for different users, ([0042] The NN component 212 can be trained using the set of triples and ratings stored in the database 216, which are regularly transmitted by the interface 210 to re-adjust the NN component 212 weights (i.e. latent representations and regression model parameters). Embeddings and weights of the regression model can be randomly initialized and trained using stochastic gradient descent (SGD) and backpropagation. [0047] The training of the machine learning model includes generating user, product, and review representations based on the stored data structure triples and their associated ratings. The NN component 212 first computes the loss of Eq. 4, and then gradients are backpropagated to update the parameters of the model (latent representations of users, items and reviews, as well as parameters of the regression model). The gradients are computed with respect to each of the parameters of the model.)
applying the item association model to the set of training examples to generate a training output corresponding to a predicted training set of catalog products and associated training scores, ([0048] At step 506, the NN component 212 predicts using the learned representations of the user, product, and review to first predict review embeddings and then predict ratings associated with the predicted review embeddings. The NN component 212 approximates the review embedding by computing the vector difference between the representation of the item and the representation of the user. Then the NN component 212 predicts the rating using the approximated review embedding as input feature vector to the regression model.)
back-propagating one or more error terms obtained from one or more loss functions to update a set of parameters of the item association model, ([0047] The training of the machine learning model includes generating user, product, and review representations based on the stored data structure triples and their associated ratings. The NN component 212 first computes the loss of Eq. 4, and then gradients are backpropagated to update the parameters of the model (latent representations of users, items and reviews, as well as parameters of the regression model). The gradients are computed with respect to each of the parameters of the model.) and one or more of the error terms are based on a difference between the interactions with the search results and the predicted training set of catalog products and associated training scores, and ([0029] The bias terms are parameters of the model in Eq. 1 that are learned/adjusted during training. Eq. 1 involves minimizing the mean squared error between the actual rating, and the output of a regression model (parameterized by w and the bias terms) that uses the review representation h.sub.rev.sub.u,i as input feature.)
building a list of recommended catalog products for the user by, for each of the item descriptors in the list, selecting one of the set of candidate catalog products ([0049] At step 508, the interface 210 makes recommendations to users based on the predicted ratings.) by generating a score for each of the set of candidate catalog products, ([0065] To visualize the correlation between words and ratings, firstly, a score was assigned to each word, the score being the average rating of the reviews that contain that word, and secondly, a two dimensional representation of the words was learned by applying t-SNE (t-Distributed Stochastic Neighbor Embedding) to the 16-dimensional word embeddings learned by TRANSREV. FIG. 10 depicts these 2-dimensional representations of the word embeddings learned for the Baby dataset where the corresponding scores are indicated on the right.)
providing the list of recommended catalog products to a user client device associated with the user, wherein providing the list of recommended catalog products to the user client device causes the user client device to display the list of recommended catalog products. ([0018] These predicted ratings can be provided to an application that displays products to users according to the predictions made by the recommender system. [0045] output devices)
But does not explicitly disclose stopping the back-propagation after the one or more loss functions satisfy one or more criteria; capturing an image at a user client device associated with a user, wherein the image depicts a list of item descriptors, wherein each item descriptor comprises text corresponding to a subset of a plurality of catalog products; applying, by the user client device, a text recognition process to the image to extract text string data for each of the list of item descriptors, wherein the text string data for an item descriptor comprises the text corresponding to a subset of a plurality of catalog products; receiving, from the user client device at an online system, the text string data for each of the list of item descriptors; wherein the historical interactions comprise data describing interactions by the user with search results through a search user interface in response to text queries provided by the user through the search user interface; wherein the linking data for different users comprises, for each of the different users, data describing interactions by the different user with search results in response to text queries provided by the different user, the text queries of the set of training examples; the interactions by the different users; wherein the score for each of the candidate catalog products is generated by applying a list recommendation model to product data associated with the candidate catalog product
However the combination of GARCIA DURAN and SARAEE does not explicitly teach wherein receiving the list that includes the one or more item descriptors comprises: instructing the user client device to capture an image of the list that includes the one or more item descriptors; and responsive to receiving the image, performing text recognition to identify one or more text strings, where each text string corresponds to a different item descriptor of the one or more item descriptors.
SARAEE, on the other hand, teaches stopping the back-propagation after the one or more loss functions satisfy one or more criteria; ([Col 129 Ln 30-35] The machine learning models can be trained using back-propagation techniques and a loss function based on the interaction data. By training the machine learning models in this way, each machine learning model may be trained to simulate a target audience and how members of the target audience will interact with specific images. wherein the historical interactions comprise data describing interactions by the user with search results through a search user interface in response to text queries provided by the user through the search user interface; ([Col 129 Ln 60-67] the content evaluation system 1005 can identify images that are posted on Website A and interaction data of users interacting with such images posted on Website A or on the web pages of Website A that include the images (e.g., did the user scroll on the web page, how long did the user spend on the web page, did the user click to access review, did the user select an “Add to Cart” option, did the user purchase an item of the web page, etc.). [Col 152 Ln 1-15] Upon receiving a query containing one or more key words from a computing device, the computer can execute a search engine machine learning model based on the keywords and the web page scores for web pages as input. The search engine machine learning model can output a set of web pages based on the keywords and web page scores. ) wherein the linking data for different users comprises, for each of the different users, data describing interactions by the different user with search results in response to text queries provided by the different user; the text queries of the set of training examples; the interactions by the different users ([Col 152 Ln 20-40] The one or more machine learning models can be configured or trained to generate image performance scores that simulate how users will interact with different images. In some embodiments, the different machine learning models may be trained to simulate how users of different target audiences will interact images on web pages. The computer can aggregate or otherwise use image performance scores of different images for individual web pages to determine web page scores for the different web pages. The computer can do so, for example, by aggregating or calculating an average or median of the image performance scores of the images of each web page. The computer can use the web page scores as input into the search engine machine learning model. In doing so, the computer can improve the search results of any query performed through the search engine machine learning model such that the search results include images that are more interactive and/or that are more likely to improve the experience of the user using the search engine.) wherein the score for each of the candidate catalog products is generated by applying a list recommendation model to product data associated with the candidate catalog product ([Col 11 Ln 50-55] The automatically published content items may come in many various forms. The content items may be through sponsored content on a news or pseudo-news website, may be native ads or editorial content on a social networking site or other web property, may be a standard banner content item, may be recommended and sponsored content on a shopping website, [Col 72 Ln 25-35] The systems and methods described herein can use artificial intelligence and other techniques to first find an audience, and then evaluate, rank, score, give recommendations, and generate new content items that are optimized for the target audience. The audience's actual behavior becomes a standard against which new and proposed content, designs, or creatives are evaluated.)
It would have been obvious to one of ordinary skill in the art to include in the method, as taught by GARCIA DURAN, the features as taught by SARAEE, since the claimed invention is merely a combination of old elements, and in the combination each element merely would have performed the same function as it did separately, and one of ordinary skill in the art would have recognized that the results of the combination were predictable. It further would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify GARCIA DURAN, to include the teachings of SARAEE, in order to select content items likely to draw attention (SARAEE, [Col 1 Ln 50-60]).
YEH, on the other hand, teaches capturing an image at a user client device associated with a user, wherein the image depicts a list of item descriptors, wherein each item descriptor comprises text corresponding to a subset of a plurality of catalog products; applying, by the user client device, a text recognition process to the image to extract text string data for each of the list of item descriptors, wherein the text string data for an item descriptor comprises the text corresponding to a subset of a plurality of catalog products; receiving, from the user client device at an online system, the text string data for each of the list of item descriptors; ([0044-0045] An image capture device 124 obtains electronic images of paper documents and communicates the electronic images to the shopping list application 125 and/or an optical character recognition (OCR) application 128. The image capture device 124 may be a camera, scanner, or other device suitable to obtain electronic images of paper documents. The image capture device 124 may be an integral component of the user computing device 120 or may be an external device in communication with the user computing device 120. The optical character recognition (OCR) application 128 receives the electronic images from the image capture device 124 or the shopping list application 125, processes the images to identify products from documents represented in the electronic images, and communicates the identified products to the shopping list application 125.)
It would have been obvious to one of ordinary skill in the art to include in the method, as taught by GARCIA DURAN and SARAEE, the features as taught by YEH, since the claimed invention is merely a combination of old elements, and in the combination each element merely would have performed the same function as it did separately, and one of ordinary skill in the art would have recognized that the results of the combination were predictable. It further would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the combination, to include the teachings of YEH, in order to create a new user generated list of items to the user computing device for display via the user computing device. (YEH, [0025]).
Regarding Claim 2, GARCIA DURAN, SARAEE and YEH teaches the method of claim 1.
SARAEE teaches wherein building the list of recommended catalog products comprises: identifying one or more sets of ranked candidate catalog product-score pairs where each of the one or more sets corresponds to a different catalog product of the plurality of catalog products; ([Col 72 Ln 20-35] marketing and other forms of content generation and/or content publishing today typically begin with an image, a video, audio, or text (or a combination of these) that becomes an ad, and then finds an audience. This process can be expensive and error-prone. The systems and methods described herein can use artificial intelligence and other techniques to first find an audience, and then evaluate, rank, score, give recommendations, and generate new content items that are optimized for the target audience. The audience's actual behavior becomes a standard against which new and proposed content, designs, or creatives are evaluated.)
retrieving pricing information for each candidate catalog product of each of the one or more sets of ranked candidate catalog product-score pairs; retrieving availability information at the one or more sources for each candidate catalog item of each of the one or more sets of ranked candidate catalog product-score pairs; ([Col 33 Ln 5-20] The system may then alert the user when there are fluctuations in variables that may explicitly influence his plans e.g., the ticket price drops or new seats with extra leg room become available, or perhaps there is a new amenity offered to him as a traveler. In the event that the user has already made the booking, the system may compare the data on his booked trip to other options occurring at the same time and notify him of any potentially preferable alternatives that may influence a booking change.)
applying the one or more sets of ranked candidate catalog product-score pairs, the pricing information, and the availability information to the list recommendation model that outputs the scores for the candidate catalog of products for each of the item descriptors in the list. ([Col 33 Ln 35-50] a custom search monitor may be used to track the publishing behavior of target groups of webpages and then alert users when these entities cross a certain threshold in content production or search ranking, viewership rate, preference or relevance metrics, etc. That is, the user may assess competing websites based on content goals, content topics, quantity and frequency of content types (blog, whitepaper, ebook, video, podcast, etc.) keyword use, or along any other attribute or action of interest. The system may also be applied to alert users when webpages located by a search query adjust in content or ranking.)
It would have been obvious to one of ordinary skill in the art to include in the method, as taught by GARCIA DURAN, the features as taught by SARAEE, since the claimed invention is merely a combination of old elements, and in the combination each element merely would have performed the same function as it did separately, and one of ordinary skill in the art would have recognized that the results of the combination were predictable. It further would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify GARCIA DURAN, to include the teachings of SARAEE, in order to select content items likely to draw attention (SARAEE, [Col 1 Ln 50-60]).
Regarding Claim 3, GARCIA DURAN, SARAEE and YEH teaches the method of claim 2.
SARAEE teaches further comprising: retrieving user data describing favorite sources of the user; wherein building the list of recommended catalog products for the user by, for each of the at least one of the item descriptors in the list, selecting one of the set of candidate catalog products based on the generated scores, further comprises: applying the user data to the list recommendation model to output the plurality of lists of recommended catalog products.. ([Col 62 Ln 45-55] important information like the user's keyword planner, display planner, current search marketing campaigns, the user's organic search rankings, the user's goals and preferences, transactions, performance history, current trends on Google™ search volumes and other industry data, the user's available budget, and other settings and information. The system may use this data to determine goals for the user and as factors for determining recommended aspects of content. )
It would have been obvious to one of ordinary skill in the art to include in the method, as taught by GARCIA DURAN, the features as taught by SARAEE, since the claimed invention is merely a combination of old elements, and in the combination each element merely would have performed the same function as it did separately, and one of ordinary skill in the art would have recognized that the results of the combination were predictable. It further would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify GARCIA DURAN, to include the teachings of SARAEE, in order to select content items likely to draw attention (SARAEE, [Col 1 Ln 50-60]).
Regarding Claim 4, GARCIA DURAN, SARAEE and YEH teaches the method of claim 1.
SARAEE teaches wherein providing the list of recommended catalog products to the user client device associated with the user comprises: instructing an ordering interface of the user client device to display an option that, once selected, adds all of the list of recommended catalog products to a shopping list. ([Col 93 Ln 30-40] the system 1000 can weight each content item by whether or not that content item led to the user completing some action of interest (e.g., filling out a form or adding a product to a shopping cart). The system 1000 can also be configured to place a higher value on content items that have received more viewer activity, comments, social media engagement or have been “Pinned” by users most often with a Pinterest-like plug in.)
It would have been obvious to one of ordinary skill in the art to include in the method, as taught by GARCIA DURAN, the features as taught by SARAEE, since the claimed invention is merely a combination of old elements, and in the combination each element merely would have performed the same function as it did separately, and one of ordinary skill in the art would have recognized that the results of the combination were predictable. It further would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify GARCIA DURAN, to include the teachings of SARAEE, in order to select content items likely to draw attention (SARAEE, [Col 1 Ln 50-60]).
Regarding Claim 8, GARCIA DURAN, SARAEE and YEH teaches the method of claim 1.
SARAEE teaches generating additional training examples using the list of recommended catalog products and a purchase by the user of a catalog product on the list responsive to the list of recommended catalog products being displayed on the user client device; and retraining the item association model based in part on the additional training examples. ([Col 111 Ln 20-40] The content evaluation system 1005 executes the neural network using the training images as inputs to generate performance scores. The content evaluation system 1005 then uses the interaction data for the individual images as labels for the correct performance scores. The content evaluation system 1005 may use back-propagation techniques based on differences between the predicted performance scores and the interaction data to tune weights of the neural network to more accurately predict performance scores for images in the future. )
It would have been obvious to one of ordinary skill in the art to include in the method, as taught by GARCIA DURAN, the features as taught by SARAEE, since the claimed invention is merely a combination of old elements, and in the combination each element merely would have performed the same function as it did separately, and one of ordinary skill in the art would have recognized that the results of the combination were predictable. It further would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify GARCIA DURAN, to include the teachings of SARAEE, in order to select content items likely to draw attention (SARAEE, [Col 1 Ln 50-60]).
Claim 9 recites a computer program product comprising a non-transitory computer readable storage medium comprising substantially similar limitations as claim 1. The claim is rejected under substantially similar grounds as claim 1.
Claim 10 recites a computer program product comprising substantially similar limitations as claim 2. The claim is rejected under substantially similar grounds as claim 2.
Claim 11 recites a computer program product comprising substantially similar limitations as claim 3. The claim is rejected under substantially similar grounds as claim 3.
Claim 12 recites a computer program product comprising substantially similar limitations as claim 4. The claim is rejected under substantially similar grounds as claim 4.
Claim 16 recites a computer program product comprising substantially similar limitations as claim 8. The claim is rejected under substantially similar grounds as claim 8.
Claim 17 recites a computer system comprising substantially similar limitations as claim 1. The claim is rejected under substantially similar grounds as claim 1.
Claim 18 recites a system comprising substantially similar limitations as claim 2. The claim is rejected under substantially similar grounds as claim 2.
Regarding Claim 19, GARCIA DURAN, SARAEE and YEH teaches the method of claim 1.
However the combination of GARCIA DURAN and SARAEE does not explicitly teach wherein receiving the list that includes the one or more item descriptors comprises: instructing the user client device to capture an image of the list that includes the one or more item descriptors; and responsive to receiving the image, performing text recognition to identify one or more text strings, where each text string corresponds to a different item descriptor of the one or more item descriptors.
YEH, on the other hand, teaches wherein receiving the list that includes the one or more item descriptors comprises: instructing the user client device to capture an image of the list that includes the one or more item descriptors; and responsive to receiving the image, performing text recognition to identify one or more text strings, where each text string corresponds to a different item descriptor of the one or more item descriptors. ([0044-0045] An image capture device 124 obtains electronic images of paper documents and communicates the electronic images to the shopping list application 125 and/or an optical character recognition (OCR) application 128. The image capture device 124 may be a camera, scanner, or other device suitable to obtain electronic images of paper documents. The image capture device 124 may be an integral component of the user computing device 120 or may be an external device in communication with the user computing device 120. The optical character recognition (OCR) application 128 receives the electronic images from the image capture device 124 or the shopping list application 125, processes the images to identify products from documents represented in the electronic images, and communicates the identified products to the shopping list application 125.)
It would have been obvious to one of ordinary skill in the art to include in the method, as taught by GARCIA DURAN and SARAEE, the features as taught by YEH, since the claimed invention is merely a combination of old elements, and in the combination each element merely would have performed the same function as it did separately, and one of ordinary skill in the art would have recognized that the results of the combination were predictable. It further would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the combination, to include the teachings of YEH, in order to create a new user generated list of items to the user computing device for display via the user computing device. (YEH, [0025]).
Claim 20 recites a system comprising substantially similar limitations as claim 8. The claim is rejected under substantially similar grounds as claim 8.
Response to Arguments
Applicant’s arguments filed with respect to the rejection of claims under 35 USC 101 have been fully considered and are found persuasive. Applicant’s arguments regarding 35 USC 101 in the Remarks dated 3/30/2026 are incorporated herein.
Applicant’s arguments with respect to rejection of the claim under 35 USC 103 have been considered but are moot in view of new grounds of rejection, necessitated by Applicant’s amendment.
Applicant argues that the references do not describe a system that trains a machine-learning model to identify candidate products for recommendation to a user base don text descriptions of the data by using user interactions with search results to generate the necessary training data. Nor do they describe identifying candidate products from text extracted from an image and using another machine-learning model to score those candidate products for presentation in a list to a user.
However, these amended limitations use a new combination of Garcia, Duran and Saraee to teach these limitations in the claims. Examiner directs Applicant’s attention to the office action, above.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Michelle T. Kringen whose telephone number is (571)270-0159. The examiner can normally be reached M-F: 11am-7pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Marissa Thein can be reached at (571)272-6764. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MICHELLE T KRINGEN/Primary Examiner, Art Unit 3689
Prosecution Insights