Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This office action is in response to submission of application on 1/26/2024.
Claims 1-20 are presented for examination.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: Is the claim to a process, machine, manufacture or composition of matter?
Claims 1-14 are directed to a method (i.e., a process); claims 18-20 are directed
to an apparatus (i.e., a machine/apparatus); and claims 15-17 are directed to an article of
manufacture (i.e., a product); therefore, all pending claims are directed to one of the four
categories of invention.
Independent Claims
Step 2A Prong One: Does the claim recite an abstract idea, law of nature, or natural phenomenon?
Yes, independent claim 1 recites an abstract idea in the form of mental processes. A mental process is a process that “can be performed in the human mind, or by a human using a pen and paper” (MPEP§ 2106.04(a)(2)(III), paragraph 1). Examples of mental processes include “observations, evaluations, judgments, and opinions” (MPEP § 2106.04(a)(2)(III), paragraph 2).
The following limitations of claim 1 are mental processes:
transforming the query into a query feature vector; [This is a mental process that can be performed by observations, evaluations, judgments, and opinions. No specific methodology for transforming the query is recited in the claim; therefore, it broadly encompasses processing that can be performed as a mental process.]
finding one or more matches to the query feature vector in each content item bucket of a plurality of content item buckets, wherein the plurality of content item buckets groups content items based on one or more attributes of the content items; and [This is a mental process that can be performed by observations, evaluations, judgments, and opinions. No specific methodology for matching the vector is recited in the claim; therefore, it broadly encompasses processing that can be performed as a mental process.]
Therefore, the independent claims recite a judicial exception.
Step 2A Prong Two: Does the claim recite additional elements that integrate the judicial exception into a practical application?
No. The judicial exception recited in the above discussed claims is not integrated into a
practical application.
receiving a query; [receiving data is sending data, which is insignificant, extra-solution activity. See MPEP 2106.05(g). Transmitting data is well-understood, routine, and conventional. See MPEP 2106.05(d)(II)(i).]
training a machine learning model using the one or more matches from each content item bucket. [Training a machine learning model are mere instructions to apply the abstract idea. Mere recitation that a judicial exception is to be performed using generic class of computer algorithms in their ordinary capacity, cannot meaningfully integrate the judicial exception into a practical application. See MPEP 2106.05(f). Additionally, this is a description of how the abstract idea is performed, using content item buckets. As such, this merely describes a technological environment. See MPEP 2106.05(h).]
Therefore, under MPEP 2106.04(d), the additional elements of the claims do not integrate
the judicial exception into a practical application.
Step 2B: Does the claim recite additional elements that amount to significantly more than the judicial exception?
No. The claims do not include additional elements that are sufficient for the claims to
amount to significantly more than the judicial exception.
Additional elements that are mere instructions to apply an exception or merely generally
linking or generally linking the use of a judicial exception to a particular technological
environment or field of use do not constitute significantly more than a judicial exception under
MPEP§2106.05(I)(A). Since the additional elements in the independent claims are all are mere
instructions to apply an exception or are merely generally linking or generally linking the use of
a judicial exception to a particular technological environment or field of use, they do not
constitute significantly more than a judicial exception.
Therefore, the additional elements identified in the Step 2A Prong Two analysis do not
constitute significantly more than a judicial exception.
Claim 15
Step 2A Prong One: Does the claim recite an abstract idea, law of nature, or natural phenomenon?
Yes, independent claim 15 recites an abstract idea in the form of mental processes. A mental process is a process that “can be performed in the human mind, or by a human using a pen and paper” (MPEP§ 2106.04(a)(2)(III), paragraph 1). Examples of mental processes include “observations, evaluations, judgments, and opinions” (MPEP § 2106.04(a)(2)(III), paragraph 2).
The following limitations of claim 15 are mental processes:
determine popularity scores based on popularity-related data of content items; [This is a mental process that can be performed by observations, evaluations, judgments, and opinions. No specific methodology for determining popularity scores is recited in the claim; therefore, it broadly encompasses processing that can be performed as a mental process.]
split the content items into content item buckets based on the popularity scores; [This is a mental process that can be performed by observations, evaluations, judgments, and opinions. No specific methodology for splitting the content items is recited in the claim; therefore, it broadly encompasses processing that can be performed as a mental process.]
Step 2A Prong Two: Does the claim recite additional elements that integrate the judicial exception into a practical application?
No. The judicial exception recited in the above discussed claims is not integrated into a
practical application.
One or more non-transitory computer-readable media having instructions stored thereon, when the instructions are executed by one or more processors, causes the one or more processors to: [A non-transitory computer-readable media having instructions executed by a processor are components recited at a high level are construed as generic computer components and algorithms used to implement the abstract idea. See MPEP 2106.05(f)(2). As such, the limitations do not integrate the abstract idea into a practical application. Nor to do they amount to significantly more.]
retrieve, using a first large language model, one or more matches in each content item bucket that semantically match a query; [receiving data is sending data, which is insignificant, extra-solution activity. See MPEP 2106.05(g). Transmitting data is well-understood, routine, and conventional. See MPEP 2106.05(d)(II)(i). Additionally, this is a description of how the abstract idea is performed, using a large language model. As such, this merely describes a technological environment. See MPEP 2106.05(h).]
generate training data based on the one or more matches from each content item bucket; and [Generating training data are mere instructions to apply the abstract idea. Mere recitation that a judicial exception is to be performed using generic class of computer algorithms in their ordinary capacity, cannot meaningfully integrate the judicial exception into a practical application. See MPEP 2106.05(f). Additionally, this is a description of how the abstract idea is performed, using content item buckets. As such, this merely describes a technological environment. See MPEP 2106.05(h).]
update parameters of a machine learning model using the training data. [Updating parameters of a machine learning model are mere instructions to apply the abstract idea. Mere recitation that a judicial exception is to be performed using generic class of computer algorithms in their ordinary capacity, cannot meaningfully integrate the judicial exception into a practical application. See MPEP 2106.05(f).]
Therefore, under MPEP 2106.04(d), the additional elements of the claims do not integrate
the judicial exception into a practical application.
Step 2B: Does the claim recite additional elements that amount to significantly more than the judicial exception?
No. The claims do not include additional elements that are sufficient for the claims to
amount to significantly more than the judicial exception.
Additional elements that are mere instructions to apply an exception or merely generally
linking or generally linking the use of a judicial exception to a particular technological
environment or field of use do not constitute significantly more than a judicial exception under
MPEP§2106.05(I)(A). Since the additional elements in the independent claims are all are mere
instructions to apply an exception or are merely generally linking or generally linking the use of
a judicial exception to a particular technological environment or field of use, they do not
constitute significantly more than a judicial exception.
Therefore, the additional elements identified in the Step 2A Prong Two analysis do not
constitute significantly more than a judicial exception.
Dependent Claims
The remaining dependent claims being rejected do not recite additional elements, whether considered individually or in combination, that are sufficient to integrate the judicial exception into a practical application or amount to significantly more than the judicial exception.
Claims 2
wherein finding the one or more matches comprises: for a first content item bucket of the content item buckets, determining a dot product of a content item feature vector of each content item in the content item bucket and the query feature vector; and returning the one or more matches having content items that have the highest dot product values. [This is a mental process that can be performed by observations, evaluations, judgments, and opinions.]
Claims 3
filtering the one or more matches found in the plurality of content item buckets based on a score computed for each match, and a pre-determined number of the one or more matches having the highest score values. [This further limitation is also a mental process. This claim does not recite any additional non-abstract elements for purposes of Step 2A Prong Two and Step 2B analysis.]
Claims 4
filtering the one or more matches found in the plurality of content item buckets based on a score computed for each match, and a threshold on score values computed for each match. [This further limitation is also a mental process. This claim does not recite any additional non-abstract elements for purposes of Step 2A Prong Two and Step 2B analysis.]
Claims 5
for a first match in the one or more matches found in the plurality of content item buckets, inputting a prompt to a large language model, the prompt comprising a question whether the first match, given metadata of the first match as context, is associated with the query; [This additional element does no more than generally link the use of a judicial exception to a particular technological environment or field of use (MPEP § 2106.05(h)). This element merely indicates a field of use or technological environment in which a judicial exception is applied, namely inputting a prompt to a large language model].
and removing the first match based on a negative response to the prompt. [Removing a match are mere instructions to apply the abstract idea. Mere recitation that a judicial exception is to be performed using generic class of computer algorithms in their ordinary capacity, cannot meaningfully integrate the judicial exception into a practical application. See MPEP 2106.05(f)]
Claims 6
determining a number of the one or more matches to find in the content item buckets based on the query. [This further limitation is also a mental process. This claim does not recite any additional non-abstract elements for purposes of Step 2A Prong Two and Step 2B analysis.]
Claims 7
updating a further model based on one or more feedback signals about the one or more matches found in each content item bucket; [Updating a model are mere instructions to apply the abstract idea. Mere recitation that a judicial exception is to be performed using generic class of computer algorithms in their ordinary capacity, cannot meaningfully integrate the judicial exception into a practical application. See MPEP 2106.05(f). Additionally, this is a description of how the abstract idea is performed, using feedback signals. As such, this merely describes a technological environment. See MPEP 2106.05(h).]
inputting the query into the further model; and [Inputting a query is sending data, which is insignificant, extra-solution activity. See MPEP 2106.05(g). Transmitting data is well-understood, routine, and conventional. See MPEP 2106.05(d)(II)(i).]
determining a number of the one or more matches to find in each content item buckets using the further model. [This further limitation is also a mental process. This claim does not recite any additional non-abstract elements for purposes of Step 2A Prong Two and Step 2B analysis.]
Claims 8
wherein the one or more feedback signals comprises a count of the one or more matches found in the plurality of content item buckets meeting a quality criterion. [This additional element does no more than generally link the use of a judicial exception to a particular technological environment or field of use (MPEP § 2106.05(h)). This element merely indicates a field of use or technological environment in which a judicial exception is applied, namely the feedback signals comprise a count of matches.].
Claims 9
wherein: the one or more attributes of the content items comprises popularity. [This additional element does no more than generally link the use of a judicial exception to a particular technological environment or field of use (MPEP § 2106.05(h)). This element merely indicates a field of use or technological environment in which a judicial exception is applied, namely the attributes of content items comprise popularity.].
Claims 10
determining scores for content items associated with the one or more attributes of the content items; and distributing the content items into the content item buckets based on the scores. [This is a mental process that can be performed by observations, evaluations, judgments, and opinions. No specific methodology for determining scores and distributing items are recited in the claim; therefore, it broadly encompasses processing that can be performed as a mental process.]
Claims 11
wherein distributing the content items into the content item buckets comprises distributing the content items using a percentile approach. [This additional element does no more than generally link the use of a judicial exception to a particular technological environment or field of use (MPEP § 2106.05(h)). This element merely indicates a field of use or technological environment in which a judicial exception is applied, namely the distribution of content items uses a percentile approach.].
Claims 12
wherein distributing the content items into the content item buckets comprises distributing the content items using a recursive Pareto distribution approach. [This additional element does no more than generally link the use of a judicial exception to a particular technological environment or field of use (MPEP § 2106.05(h)). This element merely indicates a field of use or technological environment in which a judicial exception is applied, namely the distribution of content items uses a recursive Pareto distribution.].
Claims 13
wherein distributing the content items into the content item buckets comprises distributing the content items to content item buckets, each content item bucket having a size which is set according to a geometric sequence, the geometric sequence having a base group size that is determined based on a target variance in the scores within individual content item buckets. [This additional element does no more than generally link the use of a judicial exception to a particular technological environment or field of use (MPEP § 2106.05(h)). This element merely indicates a field of use or technological environment in which a judicial exception is applied, namely the distribution of content items having a size according to a geometric sequence.]
Claims 14
clustering the content items based on the one or more attributes of the content items to generate the one or more content item buckets having cohorts of content items that share similarities in the one or more attributes. [These further limitations merely further define the mental process recited in the parent claim and are therefore considered to be part of the mental process of the parent claim. This claim does not recite any non-abstract additional elements for purposes of Step 2A Prong Two and Step 2B analysis.]
Claims 16
retrieve, using a second large language model, one or more further matches in each content item bucket that semantically match the query; [Retrieving matches in content item buckets are mere instructions to apply the abstract idea. Mere recitation that a judicial exception is to be performed using generic class of computer algorithms in their ordinary capacity, cannot meaningfully integrate the judicial exception into a practical application. See MPEP 2106.05(f). Additionally, this is a description of how the abstract idea is performed, using a large language model. As such, this merely describes a technological environment. See MPEP 2106.05(h).]
wherein the training data is generated further based on the one or more further matches from each content item bucket retrieved using the second large language model. [Generating training data are mere instructions to apply the abstract idea. Mere recitation that a judicial exception is to be performed using generic class of computer algorithms in their ordinary capacity, cannot meaningfully integrate the judicial exception into a practical application. See MPEP 2106.05(f). Additionally, this is a description of how the abstract idea is performed, using a large language model. As such, this merely describes a technological environment. See MPEP 2106.05(h).]
Claims 17
generate the training data based further on one or more further matches from each content item bucket; [Generating training data are mere instructions to apply the abstract idea. Mere recitation that a judicial exception is to be performed using generic class of computer algorithms in their ordinary capacity, cannot meaningfully integrate the judicial exception into a practical application. See MPEP 2106.05(f).]
wherein generating the training data based on the one or more matches and the one or more further matches comprises filtering out content items that do not meet a criterion. [Generating training data are mere instructions to apply the abstract idea. Mere recitation that a judicial exception is to be performed using generic class of computer algorithms in their ordinary capacity, cannot meaningfully integrate the judicial exception into a practical application. See MPEP 2106.05(f).]
Claims 19
a score calculator part to compute scores for content items; and a bucketizer part to distribute the content items into the first content item bucket and the second content item bucket based on the scores. [This is a mental process that can be performed by observations, evaluations, judgments, and opinions. No specific methodology for determining scores and distributing items are recited in the claim; therefore, it broadly encompasses processing that can be performed as a mental process.]
Claims 20
an optimizer part to determine a first number of the one or more first matches to retrieve and a second number of one or more second matches to retrieve for the query. [This further limitation is also a mental process. This claim does not recite any additional non-abstract elements for purposes of Step 2A Prong Two and Step 2B analysis.]
The prior art used for rejections are provided below:
Representation Online Matters: Practical End-to-End Diversification in Search and Recommender Systems (May 26, 2023) to Silva et al. (hereinafter Silva)
Embedding-based Retrieval in Facebook Search (July 29, 2020) to Huang et al. (hereinafter Huang).
Document Ranking with a Pretrained Sequence-to-Sequence Model (March 14, 2020) to Nogueira et al. (hereinafter Nogueira).
Sampling-Bias-Corrected Neural Modeling for Large Corpus Item Recommendations (September 16, 2019) to Yi et al. (hereinafter Yi).
Flatter is better: Percentile Transformations for Recommender Systems (July 10, 2019) to Mansoury, et al. (hereinafter Mansoury).
Learning Multi-granular Quantized Embeddings for Large-Vocab Categorical Features in Recommender Systems (August 25, 2020) to Kang, et al. (hereinafter Kang).
Maintaining Stream Statistics Over Sliding Windows (October 18, 2002) to Datar, et al. (hereinafter Datar).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1-4, 6-8, 10, 14, and 18-20 are rejected under 35 USC 103 as being unpatentable over Silva in view of Huang.
Per claim 1 A method, comprising: [Silva, pg. 1, Abstract “We develop, experiment, and deploy scalable diversification mechanisms in multiple production surfaces on the platform, including Search, Related Products, and Pinterest New User Homefeed, to improve the representation of different skin tones in beauty and fashion content. Diversification in production systems includes three components: identifying requests that will trigger diversification, ensuring diverse content is retrieved from the large content corpus during the retrieval stage, and finally balancing the diversity-utility trade-off in a self-adjusting manner in the ranking stage.”. (note: this describes a series of steps performed in sequence carried out by the system, which is a description of a method)]
receiving a query; [Silva, pg. 6, 5.2 Bucketized-ANN Retrieval “For embedding-based retrieval, the users, items, and queries are all embedded into the same space, and for applications like search and recommender systems, the system wants to retrieve the items that are closest to the query or user embedding in terms of a chosen distance metric (e.g., cosine distance).”. (note: this shows receiving a query)]
transforming the query into a query feature vector; [Silva, pg. 6, 5.2 Bucketized-ANN Retrieval “For embedding-based retrieval, the users, items, and queries are all embedded into the same space, and for applications like search and recommender systems, the system wants to retrieve the items that are closest to the query or user embedding in terms of a chosen distance metric (e.g., cosine distance).”. (note: this shows transforming queries into embedding vectors (query feature vectors) in the embedding space for retrieval purposes. An embedding is structurally identical and synonymous with a feature vector. Both represent data as a dense array of coordinates in a continuous vector space.)]
finding one or more matches to the query feature vector in each content item bucket of a plurality of content item buckets, wherein the plurality of content item buckets groups content items based on one or more attributes of the content items [Silva, pg. 7, 5.2 Bucketized-ANN Retrieval “For the Bucketized-ANN Retrieval approach, we modify the aggregation step (at the leaf and the root level) to also aggregate top 𝐾𝑑𝑖 candidates from each group 𝑑𝑖 ∈ D into buckets corresponding to each of the groups under the diversity dimension (in addition to aggregating the top-𝐾 candidates in the overall pool). In other words, each leaf now aggregates a set of K candidates, and |D| buckets with (at most) Kdi candidates each. This helps preserve the top candidates belonging to each group (whose distances are already
computed) from being dropped during the aggregation steps”; pg. 6 “Most of these approximation algorithms partition the embedding space into multiple regions and perform a search in it”. (note: this shows a Bucketized-ANN Retrieval approach where content items are divided into buckets based on their group (diversity dimension), and the system retrieves top K candidates from each bucket)]; and
Silva does not expressly disclose, but Silva combined with Huang does teach:
training a machine learning model using the one or more matches from each content item bucket. [Silva, pg. 10, 9 Conclusions and Future Work “We can analyze how diversified search results and recommendations can help mitigate serving bias in systems that generate their own training data, by creating a positive feedback loop for model retraining thanks to richer interaction data from a diverse set of Pins.”. (note: this shows using retrieved results from the retrieval system as training data for a machine learning model. Silva identifies the use of diversified retrieved results from multiple buckets to train the model.); Huang, pg. 2, 1 Introduction “…we incorporated embeddings into ranking layers and built a training data feedback loop to actively learn to identify those good and bad results from embedding-based retrieval.”. (note: this shows using retrieved results from the retrieval system as training data for a machine learning model and the training data feedback loop )]
Silva and Huang are analogous art because they are from the same field of endeavor encompassing information retrieval and recommendation systems using embedding based retrieval and machine learning model training.
Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to apply the bucketing technique of Silva using popularity as the dividing attribute as taught by Huang.
The suggestion/motivation for doing so would have been that popularity is a relevant criterion for selecting content items in a retrieval system. [Silva, pg. 6, 4.3 Query and Index Selection “On index side, we did index selection to make searching faster. For example, we only chose monthly active users, recent events, popular pages and groups.”]
Per Claim 2, Silva-Huang discloses claim 1.
Silva does not fully disclose, but with Huang does teach finding the one or more matches comprises: for a first content item bucket of the content item buckets, determining a dot product of a content item feature vector of each content item in the content item bucket and the query feature vector; and returning the one or more matches having content items that have the highest dot product values [Silva, pg. 6, 5.2 Bucketized-ANN Retrieval “For embedding-based retrieval, the users, items, and queries are all embedded into the same space, and for applications like search and recommender systems, the system wants to retrieve the items that are closest to the query or user embedding in terms of a chosen distance metric (e.g., cosine distance).”; Haung, pg. 3, 2.3 Unified Embedding Model, “…we choose cosine similarity as it is one of the commonly used in embedding learning [7]:
S
Q
,
D
=
cos
E
Q
,
E
D
=
E
Q
,
E
D
E
Q
∙
E
D
”. (note: Huang shows computing the cosine similarity, which is a normalized dot product, between query and document embeddings, and returning results with the highest similarity)].
Silva and Huang are analogous art because they are from the same field of endeavor encompassing information retrieval and recommendation systems using embedding based retrieval and machine learning model training.
Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to apply the bucketing technique of Silva using popularity as the dividing attribute as taught by Huang.
The suggestion/motivation for doing so would have been that popularity is a relevant criterion for selecting content items in a retrieval system. [Silva, pg. 6, 4.3 Query and Index Selection “On index side, we did index selection to make searching faster. For example, we only chose monthly active users, recent events, popular pages and groups.”]
Per Claim 3, Silva-Huang discloses claim 1.
Silva further teaches filtering the one or more matches found in the plurality of content item buckets based on a score computed for each match, and a pre-determined number of the one or more matches having the highest score values [Silva, pg. 7, 5.2 Bucketized-ANN Retrieval “The root is then responsible for choosing the top K candidates from the K × L candidates whose exact distances are computed during the process”; pg. 12, A.1 Round-Robin hyperparameters “In RR, score thresholds allow mitigating potential impact on utility metrics. We evaluated different values of the score threshold to determine which Pins should be included in the RR logic. The Pins that did not meet the threshold were appended towards the end in the same order they were ranked in the utility-based ranking list.”. (note: this shows selecting a predetermined number (top K) of matches from the total pool of candidates retrieved from multiple buckets based on computed distance/score values. This is filtering matches based on a score and pre-determined number having the highest scores.)].
Per Claim 4, Silva-Huang discloses claim 1.
Silva further teaches filtering the one or more matches found in the plurality of content item buckets based on a score computed for each match, and a threshold on score values computed for each match [Silva, pg. 12, A.1 Round-Robin hyperparameters “In RR, score thresholds allow mitigating potential impact on utility metrics. We evaluated different values of the score threshold to determine which Pins should be included in the RR logic. The Pins that did not meet the threshold were appended towards the end in the same order they were ranked in the utility-based ranking list.”; pg. 7, 5.3 Strong-OR Retrieval “…we can specify that the retrieved candidates must belong to either d1 or d2, but in addition, we can also specify what (minimum) percentage of candidates match each of the respective criteria.”. (note: Silva filters results based on a score threshold, items not meeting the threshold are filtered out.)].
Per Claim 6, Silva-Huang discloses claim 1.
Silva does not fully disclose, but with Huang does teach determining a number of the one or more matches to find in the content item buckets based on the query [Silva, pg. 7 “…each leaf now aggregates a set of K candidates, and |D| buckets with (at most) kdi candidates each.”; pg. 7, 5.3 Strong-OR Retrieval “If there are insufficient candidates to fulfill the criteria specified, it will match as many as possible.”; Huang, pg. 6, 4.3 Query and Index Selection “We applied the query selection technique to overcome problems like over-triggering, huge capacity cost and junkiness increase. We did not trigger EBR for certain queries as EBR would be poor at and provide no extra value for them, such as easy queries with which searchers are looking for a specific target searched and clicked before, or queries with clearly different query intents from what the embedding model was trained for.”. (note: Huang shows query dependent retrieval decisions, different queries trigger different retrieval behaviors. Silva shows varying the number of candidates per bucket (Kdi) per query. These two together determine a number of matches to find in the content item buckets based on the query.)].
Silva and Huang are analogous art because they are from the same field of endeavor encompassing information retrieval and recommendation systems using embedding based retrieval and machine learning model training.
Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to apply the bucketing technique of Silva using popularity as the dividing attribute as taught by Huang.
The suggestion/motivation for doing so would have been that popularity is a relevant criterion for selecting content items in a retrieval system. [Silva, pg. 6, 4.3 Query and Index Selection “On index side, we did index selection to make searching faster. For example, we only chose monthly active users, recent events, popular pages and groups.”]
Per Claim 7, Silva-Huang discloses claim 1.
Silva does not fully disclose, but with Huang does teach updating a further model based on one or more feedback signals about the one or more matches found in each content item bucket [Silva, pg. 10 “We can analyze how diversified search results and recommendations can help mitigate serving bias in systems that generate their own training data, by creating a positive feedback loop for model retraining thanks to richer interaction data from a diverse set of Pins.”. (note: this shows a feedback loop for model retraining based on interaction signals (feedback signals) from the diversified retrieved matches across content item buckets. ‘Richer interaction data from a diverse set of Pins’ is feedback signals about the one or more matches found in each content item bucket. The model retraining step is updating a further model.); Huang, pg. 7, 5 Later-Stage Optimization “While embedding-based retrieval can improve retrieval recall, it might have a lower precision in comparison with term matching. To address the precision issue, we built a closed feedback loop based on human rating pipeline. In particular, we logged the results after enabling embedding-based retrieval, and then sent these results to human raters to label whether they are relevant or not. We used these human rated data to re-train the relevance model so that it can be used to filter out the irrelevant results from embedding-based retrieval while keeping the relevant ones.”. (note: this shows updating (re-training) a model using feedback signals about retrieved matches, human relevance ratings of the retrieved results)];
inputting the query into the further model [Huang, pg. 3, 2.1 Evaluation Metrics “We propose to run KNN search in the whole index and then use recall@K as defined in equation 1 as the model evaluation metric. In particular, we sampled 10000 search sessions to gather the query and target result set pairs for the evaluation set and reported averaged recall@K over 10000 sessions.”. (note: Huang shows inputting queries into the retrieval model (KNN search) as the standard operational step, the further model (the retrained relevance model) receives the query as input in each search session)];
and determining a number of the one or more matches to find in each content item buckets using the further model [Huang, pg. 5, 4.1 ANN “There are several important parameters we need to tune: Coarse quantization…nprobe is the parameter to decide how many clusters will be assigned to the query embedding, which will further decide how many coarse clusters will be scanned. This parameter will affect the perf and recall”; “Tune ANN parameters when there is non-trivial model change. We observed that ANN performance is related with model characteristics. For example, when we employed ensemble techniques with model trained with non-click impressions, we found that while the model showed a better recall than baseline, the recall was worse than baseline after applying quantization to both. It should always be considered to tune the ANN parameters when there is a non-trivial change of the model training task, e.g., add more hard negatives.”. (note: Huang shows that the number of clusters scanned per query, nprobe which determines how many candidates are retrieved per cluster (bucket), is a parameter that must be tuned whenever the model changes. After the model is updated, the per-bucket retrieval count (nprobe) is re-determined using the updated model. This is determining a number of the one or more matches to find in each content item bucket using the further model. This also shows that the further (updated) model is the direct input to the determination of per bucket retrieval count.)].
Silva and Huang are analogous art because they are from the same field of endeavor encompassing information retrieval and recommendation systems using embedding based retrieval and machine learning model training.
Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to apply the bucketing technique of Silva using popularity as the dividing attribute as taught by Huang.
The suggestion/motivation for doing so would have been that popularity is a relevant criterion for selecting content items in a retrieval system. [Silva, pg. 6, 4.3 Query and Index Selection “On index side, we did index selection to make searching faster. For example, we only chose monthly active users, recent events, popular pages and groups.”]
Per Claim 8, Silva-Huang discloses claim 7.
Silva further teaches the one or more feedback signals comprises a count of the one or more matches found in the plurality of content item buckets meeting a quality criterion [Silva, pg. 7, 5.2 Bucketized-ANN Retrieval “…each leaf now aggregates a set of K candidates, and |D| buckets with (at most) Kdi candidates each. This helps preserve the top candidates belonging to each group (whose distances are already computed) from being dropped during the aggregation steps”; pg. 7, 5.3 Strong-OR Retrieval “If there are insufficient candidates to fulfill the criteria specified, it will match as many as possible.”. (note: this shows tracking counts of matches from each bucket that meet criteria (minimum threshold Kdi candidates from each group). This is counting matches meeting a quality criterion.].
Per Claim 10, Silva-Huang discloses claim 1.
Silva further teaches determining scores for content items associated with the one or more attributes of the content items; and distributing the content items into the content item buckets based on the scores [Silva, pg. 8, 6.1 Indexing “For each Pin, the skin tone range (if applicable) is computed offline using a computer vision model. An offline batch workflow periodically reads the skin tone predictions generated for each Pin from a store and adds it to the indexing pipelines of each surface for fast retrieval. The indexed diversity dimension can be passed along with the candidate Pins to the ranking stage for ranking diversification”. (note: this shows computing attribute scores (skin tone values) for each content item and then using those scores to assign items to buckets for retrieval.)].
Per Claim 14, Silva-Huang discloses claim 1.
Silva does not fully disclose, but with Huang does teach clustering the content items based on the one or more attributes of the content items to generate the one or more content item buckets having cohorts of content items that share similarities in the one or more attributes [Silva, pg. 3, 3.2 Diversity in Recommendations “Diversity dimensions may include explicit dimensions such as demographics (e.g., age, gender), geographic or cultural attributes (e.g., country, language), domain specific dimensions (e.g., skin tone ranges in beauty, cuisine type in food), business-specific dimensions (e.g., merchant sizes), but also other implicit dimensions that may not be expressed directly but can be modeled using latent representations (e.g., embedding, clustering).”; pg. 7, 5.2 Bucketized-ANN Retrieval, “…each leaf now aggregates a set of K candidates, and |D| buckets with (at most) Kdi candidates each”; Huang, pg. 5, 4.1 ANN “There are two major components for the embedding quantization, one is the coarse quantization which quantizes embedding vectors into coarse clusters typically through K-means algorithm”. (note: Silva generates buckets having cohorts of items that share similarities in their diversity dimension attributes, including via clustering. Huang shows coarse quantization via K-means clustering to group vectors into clusters, which is content item buckets with shared similarities.)].
Silva and Huang are analogous art because they are from the same field of endeavor encompassing information retrieval and recommendation systems using embedding based retrieval and machine learning model training.
Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to apply the bucketing technique of Silva using popularity as the dividing attribute as taught by Huang.
The suggestion/motivation for doing so would have been that popularity is a relevant criterion for selecting content items in a retrieval system. [Silva, pg. 6, 4.3 Query and Index Selection “On index side, we did index selection to make searching faster. For example, we only chose monthly active users, recent events, popular pages and groups.”]
Per claim 18 A computer-implemented system comprising [Silva, pg. 1, 1 Introduction “We developed and productionized a multi-stage diversification system that operates both at retrieval and ranking stages. For ranking, we developed greedy re-rankers and multi-objective optimization using Determinantal Point Process (DPP), and for retrieval, we implemented a Strong- OR operator for search over token-based indices, as well as Overfetch-and-Rerank and Bucketized-ANN Retrieval over embedding-based indices.”]:
a plurality of retrieve content item parts, comprising: a first retrieve content items part to retrieve one or more first matches to the query from a first content item bucket; and a second retrieve content items part to retrieve one or more second matches to the query from a second content item bucket [Silva, pg. 7, 5.2 Bucketized-ANN Retrieval “For the Bucketized-ANN Retrieval approach, we modify the aggregation step (at the leaf and the root level) to also aggregate top 𝐾𝑑𝑖 candidates from each group 𝑑𝑖 ∈ D into buckets corresponding to each of the groups under the diversity dimension (in addition to aggregating the top-K candidates in the overall pool).”. (note: Silva shows a plurality of retrieve content item parts, including a first and second retrieve part to pull matches from respective first and second content item buckets. The Bucketized-ANN Retrieval retrieves content item parts. By independently isolating and collecting the top Kd candidates from separate groups (di) into their respective designated buckets, the retrieval part therefore performs the steps of pulling a first set of mataaches from a first content bucket, anda second set of matches from a second content bucket. )];
a merge part to receive the one or more first matches and the one or more second matches and output a retrieved set of matches [Silva, pg. 6, 5.2 Bucketized-ANN Retrieval “Let’s say there are L number of leaves and 𝑀 number of segments per leaf; to find 𝐾 nearest neighbors for a given query embedding, each segment returns K potential nearest neighbor candidates to the corresponding leaf, which then aggregates these M × K number of candidates to only retain the top 𝐾 candidates, before passing it along to the root. The root is then responsible for choosing the top 𝐾 candidates from the K × L candidates whose exact distances are computed during the process…each leaf now aggregates a set of K candidates, and |D| buckets with (at most) Kdi candidates each. This helps preserve the top candidates belonging to each group (whose distances are already computed) from being dropped during the aggregation steps, without incurring the high cost of expanding the entire aggregation graph in the Overfetch-and-Rerank approach.”. (note: Silva shows a merge part (root node) that receives candidate from multiple leaves (bucket) and merges them into a single retrieved set)]; and
Silva does not expressly disclose, but Silva combined with Huang does teach:
a first model to receive a query [Silva, pg. 3, 3.1 Background “These systems leverage machine learning (ML) models trained to optimize certain objectives given inputs like queries, content features, user features, and past interactions between users and items that happened on the platform.”; Huang, pg. 3, 2.3 Unified Embedding Model “…our model comprises three major components: a query encoder EQ = f (Q) which produces a query embedding, a document encoder ED = g(D) which produces a document embedding, and a similarity function S(EQ, ED) which produces a score between query Q and document D”. (note: Silva and Huang show receiving a query)];
a content item retrieval system comprising a second model that is trained using the retrieved set of matches as training data [Silva, pg. 10, 9 Conclusions and Future Work “We can analyze how diversified search results and recommendations can help mitigate serving bias in systems that generate their own training data, by creating a positive feedback loop for model retraining thanks to richer interaction data from a diverse set of Pins.” (note: Silva shows that the system uses its own retrieved results to generate training date for model retraining); Huang, pg. 2, 1 Introduction “…we incorporated embeddings into ranking layers and built a training data feedback loop to actively learn to identify those good and bad results from embedding-based retrieval.”. (note: Huang shows a training data feedback loop where the results pulled from retrieval are used to train the ranking layers (second model), which are trained using the exact set of matches that were retrieved.)].
Silva and Huang are analogous art because they are from the same field of endeavor encompassing information retrieval and recommendation systems using embedding based retrieval and machine learning model training.
Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to apply the bucketing technique of Silva using popularity as the dividing attribute as taught by Huang.
The suggestion/motivation for doing so would have been that popularity is a relevant criterion for selecting content items in a retrieval system. [Silva, pg. 6, 4.3 Query and Index Selection “On index side, we did index selection to make searching faster. For example, we only chose monthly active users, recent events, popular pages and groups.”]
Per Claim 19, Silva-Huang discloses claim 18.
Silva further teaches a score calculator part to compute scores for content items; and a bucketizer part to distribute the content items into the first content item bucket and the second content item bucket based on the scores [Silva, pg. 8, 6.1 Indexing “For each Pin, the skin tone range (if applicable) is computed offline using a computer vision model. An offline batch workflow periodically reads the skin tone predictions generated for each Pin from a store and adds it to the indexing pipelines of each surface for fast retrieval. The indexed diversity dimension can be passed along with the candidate Pins to the ranking stage for ranking diversification”. (note: this shows computing attribute scores (skin tone) for each item (score calculator part) and distributing the items to their corresponding bucket (bucketizer part) based on those scores).
Per Claim 20, Silva-Huang discloses claim 18.
Silva further teaches an optimizer part to determine a first number of the one or more first matches to retrieve and a second number of one or more second matches to retrieve for the query [Silva, pg. 7, 5.2 Bucketized-ANN Retrieval “…each leaf now aggregates a set of 𝐾 candidates, and |D| buckets with (at most) 𝐾𝑑𝑖 candidates each.”; pg. 12, A.4 Choice of k in Div@k(R) “We consider two factors when choosing a value of k for the diversity metric Div@k(R): maximizing the coverage in terms of queries, and observing perceptible changes in the diversity of the rankings”. (note: this shows an optimization process for determining Kdi (the number of candidates to retrieve from each bucket), which is the optimizer part determining the first and second numbers of matches to retrieve per bucket)].
Claim 5 is rejected under 35 U.S.C. 103 as being unpatentable over Silva in view of
Huang, in further view of Nogueira.
As to claim 5, Silva-Huang discloses the method of claim 1.
Silva combined with Huang does not expressly disclose, but with Nogueira does teach for a first match in the one or more matches found in the plurality of content item buckets,
inputting a prompt to a large language model, the prompt comprising a question whether the first match, given metadata of the first match as context, is associated with the query [Nogueira, pg. 2 “Our reranking method is based on T5 [14], which is a sequence-to-sequence model that uses a similar masked language modeling objective as BERT to pretrain its encoder–decoder architecture. In this model, all target tasks are cast as sequence-to-sequence tasks. For our task, the input sequence is: Query: q Document: d Relevant: (1) where q and d are the query and document texts, respectively. The model is fine-tuned to produce the words “true” or “false” depending on whether the document is relevant or not to the query.”. (note: this shows the construction of a structured prompt, the input sequence ‘Query: q Document: d Relevant’, and providing it to a pretrained sequence-to-sequence language model (T5). The query q is the claimed query. The document d is the first match, and its text is the metadata of that match provided as context. The prompt as a whole poses the question of whether the match is associated with the query, this is what ‘Relevant:’ asks the model to complete.);
]; and
removing the first match based on a negative response to the prompt [Nogueira, pg. 2 “At inference time, to compute probabilities for each query–document pair (in a reranking setting), we apply a softmax only on the logits of the “true” and “false” tokens.”. (note: documents receiving low probability for the “true” token, for which the model’s response is effectively “false” (negative response), are ranked lower or excluded from the output set. A document for which the model outputs “false” is a match that the system removes based on that negative response. This is removing the first match based on a negative response to the prompt. The “false” output is the negative response, the removal is the result of a low “true” probability in the reranking step.)].
Silva, Huang, and Nogueira are analogous art because they are from the same field of endeavor of information retrieval and recommendation systems using embedding and language model based approaches.
Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to use two language models each independently retrieving semantic matches as taught by Nogueira from Silva’s content item buckets, and then combining both sets of matches as training data.
The suggestion/motivation for doing so would have been to improve the precision of the retrieved candidate set by removing irrelevant matches [Nogueira, pg. 1 “This gives rise to the standard multi-stage pipeline architecture of keyword retrieval followed by reranking using one or more machine learning models [1, 10].”].
Claim 9 is rejected under 35 U.S.C. 103 as being unpatentable over Silva in view of Huang, in further view of Yi.
As to claim 9, Silva-Huang discloses the method of claim 1.
Silva combined with Huang does not expressly disclose, but with Yi does teach the one or more attributes of the content items comprises popularity [Silva, pg. 3, 3.2 Diversity in Recommendations “Diversity dimensions may include explicit dimensions such as demographics (e.g., age, gender), geographic or cultural attributes (e.g., country, language), domain specific dimensions (e.g., skin tone ranges in beauty, cuisine type in food), business-specific dimensions (e.g., merchant sizes)…”; Huang, pg. 6, 4.3 Query and Index Selection “On index side, we did index selection to make searching faster. For example, we only chose monthly active users, recent events, popular pages and groups.”; Yi, pg. 3, 3 Modeling Framework “In-batch items are normally sampled from a power-law distribution in our target applications. As a result, Equation (3) introduces a large bias towards full softmax: popular items are overly penalized as negatives due to the high probability of being included in a batch.”. (note: Silva groups content items into buckets based on item attributes (diversity dimensions). Yi shows the role of popularity (power-law distribution) as an item attribute that biases retrieval and training. Huang shows selecting by popularity (monthly active users, popular pages, and groups) as a criterion for retrieval.)].
Silva, Huang, and Yi are analogous art because they are from the same field of endeavor of information retrieval and recommendation systems using embedding and language model based approaches.
Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to use popularity as the dividing attribute as taught by Yi on the bucketing technique.
The suggestion/motivation for doing so would have been that Yi explicitly teaches that item popularity (power-law distribution) creates a systematic problem in retrieval training and that correcting for popularity is a necessary design consideration. The person of ordinary skill in the art would bucket content items by popularity to address this exact bias, resulting in a more accurate retrieval model. [Yi, pg. 3, 3 Modeling Framework “In-batch items are normally sampled from a power-law distribution in our target applications. As a result, Equation (3) introduces a large bias towards full softmax: popular items are overly penalized as negatives due to the high probability of being included in a batch.”].
Claim 11 is rejected under 35 U.S.C. 103 as being unpatentable over Silva in view of Huang, in further view of Mansoury.
As to claim 11, Silva-Huang discloses the method of claim 10.
Silva combined with Huang does not expressly disclose, but with Mansoury does teach distributing the content items into the content item buckets comprises distributing the content items using a percentile approach [Silva, pg. 7, 5.2 Bucketized-ANN Retrieval “For the Bucketized-ANN Retrieval approach, we modify the aggregation step (at the leaf and the root level) to also aggregate top 𝐾𝑑𝑖 candidates from each group 𝑑𝑖 ∈ D into buckets corresponding to each of the groups under the diversity dimension”; Mansoury, pg. 1 “We propose a rating transformation model that compensates for skew in the rating distribution as well as its central tendency by converting ratings into percentile values as a pre-processing step before recommendation generation.”; pg. 5 “In statistics, given a series of measurements, percentile (or quantile) methods are used to estimate the value corresponding to a certain percentile. Given the Pth percentile, these methods attempt to put P% of the data set below and (100-P)% of the data set above… The percentile value, p, corresponding to a measurement, x, in a series of measurements, M, is computed with regard to the position of x in the ordered list M, o(M), as follows:
p
z
x
,
M
=
100
×
p
o
s
i
t
i
o
n
z
(
x
,
o
M
)
|
M
|
+
1
”; pg. 8 “…we created ten equal length bins for percentile and z-score values and aggregated each bin by its mean”. (note: Silva distributes content items into buckets based on attribute scores. Mansoury converts item associated scores (ratings) into percentile values, and further shows dividing the percentile range into equal length bins. This is a percentile approach for distributing items by their scores into distinct buckets.)].
Silva, Huang, and Mansoury are analogous art because they are from the same field of endeavor encompassing recommendation systems.
Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to transform the content item’s attribute score into a percentile value as taught by Mansoury and assign the item to the content item bucket corresponding to its percentile bin.
The suggestion/motivation for doing so would have been that percentile transformation compensates for skew in score (rating) distributions [Mansoury, pg. 3 “We propose a rating transformation model that converts users’ ratings into percentile values to compensate for skew in rating distributions and variances in users’ rating behaviors.”; pg. 10 “We hypothesize that a transformation that produces a flatter distribution will compensate for skew in the rating distribution and generate improved recommendation performance.”].
Claim 12 is rejected under 35 U.S.C. 103 as being unpatentable over Silva in view of Huang and Yi, in further view of Mansoury and Kang.
As to claim 12, Silva-Huang-Yi discloses the method of claim 10.
Silva combined with Huang and Yi does not expressly disclose, but with Mansoury and Kang does teach distributing the content items into the content item buckets [Kang, pg. 5, 4.1 Frequency-based Partitions “We first assign ascending IDs to the items from the most popular to least popular ones. Then we split the frequency ordered vocabulary V into a multi-tier partition
V
~
= (V1,V2, ...,Vm), where
∪
i
=
1
m
Vi = V , Vi ∩ Vj = ∅ for i ≠ j, m is the number of groups, V1 contains the most popular items, and Vm contains the least popular items.”. (note: this shows distributing content items into a mult-tier partition of m distinct groups, which is distributing the content items into the content item buckets)] comprises distributing the content items using a recursive Pareto distribution approach [Kang, pg. 5, 4.1 Frequency-based Partitions “As we consider recommendation datasets which usually follow power-law distributions, typically we have |V1 | < |V2 | < ... < |Vm|.”; pg. 1, Abstract “…to better handle the power-law data distribution commonly seen in recsys, we propose a Multi-Granular Quantized Embeddings (MGQE) technique which learns more compact embeddings for infrequent items.”; pg. 4, 4 Multi-Granular Quantized Embeddings “In recommendation domains, usually a few popular items dominate the training data, while the majority of the items (i.e. long-tail items) are rarely observed. In this case, allocating the same embedding capacity to all items is sub-optimal, as it could lead to overfitting on infrequent users/items due to data sparsity and high embedding dimensions.”; Mansoury, pg. 11 “…we create cumulative popularity list of items sorted from most popular to less-popular items, then we define a cutting point such that it divides the items into short-head and long-tail items. For experiments in this paper, we used cutting point of 20%, meaning that 20% of most popular items are considered as short-head items and the rest of less popular items are considered as long-tail items.”; (note: Kang shows the key structural feature of a recursive Pareto distribution approach, the multi-tier partitions produce groups of strictly increasing size, |V1 | < |V2 | < ... < |Vm|, where V1, which is the head and most popular items, is the smallest group and Vm, which is the tail and least popular items, is the largest. The size ordering is the direct mathematical result of applying a Pareto-style proportion cut successively to the remaining items. Kang also shows this structure results because recommendation data usually follow power-law distributions, which is the Pareto distribution. Mansoury shows that the standard cutting point for the Pareto based head/tail division is 20% (the 80/20 rule). Kang’s multi-tier partition with |V1 | < |V2 | < ... < |Vm| is generated by successively applying a Pareto-like proportion cut to the remaining items, a recursive Pareto distribution, since this is the only partitioning mechanism consistent with both the power-law distribution of item popularity and the 20%/80% Pareto cutting point.)].
Silva, Huang, Yi, Mansoury, and Kang are analogous art because they are from the same field of endeavor of information retrieval and recommendation systems using embedding and language model based approaches.
Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to sort items by popularity and apply a cutting point to divide them into Pareto structured groups as taught by Mansoury to create increasing tiers as taught by Kang, applied recursively to produce each successive tier.
The suggestion/motivation for doing so would have been that percentile transformation compensates for skew in score (rating) distributions [Mansoury, pg. 3 “We propose a rating transformation model that converts users’ ratings into percentile values to compensate for skew in rating distributions and variances in users’ rating behaviors.”; pg. 10 “We hypothesize that a transformation that produces a flatter distribution will compensate for skew in the rating distribution and generate improved recommendation performance.”]. Kang explicitly states that the strictly increasing group size structure, the distinctive feature of a recursive Pareto distribution, is the natural and expected outcome of partitioning items from a power-law (Pareto) popularity distribution. This directly motivates applying a Pareto based recursive partition to distribute content items into buckets, since the Pareto distribution of item popularity is the reason the group sizes must be ordered as |V1 | < |V2 | < ... < |Vm| [Kang, pg. 5, 4.1 Frequency-based Partitions “As we consider recommendation datasets which usually follow power-law distributions, typically we have |V1 | < |V2 | < ... < |Vm|.”].
Claim 13 is rejected under 35 U.S.C. 103 as being unpatentable over Silva in view of Huang and Yi, in further view of Datar.
As to claim 13, Silva-Huang-Yi discloses the method of claim 10.
Silva combined with Huang and YI does not expressly disclose, but with Datar does teach distributing the content items into the content item buckets comprises distributing the content items to content item buckets, each content item bucket having a size which is set according to a geometric sequence [Datar, pg. 1799 “In order to satisfy this and Invariant 1 with as few buckets as possible, we maintain buckets with exponentially increasing sizes so as to satisfy the following second invariant. Invariant 2. At all times, the bucket sizes are nondecreasing, i.e., C1 ≤ C2 ≤ · · · Cm-1 ≤ Cm. Further, the bucket sizes are constrained to the following: {1, 2, 4,…, 2m’} for some m’ ≤ m and m’ ≤
l
o
g
2
N
k
+
1
”; pg. 1800 “32, 32, 16, 8, 8, 4, 2, 1”. “Traverse the list of buckets in order of increasing sizes. If there are
k
2
+
2
buckets of the same size, merge the oldest two of these buckets into a single bucket of double the size.”. (note: this shows a bucket sizing scheme in which bucket sizes are constrained to the set {1, 2, 4,…, 2m}, a geometric sequence. The worked example explicitly shows bucket sizes following this geometric progression. The merge rule stated is the exact mechanism that generates a geometric sequence of bucket sizes.)],
the geometric sequence having a base group size that is determined based on a target variance in the scores within individual content item buckets [Datar, pg. 1799 “Define
k
=
[
1
∈
]
, and assume that
k
2
is an integer…We will ensure that the relative error is at most 1/k by maintaining the following invariant. Invariant 1. At all times, the bucket sizes C1,…, Cm are such that, for all j ≤ m, we have
C
j
2
1
+
∑
i
=
1
j
-
1
C
i
≤
1
k
”; Yi, pg. 1, 1 Introduction “Training data collected from users’ feedback is very sparse for most items, and thus causes model predictions to have large variance for long-tail content.”. (note: Datar shows that the bucket sizing scheme is parameterized by a target relative error (1/k), which determines the smallest bucket size (the base of the geometric sequence) and the common ratio of the geometric progression. This is a base group size that is determined based on a target variance. Yi shows that variance is the specific metric of concern for content item buckets in a popularity based recommendation and retrieval context.)].
Silva, Huang, Yi, and Datar are analogous art because they are from the same field of endeavor of content item bucketing for retrieval systems.
Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to combine Datar’s geometric bucket sizing algorithm with variance based design criterion.
The suggestion/motivation for doing so would have been to control the statistical precision with each bucket [Datar, pg. 1797 “The problem is that the distribution of 1’s in the buckets may be nonuniform. We will incur large error when the interpolation takes place in buckets with a majority of the 1’s. This suggests another scheme, in which we use buckets of nonuniform width, so as to ensure that each bucket has a near-uniform number of 1’s.”].
Per claim 15, Silva does not expressly disclose, but Silva combined with Huang, Yi, and Nogueira does teach: One or more non-transitory computer-readable media having instructions stored thereon, when the instructions are executed by one or more processors, causes the one or more processors to [Nogueira, pg. 3 “We fine-tune our T5 models (base, large, and 3B) with a constant learning rate of 10−3 for 100k iterations with class-balanced batches of size 128. To simplify our training procedure (and related hyperparameters) as well as to eliminate the need for convergence checks, we simply trained for a fixed number of iterations, selected based on the computational demands of our largest model and the (self-allotted) time for running experiments… We use a maximum of 512 input tokens and one output token.”]:
determine popularity scores based on popularity-related data of content items;
split the content items into content item buckets based on the popularity scores [Silva, pg. 7, 5.2 Bucketized-ANN Retrieval “For the Bucketized-ANN Retrieval approach, we modify the aggregation step (at the leaf and the root level) to also aggregate top 𝐾𝑑𝑖 candidates from each group 𝑑𝑖 ∈ D into buckets corresponding to each of the groups under the diversity dimension”; Yi, pg. 3, 3 Modeling Framework “In-batch items are normally sampled from a power-law distribution in our target applications. As a result, Equation (3) introduces a large bias towards full softmax: popular items are overly penalized as negatives due to the high probability of being included in a batch.”. (note: Silva splits content items into buckets based on their attribute scores computed per item. Yi discloses item popularity as a key attribute with a power-law (popularity) distribution and that popularity scores affect the training objective. Together they show splitting content items into buckets based on popularity scores.)];
retrieve, using a first large language model, one or more matches in each content item bucket that semantically match a query [Nogueira, pg. 1 “The contribution of this work is to adapt a pretrained sequence-to-sequence model (in our case, T5 [14]) to the task of document reranking”; “We show how a sequence-to-sequence model can be trained to generate relevance labels as “target words”, and how the underlying logits of these target words can be interpreted as relevance probabilities for ranking”; pg. 2 “For our task, the input sequence is: Query: q Document: d Relevant: (1) where q and d are the query and document texts, respectively. The model is fine-tuned to produce the words “true” or “false” depending on whether the document is relevant or not to the query.”. (note: this shows using a pretrained sequence-to-sequence language model (T5) to assess semantic relevance between a query (q) and a document (d) by processing the structured prompt “Query: q Document: d Relevant:” and generating a relevance label. The model operates on each candidate document to determine whether it semantically matches the query. The large language model is used to identify, from among the candidates in each bucket, those that semantically match the query, for which the model outputs “true”.)];
generate training data based on the one or more matches from each content item bucket [Silva, pg. 10, 9 Conclusions and Future Work “We can analyze how diversified search results and recommendations can help mitigate serving bias in systems that generate their own training data, by creating a positive feedback loop for model retraining thanks to richer interaction data from a diverse set of Pins.”; Huang, pg. 2, 1 Introduction “…we incorporated embeddings into ranking layers and built a training data feedback loop to actively learn to identify those good and bad results from embedding-based retrieval.”. (note: Silva and Huang show generating training data from retrieved results across multiple buckets)]; and
update parameters of a machine learning model using the training data [Silva, pg. 3, 3.1 Background “These systems leverage machine learning (ML) models trained to optimize certain objectives given inputs like queries, content features, user features, and past interactions between users and items that happened on the platform.”; Yi, pg. 3, 3 Modeling Framework “Running SGD with learning rate γ yields the model parameter update as
θ
←
θ
-
γ
∙
∇
L
B
(
θ
)
(5)”. (note: Yi shows parameter update via SGD for the machine learning model. Silva shows machine learning model training.)].
Silva, Huang, Yi, and Nogueira are analogous art because they are from the same field of endeavor of information retrieval and recommendation systems using embedding and language model based approaches.
Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to use two language models each independently retrieving semantic matches as taught by Nogueira from Silva’s content item buckets, and then combining both sets of matches as training data.
The suggestion/motivation for doing so explicitly stated in Nogueira, that a sequence-to-sequence language model can be used to generate relevance labels for query-document pairs. This provides a direct way for semantically matching documents to queries [Nogueira, pg. 1 “We show how a sequence-to-sequence model can be trained to generate relevance labels as “target words”, and how the underlying logits of these target words can be interpreted as relevance probabilities for ranking”]. Nogueira also states using T5 as a retrieval model specifically in settings with limited training data, such as highly specific content items in popularity-based buckets where training data is sparse [Nogueira, pg. 5 “T5 is able to achieve roughly 45% of the possible gain in effectiveness over the BM25 baseline with only 4% of the training data.”].
Per Claim 16, Silva-Huang-Yi-Nogueira discloses claim 15.
Silva-Huang-Yi does not fully disclose, but with Nogueira does teach retrieve, using a second large language model, one or more further matches in each content item bucket that semantically match the query [Nogueira, pg. 3 “BM25+BERT-large: We additionally compare our method against the BERT-large condition from Nogueira et al. [10], which is a two-stage pipeline with bag-of-words retrieval (BM25) followed by a BERT reranker. Architecturally, it is the same as our method, the only difference being BERT vs. T5 as the reranking model… Main results on the development set of the MS MARCO passage retrieval task are shown in Table 1, comparing BERT-large [9] and T5 models of different sizes.”. (note: this shows two distinct language model architectures, BERT and T5, that are architecturally the same for document relevance scoring, differing only in the underlying model. This discloses using a second language model, BERT as an alternative to the first T5 model, to score query-document pairs for semantic matching.); Huang, pg. 8, 6.2 Embedding Ensemble “We learned from HNM experiments that both easy and hard examples are important for EBR model training – we need hard examples to improve model precision, but easy example are also important to represent the retrieval space. The model trained using random negatives simulates the retrieval data distribution and is optimized for recall at a very large K, but it has poor precision at top K when K is small. On the other hand, the model trained to optimize precision, e.g. models trained using non-click impressions as negatives or offline hard negatives, is good at ranking for smaller set of candidates but failed for retrieval tasks. Thereafter we propose to combine models trained with different levels of hardness by a multi-stage approach, in which the first stage model focuses on recall and the second stage model specializes at differentiating more similar results returned by the first stage model.”. (note: Huang shows using a second stage model specifically to retrieve further matches that complement the first stage model’s results)];
wherein the training data is generated further based on the one or more further matches from each content item bucket retrieved using the second large language model [Silva, pg. 3, 3.1 Background “A typical recommender system employs multiple candidate generators, each satisfying different criteria, and their candidates are aggregated before being passed to the next stage.”; Huang, pg. 2, 1 Introduction “…we incorporated embeddings into ranking layers and built a training data feedback loop to actively learn to identify those good and bad results from embedding-based retrieval.”. (note: this shows that multiple candidate generators (multiple LLMs) contribute candidates that are aggregated. Huang shows that all retrieved results, from any retrieval model, are incorporated into a training data feedback loop.)].
Silva, Huang, Yi, and Nogueira are analogous art because they are from the same field of endeavor of information retrieval and recommendation systems using embedding and language model based approaches.
Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to use two language models each independently retrieving semantic matches as taught by Nogueira from Silva’s content item buckets, and then combining both sets of matches as training data.
The suggestion/motivation for doing so explicitly stated in Nogueira, that a sequence-to-sequence language model can be used to generate relevance labels for query-document pairs. This provides a direct way for semantically matching documents to queries [Nogueira, pg. 1 “We show how a sequence-to-sequence model can be trained to generate relevance labels as “target words”, and how the underlying logits of these target words can be interpreted as relevance probabilities for ranking”]. Nogueira also states using T5 as a retrieval model specifically in settings with limited training data, such as highly specific content items in popularity-based buckets where training data is sparse [Nogueira, pg. 5 “T5 is able to achieve roughly 45% of the possible gain in effectiveness over the BM25 baseline with only 4% of the training data.”].
Per Claim 17, Silva-Huang-Yi-Nogueira discloses claim 15.
Silva does not fully disclose, but with Huang does teach generate the training data based further on one or more further matches from each content item bucket; wherein generating the training data based on the one or more matches and the one or more further matches comprises filtering out content items that do not meet a criterion [Silva, pg. 12 “The Pins that did not meet the threshold were appended towards the end in the same order they were ranked in the utility-based ranking list.”; Haung, pg. 4, 2.4 Training Data Mining “We believe it is because these negatives bias towards hard cases which might match the query in one or multiple factors, while the majority of documents in index are easy cases which do not match the query at all.”; pg. 7, 6.1 Hard Mining “…we found that the top results from K embeddings given a query were usually with…hard negatives…”. (note: Silva shows filtering out content items from the set that do not met a score threshold criterion. Haung shows filtering training data to exclude items not meeting a quality criterion.)].
Silva, Huang, Yi, and Nogueira are analogous art because they are from the same field of endeavor of information retrieval and recommendation systems using embedding and language model based approaches.
Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to use two language models each independently retrieving semantic matches as taught by Nogueira from Silva’s content item buckets, and then combining both sets of matches as training data.
The suggestion/motivation for doing so explicitly stated in Nogueira, that a sequence-to-sequence language model can be used to generate relevance labels for query-document pairs. This provides a direct way for semantically matching documents to queries [Nogueira, pg. 1 “We show how a sequence-to-sequence model can be trained to generate relevance labels as “target words”, and how the underlying logits of these target words can be interpreted as relevance probabilities for ranking”]. Nogueira also states using T5 as a retrieval model specifically in settings with limited training data, such as highly specific content items in popularity-based buckets where training data is sparse [Nogueira, pg. 5 “T5 is able to achieve roughly 45% of the possible gain in effectiveness over the BM25 baseline with only 4% of the training data.”].
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Sayed M Shah whose telephone number is (571)272-9406. The examiner can normally be reached Monday-Friday 6:00 am - 2:00 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Miranda Huang can be reached at (571) 270-7092. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SAYED MUNEER SHAH/Examiner, Art Unit 2124
/MIRANDA M HUANG/Supervisory Patent Examiner, Art Unit 2124
Prosecution Insights