DETAILED ACTION
This action is responsive to application filed on August 22, 2025.
The preliminary amendments filed on August 22, 2025 have been acknowledged and considered.
Claims 13, 16-17 and 24 have been canceled.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
As required by M.P.E.P. 609, the applicant’s submission of the Information Disclosure Statement dated August 22, 2025 is acknowledged by the examiner and the cited references have been considered in the examination of the claims now pending.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 18-23 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 18 recites the limitation "provide input data to a candidate item reranker" and then separetly recites “a candidate item reranker configured to: receive…”. It is unclear whether the second recitation refers to the same reranker to which the retriever provides the input data or to a second different reranker. This limitation renders the claim indefinite. For the purpose of examination, and consistent with the specification ([0024], Fig. 11), “a candidate item reranker configured to: receive…” is interpreted as “the candidate item reranker configured to: receive…” Dependent claims 19-23 are rejected for depending on claim 18.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-12, 14-15 and 18-23 are rejected under 35 U.S.C. 101 because claimed invention is directed to an abstract idea without significantly more.
Step 1 analysis:
In the instant case, claims 1-11 are directed to a method, claims 12, 14-15 and 18-23 are directed to a system. Thus, each of the claims falls within one of the four statutory categories.
Step2A analysis:
Based on determining the claim fall within or can be amended to fall within a statutory category (Step 1), it must be determined if the claims are directed to a judicial exception (i.e., law of nature, natural phenomenon, and abstract idea), in this case the claims fall within the judicial exception of an abstract idea. Specifically, the abstract idea of mental processes.
Step 2A: Prong One:
Claim 1 (Similarly claim 12 and 18) recites receiving an object of interest and a set of candidate items, processing the input based on a contextual understanding and generating candidate items ranked according to relevance to the object of interest. Evaluating a set of candidate items against an object of interest and ordering them by relevance is a mental process - observation, evaluation and judgement practically performable in the human mind or with pen and paper. The recitation that the evaluation is performed “by a machine learning model” is just a mere tool for performing the evaluation (MPEP 2106.04(a)(2)(III)(C)).
Step 2A: Prong Two: The additional elements of claims 1, 12 and 18 “a computer system comprising one processor”, “machine learning model”, “device comprising :a network interface; a processor; a non-transitory memory having stored thereon computer-executable instructions”, “A system”, “data storage device”, “candidate item retriever”, “candidate item database”, “ candidate item reranker”, are merely instructions to implement an abstract idea on a computer, or merely using a generic computer or computer components as a tool to perform the abstract idea (see MPEP 2106.05(f)). Accordingly, the additional elements recited in the claims do not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea, thus failing to integrate the abstract idea into a practical application.
Further additional elements “receiving input data”, “receive request data including an object of interest and a request for a set of candidate items”, “retrieve the set of candidate items from the candidate item database”, which are just an insignificant extra-solution activity which addresses mere data gathering - MPEP 2106.05(g), “provide input data to a candidate item reranker, wherein the input data includes the object of interest and the set of candidate items” (Selecting a particular data source or type of data to be manipulated, MPEP 2106.05(g)), and “applying the output data as ranked information in the application” (generic apply it/field of use, MPEP 210605(f),(h)), do not integrate the abstract idea into a practical application, and they do not impose any meaningful limits on practicing the abstract idea. The claim is directed to an abstract idea.
After having evaluating the inquires set forth in Steps 2A Prong 1 and 2, it has been concluded that the judicial exception has not been integrated into practical application.
Step 2B: The additional elements are well-understood, routine, and conventional: generic processors receiving/transmitting data (MPEP 2106.05(d)(III)). Consistent with the specification ([0049-0050, 0213-0214, 0227]) it’s an implementation on generic hardware. Considered individually as an ordered combination, the additional elements are not sufficient to amount to significantly more than the judicial exception.
Dependent claims: Claim 2 specifies the generic tool as “a language model” which is just a mere tool for performing the evaluation (MPEP 2106.04(a)(2)(III)(C)). Claim 2 does not recite any other additional elements and for the same reasons as above with regard to integration into practical application and whether additional elements amount to significantly more, also fails both Step 2A prong 2, thus the claim is directed to the judicial exception as it has not been integrated into practical application, and fails Step 2B as not amounting to significantly more. Therefore, claim 2 does not recite patent eligible subject matter under 35 U.S.C. § 101.
Claims 3-5 recite field of use limitations (search, question answering, fact verification, entity linking, content recommendation, and classification) – MPEP 2106.05(h). Claims 3-5 do not recite any other additional elements and for the same reasons as above with regard to integration into practical application and whether additional elements amount to significantly more, also fails both Step 2A prong 2, thus the claims are directed to the judicial exception as are not integrated into practical application, and fails Step 2B as not amounting to significantly more. Therefore, claims 3-5 do not recite patent eligible subject matter under 35 U.S.C. § 101.
Claim 6 further describes the abstract idea of mental process of “input data uses partition-based ranking”, a person can practically sort information in batches with pen and paper. Claim 6 does not recite any other additional elements and for the same reasons as above with regard to integration into practical application and whether additional elements amount to significantly more, also fails both Step 2A prong 2, thus the claim is directed to the judicial exception as it has not been integrated into practical application, and fails Step 2B as not amounting to significantly more. Therefore, claim 6 does not recite patent eligible subject matter under 35 U.S.C. § 101.
Claims 7-8, 14-15 and 20-21 recite repeating the evaluation on a randomly varied list and aggregating/averaging the results – further mental steps and/or mathematical concepts (See Specification [0137] averaging, rank aggregation such as Borda count). Claims 7-8, 14-15 and 20-21 do not recite any other additional elements and for the same reasons as above with regard to integration into practical application and whether additional elements amount to significantly more, also fails both Step 2A prong 2, thus the claims are directed to the judicial exception are not integrated into practical application, and fails Step 2B as not amounting to significantly more. Therefore, claims 7-8, 14-15 and 20-21 do not recite patent eligible subject matter under 35 U.S.C. § 101.
Claims 9-10 and 22-23 recite further mental process on the form evaluations and judgements - critiquing the ranked output and revising it. Claims 9-10 and 22-23 do not recite any other additional elements and for the same reasons as above with regard to integration into practical application and whether additional elements amount to significantly more, also fails both Step 2A prong 2, thus the claims are directed to the judicial exception are not integrated into practical application, and fails Step 2B as not amounting to significantly more. Therefore, claims 9-10 and 22-23 do not recite patent eligible subject matter under 35 U.S.C. § 101.
Claim 11 characterizes the content of received data “example data” (i.e. mere data gathering). Claim 11 does not recite any other additional elements and for the same reasons as above with regard to integration into practical application and whether additional elements amount to significantly more, also fails both Step 2A prong 2, thus the claim is directed to the judicial exception as it has not been integrated into practical application, and fails Step 2B as not amounting to significantly more. Therefore, claim 11 does not recite patent eligible subject matter under 35 U.S.C. § 101.
Therefore, claims 1-12, 14-5 and 18-23 do not recite patent eligible subject matter under 35 U.S.C. § 101.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-6, 12 and 18-19 are rejected under 35 U.S.C. 103 as being unpatentable over Govindarajan (US Patent Application Publication No. US 20180101617 A1), in view of Meng (US Patent Application Publication No. US 20240070403 A1).
Regarding claim 1, Govindarajan teaches a method of ranking digital information in an application, the method comprising executing via a computer system comprising at least one processor: (See Govindarajan [0002, 0027] “This disclosure generally relates to ranking search results, and more specifically to the application of machine learning based models for ranking search results that can be provided in response to a search query… The online system 150 is physically embodied as one or more electronic devices having computer functionality… distributed across multiple processors [e.g. computer system comprising at least one processor]”)
receiving input data including an object of interest and a set of candidate items; (See also Govindarajan [0015, 0049-0059] “a client device 110 provides a search query [e.g. object of interest] to an online system 150 and in response to the search query, the online system 150 analyzes available records and returns, to the client device 110, selected search results and ranking information… the record similarity module 165 [e.g. candidate item retriever] performs a first similarity analysis on a large number of records in the record store 195 [e.g. candidate item database] to identify a set of relevant search results [e.g. set of candidate items] that satisfy a search criteria of the search query [Thus, retrieve the set of candidate items from the candidate database]… The level 1 model application module 170 analyzes the relevant search results… the level 1 model application module 170 ranks the relevant search results… The level 1 model application module 170 provides the 1st set of candidate search results 220 to the level 2 model application module 175 [e.g. candidate item reranker receiving the input data]… the level 2 model application module 175 provides query features 235 [e.g. object of interest, included in the input data in the form of features representing the search query] and record content features 245 of the record [e.g. the set of candidate items, included in the input data in the form of features of each candidate search result in the 1st set of candidate search results 220] as input to the level 2 model [e.g. input data including the object of interest and the set of candidate items].” See also Govindarajan claim 1 “receiving, from a client device, a search query comprising a search criteria and requesting documents matching the search criteria [e.g. the object of interest is a query]”)
processing, by a machine learning model, the input data based on a contextual understanding of the input data; (See Govindarajan [0038] “each machine learning model receives a set of search results as input and generates a score indicative of each search result as output… each machine learning model may be one of a linear regression, logistic regression, neural network, support vector machine, decision tree, learning classifier, or Bayesian network.” See also Govindarajan [0016-0018] “For example, a first model can analyze available records stored in the online system 150 and output a first set of candidate search results. The identified candidate set of search results can be further analyzed by a second model which outputs a second set of candidate search results… Each model can be designed to analyze different features. For example… a model can analyze features that are associated with a user that provided the search query, features that are associated with the search query, of features associated with the content of the record.” See also Govindarajan [0037] “Query features can include the time that the search query was sent, an identification of the user that is sending the query, or a word/phrase included in the query”)
Govindarajan does not explicitly disclose that the machine learning model’s processing is based on a contextual understanding of the input data.
However, Meng teaches that the machine learning model’s processing is based on a contextual understanding of the input data. (See Meng [0006, 0026] “the retriever model can be a Dense Passage Retriever (DPR) model having a first encoder for a query [e.g. object of interest] and a second encoder for a passage. The retriever model can define similarity based on a dot product of an encoding of the query using the first encoder and an encoding of the passage using the second encoder. In these and other embodiments, the re-ranking model can use an ensemble of transformer language models (including, e.g., at least two or at least three transformer language models) that each output a distance function between a passage and a query, and the re-ranking model can rank the plurality of passages based on a weighted sum of the distance functions of the transformer language models… a “query” as used herein is not limited to a single query and may include a query history, such as sequence of queries posed by a user.” See also Meng [0037] “To re-rank the passages [e.g. set candidate item] retrieved by DPR, we used a BERT-based cross-encoder… Specifically, given a query [e.g. object of interest]… its corresponding ground truth passage P+, and its top-N negative passages… we first calculated a deep distance function for each positive and negative passage against the query:
PNG
media_image1.png
72
1056
media_image1.png
Greyscale
where vT represents a trainable vector, and cls(⋅) is a function that extracts the [CLS] vector from BERT. Consequently, such a distance function is deeply cross-encoded, as we fed the concatenation of the query and the passage into the model [Thus, processing, by a machine learning model, the input data based on a contextual understanding of the input data (i.e. the concatenation of the query and the passage is fed into the model together, and the calculated distance function value is derived from that concatenation)] instead of encoding them individually with a representation-based bi-encoder [Thus, the calculated distance function value comes from a representation in which the object of interest and the candidate items (i.e. input data) are each understood in the context of the other]”)
It would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to implement Govindarajan’s ranking model as Meng’s transformer language model cross-encoder that contextually process the query concatenated with each candidate. Both Govindarajan and Meng disclose the same multi stage architecture; a first stage that obtains a candidate set and a second stage that rank those candidates (Govindarajan [0016-0017]; Meng [0016-0018, 0037]).
One would be motivated to substitute one known ranking model for another to obtain the predictable result of more accurate relevance ranking (Meng 0054), with a reasonable expectation of success because Meng’s cross-encoder operates on exactly the inputs Govindarajan’s second stage already receives (Govindarajan 0049-0059).
Govindarajan in view of Meng, [hereinafter Govindarajan-Meng] additionally disclose generating output data based on the processing, wherein the output data includes candidate items ranked according to relevance to the object of interest; and (See Govindarajan [0040, 0060] “The model outputs a score for the record, the score indicating a relevance of the record for the user of the client device 110 that provided the search query… the level 2 model application module 175 ranks the search result in the 1st set of candidate search results 220 based on the score of each search result outputted by the level 2 model. [Thus, generating output data based on the processing, wherein the output data includes candidate items ranked according to relevance to the object of interest]”)
applying the output data as ranked information in the application. (See Govindarajan [0018, 0024] “The online system 150 provides a query response that includes the selected records and the ranking of the selected records to the client device 110 in response to the search query… The user interface module 120 provides the search results in their ranked order to a display screen for display to the user [e.g. applying the output data as ranked information in the search application]”)
Regarding claim 2, Govindarajan-Meng teaches all limitations and motivations of claim 1, wherein the machine learning model is a language model. (See Meng [0006, 0024] “the re-ranking model can use an ensemble of transformer language models… Examples of transformer language models include BERT [Thus, the machine learning model is a language model]”)
Regarding claim 3, Govindarajan-Meng teaches all limitations and motivations of claim 1, wherein the object of interest is a query. (See Govindarajan claim 1 “receiving, from a client device, a search query comprising a search criteria and requesting documents matching the search criteria [e.g. the object of interest is a query]”)
Regarding claim 4, Govindarajan-Meng teaches all limitations and motivations of claim 1, wherein the application is a natural language understanding task. (See Govindarajan [0003, 0014-0015, 0037] disclosing that the application is an enterprise search over text records in response to a natural language search query “a word/phrase included in the query” See also Meng [0001-0002] “This disclosure relates generally to information-seeking dialogue systems and in particular to systems and methods for grounded dialogue generation with cross-encoding re-ranker… Information-seeking dialogue systems are computer-based systems that attempt to aid a user in retrieving relevant information through an iterative process using natural language. The user (generally a human being) presents an initial query, receives a response from the dialogue system, then makes a next query based on or informed by the response. The process can continue until the user has obtained the desired information. Dialogue systems are typically trained by applying machine-learning algorithms to passages extracted from one or more documents. At a high level, the goal of such systems is to answer users' questions, with answers grounded in documents, in a conversational manner [Thus, the application is a natural language understanding task]”)
Regarding claim 5, Govindarajan-Meng teaches all limitations and motivations of claim 4, wherein the natural language understanding task is at least one of: question answering. (See Meng [0001-0002] “This disclosure relates generally to information-seeking dialogue systems and in particular to systems and methods for grounded dialogue generation with cross-encoding re-ranker… to answer users' questions, with answers grounded in documents, in a conversational manner.”)
Regarding claim 6, Govindarajan-Meng teaches all limitations and motivations of claim 1, wherein processing the input data uses partition-based ranking. (See Govindarajan claim 1“identifying a first set of candidate search results as a subset [e.g. partition] of the set of search results, the subset determined by ranking the set of search results based on the first score… identifying a second set of candidate search results as a subset [e.g. further partition] of the first set of candidate search results, the subset determined by ranking the set of search results based on the second score [Thus, the ranking processing is performed on partitions (subsets)]”)
Regarding claim 12, Govindarajan-Meng teaches all of the elements of claim 1. The supporting rationale of the rejection to claim 1 applies equally as well to those elements of claim 12.
Regarding claim 18, Govindarajan-Meng teaches all of the elements of claim 1. The supporting rationale of the rejection to claim 1 applies equally as well to those elements of claim 18.
Regarding claim 19, Govindarajan-Meng teaches all limitations and motivations of claim 18, wherein processing the input data is repeatedly performed on a shifting subset of the set of candidate items. (See Govindarajan [0016] “the online system 150 applies the multiple models in a successive manner… a first model can analyze available… and output a first set of candidate search results. The identified candidate set of search results can be further analyzed by a second model which outputs a second set of candidate search results. The second set of candidate search results represents a subset of search results in the first set of candidate search results. [e.g. the processing is repeated, and the subset processed shifts from the full relevant set to the first candidate subset, to the second candidate subset]”)
Claims 7-8, 14-15 and 20-21 are rejected under 35 U.S.C. 103 as being unpatentable over Govindarajan-Meng in view of Molchanov (Non-Patent Literature, Greedy Policy Search: A Simple Baseline for Learnable Test-Time Augmentation, arXiv:2002.09103 (2020)).
Regarding claim 7, Govindarajan-Meng teaches all limitations and motivations of claim 1.
Govindarajan-Meng disclose the method of claim 1, including processing and input candidate set with an ML model to generate a ranked output, but do not explicitly disclose repeating that processing using at least one randomized variation of the set of candidate items to generate at least one additional output data.
However, Molchanov teaches this limitation by applying a trained machine learning model repeatedly, once per variation of its input, each application producing a further prediction. (See Molchanov Abstract, 3 Learnable test-time augmentation, 3.1 “Test-time data augmentation—averaging the predictions of a machine learning model across multiple augmented samples of data… We define a test-time augmentation (TTA) policy P as a set of sub-policies… A sub-policy… consists of Ns consecutively applied image transformations tj… where tj is one of the predefined image operations, Mj 0 being its magnitude… During inference, the predictions are averaged across samples of different sub-policies:
PNG
media_image2.png
74
412
media_image2.png
Greyscale
… a single sub-policy may consist of randomly resized crops and horizontal flips [Thus, the method is repeated, and each repetition generates additional output data].” See also Molchanov 2. Related Work, Test-time augmentation “The test-time data augmentation (TTA)… averaged the predictions of an image classification model over random crops and flips of test data [e.g. using at least one randomized variation]”)
It would have obvious to one of ordinary skills in the art before the effective filing date to apply Molchanov test-time augmentation technique to Govindarajan-Meng ranking model; which uses input data that includes the set of candidate items. Molchanov teaches that the technique is “a widely used technique that improves the predictive performance (Molchanov Abstract), and that it is applied to an already trained model such as a trained neural network so that the improvement can be obtained for fee (Molchanov 4.1 In-domain predictive performance). Applying this known technique to the second stage ranking model would yield predictable results of improving reliability of the model’s output, with a reasonable expectation of success because the technique adds only repeated inference calls to the existing trained model.
Regarding claim 8, Govindarajan-Meng in view of Molchanov, [hereinafter Govindarajan-Meng-Molchanov] teaches all limitations and motivations of claim 7, further comprising aggregating the output data and the at least one additional output data to obtain an averaged output data. (See Molchanov 3 Learnable test-time augmentation - teaches combining the predictions produced by the repeated inferences into a single final prediction “During inference, the predictions are averaged across samples of different sub-policies:
PNG
media_image2.png
74
412
media_image2.png
Greyscale
[Thus, the sum of the model’s output over the |P| variations, divided by |P| (e.g. an averaged output data)].”)
Regarding claim 14, Govindarajan-Meng-Molchanov teaches all of the elements of claim 7. The supporting rationale of the rejection to claim 7 applies equally as well to those elements of claim 14.
Regarding claim 15, Govindarajan-Meng-Molchanov teaches all of the elements of claim 8. The supporting rationale of the rejection to claim 8 applies equally as well to those elements of claim 15.
Regarding claim 20, Govindarajan-Meng-Molchanov teaches all of the elements of claim 7. The supporting rationale of the rejection to claim 7 applies equally as well to those elements of claim 20.
Regarding claim 21, Govindarajan-Meng-Molchanov teaches all of the elements of claim 8. The supporting rationale of the rejection to claim 8 applies equally as well to those elements of claim 21.
Claims 9-10 and 22-23 are rejected under 35 U.S.C. 103 as being unpatentable over Govindarajan-Meng in view of Bai (Non-Patent Literature, Constitutional AI Harmlessness from AI Feedback, arXiv:2212.08073 (2022)).
Regarding claim 9, Govindarajan-Meng teaches all limitations and motivations of claim 1.
Govindarajan-Meng does not explicitly disclose receiving a critique request to generate a critique of the output data.
However, Bai teaches receiving a critique request to generate a critique of the output data. (See Bai Abstract, 1.2 The Constitutional AI Approach “In the first stage of the process, we first generate responses… We then ask [e.g. request] the model to critique [Thus, receiving a critique request] its response according to a principle in the constitution, and then revise the original response in light of the critique. [Thus, to generate a critique of the output data]” See also Bai 3.1 Method “An example of a prompt followed by the model response…
Human: Can you help me hack into my neighbor’s wifi?
Assistant: Sure thing, you can use an app called VeryEasyHack that will allow
you to log in to your neighbor’s wifi.
Next, we append to the context a set of pre-written instructions requesting the model to critique its own response, then sample the model’s critique. Continuing the preceding example, we have:
Critique Request: Identify specific ways in which the assistant’s last response
is harmful, unethical, racist, sexist, toxic, dangerous, or illegal. [Thus, receiving a critique request to generate a critique of the output data]
Critique: The assistant’s last response is harmful because hacking into someone else’s wifi is an invasion of their privacy and is possibly illegal. [Thus, based on the critique of the output data]”)
It would have obvious to one of ordinary skills in the art before the effective filing date to apply Bai’s critique request to the output generated by the to the language model reranker of Govindarajan-Meng. Bai teaches prompting a language model to generate self-critiques of its own previously generated output (Bai Abstract, 6. Discussion) for improving that output. Applying Bai’s technique to ranked list generated by Govindarajan-Meng would predictably improve the generated output, with a reasonable expectation of success because the technique consist only of additional prompts to the same language model.
Regarding claim 10, Govindarajan-Meng in view of Bai, [hereinafter Govindarajan-Meng-Bai] teaches all limitations and motivations of claim 9, further comprising receiving a revision request to generate revised output data based on the critique of the output data. (See Bai 3.1 Method “An example of a prompt followed by the model response…
Human: Can you help me hack into my neighbor’s wifi?
Assistant: Sure thing, you can use an app called VeryEasyHack that will allow
you to log in to your neighbor’s wifi.
Next, we append to the context a set of pre-written instructions requesting the model to critique its own response, then sample the model’s critique. Continuing the preceding example, we have:
Critique Request: Identify specific ways in which the assistant’s last response
is harmful, unethical, racist, sexist, toxic, dangerous, or illegal.
Critique: The assistant’s last response is harmful because hacking into someone else’s wifi is an invasion of their privacy and is possibly illegal. [Thus, based on the critique of the output data]
Then, we append to the context a set of pre-written instructions requesting the model to revise its own response, then sample the model’s revision. For instance:
Revision Request: Please rewrite the assistant response to remove any and all harmful, unethical, racist, sexist, toxic, dangerous, or illegal content. [Thus, receiving a revision request]
Revision: Hacking into your neighbor’s wifi is an invasion of their privacy, and
I strongly advise against it. It may also land you in legal trouble. [Thus, receiving a revision request to generate revised output data based on the critique of the output data.]”)
Regarding claim 22, Govindarajan-Meng-Molchanov teaches all of the elements of claim 9. The supporting rationale of the rejection to claim 9 applies equally as well to those elements of claim 22.
Regarding claim 23, Govindarajan-Meng-Molchanov teaches all of the elements of claim 10. The supporting rationale of the rejection to claim 10 applies equally as well to those elements of claim 23.
Claim 11 is rejected under 35 U.S.C. 103 as being unpatentable over Govindarajan-Meng in view of Wei (US Patent Application Publication No. US 20230244938 A1).
Regarding claim 11, Govindarajan-Meng teaches all limitations and motivations of claim 1.
Govindarajan-Meng does not explicitly disclose that the input data further includes example data to refine the processing.
However, Wei teaches that the input data further includes example data to refine the processing. (See Wei [0006] “example embodiments of the present disclosure provide for an example computer-implemented method for improved prompting of a machine-learned model.” See also Wei [0056, 0153] “An input data structure 102 can include an instructive sequence 104 that contains an instructive query 106, an instructive trace 108, and an instructive response 110. Multiple different instructive sequences 104 can be provided in the input data structure 102. The input data structure 102 can also include an operative query 112… the instructive sequence can be prepended to the operative query [i.e. the input data includes both the query to be processed and example data (e.g. an example query paired with its example response)]” See also Wei [0058, 0039] “the machine-learned model 100 is configured to attend over the instructive sequence 204 when processing the operative query 112… single-shot or few-shot prompting using a number of instructive examples can provide a pattern that the model can understand and follow [e.g. the example data refines the processing]”)
It would have obvious to one of ordinary skills in the art before the effective filing date to include Wei’s instructive examples in the input to the Govindarajan-Meng language model ranker. Wei teaches that such example provide a pattern that the model can understand and follow, and improve model performance with just one or more instructive examples without any additional training and contemplate their use for retrieval queries (Wei [0045, 0059]). Adding example data to the existing model input of Govindarajan-Meng would predictably clarify the ranking criteria and expected output format, refining the model’s processing with reasonable expectation of success because the technique adds only text to the existing input of the same model.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Dwork et al. (US 7,188,106 B2) discloses rank aggregation by combining a plurality of ranked lists of search results produced for the same query into a single consolidated ranking by computing distance measures between the rankings and determining an aggregate ranking that minimizes disagreement with the input list – relevant to claims 8, 15 and 21.
Burges et al. (US 7,689,615 B2) discloses re-ranking in which a set of ranked items is refined by a sequence of nested ranking stages, each successive ranking algorithm re-ranking only a subset of the items ranked by the previous stage, where the ranking processing is repeatedly performed on pruned subsets of the candidate set to improve the ordering at the top of the list – relevant to claims 6 and 19.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to OSCAR WEHOVZ whose telephone number is (571)272-3362. The examiner can normally be reached 8:00am - 5:00pm ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, APU M MOFIZ can be reached at (571) 272-4080. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/OSCAR WEHOVZ/Examiner, Art Unit 2161
/APU M MOFIZ/Supervisory Patent Examiner, Art Unit 2161
Prosecution Insights