DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status
This instant application No. 19/091,846 has claims 1-18 pending.
Priority / Filing Date
Applicant’s claims for priority of application No. 17/898,173 (now Pat. No. US 12265561) and provisional application No. 63/239,798 (filed on September 1, 2021) is acknowledged. The effective filing date of this application is September 1, 2021.
Abstract
The abstract of the disclosure is acceptable for examination purposes.
Drawings
The drawings received on March 27, 2025 are acceptable for examination purposes.
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the claims at issue are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); and In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on a nonstatutory double patenting ground provided the reference application or patent either is shown to be commonly owned with this application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP §§ 706.02(l)(1) - 706.02(l)(3) for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/forms/. The filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to http://www.uspto.gov/patents/process/file/efs/guidance/eTD-info-I.jsp.
Claims 1-18 are rejected on the ground of nonstatutory double patenting over claims 1-20 of Pat. No. US 12265561.
Claims 1-18 of the instant application recite similar limitations and claims 1-20 of ‘561 as being compared in the table below. For the purpose of illustration, only claims 1-16 (medium claims) of the instant application are compared to the claims of the patent (underlining are used to indicate conflict limitations). The remaining claims of the instant application recite different categories (i.e., system and method claims) and are therefore not compared for simplicity purposes.
Instant Application
Pat. No. US 12265561
Claim 1
A non-transitory computer-readable medium comprising instructions that, when executed by one or more processors, cause the one or more processors to perform operations comprising:
receiving feedback for a second document in a plurality of documents, wherein the feedback comprises:
implicit feedback recorded as a user views the second document; and
explicit feedback provided from the user indicating whether the user believes the second document is relevant to a first document in the plurality of documents;
providing the implicit feedback to a classification model that determines whether the implicit feedback indicates that the second document is relevant to the first document; and
adjusting parameters of a configuration in response to the feedback, wherein the configuration defines how to generate, from the first document, a plurality of queries used to generate similarity scores for the plurality of documents.
Claim 1
A non-transitory computer-readable medium comprising instructions that, when executed by one or more processors, cause the one or more processors to perform operations comprising:
receiving feedback from a user for a second document in the plurality of documents, wherein the feedback indicates whether the second document is relevant to the first document;
Claim 2
…the feedback comprises implicit feedback recorded as the user views the second document.
Claim 5
…the feedback further comprises explicit feedback in addition to the implicit feedback…
Claim 4
…providing the implicit feedback to a classification model that determines whether the implicit feedback indicates that the second document is relevant to the first document.
Claim 1
adjusting parameters of a configuration in response to the feedback…
…using adjusted parameters of the configuration to generate a plurality of queries, and executing the plurality of queries into a document repository...
See further Ogilvy and Agichtein below for mapping and motivation to combine with the claims of ‘561.
Claim 2
The non-transitory computer-readable medium of claim 1, wherein the implicit feedback comprises a dwell time.
Claim 3
The non-transitory computer-readable medium of claim 2, wherein the implicit feedback comprises a dwell time..
Claim 3
The non-transitory computer-readable medium of claim 2, wherein the classification model determines whether a dwell time indicates that the second document is relevant to the first document.
Claim 7
The non-transitory computer-readable medium of claim 4, wherein the classification model determines whether a dwell time indicates that the second document is relevant to the first document.
Claim 4
The non-transitory computer-readable medium of claim 1, wherein the operations further comprise:
providing a plurality of documents having similarity scores that are calculated in response to receiving a first document, wherein the similarity scores indicate similarities between the first document and the plurality of documents.
Claim 1
…providing a plurality of documents having similarity scores that are calculated in response to receiving a first document, wherein the similarity scores indicate similarities between the first document and the plurality of documents…
Claim 5
The non-transitory computer-readable medium of claim 1, wherein the operations further comprise training the classification model using the implicit feedback as training data and the explicit feedback as a label for the training data.
Claim 5
… training the classification model using the implicit feedback as training data and the explicit feedback as a label for the training data.
Claim 6
The non-transitory computer-readable medium of claim 1, wherein the operations further comprise generating a pop-up window with a display of the second document.
Claim 8
The non-transitory computer-readable medium of claim 1, wherein the operations further comprise generating a pop-up window with a display of the second document.
Claim 7
The non-transitory computer-readable medium of claim 6, wherein the pop-up window comprises a control that allows the user to provide explicit feedback for the second document.
Claim 9
The non-transitory computer-readable medium of claim 8, wherein the pop-up window comprises a control that allows the user to provide explicit feedback for the second document.
Claim 8
The non-transitory computer-readable medium of claim 1, wherein the parameters of the configuration comprise a parameter that indicates a number of shingles used when generating the plurality of queries.
Claim 10
The non-transitory computer-readable medium of claim 1, wherein the parameters of the configuration comprise a parameter that indicates a number of shingles used when generating the plurality of queries.
Claim 9
The non-transitory computer-readable medium of claim 1, wherein the parameters of the configuration comprise a parameter that indicates a number of queries generated when generating the plurality of queries.
Claim 11
The non-transitory computer-readable medium of claim 1, wherein the parameters of the configuration comprise a parameter that indicates a number of queries generated when generating the plurality of queries.
Claim 10
The non-transitory computer-readable medium of claim 1, wherein the parameters of the configuration comprise a parameter that indicates weights applied to the plurality of queries.
Claim 12
The non-transitory computer-readable medium of claim 1, wherein the parameters of the configuration comprise a parameter that indicates weights applied to the plurality of queries.
Claim 11
The non-transitory computer-readable medium of claim 1, wherein the parameters of the configuration comprise a learning parameter that controls how much the parameters of the configuration are adjusted in response to the feedback.
Claim 13
The non-transitory computer-readable medium of claim 1, wherein the parameters of the configuration comprise a learning parameter that controls how much the parameters of the configuration are adjusted in response to the feedback.
Claim 12
The non-transitory computer-readable medium of claim 1, wherein the operations further comprise:
generating new similarity scores for the plurality of documents after adjusting the parameters of the configuration.
Claim 14
The non-transitory computer-readable medium of claim 1, wherein the operations further comprise:
generating new similarity scores for the plurality of documents after adjusting the parameters of the configuration.
Claim 13
The non-transitory computer-readable medium of claim 12, wherein the operations further comprise:
comparing the new similarity scores to the similarity scores to determine whether adjusting the parameters of the configuration improves the new similarity scores.
Claim 15
The non-transitory computer-readable medium of claim 14, wherein the operations further comprise:
comparing the new similarity scores to the similarity scores to determine whether adjusting the parameters of the configuration improves the new similarity scores.
Claim 14
The non-transitory computer-readable medium of claim 13, wherein determining whether adjusting the parameters of the configuration improves the new similarity scores comprises: evaluating an objective function representing a search error.
Claim 16
The non-transitory computer-readable medium of claim 15, wherein determining whether adjusting the parameters of the configuration improves the new similarity scores comprises: evaluating an objective function representing a search error.
Claim 15
The non-transitory computer-readable medium of claim 13, wherein the operations further comprise:
reverting back to original parameters of the configuration when adjusting the parameters of the configuration does not improve the new similarity scores.
Claim 17
The non-transitory computer-readable medium of claim 15, wherein the operations further comprise:
reverting back to original parameters of the configuration when adjusting the parameters of the configuration does not improve the new similarity scores.
Claim 16
The non-transitory computer-readable medium of claim 13, wherein the operations further comprise:
using adjusted parameters of the configuration when performing subsequent searches of the plurality of documents for documents of a same type as the first document when adjusting the parameters of the configuration improves the new similarity scores.
Claim 18
The non-transitory computer-readable medium of claim 15, wherein the operations further comprise:
using adjusted parameters of the configuration when performing subsequent searches of the plurality of documents for documents of a same type as the first document when adjusting the parameters of the configuration improves the new similarity scores.
It would have been obvious to a person with ordinary skills in the art at the time of the invention was made to modify the elements of claims 1-20 of ‘561 with any combination of the cited references below to arrive at claims 1-18 of the instant application for the purpose of refining search results by automatically interpreting collective behavior of users including features and predictive models from a user behavior component that are present in observed user interactions with the search results. Further, it would have been obvious to a person with ordinary skills in the art at the time of the invention was effectively filed to modify or to omit the additional elements of claims 1-20 of ‘561 to arrive at claims 1-18 of the instant application because the person would have realized that the remaining element would perform the same functions as before. “Omission of element and its function in combination is obvious expedient if the remaining elements perform same functions as before.” See In re Karlson (CCPA) 136 USPQ 184, decide Jan 16, 1963, Appl. No. 6857, U.S. Court of Customs and Patent Appeals.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
The claimed invention in claims 1-18 are directed to a judicial exception (i.e., an abstract idea) without significantly more.
Claims 1-18 pass step 1 of the 35 U.S.C. 101 analysis since each claim is either directed to a non-transitory computer-readable medium, a system comprising one or more processor and memory devices (i.e., hardware components per [0127]-[0130] of instant specification), or a method.
Claims 1, 17, and 18 recite each, in part, elements that are directed to an abstract idea (“Courts have examined claims that required the use of a computer and still found that the underlying, patent-ineligible invention could be performed via pen and paper or in a person’s mind.” Versata Dev. Group v. SAP Am., Inc., 793 F.3d 1306, 1335, 115 USPQ2d 1681, 1702 (Fed. Cir. 2015)). Each claim recites the limitations of receiving feedback for a second document…comprises: implicit feedback…; and explicit feedback…; providing the implicit feedback to a classification model…; and adjusting parameters of a configuration in response to the feedback, wherein the configuration defines how to generate…a plurality of queries… The limitations, as drafted, is a process that, under its broadest reasonable interpretation, covers performance of the limitations in the mind but for the recitation of generic computer components (e.g., receiving feedback from a user by visually observing whether the user likes or dislikes a document result based on how long the user views the result (i.e., implicit feedback) and if the user writes down a result on paper (i.e., explicit feedback); mentally processing the implicit feedback, and mentally adjusting parameters of a configuration in response to the feedback). That is, other than reciting generic components (e.g., processor, memory, and computer-executable instructions), nothing in the claim precludes the limitations from being performed in the human mind per step 2A – prong 1 of the Abstract Idea Analysis. Applicant is noted that without any explicit definition for the claimed classification model, such model is merely a group of processes occurring in the human mind. There are no additional elements for further analysis. Thus, the limitations are parts of a mental process and are ineligible
Claim 2 merely provides a definition for the implicit feedback as dwell time visually observed from the user. The claim has no additional elements for further analysis. Thus, the claim is ineligible.
Claim 3 further recites …the classification model determines whether a dwell time indicates that the second document is relevant… which can be implemented in a human mind (e.g., mentally determining that if the document result is relevant based on a duration of user viewing). The claim has no additional elements for further analysis. Thus, the claim is ineligible.
Claim 4 further recites an additional step of providing a plurality of documents… which is an extra-solution activity (per step 2A – prong 2 of the Abstract Idea Analysis) that cannot be integrated into a practical application (e.g., the elements recite trivial elements that occurred or would occur after the mental process). Each of the additional limitation(s) is no more than mere instructions to apply the exception using a generic computer component (e.g., processor, memory, and computer-executable instructions). The extra-solution activity in step 2A - prong 2 are reevaluated in step 2B to determining if each limitation is more than what is well-understood, routine, conventional activity in the field. The background of the limitations does not provide any indication that the computer components (e.g., processor, memory, and computer-executable instructions) are not off-the-shelf computer components. The Symantec, TLI, and OOP Techs court decisions cited in MPEP 2106.05(d)(II) indicate that mere receiving, generating, storing, determining, identifying, and transmitting of data over a network are a well-understood, routine, and conventional functions when claimed in a merely generic manner (as it is here). Accordingly, a conclusion that the claims are well-understood, routine, conventional (WURC) activity is supported under Berkheimer Option 2. For these reasons, there is no inventive concept in each claims, thus, the claim is ineligible.
Claim 5 recites an additional element of training the classification model using the implicit feedback…and the explicit feedback… which is implementable using a human mind and/or with aid of pen/paper (e.g., writing down the training data and labels based on the feedback data and mentally refine the model). Thus, the claim is ineligible.
Claim 6 recites an additional element of generating a pop-up… which is an extra-solution and WURC activity similar to the above analysis. Thus, the claim is ineligible.
Claim 7 merely provides definition for the pop-up window with GUI element(s) for user interaction, which is an extra-solution and WURC activity similar to the above analysis. Thus, the claim is ineligible.
Claim 8 merely provides a definition for the parameters of the configuration indicating a number of shingles used. The claim has no additional elements for further analysis. Thus, the claim is ineligible.
Claim 9 merely provides a definition for the parameters of the configuration indicating a number of queries. The claim has no additional elements for further analysis. Thus, the claim is ineligible.
Claim 10 merely provides a definition for the parameters of the configuration indicating weights applied. The claim has no additional elements for further analysis. Thus, the claim is ineligible.
Claim 11 merely provides a definition for the parameters of the configuration comprise a learning parameter. The claim has no additional elements for further analysis. Thus, the claim is ineligible
Claim 12 recites an additional element of …generating new similarity scores… which is implementable in a human mind and/or with the aid of pen/paper (e.g., mentally calculating similarity scores and writing down the results). Thus, the claim is ineligible.
Claim 13 recites an additional element of …comparing the new similarity scores… which is implementable in a human mind and/or with the aid of pen/paper (e.g., mentally comparing the scores). Thus, the claim is ineligible.
Claim 14 recites an additional element of …evaluating an objective function… which is implementable in a human mind and/or with the aid of pen/paper (e.g., mentally evaluating the function). Thus, the claim is ineligible.
Claim 15 recites an additional element of …reverting back to original parameters… which is implementable in a human mind and/or with the aid of pen/paper (e.g., mentally changing the parameters). Thus, the claim is ineligible.
Claim 16 recites additional elements of …using adjusted parameters…when performing subsequent searches… which is an extra-solution and WURC activity similar to the above analysis. Thus, the claim is ineligible.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-4, 10-12, and 17-18 are rejected under 35 U.S.C. 103 as being unpatentable over Agichtein et al. (Pub. No. US 2007/0208730, published on September 6, 2007; hereinafter Agichtein) in view of Ogilvy et al. (Pub. No. US 2012/0278341, published on November 1, 2012; hereinafter Ogilvy).
Regarding claims 1, 17, and 18, Agichtein clearly shows and discloses a non-transitory computer-readable medium comprising instructions that, when executed by one or more processors, cause the one or more processors to perform operations of a method; a system comprising: one or more memory devices comprising instructions that, when executed by the one or more processors, cause the one or more processors to perform the method of using feedback to improve similarity scores for documents (Abstract and Figure 9), the method comprising:
receiving feedback from a user for a second document in the plurality of documents, wherein the feedback comprises implicit feedback recorded as a user views the second document (the browsing behavior can be modeled, e.g., after a result is clicked, then the average page dwell time for a given query-URL pair, as well as its deviation from the expected (average) dwell time, is employed for such model, [0037]-[0039]); and
providing the implicit feedback to a classification model that determines whether the implicit feedback indicates that the second document is relevant to the first document (A general implicit feedback interpretation strategy can be employed to automatically learn a model of user preferences. The system 500 includes a ranking component 510 that can be trained from interactions with the user behavior component 515. The system 500 can employ various data mining techniques for improving search engine relevance. Such can include employing relevance classifiers in the ranker component 510, to generate high quality training data for runtime classifiers, which are employed with the search engine 540 to generate the search results 550, [0037]-[0039]); and
adjusting parameters of a configuration in response to the feedback (The weight wI is a heuristically tuned scaling factor that represents the relative "importance" of the implicit feedback. The query results can be ordered in by decreasing values of SM(r) to produce the final ranking. One particular case of such model arises when setting wI to a very large value, effectively forcing clicked results to be ranked higher than unclicked results--an intuitive and effective heuristic that can be employed as a baseline, [0033]).
Ogilvy then discloses:
receiving feedback from a user for a second document in the plurality of documents, wherein the feedback comprises explicit feedback provided from the user indicating whether the user believes the second document is relevant to a first document in the plurality of documents (providing an input means 251 for the user to interact with the results and provide additional input on the relevance or lack thereof of certain documents retrieved by the search. The user input means may be a means for assigning positive and negative relevance weights with respect to each displayed reference document, [0251]-[0252]); and
adjusting parameters of a configuration in response to the feedback (if in the set of search results, 2 documents where positively weighted and 1 document was negatively weighted and all 3 of those documents contained the term "tiger", then a multiplier of (2+1)/(1+1)=3/2=1.5 would be applied to that term. Alternatively if in the same set, 1 document was positively weighted and 1 document was negatively weighted, and both documents contained the term "tiger", then a multiplier of (1+1)/(1+1)=2/2=1 would be applied to that term, [0255]), wherein the configuration defines how to generate, from the first document, a plurality of queries used to generate the similarity scores for the plurality of documents (the re-forming of the input local term index in step 255 may comprise re-assigning the input local text term weights of the input text terms stored in the input local index. The terms for which the weights are re-assigned may be those which also appear in each of the reference documents for which user-determined input is received. In this arrangement, step 257 may comprise, on the basis of the re-assigned input local text term weights, querying the database to identify one or more relevant reference documents of enhanced relevance to the input text portion, [0293]. The user's personal interactions are recorded against a personal user term index, which records text terms which have been up- or down-weighted as a result of the user's positive or negative interactions with reference documents in previous searches. Each term in the user term index may also be associated with a term weight modifier which is used to modify the local input index of an input text portion for which the user seek to find relevant reference documents in a search query, [0303]. The document relevance score for a respective reference document is a sum of sub-calculations of adjusted weights associated with each matching term found in the respective reference document, [0240]-[0248]).
It would have been obvious to an ordinary person skilled in the art at the time of the invention was effectively filed to incorporate the teachings of Ogilvy with the teachings of Agichtein for the purpose of refining search results by automatically interpreting collective behavior of users including features and predictive models from a user behavior component that are present in observed user interactions with the search results.
Regarding claim 2, Agichtein further discloses the implicit feedback comprises a dwell time (the browsing feature 420 can capture and quantify aspects of the user web page interactions. For example, the subject innovation can compute deviation of dwell time from expected page dwell time for a query, which allows for modeling intra-query diversity of page browsing behavior, [0037]).
Regarding claim 3, Agichtein further discloses the classification model determines whether a dwell time indicates that the second document is relevant to the first document (the browsing behavior can be modeled, e.g., after a result is clicked, then the average page dwell time for a given query-URL pair, as well as its deviation from the expected (average) dwell time, is employed for such model, [0039]).
Regarding claim 4, Ogilvy further discloses providing a plurality of documents having similarity scores that are calculated in response to receiving a first document (receiving 202 via a user interface an input comprising an input text portion. The user interface may be provided on a client device as disclosed above. The input may be a text document comprising the text portion. The text portion may be received by an input means, [0208]). The user interface is adapted to display information to the user as a result of the analysis i.e. a representation of documents which are deemed by the analysis to be of relevance to the text portion, [0209]), wherein the similarity scores indicate similarities between the first document and the plurality of documents (Once a document relevance score has been determined for each document (or a subset thereof) in the database, then the documents are ranked in decreasing order of relevance and the ranked list of reference documents is output to the user and displayed on the user interface, [0248]).
Regarding claim 10, Ogilvy further discloses the parameters of the configuration comprise a parameter that indicates weights applied to the plurality of queries (if in the set of search results, 2 documents where positively weighted and 1 document was negatively weighted and all 3 of those documents contained the term "tiger", then a multiplier of (2+1)/(1+1)=3/2=1.5 would be applied to that term. Alternatively if in the same set, 1 document was positively weighted and 1 document was negatively weighted, and both documents contained the term "tiger", then a multiplier of (1+1)/(1+1)=2/2=1 would be applied to that term, [0255] The user's personal interactions are recorded against a personal user term index, which records text terms which have been up- or down-weighted as a result of the user's positive or negative interactions with reference documents in previous searches. Each term in the user term index may also be associated with a term weight modifier which is used to modify the local input index of an input text portion for which the user seek to find relevant reference documents in a search query, [0303]. The document relevance score for a respective reference document is a sum of sub-calculations of adjusted weights associated with each matching term found in the respective reference document, [0240]-[0248]).
Regarding claim 11, Ogilvy further discloses the parameters of the configuration comprise a learning parameter that controls how much the parameters of the configuration are adjusted in response to the feedback (if in the set of search results, 2 documents where positively weighted and 1 document was negatively weighted and all 3 of those documents contained the term "tiger", then a multiplier of (2+1)/(1+1)=3/2=1.5 would be applied to that term. Alternatively if in the same set, 1 document was positively weighted and 1 document was negatively weighted, and both documents contained the term "tiger", then a multiplier of (1+1)/(1+1)=2/2=1 would be applied to that term, [0255]).
Regarding claim 12, Ogilvy further discloses generating new similarity scores for the plurality of documents after adjusting the parameters of the configuration (re-forming 255 the input local term index on the basis of the user input (and/or data obtained from additional/external information sources), and, on the basis of the re-formed input local term index, querying the database to identify one or more relevant reference documents of enhanced relevance to the input text portion 257. A representation of the further identified reference documents of enhanced relevance my then be output 259 to the user interface for further viewing and inspection by the user, [0292]-[0294]).
Claim 5 is rejected under 35 U.S.C. 103 as being unpatentable over Agichtein in view of Ogilvy and further in view of Chakravarti et al. (Pub. No. US 2019/0057095, published on February 21, 2019; hereinafter Chakravarti).
Regarding claim 5, Chakravarti then discloses training the classification model using the implicit feedback as training data and the explicit feedback as a label for the training data (Unlabeled (or implicit) data collected in this step may be used to further improve the semi-supervised approaches of the previous step. Furthermore, continuously optimized ranking program 110A, 110B may prompt the system admin, manager, or domain expert with explicit labelling tasks based on the queries logged during production usage of the system, such that among the feedback collected in this step is some (smaller) set of explicitly labelled examples. The explicit data collected in this step may be used in the following step to enable supervised approaches, or to improve the semi-supervised approaches of the previous step, [0040]-[0041]).
It would have been obvious to an ordinary person skilled in the art at the time of the invention was effectively filed to incorporate the teachings of Chakravarti with the teachings of Agichtein, as modified by Ogilvy, for the purpose of enhancing query refinement based on user feedback data associated with results matching the query and a learning model using implicit and explicit feedbacks.
Claims 6-7, and 9 are rejected under 35 U.S.C. 103 as being unpatentable over Agichtein in view of Ogilvy and further in view of Marshall et al. (Pat. No. US 8005823, published on August 23, 2011; hereinafter Marshall).
Regarding claim 6, Marshall then discloses generating a pop-up window with a display of the second document (FIGS. 5A, 5B, 5C and 5D illustrate example user interface elements for allowing users to indicate user feedback regarding query results, according to certain embodiments. For example, feedback mechanism 160 may be configured to display a popup menu over a result in a results list, such as in response to a user right clicking a mouse on a result, [Column 12, Lines 9-38]).
It would have been obvious to an ordinary person skilled in the art at the time of the invention was effectively filed to incorporate the teachings of Marshall with the teachings of Agichtein, as modified by Ogilvy, for the purpose of improving search query results based on modifying and combining previous query results based, in part, on user feedback regarding previous results for similar queries.
Regarding claim 7, Marshall further discloses the pop-up window comprises a control that allows the user to provide explicit feedback for the second document (Thus, user interface 500 may be displayed allowing a user to select among a positive, negative or neutral indication of a level of correctness for a query result, [Column 12, Lines 9-38]).
Regarding claim 9, Marshall further discloses the parameters of the configuration comprise a parameter that indicates a number of queries generated when generating the plurality of queries (community search system 100 may allow a community member to provide user feedback regarding the results of a query and FIG. 6B illustrates user feedback 660 showing that a user indicated that results 610 and 625 were positive results and that result 620 was a negative result, according to one embodiment. FIG. 6C illustrates the results after being modified, such as by community search system 100 and/or feedback mechanism 160, according to various embodiments. Thus, modified results 670 illustrates that community search system 100 moved the two results that received positive user feedback (i.e., results 610 and 625) to the top of the list of results and deleted the result that received negative user feedback (i.e., result 620) from the list, [Column 15, Lines 2-14]).
Claim 8 is rejected under 35 U.S.C. 103 as being unpatentable over Agichtein in view of Ogilvy and further in view of Kwok et al. (Pub. No. US 2015/0169584, published on June 18, 2015; hereinafter Kwok).
Regarding claim 8, Kwok then discloses the parameters of the configuration comprise a parameter that indicates a number of shingles used when generating the plurality of queries (determining (1004) that a first document and a second document satisfy a similarity criterion, according to some embodiments. The re-ranking module 208 identifies (1202) a first plurality of shingles for the first document, identifies (1204) a second plurality of shingles for the second document, and determines (1206) that a predetermined quantity of shingles in the first plurality of shingles and in the second plurality of shingles of content are identical. In some implementations, the predetermined quantity is a predetermined percentage of shingles (e.g., 90% of the shingles). In some implementations, the predetermined quantity is a predetermined number of shingles (e.g., 20 shingles), [0244]-[0247]).
It would have been obvious to an ordinary person skilled in the art at the time of the invention was effectively filed to incorporate the teachings of Kwok with the teachings of Agichtein, as modified by Ogilvy, for the purpose of re-ranking search results based on determining whether a plurality of documents matching a query satisfy a similarity condition to further improve query accuracy associated with the documents.
Claims 13-16 are rejected under 35 U.S.C. 103 as being unpatentable over Agichtein in view of Ogilvy and further in view of Corvinelli et al. (Pub. No. US 2022/0004553, filed on July 1, 2020; hereinafter Corvinelli).
Regarding claim 13, Corvinelli then discloses comparing the new similarity scores to the similarity scores to determine whether adjusting the parameters of the configuration improves the new similarity scores (component 112, via machine learning engine 144, may accumulate and evaluate input data from both the discovery and feedback processes (similar to FIG. 2, evaluate step (evaluate 144a)), from which the set of models to build or modify are identified. The evaluate step (i.e., evaluate 144a) includes identifying that the appropriate model already exists but a retraining operation might be required if the model is no longer achieving the required level of performance, automatically reverting back to the previous model, [0047]).
It would have been obvious to an ordinary person skilled in the art at the time of the invention was effectively filed to incorporate the teachings of Corvinelli with the teachings of Agichtein, as modified by Ogilvy, for the purpose of refining a query optimization model based on query feedback and performance feedback associated with execution of a respective query.
Regarding claim 14, Corvinelli further discloses determining whether adjusting the parameters of the configuration improves the new similarity scores comprises: evaluating an objective function representing a search error (component 112, via optimizer component 148, may collect data from the execution of the query to identify errors in estimates of the query optimizer (i.e., optimizer component 148); and submitting the query feedback and the runtime feedback to machine learning engine 144 to update the set of potentially useful models, [0049]).
Regarding claim 15, Corvinelli further discloses reverting back to original parameters of the configuration when adjusting the parameters of the configuration does not improve the new similarity scores (The evaluate step (i.e., evaluate 144a) includes identifying that the appropriate model already exists but a retraining operation might be required if the model is no longer achieving the required level of performance, automatically reverting back to the previous model, [0047]).
Regarding claim 16, Ogilvy further discloses using adjusted parameters of the configuration when performing subsequent searches of the plurality of documents for documents of a same type as the first document when adjusting the parameters of the configuration improves the new similarity scores (If the user inputs additional relevance information regarding one or more of the documents, this additional information is sent to the processor 161 and, on the basis of the additional information, the processor 161 reforms 355 the index and performs a further query 356 of the database 111 to determine further document matches 357 of enhanced relevance to the text portion and the user's specific intention (i.e. of greater contextual relevance to the users specific requirements) and the documents of improved relevance are output 358 to the user interface for further review by the user, [0309]).
Relevant Prior Art
The following references are considered relevant to the claims:
Castelli et al. (Pat. No. US 7272593) teaches an iterative refinement algorithm for content-based retrieval of images based on low-level features such as textures, color histograms, and shapes that can be described by feature vectors. This technique adjusts the original feature space to the new application by performing nonlinear multidimensional scaling. Consequently, the transformed distance of those feature vectors which are considered to be similar is minimized in the new feature space. Meanwhile, the distance among clusters are maintained. User feedback is utilized to refine the query, by dynamically adjusting the similarity measure and modifying the linear transform of features, along with revising the feature vectors.
Legrand et al. (Pub. No. US 2017/0091319) teaches identifying a desired document is provided to include calculating a Prior probability score for each document of a candidate list including a portion of documents of an embedding space, the Prior probability score indicating a preliminary probability, for each document of the candidate list, that the document is the desired document, and identifying and refining candidate documents from the candidate list in dependence on the calculated Prior probability scores.
Contact Information
Any inquiry concerning this communication or earlier communications from the Examiner should be directed to Son Hoang whose telephone number is (571) 270-1752. The Examiner can normally be reached on Monday – Friday (7:00 AM – 4:00 PM).
If attempts to reach the Examiner by telephone are unsuccessful, the Examiner’s supervisor, Sherief Badawi can be reached on (571) 272-9782. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SON T HOANG/Primary Examiner, Art Unit 2169
November 20, 2025