Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Remarks
This Office Action is responsive to Applicants' Amendment filed on October 10, 2025, in which claims 1, 16, 17, 22, 24, are currently amended. Claims 1-30 are currently pending.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on April 11, 2025, April 22, 2025, April 25, 2025, and November 24, 2025 are in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statements are being considered by the examiner.
Response to Arguments
The previous rejections to claims 1-22 under 35 U.S.C. § 112(f)/(b) are hereby withdrawn, as necessitated by applicant's amendments and remarks made to the rejections.
Applicant’s arguments with respect to rejection of claims 1-30 under 35 U.S.C. 101 based on amendment have been considered.
With respect to Applicant's arguments on pp. 9-12 of the Remarks submitted 10/10/2025 that the claims do not recite an abstract idea, Examiner respectfully disagrees. Claim 22 for example explicitly recites "estimating, […], a contribution of a first item of content to the generated content, generated by the generative model using the prompt received over the network, based on a plurality of factual or opinion claims comprising respective statements about entities, events, or concepts, wherein the plurality of claims comprise a first claim extracted from the generated content and a second claim extracted from the first item of content" and "generating feedback based at least in part on the estimated contribution of the first item of content to the generated content" both of which are mental processes which can readily be performed entirely in the mind with or without the assistance of tools such as pen and paper (or a generic computer).
With respect to Applicant's arguments on pp. 13-16 that the claims are directed towards a practical application, while the arguments are persuasive in view of claims 1-21, Examiner notes that claims 22-30 have significantly different scope and appear to be directed entirely towards using generic and well-known computer components in a routine way to perform a mental process. Examiner notes that the mental process recited in claim 22 for example does not appear to integrate the judicial exception into a practical application as aside from the judicial exception the claims recite only mere instructions to apply the judicial exception using generic computer components and insignificant extra-solution activity which is well-understood, routine, and conventional in the art (see MPEP 2106.05(d)(II)). Applicant has not provided objective evidence for how claim 22 provides a technical improvement, Examiner notes MPEP 2106.05(a) "It is important to note, the judicial exception alone cannot provide the improvement. The improvement can be provided by one or more additional elements.", "An important consideration in determining whether a claim improves technology is the extent to which the claim covers a particular solution to a problem or a particular way to achieve a desired outcome, as opposed to merely claiming the idea of a solution or outcome.", and MPEP 2106.07(a)(II) "employing well-known computer functions to execute an abstract idea, even when limiting the use of the idea to one particular environment, does not integrate the exception into a practical application". For at least these reasons Examiner asserts that it is appropriate to maintain the rejection for claims 22-30. The rejection under 32 USC §101 for claims 1-21 is hereby withdrawn.
Applicant’s arguments with respect to rejection of claims 1-30 under 35 U.S.C. 102/103 based on amendment have been considered.
With respect to Applicant's arguments on pp. 17-18 of the Remarks submitted 10/10/2025 that Rollwage does not disclose "detect a prompt from a user device provided to a generative model, the generative model trained using content items from a plurality of corpuses of a plurality of different sources, the generative model configured to break down input text into numerical units, convert the numerical units into embeddings comprising vectors, perform positional encoding that encodes a given word in the input text, feed the embedding through a transformer that computes an attention score to determine a given numerical unit's importance", Examiner respectfully disagrees. Rollwage explicitly discloses taking user input as a prompt for a large language model ([Abstract] "an input configured to receive input data relating to speech or text provided by a user"), that the training data may be from multiple corpora ([¶0387] "the language model may be trained using a dataset comprising a large number of web pages, such as the “WebText” dataset for example. The dataset may further comprise data from other text sources as well"), that the generative model breaks down input text input numerical units ([¶0371] "The system prompt is taken as input to a tokeniser 22. The tokeniser 22 takes the input text and outputs a sequence of tokens representing the text, from a vocabulary of possible tokens. Special tokens, such as tokens representing a start or end, may also be included in the vocabulary. Each token may be represented by a different positive integer number for example. The tokeniser 22 outputs a sequence of numbers corresponding to the input system prompt. The number of tokens in the sequence will vary between different input prompts."), converts them into embeddings comprising vectors ([¶0372] "The sequence of tokens is taken as input to a vector representation module 23. The vector representation module comprises stored token representations. Each token representation is a stored vector, where each vector corresponds to a token from the vocabulary. For each token in the input sequence of tokens, the corresponding token representation is retrieved."), performs positional encoding ([¶0373] "The vector representation module may further comprise stored positional representations. For example, each positional representation may be a stored vector corresponding to an absolute position in the sequence of tokens."), and generates attention scores ([¶0377] "Scores are then calculated for each attention head. A matrix product is calculated between the query matrix and the transposed key matrix for each attention head. The scores represent, for the token being processed (the score matrix row), the attention of the model on each other token in the sequence (the score matrix columns). A higher score corresponds to more focus on the token.") in exactly the order of instant claim 1.
Applicant's arguments on p. 18 of the Remarks submitted 10/10/2025 directed towards extracting factual or opinion claims and generating embedding values is persuasive, the argument is moot in view of a new ground of rejection set forth below.
Examiner notes that claim 22 is of significantly different scope than claim 1.
Claim Objections
Claims 1 and 22 are objected to because of the following informalities:
Regarding claim 1, "associated with the corpus the corpus" should read "associated with the corpus, the corpus".
Regarding claim 1, "the response of generative model" should read "the response of the generative model".
Regarding claim 22, "a contribution of a first item of content" should read "a contribution of the first item of content".
Appropriate correction is required.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-30 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Regarding claim 1, "the embedding" lacks antecedent basis. Claim 1 introduces "embeddings" such that it would be unclear which of the embeddings (plural) was "the embedding" (singular). "The embeddings" is recommended.
Regarding claim 1, "detect a prompt from a user device provided to a generative model" is grammatically indefinite. The language appears to suggest that the user device is provided to a generative model. However, the instant specification appears to teach away from this interpretation ([¶0058] "some or all of the LLM system 104 functionality may be incorporated into the user device 110." It appears circular to provide a user device to an LLM on said user device). It's unclear if the limitation is meant to read "providing, from a user device, a prompt to a generative model", "detect, for a generative model, a prompt received from a user device", or "detecting a prompt from a generative model on a user device" (as supported by the instant specification) or something else altogether. As these interpretations are contradictory the scope of the claim cannot be reasonably determined. In the interest of further examination the claim is interpreted as "detect, for a generative model, a prompt received from a user device".
Regarding claim 1, "use the respective embedding values for the first set of one or more factual or opinion claims to estimate a contribution of the first item of content, associated with the corpus the corpus from a different source than the prompt, to the response of generative model based on a similarity of the first set of one or more claims to the second set of one or more claims" is grammatically indefinite. First, because of the comma placement after "content", it's unclear what "associated with the corpus" modifies: "a first item of content", "the response of generative model", or something else altogether. Second, it's unclear what "based on a similarity" modifies: "the estimation", "the contribution", "the response of the generative model", or something else altogether. The sentence structure makes it unclear which prepositional phrases modify which nouns such that the scope of the claim cannot reasonably be determined. In the interest of further examination, the claim limitation is interpreted as "use the respective embedding values for the first set of one or more factual or opinion claims to estimate a contribution of the first item of content associated with the corpus the corpus. The response of the generative model further based on a similarity of the first set of one or more claims to the second set of one or more claims".
Regarding claim 22, "the embedding" lacks antecedent basis. "An embedding" is recommended.
The remaining claims are rejected with respect to their dependence on the rejected claims.
Claim Rejections - 35 USC § 101
101 Rejection
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 22-30 are rejected under 35 USC § 101 because the claimed invention is directed to non-statutory subject matter.
Regarding Claim 1: Claim 22 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 22 is directed to a system which is directed to a machine, one of the statutory categories.
Step 2A Prong One Analysis: Claim 22 under its broadest reasonable interpretation is a series of mental processes. For example, but for the generic computer components language, the above limitations in the context of this claim encompass machine learning processing, including the following:
estimating, […], a contribution of a first item of content to the generated content, generated by the generative model using the prompt received over the network, based on a plurality of factual or opinion claims comprising respective statements about entities, events, or concepts, wherein the plurality of claims comprise a first claim extracted from the generated content and a second claim extracted from the first item of content generated by the generative model using the prompt received over the network (observation, evaluation, and judgement),
generating feedback based at least in part on the estimated contribution of the first item of content to the generated content (observation, evaluation, and judgement)
Therefore, Claim 22 recites an abstract idea which is a judicial exception.
Step 2A Prong Two Analysis: Claim 22 recites additional elements “a processor comprising a register, an arithmetic logic unit, and a bus”. However, these additional features are computer components recited at a high-level of generality, such that they amount to no more than mere instructions to apply the judicial exception using a generic computer component. An additional element that merely recites the words “apply it” (or an equivalent) with the judicial exception, or merely includes instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea, does not integrate the judicial exception into a practical application. Claim 22 also recites additional elements “receiving a prompt over a network at a computer system including”, “storing the item of generated content in non-transitory memory”, “accessing the item of generated content from non-transitory memory”, and “transmitting the feedback generated based at least in part on the estimated contribution of the first item of content to the generated content to one or more networked destinations, the feedback comprising a token, wherein the greater the estimated contribution of the first item of content to the response of the generative model, the greater a value of the token” which amounts to gathering and outputting data which is insignificant extra-solution activity (See MPEP 2106.05(g)). Therefore, Claim 22 is directed to a judicial exception.
Step 2B Analysis: Claim 22 does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the lack of integration of the abstract idea into a practical application, the additional elements recited in Claim 22 amount to no more than mere instructions to apply the judicial exception using a generic computer component and insignificant extra-solution activity. The gathering and outputting of data is considered well-understood, routine, and conventional in the art (See MPEP 2106.05(d)(II)). Claim 22 also recites additional elements “generating, using the prompt, an item of content using a generative model that encodes a given word in the input text, feed the embedding through a transformer that computes an attention score to determine a given numerical unit's importance” which amounts to routine operation of a well-known transformer neural network model (See Yu “WHY “CLASSIC” TRANSFORMERS ARE SHALLOW AND HOW TO MAKE THEM GO DEEP”, 2024, [p. 2] “well-known Transformer models: BERT [19] and ALBERT [21], both following the original (or classic) encoder architecture as proposed”)).
For the reasons above, Claim 22 is rejected as being directed to non-patentable subject matter under §101. This rejection applies equally to dependent claims 23-30. The additional limitations of the dependent claims are addressed briefly below:
Dependent claim 23 recites additional instructions to apply the judicial exception using generic computer components “wherein the item of content is one of a plurality of content items” as well as additional observation, evaluation, and judgement “estimate a contribution of each of the plurality of content items to the response of the generative model; estimate contribution percentages of each of the plurality of content items to the response of the generative model” and additional insignificant extra-solution activity of outputting data “transmit corresponding pro rata feedback to respective sources of the plurality of content items” which is well-understood, routine, and conventional in the art (See MPEP 2106.05(g) and MPEP 2106.05(d)(II))
Dependent claim 24 recites additional insignificant extra-solution activity (see MPEP 2106.05(g)) “wherein the first item of content comprises still image data, video image data, and/or audio data” which amounts to selection of a data type which is well-understood, routine, and conventional in the art (See MPEP 2106.05(d)).
Dependent claim 25 recites additional observation, evaluation, and judgement “to estimate the contribution of the first item of content, used to train the generative model, to the generative model output based at least in part on labels associated with the first item of content, changes in weights of the generative model caused at least partly by training of the generative model using the first item of content, and/or based on an analysis of the output of the generative model”
Dependent claim 26 recites additional observation, evaluation, and judgement “to: generate a prompt instructing the generative model to use only specified document chunks in providing a response to the generated prompt” and “determine similarities of the response to the generated prompt to the specified document chunks” as well as additional insignificant extra-solution activity of gathering data (See MPEP 2106.05(g)) “receive a response to the generated prompt from the generative model” which is well-understood, routine, and conventional in the art (See MPEP 2106.05(d)(II))
Dependent claim 27 recites additional observation, evaluation, and judgement “wherein the estimated contribution of the first item of content to the generative model output comprises an estimated contribution of vocabulary choice, sentence structure, grammar and punctuation, tone and voice, themes and topics, and/or rhetorical devices to the generative model output”
Dependent claim 28 recites additional observation, evaluation, and judgement “wherein the estimated contribution of the first item of content to the generative model output comprises an estimated contribution of symbols, shapes, motifs, and/or iconography to the generative model output”
Dependent claim 29 recites additional observation, evaluation, and judgement “wherein the estimated contribution of the first item of content to the generative model output comprises an estimated contribution to one or more claims of the generative model output”
Dependent claim 30 recites additional insignificant extra-solution activity of outputting data (See MPEP 2106.05(g)) “to transmit an aggregated feedback for a first period of time to the one or more networked destinations” which is well-understood, routine, and conventional in the art (See MPEP 2106.05(d)(II))
Therefore, when considering the elements separately and in combination, they do not add significantly more to the inventive concept. Accordingly, claims 22-30 are rejected under 35 U.S.C. § 101.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-3 and 5-21 are rejected under U.S.C. §103 as being unpatentable over the combination of Rollwage (US20240404514A1), Schuurman (“Step-by-Step Design and Simulation of a Simple CPU Architecture”, 2013), and Shaar (“Assisting the Human Fact-Checkers: Detecting All Previously Fact-Checked Claims in a Document”, 2022).
Regarding claim 1, Rollwage teaches A computer system, the computer system comprising: a network interface; at least one processing device, comprising one or more registers, ([Abstract] "A dialogue system, comprising: an input configured to receive input data relating to speech or text provided by a user; an output configured to provide output data relating to speech or text to a user; and one or more processors" [¶0326] "the storage 107 is local memory that is contained in the device")
one or more arithmetic logic units, and one or more buses, the at least one processing device configured to([¶0327] "a general central processing unit (CPU)" one of ordinary skill in the art would recognize that a typical Von Neumann architecture central processing unit has an ALU, registers, and one or more buses)
detect a prompt from a user device provided to a generative model;([Abstract] "an input configured to receive input data relating to speech or text provided by a user" [0021] "the system input is a system prompt")
the generative model trained using content items from a plurality of corpuses of a plurality of different sources, ([¶0387] "the language model may be trained using a dataset comprising a large number of web pages, such as the “WebText” dataset for example. The dataset may further comprise data from other text sources as well")
the generative model configured to break down input text into numerical units, ([¶0371] "The system prompt is taken as input to a tokeniser 22. The tokeniser 22 takes the input text and outputs a sequence of tokens representing the text, from a vocabulary of possible tokens. Special tokens, such as tokens representing a start or end, may also be included in the vocabulary. Each token may be represented by a different positive integer number for example. The tokeniser 22 outputs a sequence of numbers corresponding to the input system prompt. The number of tokens in the sequence will vary between different input prompts.")
convert the numerical units into embeddings comprising vectors, ([¶0372] "The sequence of tokens is taken as input to a vector representation module 23. The vector representation module comprises stored token representations. Each token representation is a stored vector, where each vector corresponds to a token from the vocabulary. For each token in the input sequence of tokens, the corresponding token representation is retrieved.")
perform positional encoding that encodes a given word in the input text, ([¶0373] "The vector representation module may further comprise stored positional representations. For example, each positional representation may be a stored vector corresponding to an absolute position in the sequence of tokens.")
feed the embedding through a transformer that computes an attention score to determine a given numerical unit's importance([¶0377] "Scores are then calculated for each attention head. A matrix product is calculated between the query matrix and the transposed key matrix for each attention head. The scores represent, for the token being processed (the score matrix row), the attention of the model on each other token in the sequence (the score matrix columns). A higher score corresponds to more focus on the token.")
receive a response to the prompt provided by the generative model;([¶0027] "the language model is configured to generate the subsequent words in a sequence of text beginning with the system prompt.")
extract a first set of one or more factual or opinion claims, comprising respective statements about entities, events, or concepts, from the response provided by the generative model to the prompt;([¶0771] "The output safety module 50 therefore categorises the dynamically determined system responses into four categories in this example: generally undesirable behaviour (e.g. use of slang, engaging in debates, etc), harmful or offensive utterances (e.g. racist, misogynistic, homophobic, etc), giving medical advice (or other forms of clear recommendations/opinions on health matters), or none of these (safe)." Safety system extracts information from generative model response to generate categorical opinion claim.)
receive a first item of content associated with a corpus in the plurality of corpuses, the corpus from a different source than the prompt; ([¶0387] "the language model may be trained using a dataset comprising a large number of web pages, such as the “WebText” dataset for example. The dataset may further comprise data from other text sources as well" [¶0027] "the language model is configured to generate the subsequent words in a sequence of text beginning with the system prompt." [§0621] "The second module 20 comprises a second language model that receives a second type of system prompt from the second prompt generator 52 b as input" Second training prompt or second prompt from second prompt generator using other text sources interpreted as first item of content associated with a corpus in the plurality of corpuses, the corpus from a different source than the prompt (first prompt))
extract a second set of one or more factual or opinion claims from the first item of content([¶0771] "The output safety module 50 therefore categorises the dynamically determined system responses into four categories in this example: generally undesirable behaviour (e.g. use of slang, engaging in debates, etc), harmful or offensive utterances (e.g. racist, misogynistic, homophobic, etc), giving medical advice (or other forms of clear recommendations/opinions on health matters), or none of these (safe)." Safety system extracts information from generative model response to generate categorical opinion claim.)
generate feedback based at least in part on the estimated contribution of the first item of content to the response of the generative model; and transmit, using the network interface, the feedback generated based at least in part on the estimated contribution of the first item of content to the response of the generative model to one or more networked destinations, the feedback comprising a token, ([¶0393] "A similarity measure is generated, for each entry in the knowledge bank 33. The similarity measure may be generated by embedding the input text using a language model, embedding all reference entries in the knowledge bank 33 using the same language model, computing the cosine similarity (or some other similarity measure) between the input embedding and all reference embeddings" [¶0593] "the language model 62 may be implemented on a separate system, with the system prompts being sent to and the language model outputs being received from the language model 62 system via a communication network." System output interpreted as feedback)
wherein the greater the estimated contribution of the first item of content to the response of the generative model, the greater a value of the token.([¶0377] "Scores are then calculated for each attention head. A matrix product is calculated between the query matrix and the transposed key matrix for each attention head. The scores represent, for the token being processed (the score matrix row), the attention of the model on each other token in the sequence (the score matrix columns). A higher score corresponds to more focus on the token." [¶0378] "An attention mask is then applied to the output score matrix for each attention head, to mask out the scores corresponding to future tokens. A softmax function is applied to the result, giving a final score matrix for each attention head. The matrix product of the score matrix with the value matrix is then taken for each attention head. The outputs from the attention heads are then merged. The matrix product with a stored projection matrix is taken, to give the output of the attention layer.").
While it would be obvious to one of ordinary skill in the art before the effective filing date of the claimed invention that the vast majority of modern CPU’s use the Von Neumann architecture which necessarily includes an ALU, registers, and buses, Rollwage does not explicitly teach one or more arithmetic logic units, and one or more buses, the at least one processing device configured to
generate respective embedding values for the first set of one or more factual or opinion claims;
use the respective embedding values for the first set of one or more factual or opinion claims to estimate a contribution of the first item of content, associated with the corpus the corpus from a different source than the prompt, to the response of generative model based on a similarity of the first set of one or more claims to the second set of one or more claims.
Schuurman, in the same field of endeavor, teaches one or more arithmetic logic units, and one or more buses, the at least one processing device configured to([Abstract] "This paper describes a sequence of assignments, each building upon the next, leading students to a working simulation of a simple 8-bit CPU (Central Processing Unit). The design features a classic Von Neumann architecture comprising a simple data path with a few registers, a simple ALU (Arithmetic Logic Unit), and a microprogram to direct all the control signals.").
Rollwage as well as Schuurman are directed towards computer processing. Therefore, Rollwage as well as Schuurman are reasonably pertinent analogous art. It would have been obvious before the effective filing date of the claimed invention to combine the teachings of Rollwage with the teachings of Schuurman by using any Von Neumann CPU as the CPU explicitly disclosed in Rollwage. Schuurman provides as additional motivation for combination (The design features a classic Von Neumann architecture comprising a simple data path with a few registers, a simple ALU (Arithmetic Logic Unit), and a microprogram to direct all the control signals).
However, the combination of Rollwage and Schuurman does not explicitly teach generate respective embedding values for the first set of one or more factual or opinion claims;
use the respective embedding values for the first set of one or more factual or opinion claims to estimate a contribution of the first item of content, associated with the corpus the corpus from a different source than the prompt, to the response of generative model based on a similarity of the first set of one or more claims to the second set of one or more claims.
Shaar, in the same field of endeavor, teaches generate respective embedding values for the first set of one or more factual or opinion claims;([p. 3 §3] "Given an input document and a database of previously fact-checked claims, produce a ranked list of its sentences, so that those that contain claims that can be verified by a claim from the database are ranked as high as possible. We further want the system to be able to point to the database claims that verify a claim in an input sentence." [p. 7 §6] "Cosine similarity for sentence-BERT-large embedding of the Input sentence compared to the embedding for the Verified Statement, the Title, and the Body")
use the respective embedding values for the first set of one or more factual or opinion claims to estimate a contribution of the first item of content, associated with the corpus the corpus from a different source than the prompt, to the response of generative model based on a similarity of the first set of one or more claims to the second set of one or more claims([p. 7] "we can match the Input sentence against the Verified Statement, the Title, and the Body of the verified claims’ fact-checking article in PolitiFact" [p. 9 §8] "We developed a new dataset for this task formulation, consisting of seven debates, 5,054 sentences, 16,636 target verified claims to match against" [p. 7 §6] "Cosine similarity for sentence-BERT-large embedding of the Input sentence compared to the embedding for the Verified Statement, the Title, and the Body" [p. 7 §6] "To do that, we propose to compute multiple similarity measures between all possible Input–Verified pairs, where we can match the Input sentence against the Verified Statement, the Title, and the Body of the verified claims’ fact-checking article in PolitiFact." Politifact and debate interpreted as different source corpora).
The combination of Rollwage and Schuurman as well as Shaar are directed towards LLM output verification. Therefore, the combination of Rollwage and Schuurman as well as Shaar are analogous art in the same field of endeavor. It would have been obvious before the effective filing date of the claimed invention to combine the teachings of the combination of Rollwage and Schuurman with the teachings of Shaar by incorporating the LLM in Shaar as one of the “other models” for output validation in Rollwage. Shaar provides as additional real world motivation for combination ([p. 3 §3] "Given an input document and a database of previously fact-checked claims, produce a ranked list of its sentences, so that those that contain claims that can be verified by a claim from the database are ranked as high as possible. We further want the system to be able to point to the database claims that verify a claim in an input sentence."). This motivation for combination also applies to the remaining claims which depend on this combination.
Regarding claim 2, the combination of Rollwage, Schuurman, and Shaar teaches The computer system as defined in Claim 1, wherein the item of content is one of a plurality of content items, and wherein the system is configured to: estimate a contribution of each of the plurality of content items to the response of the generative model; estimate contribution percentages of each of the plurality of content items to the response of the generative model; and transmit corresponding pro rata feedback to respective sources of the plurality of content items(Rollwage [¶0806] " If the retrieved snippets include a pre-determined percentage of snippets that are related to the specified clinical guidance, in other words corresponding to the selected intervention, then the checking module 76 determines that the response is in line with clinical guidance. That is, if the number of snippets corresponding to the correct clinical guidance database are greater than a pre-determined percentage, then the checking module 76 may determine that the response is in line with clinical guidance." [¶0807] "If all other checks are also satisfied, the dynamically determined system response is output to the user, by way of output 101. If it is determined that the response is not in line with the correct clinical guidance, then the flow module 58 passes control to the modifier module 65 to modify or change the dynamically determined system response.").
Regarding claim 3, the combination of Rollwage, Schuurman, and Shaar teaches The computer system as defined in Claim 1, wherein the generative model comprises a large language model(Rollwage [R0385] "The combination of one or more clinical, mechanistic models (such as the cognitive distortion understanding model 300) with a large language models provides improved computational efficiency compared to using a single large language model to accomplish the same task" [¶0395] "The safety module 50 contains one or more machine learning modules that evaluate the quality and safety of the utterances of the LLM.").
Regarding claim 5, the combination of Rollwage, Schuurman, and Shaar teaches The computer system as defined in Claim 1, wherein the generative model output comprises text, image data, or audio data(Rollwage [¶0321] "The user provides spoken or text inputs through the web browser or application. The audio or text signal is then processed and the data sent from the user device 200 to the dialogue system 100 over the communication network").
Regarding claim 6, the combination of Rollwage, Schuurman, and Shaar teaches The computer system as defined in Claim 1, wherein the first item of content comprises still image data, video image data, and/or audio data(Rollwage [¶0321] "The user provides spoken or text inputs through the web browser or application. The audio or text signal is then processed and the data sent from the user device 200 to the dialogue system 100 over the communication network").
Regarding claim 7, the combination of Rollwage, Schuurman, and Shaar teaches The computer system as defined in Claim 1, wherein the first item of content comprises text data(Rollwage [¶0321] "The user provides spoken or text inputs through the web browser or application. The audio or text signal is then processed and the data sent from the user device 200 to the dialogue system 100 over the communication network").
Regarding claim 8, the combination of Rollwage, Schuurman, and Shaar teaches The computer system as defined in Claim 1, wherein the estimated contribution of the first item of content to the generative model output comprises an estimated contribution of style(Rollwage [¶0008] "large language models may have many advantages over older, more classical text generation systems, like rule-based chatbots. For example, their large amounts of training data make them understand quite varied forms of language and tone and they are robust in reacting to mistakes in the input text. Additionally, this allows them to respond in varied, and potentially highly nuanced ways, adapting their tone, and response, as well as not having to rely on formulaic responses" Tone interpreted as synonymous with style.).
Regarding claim 9, the combination of Rollwage, Schuurman, and Shaar teaches The computer system as defined in Claim 1, wherein the estimated contribution of the first item of content to the generative model output comprises an estimated contribution of vocabulary choice, sentence structure, grammar and punctuation, tone and voice, themes and topics, and/or rhetorical devices to the generative model output(Rollwage [¶0008] "large language models may have many advantages over older, more classical text generation systems, like rule-based chatbots. For example, their large amounts of training data make them understand quite varied forms of language and tone and they are robust in reacting to mistakes in the input text. Additionally, this allows them to respond in varied, and potentially highly nuanced ways, adapting their tone, and response, as well as not having to rely on formulaic responses").
Regarding claim 10, the combination of Rollwage, Schuurman, and Shaar teaches The computer system as defined in Claim 1, wherein the estimated contribution of the first item of content to the generative model output comprises an estimated contribution of symbols, shapes, motifs, and/or iconography to the generative model output(Rollwage [¶0612] "In FIG. 14 a and FIG. 14 b , the app displays a widget “I'd like to talk it through”. When the user interacts (e.g., by pressing a button, by clicking on a pop-up, by touching an icon, by pressing a switch, etc.) with this widget on the app, the dialogue application 102 may automatically dial the specific number, thereby providing immediate help to the user. The app also displays a widget “Back to my session”. When the user interacts with this widget on the app, the flow module 58 may pass the control to prompt generation module 40 to initiate a session and/or dialogue with the user." touching icon to control/prompt generation module interpreted as synonymous with an estimated contribution of iconography to the estimated contribution of the first item (input).).
Regarding claim 11, the combination of Rollwage, Schuurman, and Shaar teaches The computer system as defined in Claim 1, wherein the estimated contribution of the first item of content to the generative model output comprises an estimated contribution to one or more claims of the generative model output(Rollwage [¶0388] "During the training process, sequences of text from the training dataset are processed by the language model 21 in the same manner as described above. The language model 21 comprises a number of trainable parameters, which can be expressed as a vector θ. The parameters include the token representation vector values, the position representation vector values, the attention layer weights and the neural network layer weights for example. The parameters are randomly initialised. The update process searches for a parameter vector θ so that the difference between the next token in the sequence extracted from the dataset and the prediction of the next token made by the language model 21 is minimised.").
Regarding claim 12, the combination of Rollwage, Schuurman, and Shaar teaches The computer system as defined in Claim 1, wherein transmitting the feedback generated based at least in part on the estimated contribution of the first item of content to the generative model output to one or more networked destinations, further comprises transmitting feedback to a plurality of networked destinations based at least in part on estimated percentage contributions of a plurality of items of content to the generative model output(Rollwage [¶0806] " If the retrieved snippets include a pre-determined percentage of snippets that are related to the specified clinical guidance, in other words corresponding to the selected intervention, then the checking module 76 determines that the response is in line with clinical guidance. That is, if the number of snippets corresponding to the correct clinical guidance database are greater than a pre-determined percentage, then the checking module 76 may determine that the response is in line with clinical guidance." [¶0807] "If all other checks are also satisfied, the dynamically determined system response is output to the user, by way of output 101. If it is determined that the response is not in line with the correct clinical guidance, then the flow module 58 passes control to the modifier module 65 to modify or change the dynamically determined system response.").
Regarding claim 13, the combination of Rollwage, Schuurman, and Shaar teaches The computer system as defined in Claim 1, wherein the system is configured to transmit an aggregated feedback for a first period of time to the one or more networked destinations(Rollwage [¶0398] "The interface 70 may interact with the user at predetermined times, or at predetermined time intervals").
Regarding claim 14, the combination of Rollwage, Schuurman, and Shaar teaches The computer system as defined in Claim 1, wherein the system is operable to estimate the contribution of the first item of content, used to train the generative model, (Rollwage [¶0388] "During the training process, sequences of text from the training dataset are processed by the language model 21 in the same manner as described above. The language model 21 comprises a number of trainable parameters, which can be expressed as a vector θ. The parameters include the token representation vector values, the position representation vector values, the attention layer weights and the neural network layer weights for example. The parameters are randomly initialised. The update process searches for a parameter vector θ so that the difference between the next token in the sequence extracted from the dataset and the prediction of the next token made by the language model 21 is minimised.")
to the generative model output based at least in part on labels associated with the first item of content, (Rollwage [¶0028] "One or more of the one or more subject understanding models may be trained using data that is labelled with information relating to the corresponding aspect." [¶0347] "The training dataset may comprise historical patient utterances (sequences of text) and may be used to learn the weights of the deep learning algorithm prior to implementation of the dialogue system 100. The training data may comprise clinician-labelled datasets, which may comprise a number of example patient utterances. To prepare the training dataset, a number of clinicians review the patient utterances and label them as corresponding to a “distorted thought” (these utterances are given the label 1) or not a distorted thought (these utterances are given the label 0).")
changes in weights of the generative model caused at least partly by training of the generative model using the first item of content, and/or based on an analysis of the output of the generative model(Rollwage [¶0347] "The training dataset may comprise historical patient utterances (sequences of text) and may be used to learn the weights of the deep learning algorithm prior to implementation of the dialogue system 100").
Regarding claim 15, the combination of Rollwage, Schuurman, and Shaar teaches The computer system as defined in Claim 1, wherein the system is operable to estimate the contribution of the first item of content, used to train the generative model, to the generative model output using a neural network, a Support Vector Machine, a Random Forest, a probabilistic algorithm, and/or a K-Nearest Neighbors algorithm(Rollwage [¶0340] " the cognitive distortion understanding model 300 predicts a probability that the following thought is distorted: “Everybody hates me”. This probability is then used to output an indication of whether the user input corresponds to a cognitive distortion.").
Regarding claim 16, the combination of Rollwage, Schuurman, and Shaar teaches The computer system as defined in Claim 1, wherein the feedback comprises label weights associated with labels assigned to the first item of content, an identification of a contribution to an adjustment of a weight of the generative model, and/or a token(Rollwage [¶0348] "The parameters include the neural network layer weights for example. The parameters are randomly initialised. The update process searches for a parameter vector θe so that the difference between label in the training dataset and the prediction made by the model is minimised. A process of updating θe sequentially by computing the gradient of a loss function and updating θe using the computed gradient and an optimiser function is performed. A cross entropy loss may be used, in which [...] yi is the label from the training data set [...] The gradient of the loss L with respect to each of the trainable parameters is determined through back-propagation. The gradient is then used to determine the updated parameters").
Regarding claim 17, the combination of Rollwage, Schuurman, and Shaar teaches The computer system as defined in Claim 1, wherein the system utilizes multiple models to estimate the contribution of the first item of content, to the response of the generative model(Shaar [p. 1] "Figure 1: The architecture of our system. Given an input document, it aims to detect all sentences that contain a claim that can be verified by some previously fact checked claims (from a given database). The output is a re-ranked list of the document sentences, so that those that can be verified are ranked as high as possible, together with corresponding evidence." ranked output sentences interpreted as feedback).
Regarding claim 18, the combination of Rollwage, Schuurman, and Shaar teaches The computer system as defined in Claim 1, wherein estimating the contribution of the first item of content to the generative model output further comprises estimating a style contribution of the first item of content to the generative model output(Rollwage [¶0008] "large language models may have many advantages over older, more classical text generation systems, like rule-based chatbots. For example, their large amounts of training data make them understand quite varied forms of language and tone and they are robust in reacting to mistakes in the input text. Additionally, this allows them to respond in varied, and potentially highly nuanced ways, adapting their tone, and response, as well as not having to rely on formulaic responses" Tone interpreted as synonymous with style. Response interpreted as synonymous with output.).
Regarding claim 19, the combination of Rollwage, Schuurman, and Shaar teaches The computer system as defined in Claim 1, wherein the computer system is operable to: chunk text of at least a first document into a plurality of overlapping chunks; generate embeddings comprising vectors corresponding to the plurality of overlapping chunks; (Rollwage [0806] "The checking module 76 may first transform the dynamically determined system response to a vector representation using a pre-trained sentence embedder such as for example, a SentenceBERT based model, such as described in the paper “sentence-BERT”, Reimers & Gurevych, 2019, see arXiv:1908.10084 the entire contents of which are incorporated by reference herein. The vector representation is then compared against a vector representation of the segments of text from the database, and a similarity measure determined. The similarity measure may be a cosine similarity for example" Sentence segment vector interpreted as synonymous with chunk.)