DETAILED ACTION
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 06/30/2026 has been entered.
The application contains claims 1-20, all examined and rejected.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
It is acknowledged that claims 1, 9, and 17 have been amended.
Response to Arguments
Applicant's arguments filed 06/30/2026 the claims being rejected under 35 USC 101 have been fully considered but they are not persuasive.
The amendment reciting “wherein the retraining is based on the LLM being configured to periodically retain by accessing the corrective contents broadcasted on the one or more channels” does not change the eligibility analysis.
Under Step 2A, Prong One: the claims continue to recite an abstract idea related to a mental process, where with the aid of pen and paper, using observation, evaluation, judgment, and opinion, identifying noise content, determining a theme, determining a target genre distribution, generating corrective content based on the determined information, and using the resulting information to retrain a model, can practically be performed in the human mind.
The recitation of an AI model or LLM merely identifies the environment in which the abstract idea is performed and does not remove the claim from the judicial exception.
Under Step 2A, Prong Two: the Applicant argues that the claims improve machine learning models by generating, broadcasting, and retraining using corrective content. Examiner respectfully submits that the claims fail to recite any specific technological improvement to how an LLM is trained or operates. Instead, the claims merely recite the desired result of retraining using the corrective content. The amended limitation requiring periodic training by accessing broadcasted corrective content merely describes the source of training information and a timing of retraining. The claims do not recite how the corrective content is incorporated into the training, how model parameters are updated, how the retraining differs from conventional retraining techniques, or how the alleged improvement is technologically achieved. Therefore, the additional element merely applies the abstract idea using generic computing components and to not integrate the judicial exception into practical application. Applicant’s reliance on the Specification is not persuasive, while the Specification discusses improve LLM performance, the claims do not recite the specific technical mechanisms to achieve the improved performance. Applicant’s reliance on Ex parte Desjardins is not persuasive, because the claims fail to recite a how the training is achieved or any specific machine learning training technique that improves the model operation, but instead broadly recites retraining an LLM using corrective content.
Under Step 2B: The additional elements, including the AI model, LLM, broadcasting through channels and periodic training, when considered individually and as an ordered combination, amount to no more than applying the abstract idea using generic computer technology performing its ordinary function. Therefore, does not amount to significantly more than the judicial exception.
Applicant’s arguments with respect to the amended claim(s) being rejected under 35 USC 103 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Applicant’s arguments with respect Gao teaching away from retraining is not persuasive. Even if Gao describes correction through RAG without retraining, Gao’s approach is merely one implementation choice but does not teach that retraining should not be used. One of ordinary skill in the art would have recognized that modifying Gao’s RAG system with the ability to retrain the LLM based on updated content would provide a system where Gao would provide immediate correction of generated responses while the secondary reference provides periodic retraining to reduce similar errors in future responses.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claim(s) 1-20 is/are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: Claims 1-8, are method claims. Claims 9-16, are system claims. Claim 17-20, are non-transitory computer-readable device claims. Therefore, claims 1-20 are directed to either a process, machine, manufacture, or composition of matter.
Step 2A Prong 1:
Claim 1, 9 and 17 recites the following limitation(s):
identifying,
determining,
determining, through observation and judgement the target genre distribution can be determined based on statistical analysis, which further includes mathematical concepts)
generating,
Claims 2, 10 and 18 recites the following limitations:
and determining the target genre distribution based on a genre distribution of the sample of contents, (nothing in the claims element precludes the “determining” step from being performed in the mind with the aid of pen and paper, for example, through observation and judgement the target genre distribution can be determined based on statistical analysis, which further includes mathematical concepts)
Claims 3 and 11 recites the following limitations:
wherein the genre distribution indicates a percentage of a classification of the sample of contents into a plurality of genres, wherein the plurality of genres includes at least one of a video, a short message, or a text-based message (mathematical concepts)
Claims 4, 12 and 19 recites the following limitations:
determining channel weights, wherein the channel weights correspond to a distribution of the sample of contents across the one or more channels (mathematical concepts)
Claims 5 and 13 recites the following limitations:
determining an effectiveness metric based on a comparison between the first set of responses and the second set of responses (Mental process of evaluation and judgement which can be reasonably performed in one’s mind or with the aid of pencil and paper)
and
Claims 6, 14 and 20 recites the following limitations:
determining an effectiveness metric for the corrective contents, (Mental process of evaluation and judgement which can be reasonably performed in one’s mind or with the aid of pencil and paper)
Accordingly, under its broadest reasonable interpretation, covers performance of the highlighted limitation(s) in the mind and/or using mathematical calculations but for the recitation of generic computer components. That is, other than reciting “computer implemented method,” “by at least one computing device,” “system,” “a memory,” “a processor,” “non-transitory computer-readable device,” nothing in the claim element(s) precludes the step(s) from practically being performed in the human mind using observation, evaluation, judgment, and opinion, and/or mathematical calculations. As such, the claim(s) falls within the “Mental Processes” and “Mathematical Concepts” grouping of abstract ideas. Therefore, the claim(s) recites an abstract idea.
Step 2A Prong 2: The judicial exception(s) are not integrated into a practical application. The claim(s) recites the following additional elements:
Claim 1, recites: computer-implemented method, at least one computing device; Claim 9, recites: system, comprising: a memory; and at least one processor coupled to the memory; Claim 17 recites: on-transitory computer-readable device having instructions stored thereon that, when executed by at least one computing device; (all of which are recited at high level of generality and amounts to no more than mere instructions to apply the exception using a generic computer)
Claim 1, 9 and 17 further recites the following limitation(s):
ses, represents adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea - see MPEP 2106.05(f))
corrective contents based on the target genre distribution and the theme” and without any details about how the outcomes are accomplished.)
and broadcasting, by the at least one computing device, the corrective contents via one or more channels, (“receiving” and “outputting” are mere data gathering and output recited at a high level of generality, and thus are insignificant extra-solution activity)
retraining the LLM using the corrective content, wherein the retaining is based on the LLM being configured to periodically retain by accessing the corrective contents broadcasted on the one or mor channels (recited broadly/result oriented without any technical steps related to how the LLM is retrained (e.g. claims do not recite how the corrective content is incorporated into the training, how model parameters are updated, how the retraining differs from conventional retraining techniques, or how the alleged improvement is technologically achieved), so is merely updating based on calculated/observed solution, and thus are insignificant post-solution activity)
Claims 2, 10 and 18 further recites the following limitations:
scanning data sources to obtain a sample of contents, (scanning… to obtain are mere data gathering and output recited at a high level of generality, and thus are insignificant extra-solution activity)
Claims 5 and 13 further recites the following limitations:
before transmitting the corrective contents using the one or more channels, querying the LLM to obtain a first set of responses; after publishing the corrective contents using the one or more channels, querying the LLM to obtain a second set of responses, (querying… to obtain are mere data gathering and output recited at a high level of generality, and thus are insignificant extra-solution activity)
and updating the channel weights based on the effectiveness metric (post solution activity related to applying or adjusting setting)
Claims 6, 14 and 20 recites the following limitations:
querying the AI model at preset intervals, (querying… to obtain are mere data gathering and output recited at a high level of generality, and thus are insignificant extra-solution activity)
and adjusting a publishing schedule for the corrective contents based on the effectiveness metric, (post solution activity related to applying or adjusting setting)
Claims 7 and 15 recites the following limitations:
wherein the adjusting
Claims 8 and 16 recites the following limitations:
publishing the corrective contents via a webpage of a publicly accessible website, (“receiving” and “outputting” are mere data gathering and output recited at a high level of generality, and thus are insignificant extra-solution activity)
Accordingly, these additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. Therefore, the claim(s) are directed to an abstract idea.
Step 2B: The claim(s) does not include additional element(s) that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional element(s) amounts to no more than mere instructions to apply the exception using a generic computer and thus are mere instructions to apply an exception using a generic computer component-see MPEP 2106.05(f). Also, the additional element(s) amounts to no more than mere data gathering and output recited at a high level of generality and thus are insignificant extra-solution activity - see MPEP 2106.05(g). Specifically, Parker v. Fook: adjusting a system setting after doing math (post-solution activity); Electric Power Group: selecting/analyzing information and displaying results (data gathering/output))
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-5, 8-13, 16-19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Gao, Luyu, et al. "Rarr: Researching and revising what language models say, using language models." arXiv preprint arXiv:2210.08726 (2022) in view of Goldenstein et al. (US 20190179861 A1) further in view of Vig et al. (US 20220229999 A1) and Jang et al. “TemporalWiki: A Lifelong Benchmark for Training and Evaluating Ever-Evolving Language Models” arXiv:2204.14211v3 [cs.CL] 12 Apr 2023.
Regarding claim 1, Gao discloses:
a computer implemented method, comprising: identifying, by at least one computing device, noise content in responses of a large language model (LLM), at least by (Abstract, describes LLMs generating unsupported or misleading content, Sec. 3.2 “the agreement model checks if the evidence e disagrees with the current output y regarding the issue in query q.” where the output y that evidence e disagrees with is noise content in the response y)
determining, by the at least one computing device, a theme for corrective contents to counter the noise contents, at least by (Sec. 6.1, Analyzing RARR Page 8 First Col., “unattributed claims, especially those involving entities and numbers”, where entities and numbers are theme for corrective contents)
determining, by the at least one computing device, a target genre distribution for the corrective contents, at least by (Sec. 6.2, Ablating query generation, identifies that corrective content from sentences that ‘mimic” content on the WEB is more effective… Wikipedia” where content on the WEB such as Wikipedia is a target genre for the corrective content)
generating, by the at least one computing device and using an artificial intelligence (AI) model, the corrective contents based on the target genre distribution and the theme, wherein the correct content comprise content of different genres based on the target genre distribution, at least by (Sec. 3 Approach, Page 4 First Col. “For each query qi , it retrieves web documents and selects the best evidence snippets {ei1, ei2, . . . }. The revision stage then revises the original text x using the retrieval results {(q1, e11), . . . }, yielding a revised text y”)
But Gao fails to describes:
(a) broadcasting… the corrective content via one or more channels
(b) wherein the correct content comprise content of different genres based on the target genre distribution
(c) retraining the LLM using the corrective contents
(d) wherein the retraining is based on the LLM being configured to periodically retrain by accessing the corrective contents broadcasted on the one or more channels.
However, Goldenstein teaches the above limitation (a) at least by (paragraph [0087] “Assembled information may be provided via an intelligence channel via a customized interface. The interface may be in the form of a hub comprising a customized dashboard. Results received for each channel ordered by a user may be combined and delivered via the customized hub”)
Furthermore, Vig teaches the above limitation (b) at least by (paragraph [0023] which describes the corrective content as “dataset is enhanced, enriched and/or augmented with one or more contributing factors”, where the contributing factors includes, different style attributes (e.g. different genres) identified based on the style classifier (see: para. 0018, “identify a list of explicit style attributes”; para. 0019 “the style classifier 170 is trained to predict whether text exhibits one or more of the identified style attributes”) (e.g. target genre distribution))
Also, Vig teaches the above limitation (c) at least by (paragraph [0025] “When the responses determined in accordance with the model 180 do not sufficiently match those in the DB 232, the process and/or algorithm defined by the model 180 is altered and/or adjusted and the dataset from the DB 232 is re-processed.” Where the alterations and adjustments are the corrective content describes by “dataset is enhanced, enriched and/or augmented with one or more contributing factors,” para. 0023)
And Jang teaches the above limitation (d) at least by (Sec. 1. Into, “continual pretraining on new and updated data as a solution for mitigating temporal misalignment” where the updates are identified from differences in snapshot from particular sources such as Wikipedia “TEMPORALWIKI is responsive to the dynamic changes in the world and can be utilized to automatically train and evaluate ever-evolving LMs on each English Wikipedia and English Wikidata snap shot update….continually training LMs only on the updated portion of English Wikipedia” (see Intro). As such, the Wikidata snap shot update in this case can be corrective content, where the updates are published to the Wikipedia channel/websource)
Therefore, before the effective filing date of the invention it would have been obvious to one of ordinary skill in the art to combine the system of Gao with the content delivery system of Goldenstein to provide customizable intelligence channel to contain the spread of disinformation (Goldenstein, para. 0137); and to combine the system of Gao with the style classifier and model tuning of Vig to improve response generation by including more response style attributes and communication channels (Vig, para. 0007). And to further combine the system of Gao with Jang to address temporal misalignment reducing similar errors in future responses (Jang, intro).
As per claim 2, claim 1 is incorporate and Gao further discloses:
further comprising: scanning data sources to obtain a sample of contents, at least by (Sec. 3 Approach, Page 4 First Col. “For each query qi , it retrieves web documents and selects the best evidence snippets {ei1, ei2, . . . }. The revision stage then revises the original text x using the retrieval results {(q1, e11), . . . }, yielding a revised text y”);
But fails to describe: and determining the target genre distribution based on a genre distribution of the sample of contents.
However, Goldenstein teaches the above limitations at least by (paragraph [0090, 0136] “intelligence channel as rendered via the interface 64 may comprise any appropriate information that identifies the intelligence channel. For example, an intelligence channel may comprise an icon, text, video, sound, or any appropriate combination thereof that identifies the intelligence channel…. generate a custom intelligence channel, to combine intelligence channels, or any appropriate combination thereof. A data feed may be provided (e.g., RSS, or other data feeds) comprising user requested intelligence channels for use on websites and displays, and potentially customized to employ user graphic formats, and user system feed requirements)
Therefore, before the effective filing date of the invention it would have been obvious to one of ordinary skill in the art to combine the system of Gao with the content delivery system of Goldenstein to provide customizable intelligence channel to contain the spread of disinformation (Goldenstein, para. 0137).
As per claim 3, claim 2 is incorporate and Gao fails to disclose:
wherein the genre distribution indicates a percentage of a classification of the sample of contents into a plurality of genres, wherein the plurality of genres includes at least one of a video, a short message, or a text-based message.
However, Goldenstein teaches the above limitations at least by (paragraph [0090, 0136] “intelligence channel as rendered via the interface 64 may comprise any appropriate information that identifies the intelligence channel. For example, an intelligence channel may comprise an icon, text, video, sound, or any appropriate combination thereof that identifies the intelligence channel…. generate a custom intelligence channel, to combine intelligence channels, or any appropriate combination thereof. A data feed may be provided (e.g., RSS, or other data feeds) comprising user requested intelligence channels for use on websites and displays, and potentially customized to employ user graphic formats, and user system feed requirements, further more paragraph [0098-0103] describes considering multiple metrics related to twitter post (short text), youtube (video), blog, (text-based message), where metrics related to influence, context, relatedness to curate the intelligence channel, describes genre distribution indicates a percentage of a classification, see also para. 0289 which describes the percentage of distribution)
Therefore, before the effective filing date of the invention it would have been obvious to one of ordinary skill in the art to combine the system of Gao with the content delivery system of Goldenstein to provide customizable intelligence channel to contain the spread of disinformation (Goldenstein, para. 0137).
As per claim 4, claim 2 is incorporate and Gao fails to disclose:
further comprising: determining channel weights, wherein the channel weights correspond to a distribution of the sample of contents across the one or more channels; and transmitting the corrective contents via the one or more channels based on the channel weights.
However, Goldenstein teaches the above limitations at least by (paragraph [0137] “techniques are provided for rating the veracity of content distributed via digital communications sources… the veracity of the content to create a veracity score for delivery with the content”, where the veracity describes the channel weight)
Therefore, before the effective filing date of the invention it would have been obvious to one of ordinary skill in the art to combine the system of Gao with the content delivery system of Goldenstein to provide customizable intelligence channel to contain the spread of disinformation (Goldenstein, para. 0137).
As per claim 5, claim 4 is incorporate and Gao further discloses:
further comprising: before transmitting the corrective contents using the one or more channels, querying the LLM to obtain a first set of responses, at least by (Sec. 3.1 Evidence retrieval, which describes querying the LLM for first result); after publishing the corrective contents using the one or more channels, at least by (Sec. 6 Analyzing RARR, Pg. 9 First Col. Describes the revision stage that provides the corrective contents) querying the LLM to obtain a second set of responses; determining an effectiveness metric based on a comparison between the first set of responses and the second set of responses; and updating the channel weights based on the effectiveness metric, at least by (Sec. 6.2, Ablating query generation, describes comparative results after making certain adjustments to source or passages (corrective content) which describes the effectiveness of the changes)
As per claim 8, claim 1 is incorporate and Gao fails to disclose:
further comprising: publishing the corrective contents via a webpage of a publicly accessible website
However, Goldenstein teaches the above limitations at least by (paragraph [0089] “Channels and news may be delivered to the user through the hub like interface 64. The hub-like interface 64 may be presented in any appropriate manner and/or format. In an example embodiment, the hub-like interface 64 may be presented as “my Channels” page, or the like, which may be accessible by being displayed on all website pages”)
Therefore, before the effective filing date of the invention it would have been obvious to one of ordinary skill in the art to combine the system of Gao with the content delivery system of Goldenstein to provide customizable intelligence channel to contain the spread of disinformation (Goldenstein, para. 0137).
Claims 9, 10, 11, 12, 13 and 16 recite equivalent claim limitations as claims 1, 2, 3, 4, 5 and 8 above, except that they set forth the claimed invention as a system; Claims 17, 18 and 19 recite equivalent claim limitations as claims 1, 2, and 4 above, except that they set forth the claimed invention as a, non-transitory computer readable device, as such they are rejected for the same reasons as applied hereinabove.
Claim(s) 6, 7, 14, 15 and 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Gao, Goldenstein, Vig and Jang further in view of Brooks (US 20100174671 A1).
As per claim 6, claim 1 is incorporate and Gao fails to discloses:
further comprising: querying the AI model at preset intervals; determining an effectiveness metric for the corrective contents; and adjusting a publishing schedule for the corrective contents based on the effectiveness metric.
However, Brooks teaches the above limitations at least by (paragraph [0077] “a machine learning routine using MLR content to enhance or optimize predetermined effectiveness metrics using time periods of the schedule not allocated for the experiment. MLR content refers to the collection of content that is available for consideration by the MLR algorithm. It may include content that is specifically designed for the MLR, experimental content, or any other content.” Paragraph [0094] “content distribution and data processing module is configured to continuously adjust 69 content distribution patterns in order to learn the relationship between the content (e.g., actions) and the states (e.g., display properties) and maximize the objective function that is specified by the relative values on the different effectiveness metrics.”);
Therefore, before the effective filing date of the invention it would have been obvious to one of ordinary skill in the art to combine the system of Gao with the ability to adjust the content distribution schedule in Brooks to maximize the objective function that is specified by the relative values on the different effectiveness metrics (Brooks, para. 0094).
As per claim 7, claim 6 is incorporate and Gao fails to discloses:
wherein the adjusting is based on a reinforcement learning algorithm.
However, Brooks teaches the above limitations at least by (Paragraph [0094] “content distribution and data processing module is configured to continuously adjust 69 content distribution patterns in order to learn the relationship between the content (e.g., actions) and the states (e.g., display properties) and maximize the objective function that is specified by the relative values on the different effectiveness metrics.”);
Therefore, before the effective filing date of the invention it would have been obvious to one of ordinary skill in the art to combine the system of Gao with the ability to adjust the content distribution schedule in Brooks to maximize the objective function that is specified by the relative values on the different effectiveness metrics (Brooks, para. 0094).
Claims 14 and 15 recite equivalent claim limitations as claims 6 and 7 above, except that they set forth the claimed invention as a system; Claim 20 recite equivalent claim limitations as claim 6 above, except that they set forth the claimed invention as a, non-transitory computer readable device, as such they are rejected for the same reasons as applied hereinabove.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Carlini et al. “Poisoning Web-Scale Training Datasets is Practical” arXiv:2302.10149v2 [cs.CR] 6 May 2024- Describes the ability to inject malicious examples to publicly available snapshot datasets that LLMs are trained on (see Intro, and Sec. 3.1) such as specific domains and web sources like Wikipedia (sec. 3.3) . As such, Carlini discloses similar idea related to publishing content that is used as training data for an LLM, but the content is malicious content not “corrective content” but effectively provides the structural concept of publishing content to particular web sources to modify the training data that the LLM is trained on to change the outcome of LLM responses. But Carlini does not specifically describe: wherein the retaining is based on the LLM being configured to periodically retain by accessing the corrective contents broadcasted on the one or mor channels.
Oros (US 20240095017 A1) paragraph [0084] “a new dataset for retraining is received, after a certain amount of data for retraining is received, after a certain amount of time has elapsed since initial training or the last retraining, etc” further describes the ability to periodically retain an LLM on new content.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DENNIS TRUONG whose telephone number is (571)270-3157. The examiner can normally be reached Monday - Friday 8:30 am - 5:30 pm PT.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Amy Ng can be reached at (571) 270-1698. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DENNIS TRUONG/Primary Examiner, Art Unit 2164 08/05/2026
Prosecution Insights