DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment & Arguments
It appears that Remarks have been cut off. Not all arguments are present in the Remarks. Regardless, claims 1, 14 and 18 stand rejected under 101 Abstract Idea. See detailed rejection below.
Applicant's arguments with respect to 35 U.S.C. 102 in regards to claims 1, 14 and 18 have been considered but are moot due to new grounds of rejection necessitated by amendments. See detailed rejection below.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 6 and 7 are dependent on cancelled claim 3. Appropriate corrections are required.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-2, 4-7 and 9-20 are rejected under 35 U.S.C. 101.
Claims 1-2, 14 and 18 are rejected under 35 U.S.C. §101 as being directed to an abstract idea. In particular, it claims come down to a fundamental human mental process. At its core, the claims describes the steps of reading a piece of text, picking out important keywords (the “hint”), and using those keywords to write a summary. Under patent law, collecting, analyzing, and summarizing information are considered “mental processes” or abstract ideas. Because a human being could theoretically perform these exact same steps in their mind or with a pen and paper, the underlying concept is simply a method organizing information, which is not patentable on its own.
It uses generic computer components as a tool, not an improvement. To make an abstract idea patentable, it must be integrated into a practical application that actually improves a piece of technology. However, these claims merely takes the abstract idea of summarizing text and tells standard, generic technology, such as a “computer system”, a “machine learning model”, and a “large language model”, to do the work. It does not explain how to improve the inner workings of the computer itself, nor does it solve a specific technical hardware problem; it just uses off-the-shelf AI components as a tool to process words faster.
The training steps are standard mathematical concepts. The second half of the claims discuss “training” the model by comparing its output to a correct answer, calculating the difference (the “loss”), and adjusting the model to be more accurate. This is known as a standard supervised machine learning. Because these mathematical steps are entirely routine, well-understood, and conventional in the field of AI, they do add an “inventive concept” that transforms the abstract idea into a patent-eligible invention.
The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception because the claims are (i) mere instructions to implement the idea on a computer, and/or (ii) recitation of generic computer structure that serves to perform generic computer functions that are well-understood, routine, and conventional activities previously known to the pertinent industry. Viewed as a whole, these additional claim element(s) do not provide meaningful limitation(s) to transform the abstract idea into a patent eligible application of the abstract idea such that the claim(s) amounts to significantly more than the abstract idea itself. Therefore, the claim(s) are rejected under 35 U.S.C. 101 as being directed to non-statutory subject matter. There is further no improvement to the computing device.
Dependent claims 4-13, 16-17 and 19-20 further recite an abstract idea performable by a human and do not amount to significantly more than the abstract idea as they do not provide steps other than what is conventionally known in summarizing data.
Claims 4 and 15: directed to the abstract idea of entity extraction and summarization; making the extractor a “second large language model” is merely using another generic tool, not significantly more.
Claim 5: directed to the abstract idea of training a model by providing examples to influence outputs; “contextual prompting” with entity/category/content examples is an instruction scheme using generic ML, not an inventive concept.
Claim 6: directed to the abstract idea of identifying entities in text for summarization; stating the model is “configured to perform entity extraction” is functional labeling and adds no significantly more.
Claim 7: directed to the abstract idea of organizing training data (content/category/hint) to train an entity extractor; supervised fine tuning on labeled datapoints is conventional ML activity and adds no inventive concept.
Claim 9: directed to the abstract idea of training using reference summaries and entity categories; reinforcement learning is generic model training math and does not supply an inventive concept.
Claim 10: directed to the abstract idea of iteratively training via prompts, predicted summaries, and comparisons to references; the steps merely use generic LLM prompting and model updating, not a technical improvement.
Claim 11: directed to the abstract idea of scoring outputs, computing a reward, and updating a model; these are mathematical evaluation steps applied with generic computing, not significantly more.
Claim 12: directed to the abstract idea of weighting tokens/entities in a scoring function; weighted scoring is a mathematical concept and adds no inventive concept beyond generic implementation.
Claim 13: directed to the abstract idea of training a model with multiple learning signals; concurrent supervised fine tuning and reinforcement learning is a generic training choice and adds no significantly more.
Claims 16 and 19: directed to the abstract idea of organizing information (training datapoints with entity labels/hints) and performing mathematical optimization (computing a loss and minimizing it to update a model); the additional elements (processors, non-transitory medium, “machine learning model,” supervised fine tuning) are generic, result oriented implementation details that do not integrate the abstract idea into a practical application or add significantly more.
Claims 17 and 20: directed to the abstract idea of collecting/analyzing text (entity/category labeling), generating a summary, and using mathematical reinforcement learning style evaluation/updating based on comparisons to a reference summary; the recited non-transitory medium, processors, LLM/prompting, and RL “updating” are generic, result oriented computer/ML functions that do not integrate the abstract idea into a practical application or add significantly more.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-2, 4, 6, 13, 14-15 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over He et al. (US 11,934,781) in view of Li et al. (“Guiding Large Language Models via Directional Stimulus Prompting”; NeurIPS 2023).
Claim 1,
He teaches a method, comprising ([Figs. 2-5] [col. 5 lines 34-44] [col. 6 lines 40-50] “Fig. 5 is a simplified logic flow diagram illustrating a method for generating a controlled summary using the keywords-based model shown in Fig. 2 during inference stage.”; He also teaches “Fig. 4 is a simplified logic flow diagram illustrating a method for training the keywords-based summarization model shown in Fig. 2.”):
receiving, by a summarization system (SS) comprising one or more computer systems, content to be summarized ([Fig. 3] [col. 4 lines 33-47] [col. 5 lines 3-18] [col. 6 lines 51-53] He teaches the computer system: “Fig. 3 is a simplified diagram of a computing device for implementing the summarization system,” and “computing device 300 include a processor 310 coupled to memory 320.”; He further teaches that “memory 320 includes instructions for control summarization module 330,” and that “the controllable summarization module 330, may receive an input 340, e.g. a source document.” He also states: “At step 502, an input document … may be received”);
generating, by the SS, a hint based upon the content to be summarized ([Figs. 1A-B] [col. 3 lines 9-39] He teaches that “additional control tokens such as the keywords z, may be used to represent user preferences,” and that “keywords 120 … are extracted from the article 110.”; He further states that, at inference, “keywords 120a are automatically extracted from the source document, e.g. article 110”),
the hint comprising one or more entities identified by the SS from the content to be summarized ([col. 2 lines 36-59] [col. 3 lines 27-39] [col. 3 lines 48-56] He teaches entity-based keywords: “the user may enter or select entity names as keywords.” He also teaches automatic extraction from the source document: “keywords 120a are automatically extracted from the source document, e.g. article 110.”; He further states that “the control center 140 may choose to only keep certain entity-related keywords if the user 150 indicates interests in the particular entity name.”),
wherein each entity in the one or more entities is a word occurring in the content to be summarized or a sequence of adjacent words occurring in the content to be summarized ([col. 4 lines 1-10] [col. 5 line 54 to col. 6 line 16] He teaches source-document token/subsequence extraction: “all the longest sub-sequences are identified in the extracted sentences that have matched sub-sequences in the ground-truth summary.”; He then states that “duplicate words and stop words are removed from the sentences, and the remaining tokens are kept as keywords.”; He further teaches that “the keywords sequence maintains the order of the keywords as they were in the source document.” He gives multi-word entity examples including “Dwyane Wade” and “Stephen Curry”).
The difference between the prior art and the claimed invention is that He does not explicitly teach the one or more entities corresponding to one or more entity categories, wherein generating the hint comprises extracting the one or more entities using a particular machine learning model in the SS; generating, by the SS, a prompt comprising the content to be summarized and the hint; providing, by the SS, the prompt as input to a large language model (LLM); responsive to the prompt, generating, by the LLM, a summary for the content to be summarized; and training the particular machine learning model using a supervised fine-tuning technique and a plurality of training datapoints, each training datapoint in the plurality of training datapoints comprising training content to be summarized and a ground truth hint comprising at least one entity identified in training content to be summarized, and wherein using the supervised fine-tuning technique further comprises, for at least a first training datapoint in the plurality of training datapoints: computing a loss based on the hint generated by the particular machine learning model and the ground truth hint associated with the first training datapoint; and minimizing the loss using a loss minimization technique, wherein the minimizing comprises updating the particular machine learning model.
Li teaches the one or more entities corresponding to one or more entity categories ([Fig. 6] [pg. 19] [Appendix B.1] Li teaches entity categories corresponding to generated hint keywords: “Named Entity Recognition (NER) tagging on the generated hints, i.e. keywords.”; Li further states: “As for the NER tagging, the most commonly generated keywords include person (PERSON), geopolitical entities (GPE), dates (DATE), organizations (ORG), and numerals (CARDINAL),”),
wherein generating the hint comprises extracting the one or more entities using a particular machine learning model in the SS ([2.] [pg. 4] [ 3.1] [pg. 5] Li teaches the particular hint-generating ML model: “To generate this stimulus for each input query, we use a small tunable policy language model…”; for summarization: Li states: “we seek to guide LLMs to generate summaries … by providing keywords that should be mentioned in the desired summaries as hints.”; Li further specifics the extraction training format: “The input format for training is “Extract the keywords: [Article]’, while the output is the target stimulus consisting of keywords.”);
generating, by the SS, a prompt comprising the content to be summarized and the hint (2.] [pg. 4] [Fig. 10] [pg. 25] Li states: “We then use this generated stimulus, z, along with the original input, x, to construct the prompt that steers the LLM toward generating its output.”; Li’s CNN/Daily Mail prompt template includes “Article: [[Question]]” and “Keywords: [[HINT]].”; see Fig. 10);
providing, by the SS, the prompt as input to a large language model (LLM) ([Abstract] [Fig. 2] [pg. 3] [ 2.] [pg. 4] Li teaches identifies “black-box large language models (LLMs)” in its framework; Li states that the prompt “steers the LLM toward generating its output…”; Li’s Fig. 2 identifies a “Black-box LLM (e.g. ChatGPT).”);
responsive to the prompt, generating, by the LLM, a summary for the content to be summarized ([2.] [pg. 3] [Fig. 1] [pg. 2] Li states that, “for the summarization task, the input x is an article, and the output y is the corresponding summary.”; Li further states: “Overall, when using the LLM with DSP to perform a downstream task, the output is obtained via y ~ …”; Li’s Fig. 1 shows the LLM output summary beginning: “On April 1, Bob Barker returned to the TV show ‘The Price is Right’…”); and
training the particular machine learning model using a supervised fine-tuning technique ([2.1] [pg. 4] [3.1] [pg. 5] Li states: “To train the policy model that generates directional stimulus for LLMs, we first perform supervised fine-turning (SFT) on a pre-trained LM (e.g. T5, GPT-2, etc.) on a small collection of labeled data.”; Li also states: “We use the constructed article- stimulus pairs to train the policy model via supervised fine-tuning.”) and
a plurality of training datapoints ([2.1] [pg. 4] [3.1] [pg. 5] Li states: “The resulting dataset D’ = {(x,z*)} consists of input-stimulus pairs.”; for summarization, Li states it trains on “a subset of 1,000, 2,000 and 4,000 article-summary pairs.”),
each training datapoint in the plurality of training datapoints comprising training content to be summarized and a ground truth hint comprising at least one entity identified in training content to be summarized ([2.1] [pg. 4] [3.1] [pg. 5] [Fig. 10] [pg. 25] Li states that, “for the summarization task, we use keywords that the reference summary includes as pseudo-stimulus.” Li further explains that it “automatically extracts the keywords from the article and summary an only keeps those that appear in the reference summary.”; Li’s prompt examples show entity-containing keyword hints, such as “Marie Ccaszar,” “BBC,” “Cardiff”, and “Data Protection Act,” and another example including “Debra Lobo,” “the Jinnah Medical and Dental College,” and “Karachi”), and
wherein using the supervised fine-tuning technique further comprises, for at least a first training datapoint in the plurality of training datapoints: computing a loss based on the hint generated by the particular machine learning model and the ground truth hint associated with the first training datapoint ([2.1] [Eq. 1] [pg. 4] Li defines the SFT objective: “We then fine-tune the policy model by maximizing the log-likelihood: see eq. 1; Li identifies the model and ground-truth stimulus relationship: “The resulting dataset D’ = {(x,y*)} consists of input-stimulus pairs.”); and
minimizing the loss using a loss minimization technique ([2.1] [pg. 4] [Table 3] [pg. 16] Li states that the policy mode is fine-tuned using the negative-log-likelihood objective…” (see eq. 1); Li also gives SFT optimization hyperparameters: “Supervised fine-tuning (SFT) – batch size: 8; epochs: 5; learning rate: 0.00002; learning rate scheduler: liner; weight decay: 0.01.”),
wherein the minimizing comprises updating the particular machine learning model ([2.1] [pg. 4] [3.1] [pg. 5] Li states: “We use the constructed article-stimulus pairs to train the policy model via supervised fine-tuning,”; Li further states: “The policy model was trained for 5 epochs with a 2 x 10-5 learning rate.”; Li also states that SFT is performed “on a pre-trained LM.”).
Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the teachings of He with teachings of Li by modifying the system and method for controllable text summarization as taught by He to include the one or more entities corresponding to one or more entity categories, wherein generating the hint comprises extracting the one or more entities using a particular machine learning model in the SS; generating, by the SS, a prompt comprising the content to be summarized and the hint; providing, by the SS, the prompt as input to a large language model (LLM); responsive to the prompt, generating, by the LLM, a summary for the content to be summarized; and training the particular machine learning model using a supervised fine-tuning technique and a plurality of training datapoints, each training datapoint in the plurality of training datapoints comprising training content to be summarized and a ground truth hint comprising at least one entity identified in training content to be summarized, and wherein using the supervised fine-tuning technique further comprises, for at least a first training datapoint in the plurality of training datapoints: computing a loss based on the hint generated by the particular machine learning model and the ground truth hint associated with the first training datapoint; and minimizing the loss using a loss minimization technique, wherein the minimizing comprises updating the particular machine learning model as taught by Li for the benefit of improving the performance of LLMs with minimal labeled data, achieving or exceeding the performance of some fully supervised state-of-the-art models and outperforming both generalized human-crafted prompts and those generated through automatic prompt engineering (Li [Abstract]).
Claim 2,
He further teaches the method of claim 1, wherein the summary generated for the content to be summarized comprises one or more entities ([Step 510] summaries that focus on entities of interest), wherein each entity in the one or more entities is extracted from the content to be summarized ([col. 8 lines 56-57] acquired every entity in the document to generate summaries), corresponds to an entity category of the one or more entity categories ([col. 2 line 38] names of keywords), and occurs at least once in the summary ([col. 8 lines57-59] the Success Rate is computed, the fraction of requested entity actually occurring in the output summaries).
Claim 4,
He further teaches the method of claim 1, wherein the particular machine learning model is a second large language model ([col. 8 lines 23-41] a first large pretrained model sued for summarization (BaRTLARGE) and a second large pretrained model used for keyword/entity extraction (BeRTLARGE)).
Claim 6,
He further teaches the method of claim 3, wherein the particular machine learning model is a model configured to perform entity extraction ([col. 8 lines 51-52] entities extracted).
Claim 13,
He further teaches the method of claim 3, further comprising training the particular machine learning model using a supervised fine-tuning technique and a reinforcement learning technique concurrently ([col. 8 lines 23-41] conditional distribution p(y|x,z) using two model: BERTLARGE (keyword based model) and BERTLARGE (keyword tagger)).
Claims 14 and 18,
Claims 14 and 18 contains subject matter similar to claims 1 and 2 and thus is rejected under similar rationale.
Claim 15,
The non-transitory computer-readable medium of claim 14, wherein the particular machine learning model is a second large language model or a model configured to perform entity extraction. (Claim 15 contains subject matter similar to claim 4, and thus is rejected under similar rationale)
Claim 5 is rejected under 35 U.S.C. 103 as being unpatentable over He et al. (US 11,934,781) in view of Li et al. (“Guiding Large Language Models via Directional Stimulus Prompting”; NeurIPS 2023) and further in view of Narayan et al. (“Planning with Learned Entity Prompts for Abstractive Summarization” pgs. 1475-1492; 12/2021).
Claim 5,
He provides a base summarization technique (claim 4) in which a language model generates summaries from a document, and it introduces control tokens (keywords or prompts) to guide the summary’s content.
The difference between the prior art and the claimed invention is that He does not explicitly teach the particular contextual prompting and training the model by providing example inputs with an entity and its category as a prompt.
Narayan teaches in-context training prompts using entities: shows that prepending an entity chain ([Introduction] key entities from the summary/document) as a prompt during training improves the model’s planning and reduces hallucinations . This is an example of a contextual prompting technique in training a summarization model. Training with an entity chain prompt suggests that even if explicit entity categories are not mentioned, a skilled artisan would readily consider including an entity’s type or category for clarity. For example, if the entity is Paris, one might prompt with Paris[City] to help the model contextually understand the entity. This minor addition of an entity’s category is seen as a straightforward design choice or obvious extension of Narayan’s prompt technique, given that classifying named entities by type is a well-established practice in NLP.
He and Narayan are both directed to controllable or guided abstractive summarization using large language models. Therefore, it would have been obvious to a person of ordinary skill in the art to combine these teachings. Narayan’s use of entity-based prompts in training would motivate applying a similar prompting approach to He’s summarization system to improve control over summary content. One of ordinary skill would recognize that providing the second LLM (summarizer) with in-context examples or cues – such as key entities (possibly labeled with their types or categories) alongside the document – is merely the use of a known technique to enhance summary relevance and correctness. Using “contextual prompting” (in the broad sense of instructional or demonstration-based input) is a known strategy to guide large language models and integrating this into training a summarizer would have been a predictable variation.
Allowable Subject Matter
Claims 7, 9-12, 16-17 and 19-20 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims AND IF the Applicant can overcome the 101 Abstract Idea rejection.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SHREYANS A PATEL whose telephone number is (571)270-0689. The examiner can normally be reached Monday-Friday 8am-5pm PST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Pierre Desir can be reached at 571-272-7799. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
SHREYANS A. PATEL
Primary Examiner
Art Unit 2653
/SHREYANS A PATEL/Examiner, Art Unit 2659
Prosecution Insights