Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claims 1-3, 7-14, and 18-20 are pending and have been examined.
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submissions filed on 11/10th/2022 (amendment) and 01/12th/2023 (RCE) have been entered.
Examiner's Note
The Examiner respectfully requests of the Applicant in preparing responses, to fully consider the entirety of the reference(s) as potentially teaching all or part of the claimed invention. It is noted, REFERENCES ARE RELEVANT AS PRIOR ART FOR ALL THEY CONTAIN. “The use of patents as references is not limited to what the patentees describe as their own inventions or to the problems with which they are concerned. They are part of the literature of the art, relevant for all they contain.” In re Heck, 699 F.2d 1331, 1332-33, 216 USPQ 1038, 1039 (Fed. Cir. 1983) (quoting In re Lemelson, 397 F.2d 1006, 1009, 158 USPQ 275, 277 (CCPA 1968)). A reference may be relied upon for all that it would have reasonably suggested to one having ordinary skill in the art, including non-preferred embodiments (see MPEP 2123). The Examiner has cited particular locations in the reference(s) as applied to the claim(s) above for the convenience of the Applicant. Although the specified citations are representative of the teachings of the art and are applied to the specific limitations within the individual claim(s), typically other passages and figures will apply as well.
Response to Arguments
Applicant’s arguments, see REMARKS page 8-9 filed 8/15th/2025, regarding the rejection of claims 1-3, 7-14, and 18-20 under 35 U.S.C. §101 have been considered and they are not persuasive.
Applicant Argument #1:
First, under Step 2A, Prong Two, even if considered as reciting an abstract idea (which Applicant does not concede), as a whole integrates the alleged abstract idea into a practical application of sequential recommendation via a trained model, and thus is eligible under Step 2A, Prong Two. The 2025 Memo states that "2A Prong Two considers the claim as a whole" and "the additional limitations should not be evaluated in a vacuum, completely separate from the recited judicial exception." The Office at step 2A Prong Two only considers the elements that do not recite an abstract idea without considering them in combination with the elements that allegedly recite an abstract idea. The elements when considered in combination integrate the alleged abstract idea into a practical application of efficient sequential recommendation via a trained model. For example, the Specification states the "result of weights ensembling is a single composite model, which in effect reduces the memory and processing needed for using the model when decoding since the multiple models have been collapsed to a single mode." (Specification, [0026]).
Examiner Response #1
The examiner respectfully disagrees. While the applicant argues that amended claim 1 reflects an improvement to a problem in computer technology such as efficient methods of creating neural network based models, and while the disclosure [0026] states that result of weights ensembling is a single composite model, which in effect reduces the memory and processing needed for using the model when decoding since the multiple models have been collapsed to a single mode, there is no improvement to the functioning of a computer nor to any other technology. At best, the claimed combination amounts to an improvement to the abstract idea of initializing a summarization model, filtering training datasets, merging models based on the weighted average of all parameters of the models, determining model weights based on quality, or weighting model parameters by the model weights rather than to an improvement on the functioning of a computer or to any other technology. See MPEP 2106.05(a). Thus, even when considering the elements in combination, the claim as a whole does not integrate the recited exception into a practical application.
Applicant Argument #2:
The Appeals Review Panel decision on appeal 2024-000567 (Ex Parte Desjardins) discusses a claim which similarly recites a training method of a machine learning model, which the panel found to be eligible. Specifically, the claim recites a mathematical way of assigning values to parameters (an approximation of a posterior distribution over possible values) and using those values in training the machine learning model. As recognized by the Appeals Review Panel, when viewed as a whole, the claimed method provided "technical improvements over conventional systems by addressing challenges in continual learning and model efficiency." (Ex Parte Desjardins p.5). Similarly, claim 1 recites a method for "an element-wise weighted average of all parameters of the first summarization model and the second summarization model" where "the combined summarization model comprises of an equal number of parameters as the first summarization model" which provides a tangible benefit including at least significant computer technology performance enhancements and resource usage.
Examiner Response #2
The examiner respectfully disagrees. Different cases are treated differently. Each case is directed to different technologies and claim different limitations. For example, the instant application is directed to controlling hallucinations in document summaries and does not claim any mathematical concepts. While appeal 2024-000567 (Ex Parte Desjardins) is directed to a unique training method.
Applicant’s arguments, see REMARKS page 9-10 filed 8/15th/2025, regarding the rejection of claims 1-3, 7-14, and 18-20 under 35 U.S.C. §103 have been considered and they are moot in light of the new rejection below.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-3, 7-14, and 18-20 are rejected under 35 USC § 101 because the claimed invention is directed to non-statutory subject matter
Regarding Claim 1:
Claim 1 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis:
Claim 1 is directed to a method, which is directed to a process, one of the statutory categories.
Step 2A Prong One Analysis:
Claim 1 recites in part process steps which, under the broadest reasonable interpretation, are a series of mental processes including an observation, evaluation, judgment or opinion that could be performed in the human mind or with the aid of pencil and paper. If a claim, under its broadest reasonable interpretation, covers a mental process or a mathematical concept but for the recitation of generic computer components, then it falls within the “Mental Process” grouping of abstract ideas. The claim recites in part:
filtering the training dataset by removing summaries with the respective first scores below a first predetermined threshold resulting in a first training data subset (Mental processes -observation, evaluation, judgment, opinion) under the broadest reasonable interpretation, this limitation is a process step that covers a mental process including observation, evaluation, judgment or opinion that could be performed in the human mind or with the aid of pencil and paper (such as an operator selecting and removing summaries that have scores below a threshold). If a claim, under its broadest reasonable interpretation, covers a mental process but for the recitation of generic computer components, then it falls within the “Mental Process” grouping of abstract ideas.
filtering the training dataset by removing summaries with the respective second scores below a second predetermined threshold resulting in a second training data subset (Mental processes -observation, evaluation, judgment, opinion). under the broadest reasonable interpretation, this limitation is a process step that covers a mental process including observation, evaluation, judgment or opinion that could be performed in the human mind or with the aid of pencil and paper (such as an operator selecting and removing summaries that have scores below a threshold). If a claim, under its broadest reasonable interpretation, covers a mental process but for the recitation of generic computer components, then it falls within the “Mental Process” grouping of abstract ideas.
determining, based on a predefined factual quality goal, a first weight and a second weight (Mental processes -observation, evaluation, judgment, opinion). under the broadest reasonable interpretation, this limitation is a process step that covers a mental process including observation, evaluation, judgment or opinion that could be performed in the human mind or with the aid of pencil and paper (such as an operator determining a weight/importance of two models based on how well they performed a task such as classifying a particular word or image). If a claim, under its broadest reasonable interpretation, covers a mental process but for the recitation of generic computer components, then it falls within the “Mental Process” grouping of abstract ideas.
with each parameter of the first summarization model weighted by the first weight and each parameter of the second summarization model weighted by the second weight (Mental processes -observation, evaluation, judgment, opinion). under the broadest reasonable interpretation, this limitation is a process step that covers a mental process including observation, evaluation, judgment or opinion that could be performed in the human mind or with the aid of pencil and paper (such as an operator applying a weight/importance to the parameters of the first model and the parameters of the second model). If a claim, under its broadest reasonable interpretation, covers a mental process but for the recitation of generic computer components, then it falls within the “Mental Process” grouping of abstract ideas.
Therefore, claim 1 recites an abstract idea which is a judicial exception.
Step 2A Prong Two Analysis:
The judicial exception is not integrated into a practical application. In particular, the claim recites the additional element of:
A method for controlling factual accuracy in abstractive summarization models (computer implementation) which amounts to no more than mere instructions to apply the exception using generic computer components (See MPEP 2106.05(f))
receiving a training dataset comprising a plurality of documents and a plurality of summaries corresponding to the plurality of documents wherein each of the plurality of summaries is associated with a respective first score indicative of a first factual characteristic quality, and a respective second score indicative of a second factual characteristic quality (data gathering) which amounts to insignificant extra-solution activity of gathering data for use in the claimed process. As described in MPEP 2106.05(g).
pre-training a baseline summarization model on the training dataset using
maximum likelihood estimation (MLE) to produce a pre-trained summarization model (computer implementation) which amounts to no more than mere instructions to apply the exception using generic computer components (See MPEP 2106.05(f)).
initializing a first summarization model and a second summarization model with
the pre-trained summarization model (computer implementation) which amounts to no more than mere instructions to apply the exception using generic computer components (See MPEP 2106.05(f)).
training the first summarization model with the first training data subset (computer implementation) which amounts to no more than mere instructions to apply the exception using a generic computer component (See MPEP 2106.05(f))
training the second summarization model with the second training data subset (computer implementation) is recited at a high-level of generality such that it amounts to no more than mere instructions to apply the exception using a generic computer component (See MPEP 2106.05(f)).
constructing a combined summarization model by ensembling the first summarization model and the second summarization model based at least in part on an element-wise weighted average of all parameters of the first summarization model and the second summarization model (computer implementation) is recited at a high-level of generality such that it amounts to no more than mere instructions to apply the exception using a generic computer component (See MPEP 2106.05(f)).
wherein the combined summarization model comprises of an equal number of parameters as the first summarization model This is a field of use limitation which amounts to no more than generally linking the use of a judicial exception to a particular technological environment or field of use (MPEP § 2106.05(h)). As explained by the Supreme Court, a claim directed to a judicial exception cannot be made eligible "simply by having the applicant acquiesce to limiting the reach of the patent for the formula to a particular technological use." Diamond v. Diehr, 450 U.S. 175, 192 n.14, 209 USPQ 1, 10 n. 14 (1981). Thus, limitations that amount to merely indicating a field of use or technological environment in which to apply a judicial exception do not amount to significantly more than the exception itself, and cannot integrate a judicial exception into a practical application.
generating, via the combined summarization model, a summary of an input
document received via a data interface (computer implementation) is recited at a high-level of generality such that it amounts to no more than mere instructions to apply the exception using a generic computer component (See MPEP 2106.05(f)).
Accordingly, at Step 2A, prong two, the additional elements individually or in combination do no integrate the judicial exception into a practical application.
Step 2B Analysis:
Claim 1 does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above, the additional elements of:
A method for controlling factual accuracy in abstractive summarization models (computer implementation) which amounts to no more than mere instructions to apply the exception using generic computer components (See MPEP 2106.05(f))
receiving a training dataset comprising a plurality of documents and a plurality of summaries corresponding to the plurality of documents wherein each of the plurality of summaries is associated with a respective first score indicative of a first factual characteristic quality, and a respective second score indicative of a second factual characteristic quality (data gathering) which amounts to insignificant extra-solution activity of gathering data for use in the claimed process. As described in MPEP 2106.05(g). The courts have found limitations directed to gathering information electronically, recited at a high level of generality, to be well-understood, routine, and conventional (see MPEP 2106.05(d)(II), “gathering statistics”, and "storing and retrieving information in memory").
pre-training a baseline summarization model on the training dataset using
maximum likelihood estimation (MLE) to produce a pre-trained summarization model (computer implementation) which amounts to no more than mere instructions to apply the exception using generic computer components (See MPEP 2106.05(f)).
initializing a first summarization model and a second summarization model with
the pre-trained summarization model (computer implementation) which amounts to no more than mere instructions to apply the exception using generic computer components (See MPEP 2106.05(f)).
training the first summarization model with the first training data subset (computer implementation) which amounts to no more than mere instructions to apply the exception using a generic computer component (See MPEP 2106.05(f))
training the second summarization model with the second training data subset (computer implementation) is recited at a high-level of generality such that it amounts to no more than mere instructions to apply the exception using a generic computer component (See MPEP 2106.05(f)).
constructing a combined summarization model by ensembling the first summarization model and the second summarization model based at least in part on an element-wise weighted average of all parameters of the first summarization model and the second summarization model (computer implementation) is recited at a high-level of generality such that it amounts to no more than mere instructions to apply the exception using a generic computer component (See MPEP 2106.05(f)).
wherein the combined summarization model comprises of an equal number of parameters as the first summarization model This is a field of use limitation which amounts to no more than generally linking the use of a judicial exception to a particular technological environment or field of use (MPEP § 2106.05(h)). As explained by the Supreme Court, a claim directed to a judicial exception cannot be made eligible "simply by having the applicant acquiesce to limiting the reach of the patent for the formula to a particular technological use." Diamond v. Diehr, 450 U.S. 175, 192 n.14, 209 USPQ 1, 10 n. 14 (1981). Thus, limitations that amount to merely indicating a field of use or technological environment in which to apply a judicial exception do not amount to significantly more than the exception itself, and cannot integrate a judicial exception into a practical application.
generating, via the combined summarization model, a summary of an input
document received via a data interface (computer implementation) is recited at a high-level of generality such that it amounts to no more than mere instructions to apply the exception using a generic computer component (See MPEP 2106.05(f)).
Accordingly, at Step 2B the additional elements individually or in combination do not amount to significantly more than the judicial exception.
The additional limitations of the dependent claims contain no additional elements that provide a practical application or amount to significantly more than the abstract idea and are addressed briefly below
Dependent claim 2:
Step 2A Prong 1: The claim recites process steps that are a mental process:
the first factual characteristic quality is a measurement of dependency arc entailment (DAE) accuracy between a document and a respective reference summary (Mental processes -observation, evaluation, judgment, opinion) under the broadest reasonable interpretation, this limitation is a process step that covers a mental process including observation, evaluation, judgment or opinion that could be performed in the human mind or with the aid of pencil and paper (such as an operator calculating an accuracy value for every document-summary pair based on how accurate the summary is). If a claim, under its broadest reasonable interpretation, covers a mental process but for the recitation of generic computer components, then it falls within the “Mental Process” grouping of abstract ideas.
Step 2A Prong 2: The claim does not include additional elements that would integrate the judicial exception into a practical application.
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception.
Dependent claim 3:
Step 2A Prong 1: The claim recites process steps that are a mental process:
the second factual characteristic quality is a measurement of a number of entity tokens in a summary not present in a respective document (Mental processes -observation, evaluation, judgment, opinion) under the broadest reasonable interpretation, this limitation is a process step that covers a mental process including observation, evaluation, judgment or opinion that could be performed in the human mind or with the aid of pencil and paper (such as an operator counting the number of words in the summary that are not in the original document). If a claim, under its broadest reasonable interpretation, covers a mental process but for the recitation of generic computer components, then it falls within the “Mental Process” grouping of abstract ideas.
Step 2A Prong 2: The claim does not include additional elements that would integrate the judicial exception into a practical application.
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception.
Dependent claim 7:
Step 2A Prong 1: The claim does not contain any limitations to analyze under this step.
Step 2A Prong 2: The judicial exception is not integrated into a practical application. In particular, the claim recites the additional element of:
the first weight and the second weight are dynamically adjusted based on a determination that a summary produced by a baseline summarization model does not contain factual errors, wherein the baseline summarization model is trained on unfiltered data (computer implementation) which amounts to no more than mere instructions to apply the exception using a generic computer component (See MPEP 2106.05(f)).
Step 2B: In accordance with Step 2B, the claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception. As discussed above, the additional elements of:
the first weight and the second weight are dynamically adjusted based on a determination that a summary produced by a baseline summarization model does not contain factual errors, wherein the baseline summarization model is trained on unfiltered data (computer implementation) which amounts to no more than mere instructions to apply the exception using a generic computer component (See MPEP 2106.05(f)).
Dependent claim 8:
Step 2A Prong 1: The claim recites process steps that are a mental process: computing an action reward score based on a summary generated by the first summarization model; computing a baseline reward score; computing a first loss based on the difference of the action reward score and the baseline reward score; and updating parameters of the first summarization model based on the first loss (Mental processes -observation, evaluation, judgment, opinion) under the broadest reasonable interpretation, this limitation is a process step that covers a mental process including observation, evaluation, judgment or opinion that could be performed in the human mind or with the aid of pencil and paper (such as an operator calculating scores, and updating parameters). If a claim, under its broadest reasonable interpretation, covers a mental process but for the recitation of generic computer components, then it falls within the “Mental Process” grouping of abstract ideas.
Step 2A Prong 2: The claim does not include additional elements that would integrate the judicial exception into a practical application.
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception.
Dependent claim 9:
Step 2A Prong 1: The claim recites process steps that are a mental process: initializing the first summarization model based on a baseline summarization model; computing a second loss based on a divergence between next token probabilities of the baseline summarization model and the first summarization model; and updating parameters of the first summarization model based on the second loss (Mental processes -observation, evaluation, judgment, opinion) under the broadest reasonable interpretation, this limitation is a process step that covers a mental process including observation, evaluation, judgment or opinion that could be performed in the human mind or with the aid of pencil and paper (such as an operator initializing a first model weights according to a second model weights, calculating loss values, and updating parameters). If a claim, under its broadest reasonable interpretation, covers a mental process but for the recitation of generic computer components, then it falls within the “Mental Process” grouping of abstract ideas.
Step 2A Prong 2: The claim does not include additional elements that would integrate the judicial exception into a practical application.
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception.
Dependent claim 10:
Step 2A Prong 1: The claim does not contain any limitations to analyze under this step.
Step 2A Prong 2: The judicial exception is not integrated into a practical application. In particular, the claim recites the additional element of:
training the first summarization model comprises: training the first summarization model using a maximum likelihood estimation based on the first training data subset (computer implementation) which amounts to no more than mere instructions to apply the exception using a generic computer component (See MPEP 2106.05(f)).
Step 2B: In accordance with Step 2B, the claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception. As discussed above, the additional elements of:
training the first summarization model comprises: training the first summarization model using a maximum likelihood estimation based on the first training data subset (computer implementation) which amounts to no more than mere instructions to apply the exception using a generic computer component (See MPEP 2106.05(f)).
Dependent claim 11:
Step 2A Prong 1: The claim does not contain any limitations to analyze under this step.
Step 2A Prong 2: The judicial exception is not integrated into a practical application. In particular, the claim recites the additional element of:
training a third summarization model with the second training data subset with an objective of entity informativeness; and constructing a combined summarization model by ensembling the first summarization model, the second summarization model, and the third summarization model (computer implementation) which amounts to no more than mere instructions to apply the exception using a generic computer component (See MPEP 2106.05(f)).
Step 2B: In accordance with Step 2B, the claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception. As discussed above, the additional elements of:
training a third summarization model with the second training data subset with an objective of entity informativeness; and constructing a combined summarization model by ensembling the first summarization model, the second summarization model, and the third summarization model (computer implementation) which amounts to no more than mere instructions to apply the exception using a generic computer component (See MPEP 2106.05(f)).
Regarding Claim 12:
Claim 12 is rejected under 35 U.S.C 101 because the claimed invention is directed to an abstract idea without significantly more. The claim recites similar steps to claim 1 (see above for analysis), with the additional element of A system.
Step 2A Prong 2, Step 2B:
The additional element of A system is recited at a high-level of generality such that it amounts to no more than mere instructions to apply the exception using a generic computer component (See MPEP 2106.05(f)). Implementing an abstract idea on generic computer components does not integrate the abstract idea into a practical application, nor does it add significantly more to the exception. Thus, the claim is not patent eligible.
Dependent claim 13 recites:
Claim 13 is rejected under 35 U.S.C 101 because the claimed invention is directed to an abstract idea without significantly more. The claim recites A system with similar steps to claim 2, and thus is not patent eligible for the same reasons (see above).
Dependent claim 14 recites:
Claim 14 is rejected under 35 U.S.C 101 because the claimed invention is directed to an abstract idea without significantly more. The claim recites A system with similar steps to claim 3, and thus is not patent eligible for the same reasons (see above).
Dependent claim 18 recites:
Claim 18 is rejected under 35 U.S.C 101 because the claimed invention is directed to an abstract idea without significantly more. The claim recites A system with similar steps to claim 8, and thus is not patent eligible for the same reasons (see above).
Dependent claim 19 recites:
Claim 19 is rejected under 35 U.S.C 101 because the claimed invention is directed to an abstract idea without significantly more. The claim recites A system with similar steps to claim 9, and thus is not patent eligible for the same reasons (see above).
Dependent claim 20 recites:
Claim 20 is rejected under 35 U.S.C 101 because the claimed invention is directed to an abstract idea without significantly more. The claim recites A system with similar steps to claim 11, and thus is not patent eligible for the same reasons (see above).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all
obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-3, 7, 10, and 12-14 are rejected under 35 U.S.C. 103 as being unpatentable over Goyal (Evaluating Factuality in Generation with Dependency-level Entailment), in view of Dickie (US20220207430A1), further in view of Baron (US20210350930A1), further in view of Wu (US20220058494A1), further in view of Azar (Query-Based Single Document Summarization Using an Ensemble Noisy Auto-Encoder - 2015.), further in view of Krishnamurthy (US20220230013A1), and further in view of Baughman (US9542646), further in view of GUNNERUD (US20230167717A1).
Regarding claim 1 Goyal teaches receiving a training dataset comprising a plurality of documents and a plurality of summaries corresponding to the plurality of documents ([P. 3595, Sec. 4] We now describe our method for automatically collecting dependency-level DAE annotations from paraphrase or entailment corpora, avoiding manual annotation. In this creation process, we want data featuring a range of paraphrasing phenomenon, such as passivization, clausal reordering, synonym replacement, and more. Furthermore, we want a natural distribution of errors produced by generation models, such as wrong subject or objects for a verb or hallucination of new content).
wherein each of the plurality of summaries is associated with a respective first score indicative of a first factual characteristic quality, and a respective second score indicative of a second factual characteristic quality ([P. 3597, Sec. 6.1] We perform our evaluation on an abstractive summarization test dataset introduced in Falke et al. (2019) and used in other previous work. It contains 373 test samples, each containing an input source sentence from CNN/DM and two summary sentences covering the same content generated using the model from Chen and Bansal (2018). One of these summary sentences is factually correct and the other is factually incorrect. The examiner interprets the factually correct and incorrect summary sentences taught by Goyal to be the claimed first factual characteristic quality, and the second factual characteristic quality).
However, Goyal is not relied upon to explicitly teach pre-training a baseline summarization model on the training dataset using maximum likelihood estimation (MLE) to produce a pre-trained summarization model. Goyal is also not relied upon to explicitly teach initializing a first summarization model and a second summarization model with the pre-trained summarization model. Goyal is also not relied upon to explicitly teach filtering the training dataset by removing summaries with the respective first scores below a first predetermined threshold resulting in a first training data subset; filtering the training dataset by removing summaries with the respective second scores below a second predetermined threshold resulting in a second training data subset. Goyal is also not relied upon to explicitly teach training a first summarization model with the first training data subset; training a second summarization model with the second training data subset. Goyal is also not relied upon to explicitly teach determining, based on a predefined factual quality goal, a first weight and a second weight. Goyal is also not relied upon to explicitly teach with each parameter of the first summarization model weighted by the first weight and each parameter of the second summarization model weighted by the second weight. Goyal is also not relied upon to explicitly teach wherein the combined summarization model comprises of an equal number of parameters as the first summarization model. Goyal is also not relied upon to explicitly teach constructing a combined summarization model by ensembling the first summarization model and the second summarization model. Goyal is also not relied upon to explicitly teach based at least in part on an element-wise weighted average of all parameters of the first summarization model and the second summarization model. Goyal is also not relied upon to explicitly teach generating, via the combined summarization model, a summary of an input document received via a data interface.
On the other hand, Azar teaches pre-training a baseline summarization model on the training dataset using maximum likelihood estimation (MLE) to produce a pre-trained summarization model ([Page 4-5, Section 3.1] To estimate the parameters of the network, maximum likelihood estimation (equivalent to minimizing the negative log-likelihood) can be applied. Taking the derivative of the negative log-probability of the inputs with respect to the weights leads to a learning algorithm where the update rule for the weights of a RBM is given by:
PNG
media_image1.png
56
324
media_image1.png
Greyscale
The examiner notes that Azar teaches pretraining [Page 4, Section 3.1 Pre-training Phase] a model using maximum likelihood estimation. The examiner further notes that Goyal and Azar are both directed towards machine learning and are seen as reasonably pertinent analogous art. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Goyal’s model learning method to incorporate pre-training a baseline summarization model on the training dataset using maximum likelihood estimation (MLE) to produce a pre-trained summarization model as taught by Azar [Page 4-5, Section 3.1] to estimate the parameters of the network [Page 5, Paragraph 2]).
Furthermore, Krishnamurthy teaches initializing a first summarization model and a second summarization model with the pre-trained summarization model ([0051] The ResNet-101 backbones 504a and 504b are loaded with pre-trained weights 503. The examiner notes that Krishnamurthy teaches initializing a first model and a second model with a pre-trained model. The examiner also notes that Goyal and Krishnamurthy are both directed towards machine learning and are seen as reasonably pertinent analogous art. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Goyal’s model training to incorporate initializing a first summarization model and a second summarization model with the pre-trained summarization model as taught by Krishnamurthy [0052] in order to use a custom faster R-CNN model [0051]).
Furthermore, Dickie teaches filtering the training dataset by removing summaries with the respective first scores below a first predetermined threshold resulting in a first training data subset; filtering the training dataset by removing summaries with the respective second scores below a second predetermined threshold resulting in a second training data subset ([0088] In some instances, executed training input module 180 may also perform operations that filter the consolidated data records of first subset 182A and second subset 182B in accordance with one or more filtration criteria. By way of example, the one or more filtration criteria may cause executed training input module 180 to perform operations that exclude, from first subset 182A and second subset 182B, a consolidated data record of any customer associated with an occurrence of a default event during, or prior to, the temporal interval associated with the corresponding temporal identifier. The examiner notes that Dickie teaches filtering the a first and second training data subsets according to a filtration criteria such as a temporal identifier. The examiner also notes that Goyal and Dickie are both directed towards machine learning and are seen as reasonably pertinent analogous art. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Goyal’s data processing to incorporate filtering the training dataset by removing summaries with the respective first scores below a first predetermined threshold resulting in a first training data subset; filtering the training dataset by removing summaries with the respective second scores below a second predetermined threshold resulting in a second training data subset as taught by Dickie [0088] to exclude, from first subset 182A and second subset 182B, a consolidated data record of any customer associated with an occurrence of a default event during, or prior to, the temporal interval associated with the corresponding temporal identifier [0088]).
Furthermore, Baron teaches training a first summarization model with the first training data subset; training a second summarization model with the second training data subset ([0019] For example, the first machine learning model is trained using data of a first subset of the plurality of data categories, whereas the second machine learning model is trained using data of a second subset of the plurality of data categories. The examiner notes that Goyal and Baron are both directed towards machine learning and are seen as reasonably pertinent analogous art. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Goyal’s model learning method to incorporate training a first summarization model with the first training data subset; training a second summarization model with the second training data subset as taught by Baron [0019] in order for a plurality of machine learning models to be used to perform a clinical prediction for a patient [0019]).
Furthermore, Baron teaches determining, based on a predefined factual quality goal, a first weight and a second weight ([0119] In step 510, prediction module 306 generates a combined prediction result based on the first prediction result, the second prediction result, a first weight indicative of the first performance metric, and a second weight indicative of the second performance metric. For example, prediction module 306 can generate a combined survival rate for the patient based on a weighted average of the first survival rate and the second survival rate based on the first weight and the second weight. The first weight can reflect a first AUC of first machine learning model 304a, whereas the second weight can reflect a second AUC of second machine learning model 304b. The examiner notes that Goyal and Baron are both directed towards machine learning and are seen as reasonably pertinent analogous art. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Goyal’s model learning method to incorporate determining, based on a predefined factual quality goal, a first weight and a second weight as taught by Baron [0119] to perform a clinical prediction of the patient based on the combined prediction result [0120]).
Furthermore, Chen teaches with each parameter of the first summarization model weighted by the first weight and each parameter of the second summarization model weighted by the second weight ([0084] In a second manner, the model parameters of the matching models are weighted according to the matching coefficients to obtain weighted model parameters, and the first illumination field distribution model is generated according to the weighted model parameters. The examiner notes that Goyal and Chen are both directed towards data modeling and are seen as reasonably pertinent analogous art. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Goyal’s modeling method to incorporate with each parameter of the first summarization model weighted by the first weight and each parameter of the second summarization model weighted by the second weight as taught by Chen [0084] to generate a first illumination field distribution model is according to the weighted model parameters [0084]).
Furthermore, GUNNERUD teaches wherein the combined summarization model comprises of an equal number of parameters as the first summarization model ([0139] In this eventuality, step (iii) of the training of the first/combined model may comprise obtaining a set of first parameters the same or closely comparable to those originally generated in step (i). The examiner notes that Goyal and GUNNERUD are both directed towards data modeling and are seen as reasonably pertinent analogous art. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Goyal’s modeling method to incorporate wherein the combined summarization model comprises of an equal number of parameters as the first summarization model as taught by GUNNERUD [0139] to accurately represent the properties common to all of the plurality of production wells [0139]).
Furthermore, Wu teaches constructing a combined summarization model by ensembling the first summarization model and the second summarization model ([0013] Currently, model performance can be optimized by optimizing the single model using the model itself, but it is hard to optimize the single model continuously. Ensemble learning, or running two or more models, on the server-side can also be used to optimize model performance. Ensemble learning uses at least two models on the server-side, which is a simple but useful method for optimization because different models can be complementary. The examiner notes that Goyal and Wu are both directed towards machine learning and are seen as reasonably pertinent analogous art. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Goyal’s model design to incorporate constructing a combined summarization model by ensembling the first summarization model and the second summarization model as taught by Wu [0013] to optimize model performance [0013]).
Furthermore, Baughman teaches based at least in part on an element-wise weighted average of all parameters of the first summarization model and the second summarization model ([Col. 5, Line 35-38] In block 243, the prediction modeling engine 150 creates 35 a cross-cohort ensemble as a weighted combination of respective models from all ensembles corresponding to respective cohorts. The examiner notes that Goyal and Baughman are both directed towards machine learning and are seen as reasonably pertinent analogous art. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Goyal’s model learning method to incorporate based at least in part on an element-wise weighted average of all parameters of the first summarization model and the second summarization model as taught by Baughman [Col. 5, Line 35-38] to boosting degrees of predictor functions in case of multiple cohorts [Col. 5, Line 30-31]).
Furthermore, Krishnamurthy teaches generating, via the combined summarization model, a summary of an input document received via a data interface ([0046] FIG. 3 is a flow diagram showing a process for extracting fields of interest from a document 302, according to some implementations of the present disclosure. Information extraction is typically seen as a language problem or layout problem. Embodiments of the present disclosure provide an ensemble model which uses a language model with layout features and a vision model with language features. The examiner notes that Krishnamurthy teaches extracting text of interest using an ensemble model. The examiner also notes that Goyal and Krishnamurthy are both directed towards machine learning and are seen as reasonably pertinent analogous art. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Goyal’s model training to incorporate generating, via the combined summarization model, a summary of an input document received via a data interface as taught by Krishnamurthy [0046] to allows highlighting labels of the fields in the document [0046]).
Regarding claim 2 Goyal teaches the first factual characteristic quality is a measurement of dependency arc entailment (DAE) accuracy between a document and a respective reference summary ([P. 3594, Fig. 2]
PNG
media_image2.png
590
1088
media_image2.png
Greyscale
The examiner notes that Goyal teaches the use of an arc entailment decision between a document and a respective summary.
Regarding claim 3 Goyal teaches the second factual characteristic quality is a measurement of a number of entity tokens in a summary not present in a respective document ([P. 3594, Fig. 2]
PNG
media_image2.png
590
1088
media_image2.png
Greyscale
The examiner notes that Goyal teaches the use of entity tokens that are not present in the summary.
Regarding claim 7, Goyal teaches the method of claim 1, however, Goyal is not relied upon to explicitly teach the first weight and the second weight are dynamically adjusted based on a determination that a summary produced by a baseline summarization model does not contain factual errors, wherein the baseline summarization model is trained on unfiltered data.
However, Wu teaches the first weight and the second weight are dynamically adjusted based on a determination that a summary produced by a baseline summarization model does not contain factual errors, wherein the baseline summarization model is trained on unfiltered data ([0038] In an embodiment, ensemble inference program 112 utilizes the ensemble model to dynamically adjust the set of weights to be applied to the results from the models. In an embodiment, ensemble inference program 112 applies the first weight to the result from the first model to produce a first weighted result and applies the second weight to the result from the second model to produce a second weighted result The examiner notes that Goyal and Wu are both directed towards machine learning and are seen as reasonably pertinent analogous art. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Goyal’s model design to the first weight and the second weight are dynamically adjusted based on a determination that a summary produced by a baseline summarization model does not contain factual errors, wherein the baseline summarization model is trained on unfiltered data as taught by Wu [0038] to apply prior knowledge as a weight to a model [0035]).
Regarding claim 10, Goyal teaches the method of claim 1, however, Goyal is not relied upon to explicitly teach training the first summarization model comprises: training the first summarization model using a maximum likelihood estimation based on the first training data subset.
However, Baron teaches training the first summarization model comprises: training the first summarization model using a maximum likelihood estimation based on the first training data subset ([0018] In a case where the machine learning models are trained to predict the survival rates of the patients, the training of the machine learning models can be based on, for example, classifying patients into groups such that the similarity in the survival statistics of patients within a group is maximized, while the difference in the survival statistics of patients between the groups is maximized. The examiner notes that Goyal and Baron are both directed towards machine learning and are seen as reasonably pertinent analogous art. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Goyal’s model learning method to incorporate training the first summarization model comprises: training the first summarization model using a maximum likelihood estimation based on the first training data subset as taught by Baron [0018] in order for a plurality of machine learning models to be used to perform a clinical prediction for a patient [0019]).
Claim 12 is rejected based upon the same rationale as the rejection of claim 1 since it’s the system claim corresponding to the method claim.
Claim 13 is rejected based upon the same rationale as the rejection of claim 2 since it’s the system claim corresponding to the method claim.
Claim 14 is rejected based upon the same rationale as the rejection of claim 3 since it’s the system claim corresponding to the method claim.
Claims 8, and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Goyal (Evaluating Factuality in Generation with Dependency-level Entailment), in view of Dickie (US20220207430A1), further in view of Baron (US20210350930A1), further in view of Wu (US20220058494A1), further in view of Azar (Query-Based Single Document Summarization Using an Ensemble Noisy Auto-Encoder), and further in view of Matthews (US20110015538A1).
Regarding claim 8, Goyal teaches the method of claim 1, however, Goyal is not relied upon to explicitly teach computing an action reward score based on a summary generated by the first summarization model; computing a baseline reward score; computing a first loss based on the difference of the action reward score and the baseline reward score; and updating parameters of the first summarization model based on the first loss.
However, Matthews teaches computing an action reward score based on a summary generated by the first summarization model; computing a baseline reward score; computing a first loss based on the difference of the action reward score and the baseline reward score; and updating parameters of the first summarization model based on the first loss ([0135] In one embodiment, after obtaining a pre-training baseline measure of deviance and relying on the baseline procedure in which there is feedback of artifact-reduction only, a fourth method for calculating MMD is accomplished by calculating the difference between the baseline deviance and a deviance corresponding to the desired z-level criterion. A discrete-trials design is implemented by conducting a series of brief training trails, followed by a post-session baseline. Incremental deviance improvements are rewarded from the pre-baseline toward criterion; that is, reward is provided as the score for each short training interval approaches criterion by a fraction of the difference, and reward is withheld if there is no incremental improvement . The examiner notes that Goyal and Matthews are both directed towards data modeling and are seen as reasonably pertinent analogous art. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Goyal’s data model to incorporate computing an action reward score based on a summary generated by the first summarization model; computing a baseline reward score; computing a first loss based on the difference of the action reward score and the baseline reward score; and updating parameters of the first summarization model based on the first loss as taught by Matthews [0135] to allow the invention to provide valid and meaningful outcome measures [0134]).
Claim 18 is rejected based upon the same rationale as the rejection of claim 8 since it’s the system claim corresponding to the method claim.
Claims 9, and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Goyal (Evaluating Factuality in Generation with Dependency-level Entailment), in view of Dickie (US20220207430A1), further in view of Baron (US20210350930A1), further in view of Wu (US20220058494A1), further in view of Azar (Query-Based Single Document Summarization Using an Ensemble Noisy Auto-Encoder), and further in view of Cowan (US20230010164A1).
Regarding claim 9, Goyal teaches the method of claim 8, however, Goyal is not relied upon to explicitly teach initializing the first summarization model based on a baseline summarization model. Goyal is also not relied upon to explicitly teach computing a second loss based on a divergence between next token probabilities of the baseline summarization model and the first summarization model; and updating parameters of the first summarization model based on the second loss.
However, Azar teaches initializing the first summarization model based on a baseline summarization model ([Page 5, Section 3.2] In this phase, the weights obtained from the pretraining are used to initialize the deep AE. For that purpose, the individual RBMs are stacked on top of each other and unrolled, i.e. the recognition and generation weights are tied. The examiner notes that Goyal and Azar are both directed towards machine learning and are seen as reasonably pertinent analogous art. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Goyal’s model learning method to incorporate initializing the first summarization model based on a baseline summarization model as taught by Azar [page 5, Section 3.2] to simplify and speed up different types of Auto-Encoders [Page 5, Section 3.2]).
Furthermore, Cowan teaches computing a second loss based on a divergence between next token probabilities of the baseline summarization model and the first summarization model; and updating parameters of the first summarization model based on the second loss ([0053] In the first step, the training engine 122 updates a first set of network parameters (e.g., weighting and bias parameters) of the discriminator neural network 121b based on a loss function that measures a difference between the prediction outputted by the discriminator and whether the input to the discriminator neural network includes the high-resolution training image 110c in one of the training examples 110, or a high-resolution synthesized image 155 outputted by the generator neural network. The examiner notes that Cowan teaches updating a loss function based on the divergence of a discriminator and a generator network and updating the parameters based on said loss function. The examiner also notes that Goyal and Cowan are both directed towards machine learning and are seen as reasonably pertinent analogous art. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Goyal’s model learning method to incorporate computing a second loss based on a divergence between next token probabilities of the baseline summarization model and the first summarization model; and updating parameters of the first summarization model based on the second loss as taught by Cowan [0053] to ensure that the low-resolution training images 110a in the training examples have a same spatial resolution as the low-resolution input image [0051]).
Claim 19 is rejected based upon the same rationale as the rejection of claim 9 since it’s the system claim corresponding to the method claim.
Claims 11, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Goyal (Evaluating Factuality in Generation with Dependency-level Entailment), in view of Dickie (US20220207430A1), further in view of Baron (US20210350930A1), further in view of Wu (US20220058494A1), further in view of Baughman (US9542646).
Regarding claim 11, Goyal teaches the method of claim 1, however, Goyal is not relied upon to explicitly teach training a third summarization model with the second training data subset with an objective of entity informativeness; and constructing a combined summarization model by ensembling the first summarization model, the second summarization model, and the third summarization model.
However, Baughman teaches training a third summarization model with the second training data subset with an objective of entity informativeness; and constructing a combined summarization model by ensembling the first summarization model, the second summarization model, and the third summarization model ([P. 1, Col. 1, Line 40-51] The method for generating a drift annealed time series prediction model based on training data includes, for example: recording an ensemble of candidate models for at least one predictor variable of the training data, in a memory of a computer, responsive to creating the ensemble based on the training data, the ensemble comprising a first candidate model, a second candidate model, and a third candidate model, wherein the first candidate model is represented by a linear prediction function, the second candidate model is represented by a quadratic prediction function, and the third candidate model is represented by a cubic prediction function. The examiner notes that Goyal and Baughman are both directed towards machine learning and are seen as reasonably pertinent analogous art. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Goyal’s model learning method to incorporate training a third summarization model with the second training data subset with an objective of entity informativeness; and constructing a combined summarization model by ensembling the first summarization model, the second summarization model, and the third summarization model as taught by Baughman [P. 1, Col. 1, Line 40-51] to estimate an unknown dependent variable from the one or more dependent predictor variables [P. 1, Col. 1, Line 20-21]).
Claim 20 is rejected based upon the same rationale as the rejection of claim 11 since it’s the system claim corresponding to the method claim.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
WEN (BATCHENSEMBLE: AN ALTERNATIVE APPROACH TO
EFFICIENT ENSEMBLE AND LIFELONG LEARNING)
“WEN Teaches BatchEnsemble, an ensemble method whose computational and memory costs are significantly lower than typical ensembles”
Wilber (To Point or Not to Point: Understanding How Abstractive
Summarizers Paraphrase Text)
“Wilber teaches a method to explicitly switch between abstractive and extractive modes and to understand how abstractive summarizers paraphrase text”
Ronen (US 2022/0391591 Al)
“Ronen teaches a method for determining topics of communication transcripts using trained summarization models”
Fabbri (SummEval: Re-evaluating Summarization Evaluation)
“Fabbri re-evaluate 14 automatic evaluation metrics in a comprehensive and consistent fashion using neural summarization model outputs along with expert and crowdsourced human annotations”
Simske (US 2017 /0249289 Al)
“Simske teaches a method to generate a plurality of re-structured version of texts for each one of a plurality of different documents by applying a plurality of text summarization methods to each one of the plurality of different documents”
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SHAMCY ALGHAZZY whose telephone number is (571)272-8824. The examiner can normally be reached on M-F 7:30am-5:00pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, OMAR FERNANDEZ RIVAS can be reached on (571) 272-2589. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see https://ppair-my.uspto.gov/pair/PrivatePair. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SHAMCY ALGHAZZY/Examiner, Art Unit 2128
/OMAR F FERNANDEZ RIVAS/Supervisory Patent Examiner, Art Unit 2128
Prosecution Insights