DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 3/20/2026 has been entered.
Claims 1, 9, 10, and 18 have been amended and no claims have been added and/or canceled.
In light of Applicant’s amendment, previous claim rejections based on 35 USC 103 with respect to claims 1-20 have been withdrawn.
Claims 1-20 are pending with claims 1, 10, and 18 as independent claims.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 3, 4, and 7-8 are rejected under 35 U.S.C. 103 as being unpatentable over Huang et al. (INSET: Sentence Infilling with INter-SEntential Transformer, published June-2020, pages 1-15, hereinafter as Huang) in view of Paul et al. (COINS: Dynamically Generating Contextualized Inference Rules for Narrative Story Completion, published August-2021, pages 1-14, hereinafter as Paul).
Claim 1. A computer-implemented method comprising:
generating a long-range event relation dataset that separates event pairs by distances that satisfy a long-range event relation threshold to train an event relation extraction model to determine long-range event relations in a digital document by:
accessing, from a short-range event relation dataset, a digital document comprising an event pair that includes a first event within a first host sentence of the digital document and a second event within a second host sentence within the digital document; Huang teaches in [page 1, col. 2] fig. 1 illustrates a first sentence, indicated as preceding sentence, and a second sentence, indicated as following sentence. Thus, the first sentence and the second sentence may be a digital document comprising the first host sentence and the second host sentence. The first event may emphasize the hotel restaurant when visited UVA for a birthday party and the second event may emphasize the quality of the food in the hotel restaurant. In [page 3, col. 2] a system comprising BERT and GPT-2 models may receive or access the task of sentence infilling, which a dataset of N paragraphs, wherein the dataset has missing sentences indicated by Smk. The task may generate one or more sentences that fit the context of the other given sentences in the dataset,
generating, utilizing a generative language model, a set of synthetic sentences for inserting within the digital document between the first host sentence and the second host sentence to separate the event pair by a number of sentences that satisfies a long-range event relation threshold, wherein [the set of synthetic sentences comprises more than two synthetic sentences]; Huang teaches in [Abstract] “This task asks the model to generate intermediate missing sentences that can syntactically and semantically bridge the surrounding context.” And in [page 3, col. 2] “The criteria for successful generation are: The sentence ˆsm is fluent and meaningful. Inserting the generated sentence into the context, we obtain a semantically coherent paragraph (s1, s2, . . . , sm−1, ˆsm, sm+1, . . . , sM). ˆsm is written in the same style as contextual sentences… Since there could be multiple semantically different sentences that fit the same context well, it is not necessary for ˆsm to be close to the ground truth sm. Rather, ˆsm is considered successful as long as it satisfies the criteria above.” And in [page 2, col. 1] “MASS (Song et al., 2019) obtains sentence representations by predicting a span of missing tokens. It can be used to generate missing text, but the missing span length needs to be pre-specified.” And in [page 6, col. 1] “Our strategy of always masking the middle sentence out of 7 sentences is not only the simplest but also without loss of generality. Our model is directly applicable to the situation where we randomly mask, e.g., 3 out of 20 sentences. However, the quality of human evaluation may be affected because the patience and attention of human evaluators may decrease as the context length increases. For the effectiveness of human evaluation, we use the simplest strategy to mask sentences.” And in [page 6, col. 2] “We set the maximum number of tokens in a sentence to be 32 and the minimum number of sentences in a review to be 7 (so that the context is not too short). Any review with longer sentences or having a smaller number of sentences is discarded.”(emphasis added) Examiner note: the generated missing or intermediate sentence may be interpreted as the generated “synthetic sentences” that has been inserted between the preceding sentence(s) and the following sentence(s) as shown in figure 1. Pre-specifying the missing span length of the missing sentence may indicate that the number of missing sentences may be preset as a threshold and that the missing sentences may satisfy the criterion “threshold” of being fluent and meaningful, semantically coherent, and written in the same style. The model may be applicable to generate more than two intermediate sentences. But the quality of human evaluation may be affected because of the patience and attention of human evaluators may decrease as the number of sentences between the first and second sentences increases,
generating a long-range event relation dataset by augmenting the digital document within the short-range event relation dataset to include the set of synthetic sentences between the first host sentence and the second host sentence; Huang teaches in [Abstract] “This task asks the model to generate intermediate missing sentences that can syntactically and semantically bridge the surrounding context… generating a missing sentence that fits the context.” And in [Introduction] “the task is to generate the missing pieces that can smoothly blend into and fit the context both syntactically and semantically.” (emphasis added) Examiner note: the generated intermediate sentences may be augmented to transition from the first sentence to the second sentence as displayed in figure 1 and Table 4 in [page 9], and
generating the event relation extraction model to determine long-range event relations from digital documents by learning model parameters for the event relation extraction model from the long-range event relation dataset. Huang teaches in [Abstract] “we propose a framework to decouple the challenge and address these three aspects respectively, leveraging the power of existing largescale pre-trained models such as BERT and GPT-2. We empirically demonstrate the effectiveness of our model in learning a sentence representation for generation and further generating a missing sentence that fits the context.” And in [page 4, col. 1-2] “we learn sentence representations via autoencoding. This naturally integrates BERT and GPT-2, and combines sentence representation learning and generation… The training parameters of E and D are initialized with those of BERT and GPT-2, respectively… To train the autoencoder, we use teacher forcing and minimize the negative log-likelihood loss by (fine-tuning the parameters of E and D jointly.” And in [page 6,col. 1] “the size of the dataset (training, validation, test) is (1108134, 62543, 533) data points… The Recipe dataset is obtained from (https://commoncrawl.org), where the metadata is formatted according to Schema.org (https://schema.org/Recipe).” And in [page 6, col. 2] “Training the baseline model on our dataset, we use the same set of hyperparameters as in the original reference except that the batch size is set to 250”. (emphasis added) Examiner note: the BERT and GPT-2 model parameters may be trained on recipe documents (dataset indicated by the hyperlink) in order to generate long-range relation by combining the first sentence and the second sentence using one or more transitioning sentences as displayed in figure 1 on [page 1, col. 1] and table 4 on [page 9].
Huang does not expressly disclose wherein the set of synthetic sentences comprises more than two synthetic sentences. However, Paul, in an analogous art, teaches in [page 5088, col. 2] “Our take on this task is to incrementally generate contextualized inference rules from the given con text, and to make use of this knowledge to generate missing story sentences.” And in [page 5090] “We concatenate the generated inference rules… and store the last hidden representation MemIR ∈ IRN× L× H… where N is the number of sentences, L the maximum inference sequence length and H the hidden state dimensions. MemIR is updated with the hidden representations of inference rules in each iteration. Hence, MemIR could act as an intermediate representation, and as a basis for providing explanations for observed story sentence generations. MemIR may also be used as a memory for long-form text generation tasks, to keep track of implicit knowledge triggered by previously generated text, and could support flexible discourse serialization patterns.” (emphasis added) Examiner note: the generated inference rules may include a parameter to determine the number of the missing sentences to be generated. Accordingly, the number of the synthetic sentences can be set to be more than two sentences.
Accordingly, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of Huang with the teaching of Paul because “generating interim inference rules, and using them to guide the generation of task-specific textual outputs… We apply COINS to a Narrative Story Completion task that asks a model to complete a story with missing sentences, to produce a coherent story with plausible logical connections, causal relationships, and temporal dependencies.” Paul [Abstract].
Claim 3. The rejection of the computer-implemented method of claim 1 is incorporated, wherein augmenting the digital document comprises inserting the set of synthetic sentences between the first host sentence and the second host sentence to increase a separation between the first host sentence and the second host sentence from a short-range event relation distance to a long-range event relation distance, the long-range event relation distance satisfies the long-range event relation threshold. Huang teaches in [page 1, col. 2] figure 1 illustrates two generated intermediate sentences (synthetic sentences) have been inserted between the top first sentence and the bottom sentence to provide a smooth semantic transition from the first sentence to the second sentence, wherein the generated intermediate sentences satisfy the threshold by being fluent and meaningful, semantically coherent, and written in the same style as contextual to the other two sentences.
Claim 4. The rejection of the computer-implemented method of claim 1 is incorporated, wherein generating the set of synthetic sentences comprises utilizing the generative language model to generate a first synthetic sentence to insert between the first host sentence and the second host sentence from a set of pre-context sentences occurring before the first host sentence and a set of post-context sentences occurring after the second host sentence. Huang teaches in [page 1, col. 2] figure 1 illustrates two generated intermediate sentences (synthetic sentences) have been inserted between the top first sentence and the bottom sentence to provide a smooth semantic transition from the first sentence to the second sentence, wherein the generated intermediate sentences satisfy the threshold by being fluent and meaningful, semantically coherent, and written in the same style as contextual to the other two sentences.
Claim 7. The rejection of the computer-implemented method of claim 1 is incorporated, wherein generating the set of synthetic sentences that satisfies the long-range event relation threshold comprises:
determining a number of sentences between the first host sentence and the second host sentence; Huang teaches in [page 1, col. 2] “Figure 1: Sentence infilling: generating an intermediate sentence that provides a smooth semantic transition from the preceding to the following context.” And in [page 2, col. 2] “Our model predicts a feature vector in the latent semantic space for the missing sentence and maps the vector to text. Thus, it takes care of semantic smoothness and appropriateness… Our model allows the generation to be of arbitrary length… Compared with directly processing text, our approach significantly reduces computation time and memory usage during training, as (after pre-computing sentence features) the sequence length is the number of sentences rather than that of tokens.” (emphasis added) Examiner note: the system models generate intermediate sentences with arbitrary length and may select the number of sentences based on the semantic smoothness and appropriateness (threshold). For example, the system models selected two sentences to be the transition from the first paragraph and/or sentence to the second paragraph and/or sentence, and
generating a number of synthetic sentences to insert between the first host sentence and the second host sentence to increase the number of sentences between the first host sentence and the second host sentence to satisfy the long-range event relation threshold. Huang teaches in [page 1, col. 2] “Figure 1: Sentence infilling: generating an intermediate sentence that provides a smooth semantic transition from the preceding to the following context.” And in [page 2, col. 2] “Our model predicts a feature vector in the latent semantic space for the missing sentence and maps the vector to text. Thus, it takes care of semantic smoothness and appropriateness… Our model allows the generation to be of arbitrary length… Compared with directly processing text, our approach significantly reduces computation time and memory usage during training, as (after pre-computing sentence features) the sequence length is the number of sentences rather than that of tokens.” (emphasis added) Examiner note: the system models generate intermediate sentences with arbitrary length and may select the number of sentences based on the semantic smoothness and appropriateness (threshold). For example, the system models selected two sentences to be the transition from the first paragraph and/or sentence to the second paragraph and/or sentence.
Claim 8. The rejection of the computer-implemented method of claim 1 is incorporated, wherein generating the long-range event relation dataset comprises:
accessing, from the short-range event relation dataset, a second digital document comprising a second event pair; Huang teaches in [page 9] Table 4 provides different examples (second digital document) comprising preceding context and following context as the short-range event relation dataset,
generating, utilizing the generative language model, a second set of synthetic sentences; Huang teaches in [page 9] row five of Table 4 provide generated second intermediate sentences to provide a smooth semantic transition from the preceding content sentence/paragraph to the following context sentence/paragraph, and
generating the long-range event relation dataset by augmenting the second digital document to include the second set of synthetic sentences. Huang teaches in [page 9] row five of Table 4 provide generated second intermediate sentences to provide a smooth semantic transition from the preceding content sentence/paragraph to the following context sentence/paragraph.
Claims 2 and 5 are rejected under 35 U.S.C. 103 as being unpatentable over Huang and Paul as applied to claim 1 above, and further in view of Zhu et al. (Text Infilling, published January-2019, pages 1-10, hereinafter as Zhu).
Claim 2. The rejection of the computer-implemented method of claim 1 is incorporated, wherein Huang does not expressly disclose augmenting the digital document comprises inserting the set of synthetic sentences between the first host sentence and the second host sentence uniformly across a plurality of insertion locations. However, Zhu, in an analogous art, teaches in [page 2, col. 2] “Note that for each input template, the number of blanks and their positions are known, but the number of tokens to be infilled for each blank is not given. A model must decide by itself how many tokens to generate for a blank.” (emphasis added) Examiner note: the blanks may be insertion locations for generated sentences.
Accordingly, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of Huang with the teaching of Zhu because Zhu teaches insertion locations indicated by blank placeholders for inserting generated sentences. The technique would restoring historical damaged documents, contracts or article writing with templates, text editing and so forth. Zhu [page 1, col. 2].
Claim 5. The rejection of the computer-implemented method of claim 4 is incorporated, Huang does not expressly disclose wherein generating the set of synthetic sentences further comprises utilizing the generative language model to generate a second synthetic sentence to insert between the first host sentence and the second host sentence from a modified set of pre-context sentences that includes the first synthetic sentence. However, Zhu, in an analogous art, teaches in [page 3, col. 1] “A blank is completed when a special <End-of-Blank>token is generated. Then
the model moves on to fill other blanks.” (emphasis added) Examiner note: the system appears that a first token/sentence may be generated a first blank/placeholder first, then the system may generate a second token/sentence to fill in the next blank/placeholder. Thus, the second blank/placeholder may be filled with a second generated token/sentence to the modified first sentence with the first filled in token/sentence. In other words, the Huang reference may be modified such that the first sentence “she was extremely happy with our hotel” may be generated and filled in first position/placeholder and then the second generated sentence “and we had a complimentary buffet” may be filled in the next blank/placeholder after the first modified first sentence.
Accordingly, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of Huang with the teaching of Zhu because Zhu teaches insertion locations indicated by blank placeholders for inserting generated sentences. The technique would restoring historical damaged documents, contracts or article writing with templates, text editing and so forth. Zhu [page 1, col. 2].
Claim 6 is rejected under 35 U.S.C. 103 as being unpatentable over Huang and Paul, as applied to claim 1 above, and further in view of Yasui et al. (Using Semantic Similarity as Reward for Reinforcement Learning in Sentence Generation, published July 2019, pages 1-8, hereinafter as Yasui).
Claim 6. The rejection of the computer-implemented method of claim 5 is incorporated, Huang does not expressly disclose wherein generating the set of synthetic sentences further comprises generating the set of synthetic sentences to satisfy a performance-based reward function. However, Yasui, in an analogous art, teaches in [page 3, col. 2] “In the context of sentence generation, the goal of the agent is to maximize the expectation of the reward provided as the function r… The loss function is the negative of the reward expectation, but the expectation is typically approximated
by a single sample sequence… where rb is the baseline reward which counters the large variance of reward caused by sampling. rb can be any function that does not contain the parameter of the sentence generation model, but usually is kept to a simple model or function to not hinder the training.” (emphasis added) Examiner note: semantic similarity may be used in generating a sentence that may be semantically similar the input text (sentences).
Accordingly, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of Huang with the teaching of Yasui because “The advantages of using RL are that the reward for an action does not have to be returned spontaneously and that the reward function does not have to be differentiable by the parameter of the agent model. Because of these advantages, RL has often been used as a means to train sentence generation model against sentence-level metrics… Sentence-level metrics commonly used in RL settings, such as BLEU, ROUGE and METEOR, are typically not differentiable, and thus are not usable under the regular supervised training.” Yasui [page 3, col. 1-2].
Claim 9 is rejected under 35 U.S.C. 103 as being unpatentable over Huang and Paul as applied to claim 1 above, and further in view of Veyseh et al. (US 2022/0050967, published 2/17/2022, hereinafter as Veyseh).
Claim 9. The rejection of the computer-implemented method of claim 1 is incorporated, further comprising
Huang does not explicitly disclose
generating, utilizing the event relation extraction model, an event relation graph indicating a relationship between a pair of events separated by a number of sentences that satisfies the long-range event relation threshold, wherein the event relation graph comprises nodes representing events and edges representing relationships between nodes. However, Veyseh, in an analogous art, teaches in [0053-0056] “FIG. 2A illustrates that the document 200a includes a word sequence arranged in a plurality of sentences 201a-201n. In particular, the document 200a includes the sentences 201a-201n arranged in a specific order. In one or more embodiments, the definition extraction system 102 can determine the boundaries of the sentences 201a-201n based on natural language principles including punctuation, capitalization, or other elements of speech or writing that define separate sentences in a word sequence… the definition extraction system 102 can also utilize the machine-learning model 114 to extract term definitions from individual sentences.” And in [0084-0088] “the global dependency tree 338 can include one or more dependency trees associated with one or more sentences of the word sequence. For instance, FIG. 3B illustrates that the global dependency tree 338 includes a first dependency tree 340a associated with a first sentence from the word sequence and a second dependency tree 340b associated with a second sentence from the word sequence… the definition extraction system 102 can generate a dependency tree for a sentence in a word sequence by analyzing the speech/grammatical structure of the sentence. For example, the definition extraction system 102 can utilize natural language processing to parse text in the word sequence to determine sentence boundaries within the word sequence and sentence structure within each sentence… the first dependency tree 340a includes a plurality of nodes associated with words from the first sentence… To capture possible dependency path information across a plurality of sentences within a word sequence, the definition extraction system 102 can link a plurality of dependency trees together. Specifically, FIG. 3B illustrates that the definition extraction system 102 generates a global root node 346 and connects the global root node 346 to the first root node 342a of the first dependency tree 340a and the second root node 342b of the second dependency tree 340b. Thus, the definition extraction system 102 can link the first dependency tree 340a to the second dependency tree 340b via the global root node 346.” (emphasis added) Examiner note: global dependency tree 338 generate an event graph indicating a relationship between a pair of events such as the first dependency tree 340a, representing one sentence, and the second dependency tree 340b, representing another sentence, wherein the global dependency tree 338 comprises nodes and edges representing relationships through the global root node 346, as shown in fig. 3b.
Accordingly, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of Huang with the teaching of Veyseh because “the definition extraction system uses a machine-learning model that exploits both (i) global structures of sentences from the source document and (ii) semantic consistencies between terms and corresponding definitions to improve the feature representations used for extracting term definitions. To capture such improved feature representations, in some implementations, the definition extraction system uses a multi-task-machine-learning model—comprising a graph convolutional neural network—to generate vectors predicting dependency paths between terms and definitions as a basis for extracting term definitions.” Veyseh [0012].
Claims 10-14 and 16-19 are rejected under 35 U.S.C. 103 as being unpatentable over Huang and further in view of Veyseh et al. (US 2022/0050967, published 2/17/2022, hereinafter as Veyseh).
Claim 10. A system comprising:
one or more memory devices comprising an event relation extraction model and a shortrange event relation dataset that includes event pairs that are within a threshold number of sentences apart; Huang teaches in [page 2, col. 1-2] “Compared with directly processing text, our approach significantly reduces computation time and memory usage during training… we propose INter-Sentential Transformer (INSET), a novel approach to sentence infilling. Our model first produces sentence-level semantic features that capsulate the missing high-level information. Then, grounded on the predicted semantic features, the model generates the syntactic and lexical features to embody the predicted sentence. Specifically, understanding, planning, and generation are handled by three modules in a synergistic manner:
• a BERT-based encoder to map each sentence to the latent semantic space.
• a sentence-level semantic planner to infer the missing information that can bridge the semantics of preceding and following context.
• a GPT-based generator (decoder) to map semantic features back to the text domain.” (emphasis added) Examiner note: memory device may be used to process text using language models and dataset at least from TripAdvisor as shown in figure 1, and
one or more processors configured to cause the system to:
access, from the short-range event relation dataset, a digital document comprising an event pair that includes a first event within a first host sentence of the digital document and a second event within a second host sentence within the digital document; Huang teaches in [page 1, col. 2] fig. 1 illustrates a first sentence, indicated as preceding sentence, and a second sentence, indicated as following sentence. Thus, the first sentence and the second sentence may be a digital document comprising the first host sentence and the second host sentence. The first event may emphasize the hotel restaurant when visited UVA for a birthday party and the second event may emphasize the quality of the food in the hotel restaurant. In [page 3, col. 2] a system comprising BERT and GPT-2 models may receive or access the task of sentence infilling, which a dataset of N paragraphs, wherein the dataset has missing sentences indicated by Smk. The task may generate one or more sentences that fit the context of the other given sentences in the dataset,
generate, utilizing a generative language model, a set of synthetic sentences for [uniformly inserting within the digital document across a plurality of insertion locations between the first host sentence and the second host sentence] to separate the event pair by a number of sentences that satisfies a long-range event relation threshold; Huang teaches in [Abstract] “This task asks the model to generate intermediate missing sentences that can syntactically and semantically bridge the surrounding context.” And in [page 3, col. 2] “The criteria for successful generation are: The sentence ˆsm is fluent and meaningful. Inserting the generated sentence into the context, we obtain a semantically coherent paragraph (s1, s2, . . . , sm−1, ˆsm, sm+1, . . . , sM). ˆsm is written in the same style as contextual sentences… Since there could be multiple semantically different sentences that fit the same context well, it is not necessary for ˆsm to be close to the ground truth sm. Rather, ˆsm is considered successful as long as it satisfies the criteria above.” (emphasis added) Examiner note: the generated one or more sentences satisfy the criterion of being fluent and meaningful, semantically coherent, and written in the same style. Thus, the criterion may be the long range relation threshold,
generate a long-range event relation dataset by augmenting the digital document within the short-range event relation dataset to include the set of synthetic sentences between the first host sentence and the second host sentence; Huang teaches in [Abstract] “This task asks the model to generate intermediate missing sentences that can syntactically and semantically bridge the surrounding context… generating a missing sentence that fits the context.” And in [Introduction] “the task is to generate the missing pieces that can smoothly blend into and fit the context both syntactically and semantically.” (emphasis added) Examiner note: the generated intermediate sentences may be augmented to transition from the first sentence to the second sentence as displayed in figure 1 and Table 4 in [page 9],
modify the event relation extraction model to determine long-range event relations from digital documents by learning model parameters for the event relation extraction model from the long-range event relation dataset; Huang teaches in [Abstract] “we propose a framework to decouple the challenge and address these three aspects respectively, leveraging the power of existing largescale pre-trained models such as BERT and GPT-2. We empirically demonstrate the effectiveness of our model in learning a sentence representation for generation and further generating a missing sentence that fits the context.” And in [page 4, col. 1-2] “we learn sentence representations via autoencoding. This naturally integrates BERT and GPT-2, and combines sentence representation learning and generation… The training parameters of E and D are initialized with those of BERT and GPT-2, respectively… To train the autoencoder, we use teacher forcing and minimize the negative log-likelihood loss by (fine-tuning the parameters of E and D jointly.” And in [page 6,col. 1] “the size of the dataset (training, validation, test) is (1108134, 62543, 533) data points… The Recipe dataset is obtained from (https://commoncrawl.org), where the metadata is formatted according to Schema.org (https://schema.org/Recipe).” And in [page 6, col. 2] “Training the baseline model on our dataset, we use the same set of hyperparameters as in the original reference except that the batch size is set to 250”. (emphasis added) Examiner note: the BERT and GPT-2 model parameters may be trained (modifying parameters) on recipe documents (dataset indicated by the hyperlink) in order to generate long-range relation by combining the first sentence and the second sentence using one or more transitioning sentences as displayed in figure 1 on [page 1, col. 1] and table 4 on [page 9], and
Huang does not expressly
generate, utilizing the event relation extraction model, an event relation graph indicating a relationship between a long-range event pair, wherein the event relation graph comprises nodes representing events and edges representing relationships between nodes. However, Veyseh, in an analogous art, teaches in [0053-0056] “FIG. 2A illustrates that the document 200a includes a word sequence arranged in a plurality of sentences 201a-201n. In particular, the document 200a includes the sentences 201a-201n arranged in a specific order. In one or more embodiments, the definition extraction system 102 can determine the boundaries of the sentences 201a-201n based on natural language principles including punctuation, capitalization, or other elements of speech or writing that define separate sentences in a word sequence… the definition extraction system 102 can also utilize the machine-learning model 114 to extract term definitions from individual sentences.” And in [0084-0088] “the global dependency tree 338 can include one or more dependency trees associated with one or more sentences of the word sequence. For instance, FIG. 3B illustrates that the global dependency tree 338 includes a first dependency tree 340a associated with a first sentence from the word sequence and a second dependency tree 340b associated with a second sentence from the word sequence… the definition extraction system 102 can generate a dependency tree for a sentence in a word sequence by analyzing the speech/grammatical structure of the sentence. For example, the definition extraction system 102 can utilize natural language processing to parse text in the word sequence to determine sentence boundaries within the word sequence and sentence structure within each sentence… the first dependency tree 340a includes a plurality of nodes associated with words from the first sentence… To capture possible dependency path information across a plurality of sentences within a word sequence, the definition extraction system 102 can link a plurality of dependency trees together. Specifically, FIG. 3B illustrates that the definition extraction system 102 generates a global root node 346 and connects the global root node 346 to the first root node 342a of the first dependency tree 340a and the second root node 342b of the second dependency tree 340b. Thus, the definition extraction system 102 can link the first dependency tree 340a to the second dependency tree 340b via the global root node 346.” (emphasis added) Examiner note: global dependency tree 338 generate an event graph indicating a relationship between a pair of events such as the first dependency tree 340a, representing one sentence, and the second dependency tree 340b, representing another sentence, wherein the global dependency tree 338 comprises nodes and edges representing relationships through the global root node 346, as shown in fig. 3b.
Accordingly, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of Huang with the teaching of Veyseh because “the definition extraction system uses a machine-learning model that exploits both (i) global structures of sentences from the source document and (ii) semantic consistencies between terms and corresponding definitions to improve the feature representations used for extracting term definitions. To capture such improved feature representations, in some implementations, the definition extraction system uses a multi-task-machine-learning model—comprising a graph convolutional neural network—to generate vectors predicting dependency paths between terms and definitions as a basis for extracting term definitions.” Veyseh [0012].
a set of synthetic sentences for uniformly inserting within the digital document across a plurality of insertion locations between the first host sentence and the second host sentence to separate the event pair by a number of sentences. Further, Huang teaches in [page 2, col. 1] “sentence-level semantic planner to infer the missing information that can bridge the semantics of preceding and following context.” (emphasis added) Examiner note: the inferred missing sentences may be inserted in locations between preceding and following sentences.
Claim 11. The rejection of the system of claim 10 is incorporated, wherein the one or more processors are configured to cause the system to:
determine a number of sentences between the first host sentence and the second host sentence; Huang teaches in [page 1, col. 2] “Figure 1: Sentence infilling: generating an intermediate sentence that provides a smooth semantic transition from the preceding to the following context.” And in [page 2, col. 2] “Our model predicts a feature vector in the latent semantic space for the missing sentence and maps the vector to text. Thus, it takes care of semantic smoothness and appropriateness… Our model allows the generation to be of arbitrary length… Compared with directly processing text, our approach significantly reduces computation time and memory usage during training, as (after pre-computing sentence features) the sequence length is the number of sentences rather than that of tokens.” (emphasis added) Examiner note: the system models generate intermediate sentences with arbitrary length and may select the number of sentences based on the semantic smoothness and appropriateness (threshold). For example, the system models selected two sentences to be the transition from the first paragraph and/or sentence to the second paragraph and/or sentence,
generate the set of synthetic sentences to insert between the first host sentence and the second host sentence to increase the number of sentences between the first host sentence and the second host sentence to satisfy the long-range event relation threshold; Huang teaches in [page 1, col. 2] “Figure 1: Sentence infilling: generating an intermediate sentence that provides a smooth semantic transition from the preceding to the following context.” And in [page 2, col. 2] “Our model predicts a feature vector in the latent semantic space for the missing sentence and maps the vector to text. Thus, it takes care of semantic smoothness and appropriateness… Our model allows the generation to be of arbitrary length… Compared with directly processing text, our approach significantly reduces computation time and memory usage during training, as (after pre-computing sentence features) the sequence length is the number of sentences rather than that of tokens.” (emphasis added) Examiner note: the system models generate intermediate sentences with arbitrary length and may select the number of sentences based on the semantic smoothness and appropriateness (threshold). For example, the system models selected two sentences to be the transition from the first paragraph and/or sentence to the second paragraph and/or sentence, and
insert the set of synthetic sentences between the first host sentence and the second host sentence. Huang teaches in [page 2, col. 1] “sentence-level semantic planner to infer the missing information that can bridge the semantics of preceding and following context.” (emphasis added) Examiner note: the inferred missing sentences may be inserted in locations between preceding and following sentences.
Claim 12. The rejection of the system of claim 10 is incorporated, wherein the one or more processors are configured to cause the system to generate the set of synthetic sentences by:
generating, utilizing the generative language model, a first synthetic sentence to insert between the first host sentence and the second host sentence from a set of pre-context sentences occurring before the first host sentence and a set of post-context sentences occurring after the second host sentence; Huang teaches in [Introduction and page 2, col. 1] “generate the missing pieces that can smoothly blend into and fit the context both syntactically and semantically… sentence-level semantic planner to infer the missing information that can bridge the semantics of preceding and following context.” (emphasis added) Examiner note: the inferred missing sentences may be inserted in locations between preceding and following input sentences such that a first inferred sentence would fit and/or blend the context of the preceding sentence, which is the preceding input sentence, and
generating, utilizing the generative language model, a second synthetic sentence to insert between the first host sentence and the second host sentence from a modified set of pre-context sentences that includes the first synthetic sentence. Huang teaches in [Introduction and page 2, col. 1] “generate the missing pieces that can smoothly blend into and fit the context both syntactically and semantically… sentence-level semantic planner to infer the missing information that can bridge the semantics of preceding and following context.” (emphasis added) Examiner note: the inferred missing sentences may be inserted in locations between preceding and following sentences such that a second inferred sentence would fit and/or blend the context of the preceding sentence, which is the first inferred sentence.
Claim 13. The rejection of the system of claim 10 is incorporated, wherein the one or more processors are configured to cause the system to generate the set of synthetic sentences via successively shifting a set of pre-context sentences by:
utilizing the generative language model to generate a first synthetic sentence from a set of pre-context sentences occurring before the first host sentence and a set of post-context sentences occurring after the second host sentence; inserting the first synthetic sentence between the first host sentence and the second host sentence; Huang teaches in [Abstract and page 1, col. 2] “This task asks the model to generate intermediate missing sentences that can syntactically and semantically bridge the surrounding context. Solving the sentence infilling task requires techniques in natural language processing ranging from understanding to discourse-level planning to generation. In this paper, we propose a framework to decouple the challenge and address these three aspects respectively, leveraging the power of existing largescale pre-trained models such as BERT and GPT-2. We empirically demonstrate the effectiveness of our model in learning a sentence representation for generation and further generating a missing sentence that fits the context… generating an intermediate sentence that provides a smooth semantic transition from the preceding to the following context. This example is generated by our model on the TripAdvisor dataset.” (emphasis added) Examiner note: two generated intermediate sentences have been positioned to transition pre-context on the top first sentence/paragraph and post=context on the bottom second sentence/paragraph. The pre-context may be indicated the keywords in red color and the post-context may be indicated by the keywords in blue color,
utilizing the generative language model to generate a second synthetic sentence from a modified set of pre-context sentences different than the set of pre-context sentences, wherein the modified set of pre-context sentences includes the first synthetic sentence; and inserting the second synthetic sentence after the first synthetic sentence and between the first host sentence and the second host sentence. Huang teaches in [Introduction and page 2, col. 1] “generate the missing pieces that can smoothly blend into and fit the context both syntactically and semantically… sentence-level semantic planner to infer the missing information that can bridge the semantics of preceding and following context.” (emphasis added) Examiner note: the inferred missing sentences may be inserted in locations between preceding and following sentences such that a second inferred sentence would fit and/or blend the context of the preceding sentence, which is the first inferred sentence.
Claim 14. The rejection of the system of claim 10 is incorporated, wherein augmenting the digital document to include the set of synthetic sentences further causes the system to:
generate, as part of the set of synthetic sentences, a first number of synthetic sentences to insert at a first insertion point defined by the first host sentence, wherein the first number of synthetic sentences depends on the long-range event relation threshold; Huang teaches in [page 1, col. 2] figure 1 displays how the two intermediate sentences have been generated to provide a smooth semantic transition between the first sentence and/or paragraph, which talks about the keywords “hotel” and “friend” as the keywords provide context between the first sentence/paragraph and the first generated intermediate sentence, and
generate, as a further part of the set of synthetic sentences, a second number of synthetic sentences to insert at a second insertion point defined by the second host sentence. Huang teaches in [page 1, col. 2] figure 1 displays how the two intermediate sentences have been generated to provide a smooth semantic transition between the first sentence and/or paragraph, which talks about the hotel, and the second sentence and/or paragraph, which talks about the food, wherein the keywords “food” and “happy” provide context for generating the second sentence in the generated intermediate sentences.
Claim 16. The rejection of the system of claim 10 is incorporated, wherein the one or more processors are configured to cause the system to:
access, from the short-range event relation dataset, a second digital document comprising a second event pair; and generate, utilizing the generative language model, a second set of synthetic sentences. Huang teaches in [page 9] row five of Table 4 provide second generated intermediate sentences to provide a smooth semantic transition from the preceding context sentence/paragraph to the following context sentence/paragraph, wherein the second event pair may be “dinner” event in the preceding context and “describing the type of food in the dinner” event in the following context as shown in table 4.
Claim 17. The rejection of the system of claim 16 is incorporated, wherein the one or more processors are configured to cause the system to generate the long-range event relation dataset by augmenting the second digital document to include the second set of synthetic sentences. Huang teaches in [page 9] row five of Table 4 provide second generated intermediate sentences to provide a smooth semantic transition from the preceding context sentence/paragraph to the following context sentence/paragraph, wherein the second event pair may be “dinner” event in the preceding context and “describing the type of food in the dinner” event in the following context as shown in table 4. The generated sentences, in row 5 in table 4, may be augmented as an intermediate sentence between the proceeding context sentence/paragraph and the following context sentence/paragraph.
Claim 18. A non-transitory computer-readable medium storing executable instructions which, when executed by a processing device, cause the processing device to perform operations comprising:
accessing a digital document comprising a first sentence and a second sentence separated by a number of sentences that satisfies a long-range event relation threshold; Huang teaches in [Abstract] “This task asks the model to generate intermediate missing sentences that can syntactically and semantically bridge the surrounding context.” And in [page 3, col. 2] “The criteria for successful generation are: The sentence ˆsm is fluent and meaningful. Inserting the generated sentence into the context, we obtain a semantically coherent paragraph (s1, s2, . . . , sm−1, ˆsm, sm+1, . . . , sM). ˆsm is written in the same style as contextual sentences… Since there could be multiple semantically different sentences that fit the same context well, it is not necessary for ˆsm to be close to the ground truth sm. Rather, ˆsm is considered successful as long as it satisfies the criteria above.” (emphasis added) Examiner note: the generated one or more sentences satisfy the criterion of being fluent and meaningful, semantically coherent, and written in the same style. Thus, the criterion may be the long range relation threshold,
determining, utilizing an event relation extraction model comprising parameters learned from a synthetically augmented long-range event relation dataset, a long-range event pair that includes the first sentence and the second sentence from the digital document; Huang teaches in [Abstract] “we propose a framework to decouple the challenge and address these three aspects respectively, leveraging the power of existing largescale pre-trained models such as BERT and GPT-2. We empirically demonstrate the effectiveness of our model in learning a sentence representation for generation and further generating a missing sentence that fits the context.” And in [page 4, col. 1-2] “we learn sentence representations via autoencoding. This naturally integrates BERT and GPT-2, and combines sentence representation learning and generation… The training parameters of E and D are initialized with those of BERT and GPT-2, respectively… To train the autoencoder, we use teacher forcing and minimize the negative log-likelihood loss by (fine-tuning the parameters of E and D jointly.” And in [page 6,col. 1] “the size of the dataset (training, validation, test) is (1108134, 62543, 533) data points… The Recipe dataset is obtained from (https://commoncrawl.org), where the metadata is formatted according to Schema.org (https://schema.org/Recipe).” And in [page 6, col. 2] “Training the baseline model on our dataset, we use the same set of hyperparameters as in the original reference except that the batch size is set to 250”. (emphasis added) Examiner note: the BERT and GPT-2 model parameters may be trained on recipe documents (dataset indicated by the hyperlink) in order to generate long-range relation by combining the first sentence and the second sentence using one or more transitioning sentences as displayed in figure 1 on [page 1, col. 1] and table 4 on [page 9], and
Huang does not expressly disclose
generating an event relation graph indicating a long-range event relation between the long-range event pair, wherein the event relation graph comprises nodes representing events and edges representing relationships between nodes. However, Veyseh, in an analogous art, teaches in [0053-0056] “FIG. 2A illustrates that the document 200a includes a word sequence arranged in a plurality of sentences 201a-201n. In particular, the document 200a includes the sentences 201a-201n arranged in a specific order. In one or more embodiments, the definition extraction system 102 can determine the boundaries of the sentences 201a-201n based on natural language principles including punctuation, capitalization, or other elements of speech or writing that define separate sentences in a word sequence… the definition extraction system 102 can also utilize the machine-learning model 114 to extract term definitions from individual sentences.” And in [0084-0088] “the global dependency tree 338 can include one or more dependency trees associated with one or more sentences of the word sequence. For instance, FIG. 3B illustrates that the global dependency tree 338 includes a first dependency tree 340a associated with a first sentence from the word sequence and a second dependency tree 340b associated with a second sentence from the word sequence… the definition extraction system 102 can generate a dependency tree for a sentence in a word sequence by analyzing the speech/grammatical structure of the sentence. For example, the definition extraction system 102 can utilize natural language processing to parse text in the word sequence to determine sentence boundaries within the word sequence and sentence structure within each sentence… the first dependency tree 340a includes a plurality of nodes associated with words from the first sentence… To capture possible dependency path information across a plurality of sentences within a word sequence, the definition extraction system 102 can link a plurality of dependency trees together. Specifically, FIG. 3B illustrates that the definition extraction system 102 generates a global root node 346 and connects the global root node 346 to the first root node 342a of the first dependency tree 340a and the second root node 342b of the second dependency tree 340b. Thus, the definition extraction system 102 can link the first dependency tree 340a to the second dependency tree 340b via the global root node 346.” (emphasis added) Examiner note: global dependency tree 338 generate an event graph indicating a relationship between a pair of events such as the first dependency tree 340a, representing one sentence, and the second dependency tree 340b, representing another sentence, wherein the global dependency tree 338 comprises nodes and edges representing relationships through the global root node 346, as shown in fig. 3b.
Accordingly, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of Huang with the teaching of Veyseh because “the definition extraction system uses a machine-learning model that exploits both (i) global structures of sentences from the source document and (ii) semantic consistencies between terms and corresponding definitions to improve the feature representations used for extracting term definitions. To capture such improved feature representations, in some implementations, the definition extraction system uses a multi-task-machine-learning model—comprising a graph convolutional neural network—to generate vectors predicting dependency paths between terms and definitions as a basis for extracting term definitions.” Veyseh [0012].
a first sentence and a second sentence separated by a number of sentences that satisfies a long-range event relation threshold. Huang teaches in [Introduction and page 2, col. 1] “generate the missing pieces that can smoothly blend into and fit the context both syntactically and semantically… sentence-level semantic planner to infer the missing information that can bridge the semantics of preceding and following context.” (emphasis added) Examiner note: the inferred missing sentences may be inserted in locations between preceding and following input sentences such that a first inferred sentence would fit and/or blend the context of the preceding sentence, which is the preceding input sentence.
Claim 19. The rejection of the non-transitory computer-readable medium of claim 18 is incorporated, Huang does not expressly wherein the operations further comprise providing the event relation graph to a client device in response to receiving a query pertaining to the digital document from the client device. However, Veyseh, in an analogous art, teaches in [0053-0056] “FIG. 2A illustrates that the document 200a includes a word sequence arranged in a plurality of sentences 201a-201n. In particular, the document 200a includes the sentences 201a-201n arranged in a specific order. In one or more embodiments, the definition extraction system 102 can determine the boundaries of the sentences 201a-201n based on natural language principles including punctuation, capitalization, or other elements of speech or writing that define separate sentences in a word sequence… the definition extraction system 102 can also utilize the machine-learning model 114 to extract term definitions from individual sentences.” And in [0084-0088] “the global dependency tree 338 can include one or more dependency trees associated with one or more sentences of the word sequence. For instance, FIG. 3B illustrates that the global dependency tree 338 includes a first dependency tree 340a associated with a first sentence from the word sequence and a second dependency tree 340b associated with a second sentence from the word sequence… the definition extraction system 102 can generate a dependency tree for a sentence in a word sequence by analyzing the speech/grammatical structure of the sentence. For example, the definition extraction system 102 can utilize natural language processing to parse text in the word sequence to determine sentence boundaries within the word sequence and sentence structure within each sentence… the first dependency tree 340a includes a plurality of nodes associated with words from the first sentence… To capture possible dependency path information across a plurality of sentences within a word sequence, the definition extraction system 102 can link a plurality of dependency trees together. Specifically, FIG. 3B illustrates that the definition extraction system 102 generates a global root node 346 and connects the global root node 346 to the first root node 342a of the first dependency tree 340a and the second root node 342b of the second dependency tree 340b. Thus, the definition extraction system 102 can link the first dependency tree 340a to the second dependency tree 340b via the global root node 346.” (emphasis added) Examiner note: global dependency tree 338 generate an event graph indicating a relationship between a pair of events such as the first dependency tree 340a, representing one sentence, and the second dependency tree 340b, representing another sentence, wherein the global dependency tree 338 comprises nodes and edges representing relationships through the global root node 346, as shown in fig. 3b.
Accordingly, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of Huang with the teaching of Veyseh because “the definition extraction system uses a machine-learning model that exploits both (i) global structures of sentences from the source document and (ii) semantic consistencies between terms and corresponding definitions to improve the feature representations used for extracting term definitions. To capture such improved feature representations, in some implementations, the definition extraction system uses a multi-task-machine-learning model—comprising a graph convolutional neural network—to generate vectors predicting dependency paths between terms and definitions as a basis for extracting term definitions.” Veyseh [0012].
Claims 15 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Huang and Veyseh, as applied to claim 10 above, and further in view of Yasui et al. (Using Semantic Similarity as Reward for Reinforcement Learning in Sentence Generation, published July 2019, pages 1-8, hereinafter as Yasui).
Claim 15. The rejection of the system of claim 10 is incorporated, wherein the one or more processors are configured to cause the system to generate, as part of the set of synthetic sentences, a first synthetic sentence and a second synthetic sentence based on [a performance-based reward function]. Huang teaches in [Abstract] “This task asks the model to generate intermediate missing sentences that can syntactically and semantically bridge the surrounding context.” And in [page 3, col. 2] “The criteria for successful generation are: The sentence ˆsm is fluent and meaningful. Inserting the generated sentence into the context, we obtain a semantically coherent paragraph (s1, s2, . . . , sm−1, ˆsm, sm+1, . . . , sM). ˆsm is written in the same style as contextual sentences… Since there could be multiple semantically different sentences that fit the same context well, it is not necessary for ˆsm to be close to the ground truth sm. Rather, ˆsm is considered successful as long as it satisfies the criteria above.” (emphasis added) Examiner note: the generated one or more sentences satisfy the criterion of being fluent and meaningful, semantically coherent, and written in the same style. Thus, the criterion may be the long range relation threshold,
Huang does not expressly disclose a performance-based reward function. However, Yasui, in an analogous art, teaches in [page 3, col. 1-2] “Reinforcement learning, a framework in which the agent must choose a series of discrete actions to maximize the reward returned from its surrounding environment, is one of such approaches… In the context of sentence generation, the goal of the agent is to maximize the expectation of the reward provided as the function r… The loss function is the negative of the reward expectation, but the expectation is typically approximated by a single sample sequence… where rb is the baseline reward which counters the large variance of reward caused by sampling. rb can be any function that does not contain the parameter of the sentence generation model, but usually is kept to a simple model or function to not hinder the training.” (emphasis added) Examiner note: semantic similarity may be used in generating a sentence that may be semantically similar the input text (sentences).
Accordingly, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of Huang with the teaching of Yasui because “The advantages of using RL are that the reward for an action does not have to be returned spontaneously and that the reward function does not have to be differentiable by the parameter of the agent model. Because of these advantages, RL has often been used as a means to train sentence generation model against sentence-level metrics… Sentence-level metrics commonly used in RL settings, such as BLEU, ROUGE and METEOR, are typically not differentiable, and thus are not usable under the regular supervised training.” Yasui [page 3, col. 1-2].
Claim 20. The rejection of the non-transitory computer-readable medium of claim 19 is incorporated, wherein the event relation extraction model comprises model parameters learned by:
generating a set of synthetic sentences for inserting within a digital document of a short-range event relation dataset; Huang teaches in [page 1, col. 2] “Figure 1: Sentence infilling: generating an intermediate sentence that provides a smooth semantic transition from the preceding to the following context. This example is generated by our model on the TripAdvisor dataset. The colors mark the correspondence between the generated sentence and the context.” (emphasis added) Examiner note: the preceding and following context may be a digital document of a short-range event relation dataset and the generated intermediate sentences may be the synthetic sentences to be inserted between the preceding and following sentences,
generating a long-range event relation dataset by augmenting the digital document within the short-range event relation dataset to include the set of synthetic sentences; Huang teaches in [page 1, col. 2] “Figure 1: Sentence infilling: generating an intermediate sentence that provides a smooth semantic transition from the preceding to the following context. This example is generated by our model on the TripAdvisor dataset. The colors mark the correspondence between the generated sentence and the context.” (emphasis added) Examiner note: the preceding and following context may be a digital document of a short-range event relation dataset and the generated intermediate sentences may be the synthetic sentences to be inserted between the preceding and following sentences. When inserting the generated intermediate sentences between the preceding and following sentences, a long-range event relation dataset may be generated by providing a smooth semantic transition from the preceding sentence/paragraph to the following sentence/paragraph, wherein the colors in the keywords provide context, and
Huang does not expressly disclose updating the model parameters based on the long-range event relation dataset according to a reinforcement learning algorithm. However, Yasui, in an analogous art, teaches in [Abstract] “We use the BERT-based scorer fine-tuned to the Semantic Textual Similarity (STS) task for semantic similarity estimation, and train the model with the estimated scores through reinforcement learning (RL)… In the proposed framework, semantic similarity of sentence pairs is estimated by a BERT-based (Devlin et al., 2018) regression model fine-tuned against Semantic Textual Similarity (Agirre et al., 2012) dataset, and the resulting score is passed back to the model using reinforcement learning strategies.” And in [page 5, col. 1-2] “the model training is separated into three stages… The final stage is the RL stage where we apply REINFORCE to NMT model. The loss function for REINFORCE is rewritten from Eq. (5)… where Rt is the difference between the reward rψ and the expected reward Ωω… During the RL stage, the reward prediction model Ωω is trained using the MSE loss”. (emphasis added) Examiner note: Bidirectional Encoder Representations from Transformer (BERT), may be fine-tuned (updated) utilizing Reinforcement Learning strategies, for sentence generation.
Accordingly, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of Huang with the teaching of Yasui because “The advantages of using RL are that the reward for an action does not have to be returned spontaneously and that the reward function does not have to be differentiable by the parameter of the agent model. Because of these advantages, RL has often been used as a means to train sentence generation model against sentence-level metrics… Sentence-level metrics commonly used in RL settings, such as BLEU, ROUGE and METEOR, are typically not differentiable, and thus are not usable under the regular supervised training.” Yasui [page 3, col. 1-2].
Response to Arguments
Applicant’s arguments with respect to claims 1have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Argument: Applicant argues “The combination of references fails to teach or suggest "generating… a set of synthetic sentences… that satisfies the long-range event relation threshold, wherein the set of synthetic sentences comprises more than two synthetic sentences," as recited by currently amended independent claim 1.”
Response: the specification of the instant application recites “[0061] In one or more embodiments, the long-range event relation system 102 generates the set of synthetic sentences 402 (e.g., intermediate missing sentences) to semantically and syntactically bridge the surrounding context. In particular, as previously discussed, the long-range event relation system 102 utilizes the generative language model 400 to generate the set of synthetic sentences 402 to fill a gap between host sentences containing related events. To illustrate, the long-range event relation system 102 utilizes the generative language model 400 implemented as an Inter-Sentential Transformer model (INSET) which is described by Huang, Y., Zhang, Y., Elachqar, O., & Cheng, Y. in INSET: Sentence infilling with INter-Sentential transformer, arXiv: 1911.03892 (2019), which is fully incorporated by reference herein.”
As indicated by Huang in [page 8, Table 3], the generated one or more sentences satisfy the “criterion” column as threshold of being fluent and meaningful, semantically coherent, and written in the same style. Thus, the criterion may be the long range relation threshold. The model may be applicable to generate more than two intermediate sentences. But the quality of human evaluation may be affected because of the patience and attention of human evaluators may decrease as the number of sentences between the first and second sentences increases. Huang, also, teaches in [page 2, col. 1] “obtains sentence representations by predicting a span of missing tokens. It can be used to generate missing text, but the missing span length needs to be pre-specified.” This indicates that Huang at least suggest specifying the number tokens, which would approximately the number of sentences to be generated and inserted between the preceding and following sentences.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. See PTO-892.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to AHAMED I NAZAR whose telephone number is (571)270-3174. The examiner can normally be reached 10 am to 7 pm Mon-Fri.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Stephen Hong can be reached at 571-272-4124. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/AHAMED I NAZAR/Examiner, Art Unit 2178 4/2/2026
/STEPHEN S HONG/Supervisory Patent Examiner, Art Unit 2178
Prosecution Insights