DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after December 18, 2021, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-17 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Subject Matter Eligibility Analysis Step 1:
Claim 1 recites a method, which is a process, one of the four statutory categories of patentable subject matter.
Subject Matter Eligibility Analysis Step 2A Prong 1:
Claim 1 further recites the method comprising 1) using a dependency parsing process on the labeled sentence to generate first augmented training data, 2) using a constituency parsing process on the labeled sentence to generate second augmented training data, and 3) using a scoring function to order a training set…, which are all mental process steps that can practically be performed in the human mind, by pen and paper, which falls within the mental process grouping of abstract ideas. The aforementioned steps can be performed by a human/using pen and paper with acquired labeled sentences. One can perform a dependency parsing processes by hand by creating a dependency tree to identify the relationships between words when the parts of speech of words in the sentence have been identified, then new variations of the sentence can be created as desired using synonym substitution, reordering, etc. One can perform a constituency parsing process by hand by creating a hierarchal tree of phrases (noun phrases, verb phrases, etc.) to determine the phrase-level structure of the sentence, then new variations of the sentence can be created as desired using noun phrase expansion, phrase splitting, etc. One can order a training set manually with a scoring function, using pen and paper, by assigning (difficulty) scores to every sentence in the training set based on some predefined criteria they decide, and sort the sentences according the scores.
Subject Matter Eligibility Analysis Step 2A Prong 2:
receiving an original labeled sentence as input… (insignificant extra-solution activity (necessary data gathering) (see MPEP 2106.05(g) (“Insignificant Extra-Solution Activity”)))
using a curriculum learning process to train the relation extraction model by feeding the scored training set to the machine learnable model (implements the abstract idea on a computer or “other machinery” to carry out the abstract method (using a computer to extract relationships from labeled sentence data) (see MPEP 2106.05(f) (“Apply It”)))
storing the trained relation extraction model in a memory (insignificant extra-solution activity (storing information) (see MPEP 2106.05(g) (“Insignificant Extra-Solution Activity”)))
Subject Matter Eligibility Analysis Step 2B:
The additional elements of claim 1 do not provide significantly more than the abstract idea itself. Therefore, claim 1 is subject matter ineligible.
Claim 2, dependent upon claim 1, further recites a mental process and additional element where the method further comprises using a lexically constrained paraphrasing process on the labeled sentence to generate third augmented training data, (a mental process, a lexically constrained paraphrasing process can done by hand (creating new sentences while ensuring certain words/phrases/entities are included or excluded)) and wherein the training set includes the original labeled sentence, the first augmented training data, the second augmented training data, and the third augmented training data (limits the data included in the training dataset, see MPEP 2106.05(h) (“Field of Use and Technological Environment”)). Thus, the claim remains an abstract idea, and the additional element of the claim does not provide a practical application or significantly more than the abstract idea itself.
Claims 3-6 further recite additional elements. Claim 3 recites the method of claim 2 wherein the lexically constrained paraphrasing process is constrained such that the third augmented training data retains the entities from the original labeled sentence (limits the lexical constraint applied to data to a positive entity-preserving constraint, see MPEP 2106.05(h) (“Field of Use and Technological Environment”)). Claim 4 recites the method of claim 3 wherein the lexically constrained paraphrasing process uses back-translation to generate the third augmented training data (limits the applied augmentation data generation technique, see MPEP 2106.05(h) (“Field of Use and Technological Environment”)). Claim 5 recites the method of claim 1 wherein the constituency parsing process uses least common ancestor detection to generate the second augmented training data (limits the sentence structure analysis technique used, see MPEP 2106.05(h) (“Field of Use and Technological Environment”)). Claim 6 recites the method of claim 1 wherein the dependency parsing process uses shortest dependency path detection to generate the first augmented training data (limits the syntactic relationship analysis technique used, see MPEP 2106.05(h) (“Field of Use and Technological Environment”)). Thus, none of the additional elements of the claims provide a practical application or significantly more than the abstract idea itself.
Subject Matter Eligibility Analysis Step 1:
Claim 7 recites a method, which is a process, one of the four statutory categories of patentable subject matter.
Subject Matter Eligibility Analysis Step 2A Prong 1:
Claim 7 further recites the method comprising 1) using a lexically constrained paraphrasing process on the labeled sentence to generate first augmented training data (a mental process, a lexically constrained paraphrasing process can done by hand (creating new sentences while ensuring certain words/phrases/entities are included or excluded)) and 2) using a scoring function to order a training set, the training set including the original labeled sentence and the first augmented training data (a human can order a training set manually by assigning (difficulty) scores to sentences based on some predefined criteria and sort the sentence data accordingly). These are both mental process steps that can practically be performed in the human mind/by pen and paper; therefore they fall within the mental process grouping of abstract ideas.
Subject Matter Eligibility Analysis Step 2A Prong 2:
receiving an original labeled sentence as input… (insignificant extra-solution activity (necessary data gathering) (see MPEP 2106.05(g) (“Insignificant Extra-Solution Activity”)))
using a curriculum learning process to train the relation extraction model by feeding the scored training set to the machine learnable model (implements the abstract idea on a computer or “other machinery” to carry out the abstract method (using a computer to extract relationships from labeled sentence data) (see MPEP 2106.05(f) (“Apply It”)))
storing the trained relation extraction model in a memory (insignificant extra-solution activity (storing information) (see MPEP 2106.05(g) (“Insignificant Extra-Solution Activity”)))
Subject Matter Eligibility Analysis Step 2B:
The additional elements of claim 7 do not provide significantly more than the abstract idea itself. Therefore, claim 7 is subject matter ineligible.
Claims 8-9 further recite additional elements. Claim 8 recites the method of claim 7 wherein the lexically constrained paraphrasing process is constrained such that the first augmented training data retains the entities from the original labeled sentence (limits the lexical constraint applied to data to a positive entity-preserving constraint, see MPEP 2106.05(h) (“Field of Use and Technological Environment”)). Claim 9 recites the method of claim 7 wherein the lexically constrained paraphrasing process uses back-translation to generate the first augmented training data (limits the applied augmentation data generation technique, see MPEP 2106.05(h) (“Field of Use and Technological Environment”)). Thus, none of the additional elements of the claims provide a practical application or significantly more than the abstract idea itself.
Claim 10, dependent upon claim 7, further recites a mental process and additional element where the method further comprises using a dependency parsing process on the labeled sentence to generate second augmented training data, (a dependency tree can be created using pen and paper to identify relationships between words, and new variations of sentences can be created as desired) and wherein the training set includes the original labeled sentence, the first augmented training data and the second augmented training data (limits the data included in the training dataset, see MPEP 2106.05(h) (“Field of Use and Technological Environment”)). Thus, the claim remains an abstract idea, and the additional element of the claim does not provide a practical application or significantly more than the abstract idea itself.
Claim 11, dependent upon claim 10, further recites an additional element wherein the dependency parsing process uses shortest dependency path detection to generate the first augmented training data (limits the syntactic relationship analysis technique used, which by MPEP 2106.05(h) (“Field of Use and Technological Environment”) does not integrate the abstract idea into a practical application or amount to significantly more than the abstract idea itself).
Claim 12, dependent upon claim 10, further recites a mental process and additional element where the method further comprises using a constituency parsing process on the labeled sentence to generate third augmented training data, (a hierarchal tree of phrases can be created using pen and paper to determine the phrase-level structures of sentences, and new variations sentences can be created as desired) and wherein the training set includes the original labeled sentence, the first augmented training data, the second augmented training data, and the third augmented training data (limits the data included in the training dataset, see MPEP 2106.05(h) (“Field of Use and Technological Environment”)). Thus, the claim remains an abstract idea, and the additional element of the claim does not provide a practical application or significantly more than the abstract idea itself.
Claim 13, dependent upon claim 12, further recites an additional element wherein the constituency parsing process uses least common ancestor detection to generate the second augmented training data (limits the sentence structure analysis technique used, which by MPEP 2106.05(h) (“Field of Use and Technological Environment”) does not integrate the abstract idea into a practical application or amount to significantly more than the abstract idea itself).
Subject Matter Eligibility Analysis Step 1:
Claim 14 recites a method, which is a process, one of the four statutory categories of patentable subject matter.
Subject Matter Eligibility Analysis Step 2A Prong 1:
Claim 14 further recites the method comprising 1) b) using at least one of a dependency parsing process, a constituency parsing process, and a lexically constrained paraphrasing process on the labeled sentence to generate augmented training data (all mental processes that can practically be done by hand) and 2) c) selecting a first scoring function from a plurality of scoring functions to order a training set based on difficulty… (a human can select a desired first scoring function from a plurality of scoring functions), 3) e) determining a respective performance metric for each of the scoring functions in the plurality by evaluating a performance of the intermediate model… (determining a performance metric based on an observation of output results), and 4) f) selecting another scoring function from the plurality of scoring functions to order the training set… (a human can select another desired scoring function from a plurality of scoring functions). These are all mental process steps that can practically be performed in the human mind/by pen and paper; therefore they fall within the mental process grouping of abstract ideas.
Subject Matter Eligibility Analysis Step 2A Prong 2:
a) receiving an original labeled sentence as input… (insignificant extra-solution activity (necessary data gathering) (see MPEP 2106.05(g) (“Insignificant Extra-Solution Activity”)))
d) training the relation extraction model using a curriculum learning process by feeding the scored training set to the relation extraction model in an order determined by the selected scoring function to generate an intermediate model (implements the abstract idea on a computer or “other machinery” to carry out the abstract method (using a computer to extract relationships from sentences in an ordered training dataset) (see MPEP 2106.05(f) (“Apply It”)))
g) training the relation extraction model again using the scored training set data from the another scoring function (implements the abstract idea on a computer or “other machinery” to carry out the abstract method (using a computer to extract relationships from sentences in a scored/ordered training dataset) (see MPEP 2106.05(f) (“Apply It”)))
Subject Matter Eligibility Analysis Step 2B:
The additional elements of claim 14 do not provide significantly more than the abstract idea itself. Therefore, claim 14 is subject matter ineligible.
Claim 15, dependent upon claim 14, further recites a mental process and additional element wherein steps f) and g) are repeated until convergence of the relation extraction model. Repeatedly performing the mental process of step f) and additional element of step g) is still a mental process/additional element in itself. Thus, the claim remains an abstract idea, and the additional element of the claim does not provide a practical application or significantly more than the abstract idea itself.
Claims 16-17 further recite additional elements. Claim 16 recites the method of claim 14 wherein the performance metric corresponds to negative correlation, such that the scoring function having a larger negative correlation is selected as the another scoring function (limits the range of the performance metric focusing on those that are negatively correlated, see MPEP 2106.05(h) (“Field of Use and Technological Environment”)). Claim 17 recites the method of claim 15 wherein the plurality of scoring functions include at least a distance between two entities function, a sentence length function, a word rarity function, and a perplexity of sentence function (limits the selection of scoring functions, see MPEP 2106.05(h) (“Field of Use and Technological Environment”)). Thus, none of the additional elements of the claims provide a practical application or significantly more than the abstract idea itself.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim 1 is rejected under 35 U.S.C. 103 as being unpatentable over Park et al. “Improving Sentence-Level Relation Extraction through Curriculum Learning” (2021) (hereinafter Park) in view of Yu et al. “Improving Relation Extraction with Relational Paraphrase Sentences” (2020) (hereinafter Yu), Tiwari et al. U.S. Patent Publication No. 20210133251 (“Data Processing Systems and Methods”) (2021) (hereinafter Tiwari), and Dai et al. U.S. Patent Publication No. 20210216722 (“Method and apparatus for processing sematic description of text entity, and storage medium”) (2021) (hereinafter Dai).
Claim 1:
Regarding claim 1, Park discloses: A computer-implemented method for training a relation extraction model using [[data augmentation] of] training data, the method comprising:
Park, pg. 1, Column 1, Abstract “…In this paper, we propose a curriculum learning-based relation extraction model that splits data by difficulty and utilizes them for learning...”
Discloses a computer-implemented method for training a relation extraction model using training data.
receiving an original labeled sentence as input, the labeled sentence including entities and at least one relation;
Park, pg. 1, Column 2, Section 2.1 “…We used the marking method proposed by Zhou and Chen (2021) [8] among various entity marking methods. Their method uses several punctuations as symbols for the entities and marks the entities in the sentence with the symbols. Sentences are inputted to the language model, and tokens containing each entity symbol are encoded. An entity encoding uses a symbol encoding vector that corresponds to the symbol of the entity. We also used a graph attention network (GAT) [9] using dependency graphs for more effective relationship extraction. GAT encodes syntactic structural information between two entities and predicts a suitable relation label through a bilinear classifier by concatenating the output of GAT and the encoding vector.”
Discloses receiving an original labeled sentence as input, the labeled sentence including entities and at least one relation.
[using a dependency parsing process on the labeled sentence to generate first augmented training data;]
[using a constituency parsing process on the labeled sentence to generate second augmented training data;]
using a scoring function to order a training set, [the training set including the original labeled sentence, the first augmented training data, and the second augmented training data;]
Park, pg. 1-2, Column 2, Section 2.2 “To measure the difficulty of data, we used the cross review method based on the prediction of the model. First, we split corpus into {0, 1, … , 𝑛} subsets by sampling without replacement. Then we created {0, 1, … , 𝑛} independent models trained on each subset and predicted the relation labels of the remaining subsets except the subset that was used for the training…”
Park, pg. 2, Column 1, Paragraph 1. “…In instances where most of the sub-models predicted the correct relation labels, it was determined that the difficulty of the data was easy and that the more the relation prediction was wrong, the more noisy or difficult the data became. Finally, we divided the entire corpus into several groups according to the difficulty of the data and allowed the model to gradually learn the entire data according to the difficulty.”
Discloses using a scoring function (based on the prediction accuracy of the sub-models) to order a training set.
using a curriculum learning process to train the relation extraction model by feeding the scored training set to the machine learnable model; and
Park, pg. 1, Column 1, Section 1 “…In this paper, we propose a curriculum learning-based relation extraction model that splits data by difficulty and utilizes them for learning. In the curriculum learning process, general parameters for relation extraction are learned through easy examples and high-level inference becomes possible while difficult examples are gradually learned [3]…”
Park, pg. 2, Column 1, Paragraph 1. “…Finally, we divided the entire corpus into several groups according to the difficulty of the data and allowed the model to gradually learn the entire data according to the difficulty.”
Discloses using a curriculum learning process to train the relation extraction model by feeding the scored training set to the machine learnable model.
[storing the trained relation extraction model in a memory.]
Thus far, Park does not explicitly teach training a relation extraction model using data augmentation of training data
Yu teaches training a relation extraction model using data augmentation of training data
Yu, pg. 1687, Abstract “…human-annotated data is costly and non-scalable while the distantly supervised data contains many noises. In this paper, we propose an alternative approach to improve RE systems via enriching diverse expressions by relational paraphrase sentences. Based on an existing labeled data, we first automatically build a task-specific paraphrase data.”
Yu, pg. 1688, Section 2 “In this section, we describe how to build the Relational Paraphrase (ReP) data, which is a task-specific paraphrase data for RE. As shown in Figure 1, we build the ReP data by generating paraphrase sentences for human-annotated sentences from an existing RE data. In this way, our ReP data contains two parts: ReP-GOLD and ReP-AUTO. ReP-GOLD is the original training set of the existing RE data and RePAUTO is the auto-generated paraphrase data…”
Yu, pg. 1689, Section 2.1 “In this paper, we take a widely used relation extraction data: TACRED (Zhang et al., 2017), which contains about 105k sentences in total. There are 41 pre-defined relation types (e.g., “person:city of birth”, “organization:founded by”) and a special type no relation. In each sentence, two entities and one relation are labeled by human…”
Yu, pg. 1689, Section 2.2 “…we take more than one public NMT systems to perform back-translation on the training set of TACRED. As the NMT systems provide end-to-end translations, entities in sentences may be replaced by other words after back-translation…”
Discloses augmented training data (an augmented training dataset), and training a relation extraction model using data augmentation of training data.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the augmented TACRED training data taught by Yu with the TACRED training dataset taught by Park because it would create a larger and more diverse dataset, which would help create a more robust model/improve generalization while avoiding the high cost of obtaining large amounts of data (See Yu, pg. 1687, Abstract “…Due to the limited size, the human-annotated data is usually incapable of covering diverse relation expressions, which could limit the performance of RE [relation extraction]. To increase the coverage of relation expressions, we may enlarge the labeled data by hiring annotators or applying Distant Supervision (DS). However, the human-annotated data is costly and non-scalable while the distantly supervised data contains many noises. In this paper, we propose an alternative approach to improve RE systems via enriching diverse expressions by relational paraphrase sentences.”).
Also, thus far, the combination of Park/Yu does not teach the following limitations of:
using a dependency parsing process on the labeled sentence to generate first augmented training data;
using a constituency parsing process on the labeled sentence to generate second augmented training data;
[using a scoring function to order a training set,] the training set including the original labeled sentence, the first augmented training data, and the second augmented training data;
Tiwari teaches the above limitations:
using a dependency parsing process on the labeled sentence to generate first augmented training data;
Tiwari [0050] “…Using relevant utterances as features in answering questions has shown to improve both the precision and recall for retrieving the right answer by a conversational bot. Therefore, utterance generation has become an important problem with the goal of generating relevant utterances (e.g., sentences or phrases) from a knowledge base article that consists of a title and a description...”
Tiwari [0056] “FIG. 7 is a process diagram depicting an embodiment of a method 700 for generating paraphrases and selecting candidates. Paraphrase generation uses many different methods for generating paraphrases, such as full backtranslation 702, noun/verb phrase backtranslation 704 using constituency parsing, synonym replacement 706, and phrase replacement 708.”
Tiwari [0069] “As discussed herein, the method proposed for utterance generation uses paraphrase generation and extractive summarization techniques to generate utterances. Paraphrase generation is used to generate multiple paraphrases of the title of an article, whereas extractive summarization is used to select the relevant sentences from the description of the article.”
Tiwari [0073] “For generating utterances from the description, the systems and methods use extractive summarization to select salient and important sentences from the description. For each of the sentence in the summary, the systems and methods generate paraphrases using the paraphrase generation method described above. The systems and methods also perform question generation (using syntactic rules based on dependency parsing) from the summary sentences.”
So, for relevant sentences selected using extractive summarization, the system performs question generation on the sentences, which uses a dependency parsing process on the relevant sentences.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the multi-technique text/sentence augmentation process taught by Tiwari with the method taught by taught by Park/Yu to specifically teach a text/sentence augmentation method that uses multiple different techniques on the same input text, because it would create a greater variety of sentence transformations resulting in a more diverse training dataset. As different techniques have their own nuances, exposing a model to different variations of the same input sentence would improve the model’s robustness. Since Tiwari’s utterances can be generated from full text/full sentences in a knowledge base that inherently capture relational information, they can seamlessly be extracted for relation extraction (See Tiwari [00500] “…utterance generation has become an important problem with the goal of generating relevant utterances (e.g., sentences or phrases) from a knowledge base article that consists of a title and a description. However, generating good utterances typically requires a significant amount of manual effort, creating the need for an automated utterance generation. The systems and methods discussed herein 1) use extractive summarization to extract important sentences from the description, 2) use multiple paraphrasing techniques to generate a diverse set of paraphrases of the title and summary sentences, and 3) select good candidate paraphrases with the help of a candidate selection algorithm.”).
using a constituency parsing process on the labeled sentence to generate second augmented training data;
Tiwari [0056] “FIG. 7 is a process diagram depicting an embodiment of a method 700 for generating paraphrases and selecting candidates. Paraphrase generation uses many different methods for generating paraphrases, such as full backtranslation 702, noun/verb phrase backtranslation 704 using constituency parsing, synonym replacement 706, and phrase replacement 708.”
Tiwari [0058] “The described systems and methods use noun/verb phrase backtranslation 704 by generating paraphrases for a certain meaningful phrase from the input sentence. In some embodiments, the systems and methods use the Berkeley Neural Parser (Kitaev and Klein 2018) to perform constituency parsing and extract all noun and verb phrases from the input sentence. For each of these extracted phrases, the systems and methods generate backtranslations and replace the phrase in the original sentence with its respective backtranslations.”
Discloses using a constituency parsing process (phrase-based backtranslation) to generate augmented sentences.
[using a scoring function to order a training set,] the training set including the original labeled sentence, the first augmented training data, and the second augmented training data;
Tawari [0055] “…The goal is to generate a diverse set of paraphrases for a sentence and, therefore, the systems and methods attempt to generate a large number of diverse paraphrases. The systems and methods first use multiple paraphrasing techniques to generate a large pool of paraphrases followed by implementation of a candidate selection algorithm to select useful and relevant paraphrases for each input sentence.”
Tiwari [0056] “FIG. 7 is a process diagram depicting an embodiment of a method 700 for generating paraphrases and selecting candidates. Paraphrase generation uses many different methods for generating paraphrases, such as full backtranslation 702, noun/verb phrase backtranslation 704 using constituency parsing, synonym replacement 706, and phrase replacement 708.”
Tiwari [0061] “The multiple techniques for paraphrasing discussed above generates a large pool 710 of paraphrases that could potentially contain sentences which are semantically different from the input sentence or synonyms replaced in the wrong context as well as duplicates of the title and each other.”
Discloses a text/sentence dataset including the original labeled sentence, the first augmented data (created using question generation), and the second augmented data (created using phrase-based backtranslation). This use of multiple augmentation techniques on the labeled sentences taught by Park/Yu would generate the many variations to be used in the training dataset (taught by Park/Yu).
Also, thus far, the combination of Park/Yu/Tiwari does not teach the remaining limitation of storing the trained relation extraction model in a memory.
Dai teaches storing the trained relation extraction model in a memory.
Dai [0027] “In an embodiment of the present disclosure, the target text, the main entity and each related entity are processed based on a pre-trained relation extraction model, and a probability distribution of a relation between the main entity and each related entity is acquired.”
Dai [0071] “In an embodiment of the present disclosure, the acquiring module 20 is specifically configured to: process the target text, the main entity and each related entity based on a pre-trained relation extraction model…”
Dai [0079] “The memory 602 is a non-transitory computer-readable storage medium according to the present disclosure. The memory stores instructions executable by at least one processor, so that the at least one processor executes the method for processing a sematic description of a text entity according to the present disclosure.”
The above discloses storing a relation extraction model in a memory. The system employs a pre-trained relation extraction model and stores the instructions (which encapsulates the model) in memory.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the trained relation extraction model taught by Dai with the relation extraction method taught by Park/Yu/Tiwari to explicitly disclose storing a relation extraction model in a memory, because it is an obvious application of a known principle and standard practice in computer system implementations. It is understood that a model’s parameters and associated instructions may be stored in memory as a part of the execution environment.
Claims 2-4 are rejected under 35 U.S.C. 103 as being unpatentable over Park, Yu, Tiwari, and Dai in view of Hu et al. “Improved Lexically Constrained Decoding for Translation and Monolingual Rewriting” (2019) (hereinafter Hu).
Claim 2:
Regarding claim 2, Tiwari discloses: The computer-implemented method of claim 1, further comprising:
using a [lexically constrained] paraphrasing process on the labeled sentence to generate third augmented training data,
Tiwari [0055] “…The goal is to generate a diverse set of paraphrases for a sentence and, therefore, the systems and methods attempt to generate a large number of diverse paraphrases. The systems and methods first use multiple paraphrasing techniques to generate a large pool of paraphrases followed by implementation of a candidate selection algorithm to select useful and relevant paraphrases for each input sentence.”
Tiwari [0056] “FIG. 7 is a process diagram depicting an embodiment of a method 700 for generating paraphrases and selecting candidates. Paraphrase generation uses many different methods for generating paraphrases, such as full backtranslation 702, noun/verb phrase backtranslation 704 using constituency parsing, synonym replacement 706, and phrase replacement 708.”
Discloses using a paraphrasing process on the labeled sentence (taught by Park and Yu) to generate additional/third augmented training data.
wherein the training set includes the original labeled sentence, the first augmented training data, the second augmented training data, and the third augmented training data.
Tiwari [0061] “The multiple techniques for paraphrasing discussed above generates a large pool 710 of paraphrases that could potentially contain sentences which are semantically different from the input sentence or synonyms replaced in the wrong context as well as duplicates of the title and each other.”
Thus far, the combination of Park/Yu/Tiwari/Dai does not explicitly teach using a lexically constrained paraphrasing process to generate augmented training data
Hu teaches using a lexically constrained paraphrasing process to generate augmented training data
Hu, pg. 839, Abstract “Lexically-constrained sequence decoding allows for explicit positive or negative phrase-based constraints to be placed on target output strings in generation tasks such as machine translation or monolingual text rewriting. We describe vectorized dynamic beam allocation, which extends work in lexically-constrained decoding to work with batching, leading to a five-fold improvement in throughput when working with positive constraints. Faster decoding enables faster exploration of constraint strategies: we illustrate this via data augmentation experiments with a monolingual rewriter applied to the tasks of natural language inference, question answering and machine translation, showing improvements in all three.”
Hu, pg. 846, Column 2, Section 5.2 “We apply our paraphrastic rewriter to the task of question answer sentence selection to see if augmenting with paraphrases leads to improvements.”
Discloses using a lexically constrained paraphrasing process (generation tasks using positive and negative constraints) to generate augmented training data.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the paraphrasing process taught by Hu with the set of multiple paraphrasing techniques taught by Park/Yu/Tiwari/Dai because it would further generate more varied/diverse training data and reduce possible bias in using one method, which would provide a good representation of language variation and improve model robustness.
Claim 3:
Regarding claim 3, Hu discloses: The computer-implemented method of claim 2, wherein the lexically constrained paraphrasing process is constrained such that the third augmented training data retains the entities from the original labeled sentence.
Hu, pg. 847, Column 1, Paraphrase Generation “We augment each answer candidate sentence with exactly 1 paraphrase in the dataset using the following heuristics: (1) named entities shared between a specific answer and its corresponding question are retained as positive constraints; (2) correct answer spans are retained as positive constraints; (3) words with the top-k IDFs (inverse document frequencies; hence “important” words) that are not positive constraints are selected as negative constraints to promote the lexical diversity of the paraphrases.9”
Discloses that the lexically constrained paraphrasing process is constrained such that the augmented training data retains the entities from the original labeled sentence.
Claim 4:
Regarding claim 4, Hu discloses: The computer-implemented method of claim 3, wherein the lexically constrained paraphrasing process uses back-translation to generate the third augmented training data.
Hu, pg. 843, “Inspired by the approach described in PARABANK (Hu et al., 2019), we trained a more powerful English monolingual rewriter by using a multi-head self-attention NMT model, Transformer (Vaswani et al., 2017)…We retain 141,381,887 paraphrastic pairs, out of over 220 million, as training data after applying these filters. To ensure output quality, we only use back-translated paraphrases as source.”
So, data augmented by the rewriter are sourced through back-translation.
Hu, pg. 846, Column 2, Section 5.2 “We apply our paraphrastic rewriter to the task of question answer sentence selection to see if augmenting with paraphrases leads to improvements.”
The above discloses that the lexically constrained paraphrasing process (that generates answer candidates using the rewriter) uses back-translation to generate augmented training data.
Claims 5 and 6 are rejected under 35 U.S.C. 103 as being unpatentable over Park, Yu, Tiwari, Dai and Hu in view of Pawar et al. “Relation Extraction : A Survey” (2017) (hereinafter Pawar).
Claim 5:
Regarding claim 5, Pawar discloses: The computer-implemented method of claim 1, wherein the constituency parsing process uses least common ancestor detection to generate the second augmented training data.
Pawar, pg. 1, Abstract “…In this paper, we survey several important supervised, semi-supervised and unsupervised RE techniques…”
Pawar, pg. 10, Section 2.2.2., Paragraph 1 “Structural properties of a sentence are encoded by its constituent parse tree… The task of constructing a constituent parse tree for a given sentence, is called as parsing…We focus on the approaches which make use of parse trees already produced by some parsers.”
Pawar, pg. 11, Relation Instance Representation “A sentence containing Ne entity mentions gives rise to Ne relation instances. Hence, it is also important to decide which part of the complete syntactic tree characterizes a particular relation instance. Zhang et al. [146] described five cases to construct a tree representation for a given relation instance which are shown in the figure 3…1. Minimum Complete Tree (MCT): It is the complete subtree formed by the lowest common ancestor of the two entities.”
Discloses a constituency parsing process using least common ancestor detection.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the tree structure concept taught by Pawar with the constituency parsing process taught by Park/Yu/Tiwari/Dai/Hu to explicitly disclose a constituency parsing process using least common ancestor detection, because it is an obvious application of a known principle. Pawar discloses least common ancestor detection (a known principle in graph theory/data structures) which is useful for identifying syntactical relationships between words/phrases for relation extraction.
Claim 6:
Regarding claim 6, Pawar discloses: The computer-implemented method of claim 1, wherein the dependency parsing process uses shortest dependency path detection to generate the first augmented training data.
Pawar, pg. 13, Section 2.2.3. “Grammatical relations between words in a sentence are encoded by its dependency tree.”
Pawar, pg. 13, Section 2.2.3., Relation Instance Representation “For each entity mention pair in a sentence, smallest subtree of the sentence’s dependency tree which contains both the mentions, is considered”
Pawar, pg. 14, Section 2.2.4. “Bunescu and Mooney [14] proposed a novel dependency path based kernel for RE. The main intuition was that the information required to assert a relationship between two entities in a sentence is typically captured by the shortest path between the two entities in the dependency graph.”
Discloses a dependency parsing process using shortest dependency path detection.
The obviousness statement recited in the rejection of claim 5 above also applies here, as shortest dependency path detection is another well-established principle.
Claims 7-9 are rejected under 35 U.S.C. 103 as being unpatentable over Park and Yu in view of Hu and Dai.
Claim 7:
Regarding claim 7, Park and Yu disclose: A computer-implemented method for training a relation extraction model using data augmentation of training data, the method comprising:
Park, pg. 1, Column 1, Abstract “…In this paper, we propose a curriculum learning-based relation extraction model that splits data by difficulty and utilizes them for learning...”
Discloses a computer-implemented method for training a relation extraction model using training data.
Yu, pg. 1689, Section 2.2 “…we take more than one public NMT systems to perform back-translation on the training set of TACRED. As the NMT systems provide end-to-end translations, entities in sentences may be replaced by other words after back-translation…”
Discloses augmented training data (an augmented training dataset), and training a relation extraction model using data augmentation of training data.
In combination Park/Yu teaches training a machine learnable model using data augmentation of training data.
receiving an original labeled sentence as input, the labeled sentence including entities;
Park, pg. 1, Column 2, Section 2.1 “…We used the marking method proposed by Zhou and Chen (2021) [8] among various entity marking methods. Their method uses several punctuations as symbols for the entities and marks the entities in the sentence with the symbols. Sentences are inputted to the language model, and tokens containing each entity symbol are encoded. An entity encoding uses a symbol encoding vector that corresponds to the symbol of the entity…”
Discloses receiving an original labeled sentence as input, the labeled sentence including entities.
[using a lexically constrained paraphrasing process on the labeled sentence to generate first augmented training data;]
using a scoring function to order a training set, [the training set including the original labeled sentence and the first augmented training data;]
Park, pg. 1-2, Column 2, Section 2.2 “To measure the difficulty of data, we used the cross review method based on the prediction of the model. First, we split corpus into {0, 1, … , 𝑛} subsets by sampling without replacement. Then we created {0, 1, … , 𝑛} independent models trained on each subset and predicted the relation labels of the remaining subsets except the subset that was used for the training…”
Park, pg. 2, Column 1, Paragraph 1. “…In instances where most of the sub-models predicted the correct relation labels, it was determined that the difficulty of the data was easy and that the more the relation prediction was wrong, the more noisy or difficult the data became. Finally, we divided the entire corpus into several groups according to the difficulty of the data and allowed the model to gradually learn the entire data according to the difficulty.”
Discloses using a scoring function (based on the prediction accuracy of the sub-models) to order a training set.
using a curriculum learning process to train the relation extraction model by feeding the scored training set to the machine learnable model; and
Park, pg. 1, Column 1, Section 1 “…In this paper, we propose a curriculum learning-based relation extraction model that splits data by difficulty and utilizes them for learning. In the curriculum learning process, general parameters for relation extraction are learned through easy examples and high-level inference becomes possible while difficult examples are gradually learned [3]…”
Park, pg. 2, Column 1, Paragraph 1. “…Finally, we divided the entire corpus into several groups according to the difficulty of the data and allowed the model to gradually learn the entire data according to the difficulty.”
Discloses using a curriculum learning process to train the relation extraction model by feeding the scored training set to the machine learnable model.
[storing the trained relation extraction model in a memory.]
Thus far, the combination of Park/Yu does not explicitly teach using a lexically constrained paraphrasing process on the labeled sentence to generate first augmented training data;
Hu teaches using a lexically constrained paraphrasing process on the labeled sentence to generate first augmented training data;
Hu, pg. 839, Abstract “Lexically-constrained sequence decoding allows for explicit positive or negative phrase-based constraints to be placed on target output strings in generation tasks such as machine translation or monolingual text rewriting. We describe vectorized dynamic beam allocation, which extends work in lexically-constrained decoding to work with batching, leading to a five-fold improvement in throughput when working with positive constraints. Faster decoding enables faster exploration of constraint strategies: we illustrate this via data augmentation experiments with a monolingual rewriter applied to the tasks of natural language inference, question answering and machine translation, showing improvements in all three.”
Hu, pg. 846, Column 2, Section 5.2 “We apply our paraphrastic rewriter to the task of question answer sentence selection to see if augmenting with paraphrases leads to improvements.”
Hu, pg. 847, Column 1, Data Setup “We augment the raw TREC-QA dataset (Wang et al., 2007) under the following orthogonal strategies: (1) augmenting the training set with the paraphrases generated via the approach described above; (2) augmenting the answer candidates at evaluation time, and choosing the max score among the paraphrases as the score (aggregation by voting).”
Discloses using a lexically constrained paraphrasing process (generation tasks using positive and negative constraints) on labeled sentences (obtained from the raw TREC-QA dataset) to generate first augmented training data.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the paraphrasing process by Hu with the relation extraction model training method taught by Park because it would create a larger and more diverse dataset, which would help create a more robust model/improve generalization while avoiding the high cost of obtaining large amounts of data (See Hu, pg. 840, Column 2, Data Augmentation “Data augmentation has been used to improve performance and robustness in deep neural models.”).
Also, thus far, the combination of Park/Yu does not teach the training set including the original labeled sentence and the first augmented training data;
Hu also teaches the training set including the original labeled sentence and the first augmented training data;
Hu, pg. 847, Column 1, Data Setup “We augment the raw TREC-QA dataset (Wang et al., 2007) under the following orthogonal strategies: (1) augmenting the training set with the paraphrases generated via the approach described above; (2) augmenting the answer candidates at evaluation time, and choosing the max score among the paraphrases as the score (aggregation by voting).”
Discloses the training set includes the original labeled sentences and the first augmented training data.
Also, thus far, the combination of Park/Yu/Hu does not teach the remaining limitation of storing the trained relation extraction model in a memory.
Dai teaches storing the trained relation extraction model in a memory.
Dai [0027] “In an embodiment of the present disclosure, the target text, the main entity and each related entity are processed based on a pre-trained relation extraction model, and a probability distribution of a relation between the main entity and each related entity is acquired.”
Dai [0071] “In an embodiment of the present disclosure, the acquiring module 20 is specifically configured to: process the target text, the main entity and each related entity based on a pre-trained relation extraction model…”
Dai [0079] “The memory 602 is a non-transitory computer-readable storage medium according to the present disclosure. The memory stores instructions executable by at least one processor, so that the at least one processor executes the method for processing a sematic description of a text entity according to the present disclosure.”
The above discloses storing a relation extraction model in a memory. The system employs a pre-trained relation extraction model and stores the instructions (which encapsulates the model) in memory.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the trained relation extraction model taught by Dai with the relation extraction method taught by Park/Yu/Hu to explicitly disclose storing a relation extraction model in a memory, because it is an obvious application of a known principle and standard practice in computer system implementations. It is understood that a model’s parameters and associated instructions may be stored in memory as a part of the execution environment.
Claim 8:
Regarding claim 8, Hu discloses: The computer-implemented method of claim 7, wherein the lexically constrained paraphrasing process is constrained such that the first augmented training data retains the entities from the original labeled sentence.
Hu, pg. 847, Column 1, Paraphrase Generation “We augment each answer candidate sentence with exactly 1 paraphrase in the dataset using the following heuristics: (1) named entities shared between a specific answer and its corresponding question are retained as positive constraints; (2) correct answer spans are retained as positive constraints; (3) words with the top-k IDFs (inverse document frequencies; hence “important” words) that are not positive constraints are selected as negative constraints to promote the lexical diversity of the paraphrases.9”
Discloses that the lexically constrained paraphrasing process is constrained such that the augmented training data retains the entities from the original labeled sentence.
Claim 9:
Regarding claim 9, Hu discloses: The computer-implemented method of claim 7, wherein the lexically constrained paraphrasing process uses back-translation to generate the first augmented training data.
Hu, pg. 843, “Inspired by the approach described in PARABANK (Hu et al., 2019), we trained a more powerful English monolingual rewriter by using a multi-head self-attention NMT model, Transformer (Vaswani et al., 2017)…We retain 141,381,887 paraphrastic pairs, out of over 220 million, as training data after applying these filters. To ensure output quality, we only use back-translated paraphrases as source.”
So, data augmented by the rewriter are sourced through back-translation.
Hu, pg. 846, Column 2, Section 5.2 “We apply our paraphrastic rewriter to the task of question answer sentence selection to see if augmenting with paraphrases leads to improvements.”
The above discloses that the lexically constrained paraphrasing process (that generates answer candidates using the rewriter) uses back-translation to generate augmented training data.
Claims 10 and 12 are rejected under 35 U.S.C. 103 as being unpatentable over Park, Yu, Hu, and Dai in view of Tiwari.
Claim 10:
Regarding claim 10, Tiwari discloses: The computer-implemented method of claim 7, further comprising:
using a dependency parsing process on the labeled sentence to generate second augmented training data,
Tiwari [0050] “…Using relevant utterances as features in answering questions has shown to improve both the precision and recall for retrieving the right answer by a conversational bot. Therefore, utterance generation has become an important problem with the goal of generating relevant utterances (e.g., sentences or phrases) from a knowledge base article that consists of a title and a description...”
Tiwari [0056] “FIG. 7 is a process diagram depicting an embodiment of a method 700 for generating paraphrases and selecting candidates. Paraphrase generation uses many different methods for generating paraphrases, such as full backtranslation 702, noun/verb phrase backtranslation 704 using constituency parsing, synonym replacement 706, and phrase replacement 708.”
Tiwari [0069] “As discussed herein, the method proposed for utterance generation uses paraphrase generation and extractive summarization techniques to generate utterances. Paraphrase generation is used to generate multiple paraphrases of the title of an article, whereas extractive summarization is used to select the relevant sentences from the description of the article.”
Tiwari [0073] “For generating utterances from the description, the systems and methods use extractive summarization to select salient and important sentences from the description. For each of the sentence in the summary, the systems and methods generate paraphrases using the paraphrase generation method described above. The systems and methods also perform question generation (using syntactic rules based on dependency parsing) from the summary sentences.”
So, for relevant sentences selected using extractive summarization, the system performs question generation on the sentences, which uses a dependency parsing process on the relevant sentences.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the augmented training data taught by Tiwari with the training dataset taught by Park/Yu/Hu/Dai because it would create a larger and more diverse dataset, which would help create a more robust model/improve generalization while avoiding the high cost of obtaining large amounts of data (See Tiwari [0050] “…Using relevant utterances as features in answering questions has shown to improve both the precision and recall for retrieving the right answer by a conversational bot. Therefore, utterance generation has become an important problem with the goal of generating relevant utterances (e.g., sentences or phrases) from a knowledge base article that consists of a title and a description. However, generating good utterances typically requires a significant amount of manual effort, creating the need for an automated utterance generation...”).
wherein the training set includes the original labeled sentence, the first augmented training data and the second augmented training data.
Tiwari [0056] “FIG. 7 is a process diagram depicting an embodiment of a method 700 for generating paraphrases and selecting candidates. Paraphrase generation uses many different methods for generating paraphrases, such as full backtranslation 702, noun/verb phrase backtranslation 704 using constituency parsing, synonym replacement 706, and phrase replacement 708.”
Tiwari [0061] “The multiple techniques for paraphrasing discussed above generates a large pool 710 of paraphrases that could potentially contain sentences which are semantically different from the input sentence or synonyms replaced in the wrong context as well as duplicates of the title and each other.”
Discloses a training set including the original labeled sentence, first augmented training data (generated using a form of back-translation) and second augmented training data.
Claim 12:
Regarding claim 12, Tiwari discloses: The computer-implemented method of claim 10, further comprising: using a constituency parsing process on the labeled sentence to generate third augmented training data,
Tiwari [0056] “FIG. 7 is a process diagram depicting an embodiment of a method 700 for generating paraphrases and selecting candidates. Paraphrase generation uses many different methods for generating paraphrases, such as full backtranslation 702, noun/verb phrase backtranslation 704 using constituency parsing, synonym replacement 706, and phrase replacement 708.”
Tiwari [0058] “The described systems and methods use noun/verb phrase backtranslation 704 by generating paraphrases for a certain meaningful phrase from the input sentence. In some embodiments, the systems and methods use the Berkeley Neural Parser (Kitaev and Klein 2018) to perform constituency parsing and extract all noun and verb phrases from the input sentence. For each of these extracted phrases, the systems and methods generate backtranslations and replace the phrase in the original sentence with its respective backtranslations.”
Discloses using a constituency parsing process (phrase-based backtranslation) on labeled sentences to generate augmented data.
wherein the training set includes the original labeled sentence, the first augmented training data, the second augmented training data, and the third augmented training data.
Tiwari [0061] “The multiple techniques for paraphrasing discussed above generates a large pool 710 of paraphrases that could potentially contain sentences which are semantically different from the input sentence or synonyms replaced in the wrong context as well as duplicates of the title and each other.”
The obviousness statement recited in the rejection of claim 10 above also applies here, as the application of a constituency parsing process produce the same benefits of using augmented training data.
Claims 11 and 13 are rejected under 35 U.S.C. 103 as being unpatentable over Park, Yu, Hu, Dai and Tiwari in view of Pawar.
Claim 11:
Regarding claim 11, Pawar discloses: The computer-implemented method of claim 10, wherein the dependency parsing process uses shortest dependency path detection to generate the first augmented training data.
Pawar, pg. 13, Section 2.2.3. “Grammatical relations between words in a sentence are encoded by its dependency tree.”
Pawar, pg. 13, Section 2.2.3., Relation Instance Representation “For each entity mention pair in a sentence, smallest subtree of the sentence’s dependency tree which contains both the mentions, is considered”
Pawar, pg. 14, Section 2.2.4. “Bunescu and Mooney [14] proposed a novel dependency path based kernel for RE. The main intuition was that the information required to assert a relationship between two entities in a sentence is typically captured by the shortest path between the two entities in the dependency graph.”
Discloses a dependency parsing process using shortest dependency path detection.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the tree structure concept taught by Pawar with the dependency parsing process taught by Park/Yu/Hu/Dai/Tiwari to explicitly disclose a dependency parsing process using least shortest dependency path detection, because it is an obvious application of a known principle. Pawar discloses shortest dependency path detection (a known principle in graph theory/data structures) which is useful for determining and classifying structural relationships between words/entities for relation extraction.
Claim 13:
Regarding claim 13, Pawar discloses: The computer-implemented method of claim 12, wherein the constituency parsing process uses least common ancestor detection to generate the second augmented training data.
Pawar, pg. 1, Abstract “…In this paper, we survey several important supervised, semi-supervised and unsupervised RE techniques…”
Pawar, pg. 10, Section 2.2.2., Paragraph 1 “Structural properties of a sentence are encoded by its constituent parse tree… The task of constructing a constituent parse tree for a given sentence, is called as parsing…We focus on the approaches which make use of parse trees already produced by some parsers.”
Pawar, pg. 11, Relation Instance Representation “A sentence containing Ne entity mentions gives rise to Ne relation instances. Hence, it is also important to decide which part of the complete syntactic tree characterizes a particular relation instance. Zhang et al. [146] described five cases to construct a tree representation for a given relation instance which are shown in the figure 3…1. Minimum Complete Tree (MCT): It is the complete subtree formed by the lowest common ancestor of the two entities.”
Discloses a constituency parsing process using least common ancestor detection.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the tree structure concept taught by Pawar with the constituency parsing process taught by Park/Yu/Hu/Dai/Tiwari to explicitly disclose a constituency parsing process using least common ancestor detection, because it is an obvious application of a known principle. Pawar discloses least common ancestor detection (a known principle in graph theory/data structures) which is useful for identifying syntactical relationships between words/phrases for relation extraction.
Claims 14-17 are rejected under 35 U.S.C. 103 as being unpatentable over Park, Yu, and Tiwari in view of Wang et al. “A Survey on Curriculum Learning”) (2021) (hereinafter Wang) and Auer et al. “Finite-time Analysis of the Multiarmed Bandit Problem” (2002).
Claim 14:
Regarding claim 14, Park, Yu, and Tiwari disclose: A computer-implemented method for training a machine learnable model using data augmentation of training data, the method comprising:
Park, pg. 1, Column 1, Abstract “…In this paper, we propose a curriculum learning-based relation extraction model that splits data by difficulty and utilizes them for learning...”
Discloses a computer-implemented method for training a relation extraction model using training data.
Yu, pg. 1689, Section 2.2 “…we take more than one public NMT systems to perform back-translation on the training set of TACRED. As the NMT systems provide end-to-end translations, entities in sentences may be replaced by other words after back-translation…”
Discloses augmented training data (an augmented training dataset), and training a relation extraction model using data augmentation of training data.
In combination Park/Yu teaches training a machine learnable model using data augmentation of training data.
a) receiving an original labeled sentence as input, the labeled sentence including entities;
Park, pg. 1, Column 2, Section 2.1 “…We used the marking method proposed by Zhou and Chen (2021) [8] among various entity marking methods. Their method uses several punctuations as symbols for the entities and marks the entities in the sentence with the symbols. Sentences are inputted to the language model, and tokens containing each entity symbol are encoded. An entity encoding uses a symbol encoding vector that corresponds to the symbol of the entity…”
Discloses receiving an original labeled sentence as input, the labeled sentence including entities.
b) using at least one of a dependency parsing process, a constituency parsing process, and a lexically constrained paraphrasing process on the labeled sentence to generate augmented training data;
Tiwari [0073] “For generating utterances from the description, the systems and methods use extractive summarization to select salient and important sentences from the description. For each of the sentence in the summary, the systems and methods generate paraphrases using the paraphrase generation method described above. The systems and methods also perform question generation (using syntactic rules based on dependency parsing) from the summary sentences.”
Tiwari [0056] “FIG. 7 is a process diagram depicting an embodiment of a method 700 for generating paraphrases and selecting candidates. Paraphrase generation uses many different methods for generating paraphrases, such as full backtranslation 702, noun/verb phrase backtranslation 704 using constituency parsing, synonym replacement 706, and phrase replacement 708.”
Discloses using a dependency parsing process and a constituency parsing process to generate augmented data.
[c) selecting a first scoring function from a plurality of scoring functions to order a training set based on difficulty,] the training set including the original labeled sentence and the augmented training data;
Tiwari [0061] “The multiple techniques for paraphrasing discussed above generates a large pool 710 of paraphrases that could potentially contain sentences which are semantically different from the input sentence or synonyms replaced in the wrong context as well as duplicates of the title and each other.”
Discloses a dataset including the original labeled sentence (taught by Park/Yu) and the augmented data.
d) training the relation extraction model using a curriculum learning process by feeding the scored training set to the relation extraction model [in an order determined by the selected scoring function to generate an intermediate model;]
Park, pg. 1, Column 1, Section 1 “…In this paper, we propose a curriculum learning-based relation extraction model that splits data by difficulty and utilizes them for learning. In the curriculum learning process, general parameters for relation extraction are learned through easy examples and high-level inference becomes possible while difficult examples are gradually learned [3]…”
Park, pg. 1-2, Column 2, Section 2.2 “To measure the difficulty of data, we used the cross review method based on the prediction of the model. First, we split corpus into {0, 1, … , 𝑛} subsets by sampling without replacement. Then we created {0, 1, … , 𝑛} independent models trained on each subset and predicted the relation labels of the remaining subsets except the subset that was used for the training…”
Park, pg. 2, Column 1, Paragraph 1. “…In instances where most of the sub-models predicted the correct relation labels, it was determined that the difficulty of the data was easy and that the more the relation prediction was wrong, the more noisy or difficult the data became. Finally, we divided the entire corpus into several groups according to the difficulty of the data and allowed the model to gradually learn the entire data according to the difficulty.”
Park, pg. 2, Column 1, Paragraph 1. “…Finally, we divided the entire corpus into several groups according to the difficulty of the data and allowed the model to gradually learn the entire data according to the difficulty.”
Discloses training the relation extraction model using a curriculum learning process by feeding the scored training set to the relation extraction model.
[e) determining a respective performance metric for each of the scoring functions in the plurality by evaluating a performance of the intermediate model using a validation data set ordered respectively by the plurality of scoring functions;]
[f) selecting another scoring function from the plurality of scoring functions to order the training set, the another scoring function being selected based on the determined performance metric of the second scoring function; and]
[g) training the relation extraction model again using the scored training set data from the another scoring function.]
Thus far, the combination of Park/Yu/Tiwari does not explicitly teach c) selecting a first scoring function from a plurality of scoring functions to order a training set based on difficulty,
Wang teaches c) selecting a first scoring function from a plurality of scoring functions to order a training set based on difficulty,
Wang, pg. 5, Column 2, Section 4.1 “Recall that the core definition of CL (Definition 1) lies in the strategy of “training from easier data to harder data”. In essence, to design such a curriculum, we need to decide two things: 1) What kind of training data is supposed to be easier than other data? 2) When should we present more harder data for training, and how much more? Issue 1) can be abstracted to a Difficulty Measurer, which decides the relative “easiness” of each data example. Issue 2) can be abstracted to a Training Scheduler, which decides the sequence of data subsets throughout the training process based on the judgment from the Difficulty Measurer.”
Wang, pg. 6, Column 1, Section 4.2.1 “Researchers have manually designed various Difficulty Measurers mainly based on the data characteristics of specific tasks. We summarize common types of Difficulty Measurers in Table 2.”
Discloses selecting a scoring function from a plurality of scoring functions to order a training set based on difficulty.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine method of selecting a scoring function taught by Wang with the method taught by Park/Yu/Tiwari because the selection of a scoring function/difficulty measure is a common practice in curriculum learning (See Wang, pg. 4560, Column 1, Section 4 “Since we have understood why CL is effective and why researchers apply CL to different scenes, a natural and important question should be: how to design an appropriate curriculum for a specific learning task? In this section, we provide a general framework of “Difficulty Measurer + Training Scheduler” (Section 4.1), which unifies most of CL methodologies.”).
Also, thus far, the combination of Park/Yu/Tiwari does not teach feeding a scored training set to a model in an order determined by the selected scoring function to generate an intermediate model;
Wang also teaches feeding a scored training set to a model in an order determined by the selected scoring function to generate an intermediate model;
Wang, pg. 5, Column 2, Section 4.1 “…a general framework for curriculum design consists of these two core components: Difficulty Measurer + Training Scheduler, which is illustrated in Fig. 2a. To begin with, all the training examples are sorted by the Difficulty Measurer from the easiest to the hardest and passed to the Training Scheduler…”
This discloses feeding a scored training set to a model in an order determined by the selected scoring function (difficulty measure) to generate an intermediate model.
Thus far, the combination of Park/Yu/Tiwari/Wang does not teach the following remaining limitations,
Auer teaches the following remaining limitations:
e) determining a respective performance metric for each of the scoring functions in the plurality by evaluating a performance of the intermediate model using a validation data set ordered respectively by the plurality of scoring functions;
Auer, pg. 235, Abstract “Reinforcement learning policies face the exploration versus exploitation dilemma, i.e. the search for a balance between exploring the environment to find profitable actions while taking the empirically best action as often as possible. A popular measure of a policy’s success in addressing this dilemma is the regret, that is the loss due to the fact that the globally optimal policy is not followed all the times…”
Auer, pg. 236, Paragraph 2 “…A policy, or allocation strategy, A is an algorithm that chooses the next machine to play based on the sequence of past plays and obtained rewards…Then the regret of A after n plays is defined by µ∗n − µj K j=1 IE[Tj(n)] where µ∗ def = max 1≤i≤K µi and IE[·] denotes expectation. Thus the regret is the expected loss due to the fact that the policy does not always play the best machine.”
Auer, pg. 237, Section 2 “Our first result shows that there exists an allocation strategy, UCB1, achieving logarithmic regret uniformly over n and without any preliminary knowledge about the reward distributions (apart from the fact that their support is in [0, 1])…”
Discloses determining a respective performance metric (cumulative reward/regret) for each of the scoring functions (that evaluates individual actions or strategies) in the plurality by evaluating a performance of the intermediate model (policy).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine method of determining a respective performance measure of a plurality of scoring functions as taught by Auer with the method taught by Park/Yu/Tiwari/Wang because it would create an adaptive learning process (being able to self-optimize and refine models using the plurality of scoring functions) that would accelerate convergence, resulting in a more robust learning system. (See Auer, Abstract “Reinforcement learning policies face the exploration versus exploitation dilemma, i.e. the search for a balance between exploring the environment to find profitable actions while taking the empirically best action as often as possible…”).
f) selecting another scoring function from the plurality of scoring functions to order the training set, the another scoring function being selected based on the determined performance metric of the second scoring function; and
Auer, pg. 237, Section 2 “The policy UCB1 (sketched in figure 1) is derived from the index-based policy of Agrawal (1995). The index of this policy is the sum of two terms. The first term is simply the current average reward. The second term is related to the size (according to Chernoff-Hoeffding bounds, see Fact 1) of the one-sided confidence interval for the average reward within which the true expected reward falls with overwhelming probability.”
Auer, pg. 239, Paragraph 2 “A simple and well-known policy for the bandit problem is the so-called ε-greedy rule (see Sutton, & Barto, 1998). This policy prescribes to play with probability 1−ε the machine with the highest average reward, and with probability ε a randomly chosen machine.”
Discloses selecting another scoring function (the index used by the UCB1 policy) from the plurality of scoring functions to order the training set, the another scoring function being selected based on the determined performance metric of the second scoring function (selecting machines based on the past reward/performance).
g) training the relation extraction model again using the scored training set data from the another scoring function.
Auer, pg. 238, Paragraphs 2-3 “Using a slightly more complicated policy, which we call UCB2 (see figure 2), we can bring the main constant of (2) arbitrarily close to 1/(2 2 j). The policy UCB2 works as follows. The plays are divided in epochs. In each new epoch a machine i is picked and then played τ(ri + 1) − τ(ri) times, where τ is an exponential function and ri is the number of epochs played by that machine so far.”
Auer, pg. 239, Theorem 3 “For all K > 1 and for all reward distributions P1,..., PK with support in [0, 1], if policy εn-GREEDY is run with input parameter 0 < d ≤ min … then the probability that after any number n ≥ cK/d of plays εn-GREEDY chooses a suboptimal machine…”
Discloses training model/policy again using the scored data from the another scoring function. After applying the initial scoring function and training the model a new scoring function is applied to data.
Claim 15:
Regarding claim 15, Auer discloses: The method of claim 14, wherein steps f) and g) are repeated until convergence of the relation extraction model.
Auer, pg. 238, Paragraph 3 “The plays are divided in epochs. In each new epoch a machine i is picked and then played τ(ri + 1) − τ(ri) times, where τ is an exponential function and ri is the number of epochs played by that machine so far.”
Claim 16:
Regarding claim 16, Auer discloses: The method of claim 14, wherein the performance metric corresponds to negative correlation, such that the scoring function having a larger negative correlation is selected as the another scoring function.
PNG
media_image1.png
315
656
media_image1.png
Greyscale
Auer Figure 1. Sketch of the deterministic policy UCB1 (see Theorem 1).
The UCB1 policy with selection rule implicitly balances the trade-off between exploiting known high rewards and exploring less-certain options, analogous to selecting a scoring function based on its performance metric (an effective negative correlation with uncertainty). The confidence bones is added to the average reward to decide which machine to play.
Claim 17:
Regarding claim 17, Wang discloses: The method of claim 15, wherein the plurality of scoring functions include at least a distance between two entities function, a sentence length function, a word rarity function, and a perplexity of sentence function.
Wang, pg. 6, Column 1, Section 4.2.1 “…We summarize common types of Difficulty Measurers in Table 2. Most of the predefined Difficulty Measurers are designed for image and text data in various CV and NLP scenarios, while other data types include audio data, programs, tabular data, etc...”
Wang, pg. 6, Column 1, Section 4.2.1, Paragraph 2 “First, complexity stands for the structural complexity of a particular data example, such that examples with higher complexity have more dimensions and are thus harder to be captured by models. For instance, sentence length, the most popular Difficulty Measurer in NLP tasks [86], [107], [112], intuitively expresses the complexity of a sentence/paragraph. Therefore, longer sentences are often supposed as harder training data. Other examples include…the parse tree depth [113] that measures the sentence complexity in the view of grammar;…”
Wang, pg. 6, Column 2, Paragraph 2 “Secondly, the angle of diversity here stands for the distributional diversity of a group of data (e.g., regular or irregular shapes [6]) or the elements (e.g., words) of a data point (e.g., sentence). A larger value of diversity means the data is
more various, including more (rare) types/styles of data or elements, and is thus more difficult for model learning. For example, a sentence with more rare words is usually considered harder to learning [86].
Discloses wherein the plurality of scoring functions include a sentence length function and a word rarity function.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant’s disclosure. The list references in the attached PTO-892 that are not relied upon in the above 35 U.S.C 103 rejections are pertinent to particular concepts of the dependent claims.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ISIS M BLACK whose telephone number is (703)756-1121. The examiner can normally be reached Monday - Friday 9:00 am - 5:00 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Miranda Huang can be reached on (571) 270-7092. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/I.M.B./Examiner, Art Unit 2124
/MIRANDA M HUANG/Supervisory Patent Examiner, Art Unit 2124
Prosecution Insights