Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
Claim Objections
Claim 1 objected to because of the following informalities: the claim lacks a recitation of “comprising,” subsequent to the preamble. Appropriate correction is required.
Claim Rejections - 35 USC § 101
Applicants arguments and amendments to the claims suffice to obviate the rejection of claims 1-3, 5, 6, 8-12, 14, 15, 17, 18, 20 over 35 U.S.C. 101 as the claims are considered to recite elements which integrate an improvement upon the natural language processing domain.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-3, 5, 6, 8-12, 14, 15, 17, 18, 20 rejected under 35 U.S.C. 103 as being unpatentable over Chen: 20240037339 hereinafter Che further in view of Chen: “Explicitly Capturing Relations between Entity Mentions via Graph Neural Networks for Domain-specific Named Entity Recognition,” (provided by Examiner and further descriptive of the Chen patent; copyright 2021 and hereinafter Che_2) and further in view of Fei: 20210240929.
Regarding claim 1
Che teaches:
An apparatus for deep learning-based coreference resolution comprising, one or more computer-executable modules being configured and executed by a processor using algorithms associated with at least one non-transitory storage device, the algorithms, when executed, causing the processor to execute the one or more computer-executable modules (Chen: Abstract; ¶ 16-18, 27-29; Fig 1-3: such as the disclosed computer bearing processor, instructions borne upon non transitory memory, etc.), the one or more computer-executable modules comprising:
a data generation module configured to extract one or more natural language sentences from a natural language paragraph (Che: Abstract; ¶ 15, 45, 48, 49: for each input document unstructured natural-language sentences are provided as input to a machine-learning model to generate contextual word embeddings by building a coreference relation graph based on all the words of received sentences in an input document and determining a sentence level dependency graph by parsing the document sentence wise; paragraphs are thus considered a set of sentences which comprise a subset of each/any document) and perform dependency parsing on the natural language sentences to generate dependency relation data of the natural language sentences (Che: Abstract; ¶ 2, 15, 34, 37, 45, 48, 49: sentence level dependency relations or parse sentences operable to generate a dependency tree; as such Che constructs a relation graph using coreference and dependency relations to form a coreference graph and dependency graph by extracting sentences from the input and performing dependency parsing thereon to generate dependency relation data of the sentences);
an embedding module configured to generate an integrated embedding vector for the natural language paragraph based on the natural language sentence and the dependency relation data (Che: Abstract; ¶ 5-7, 45-49, 65; Fig 5; Claim 1: system generates a graphs based on sentences, words, etc. of input documents, generates a graph of dependencies, a graph of coreferences, etc. in concert with determined embeddings and provides same to a graph neural network wherein the embedding layer of the system receives a matrix of word vectors, values etc. as input and at least the embedding and encoding layers of the system output vectors and thereby generates an entity relation graph based thereon), the integrated embedding vector including semantic information of the natural language paragraph and semantic structure information of the natural language sentence (Che: Abstract; ¶ 5-7, 45-49, 65; Fig 5; Claim 1: semantic information encoded within the word embeddings of the overall document; paragraph, sentences, etc. therein; semantic sentence structure encoded within, upon, etc. the dependency and coreference graphs which operate in concert with the graph neural network (GNN) to update embeddings that integrate both semantics and structure thereby operating in a manner similar to the integrated embedding vector);
a coreference resolution module comprising a deep learning neural network calculation unit configured to input the integrated embedding vector to the deep learning neural network to generate a calculation result of the deep learning neural network (Che: Abstract; ¶ 5-7, 45-54, 58, 65; Fig 5; Claim 1, 2, 6, 14, 17: integrated embedding vector such as the word embeddings input to the to the BiLSTM encoding layer, which produces contextualized encodings and provides same to the GNN which subsequently fuses the input word embedding within, upon, etc. the coreference and dependency graphs to produce updated word embeddings as output to the conditional random field decoding layer such that deep learning calculation results are generated at each step, layer, etc. of the processing pipeline to produce calculation, decoding, etc. results such as for prediction such as of entities, coreference entity mentions, etc.). Thus Che operates to perform named entity recognition based in an entity relation, coreference and dependency graph to obtain updated word embedding which decode to provide enriched entity predictions.
Che strongly suggests but does not explicitly name a training data generation module but merely performs the recited steps by similarly deriving dependency relations; Che additionally strongly suggests but does not explicitly teach the performance of dependency parsing on the natural language sentences to generate relation data thereof; the integrated embedding vector comprising semantic information of paragraphs and semantic structure information of the sentences; a mention detection unit configured to detect a mention in the natural language paragraph based on the calculation result of the deep learning neural network, a coreference recognition unit configured to generate a second coreference mention of the natural language paragraph based on a result of the mention detection, a training unit configured to calculate a coreference recognition error between the first coreference mention and the second coreference mention, and train the deep learning neural network based on the error to generate a coreference resolution model.
In a related field of endeavor and describing the same work Che_2 teaches a system and method for “Capturing Relations between Entity Mentions via Graph Neural Networks for Domain-specific Named Entity Recognition,” comprising: a training data generation module that extracts configured to extract one or more natural language sentences from a natural language document (Che_2: Abstract): system determines, connects, etc. entity mentions based on coreference and dependency relations to improve entity mention representation); and to perform dependency parsing on the natural language sentences to generate dependency relation data of the natural language sentences (Che_2: § 3: coreference and dependent relations between entity mentions connect mentions in syntactic dependency relations);
an embedding module configured to generate an integrated embedding vector for the natural language paragraph based on the natural language sentence and the dependency relation data, the integrated embedding vector including semantic information of the natural language paragraph and semantic structure information of the natural language sentence (Che_2: § 3; Fig 2: system uses a contextual encoder such as BERT or the pictured BiLSTM to obtain contextual word embeddings comprising semantic information of a natural language input; outputs the embeddings to build an entity relation graph based on coreference and dependency relations of the input word embeddings and applies a graph attention network/graph neural network over the constructed entity relation graph wherein nodes of the GNN comprise semantic embeddings which communicate along dependency and coreference edges to output enhanced word embeddings such as in an integrated embedding vector inclusive of sematic information and semantic structure information of sentences with documents);
a deep learning neural network calculation unit configured to input the integrated embedding vector to the deep learning neural network to generate a calculation result of the deep learning neural network, (Che_2: an input to the GNN is decoded; subsequent to the construction of the entity relation graph, mention representations and entity relations are input to the GAN to output refined mention representations based on neighboring data; said refined mention representations input to a sequence tagging decoder which outputs a set of refined representations in concert with label scores and ultimately entity tags which are used for improved named entity recognition). It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to utilized the modules described in Che_2 as an enhancement to the Che system, method, etc. for at least the purpose of improving named entity recognition based on the particular operations described therein; one of ordinary skill in the art would have expected only predictable results therefrom.
Che in view of Ch_2 does not explicitly teach a mention detection unit configured to detect a mention in the natural language paragraph based on the calculation result of the deep learning neural network, a coreference recognition unit configured to generate a second coreference mention of the natural language paragraph based on a result of the mention detection, a training unit configured to calculate a coreference recognition error between the first coreference mention and the second coreference mention, and train the deep learning neural network based on the error to generate a coreference resolution model.
In a related field of endeavor Fei teaches a deep reinforcement learning system and method (Fei: Title; ¶ 27, 29) for coreference resolution (Fei: ¶ 29, 30) such as by iteratively generating training data such as co-reference graphs for input documents, paragraphs, sentences thereof used over a plurality of epochs (Fei: ¶ 29, 30; Fig 1, 2; claims 2, 16) and a first coreference mention preset (Fei: ¶ 5, 6, etc.; claim 1, 2: such as the recited ground truth coreference information for a document and operable for training a coreference resolution model) and comprising:
a mention detection unit configured to detect a mention in the natural language paragraph based on the calculation result of the deep learning neural network (Fei: ¶ 5, 6, 41-49; Fig 4, 6: system generates attention spans, refines and prunes same, to thereby obtain mention scores using a mention neural network module by applying self-attention on the generated span representations; the mention scores used to generate coreference scores based on mention scores and antecedent scores; the generated coreference scores used to generate a probability distribution for each mention in such a way as to generate contextualized representations of words upon a document and a document with respect to words therein the recited paragraphs are considered encompassed in this way),
a coreference recognition unit configured to generate a second coreference mention of the natural language paragraph based on a result of the mention detection (Fei: ¶ 5, 6; Fig 4, 6: system determines linking actions among/between mentions and updates a coreference graph based thereon; a mention detected in an unpruned or high scored span used in a linking step that selects prior mention antecedents and creates an edge connection thereby in the coreference graph in a manner functional to generate a second coreference based on the linked prior mention resultant from subsequent mention detection), and
a training unit configured to calculate a coreference recognition error between the first coreference mention and the second coreference mention, and train the deep learning neural network based on the error to generate a coreference resolution model (Fei: ¶ 5, 6, 31, 39, 68-72; Fig 4, 6; Table 1; Claim 1, 2: a training reward based on coreference evaluation metrics comparing predicted and ground truth coreference information used to train, update, refine the policy network by generating and updating a gradient based thereon; such as by utilizing, optimizing, etc. one or more coreference metrics such as based on iterative error measurements using the coreference metrics).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to utilize the Fei taught mention detection, coreference linking and training based coreference resolution architecture; such as by applying the Fei teachings at the paragraph level to thereby improve the Che in view of Che_2 dependency and coreference resolution capabilities based on the Che in view of Che_2 integrated embeddings and annotated labelled mentions for at least the purpose of better providing improved inputs to the Fei coreference model to thereby better train or refine the Pei coreference model based thereon; one of ordinary skill in the art would have expected only predictable results therefrom.
Regarding claim 2
Che in view of Che_2 in view of Fei teaches or suggests:
The apparatus of claim 1, wherein the training data generation module includes:
a natural language sentence extraction unit that extracts the natural language sentence from the natural language paragraph (Che: ¶ 47-50; Fig 5: system performs sentence extraction from documents reliably consisting of paragraphs, sections, etc.);
a dependency parsing unit that performs the dependency parsing on the natural language sentence (Che: ¶ 47-50; Fig 5: system parses sentences to determine dependency arcs); and
a dependency relation extraction unit that generates the dependency relation data by extracting the dependency relation between words included in the natural language sentence based on a dependency parsing result of the dependency parsing unit (Che: ¶ 47-50; Fig 5: system parses sentences to determine dependency arcs thereby generating token-wise or word-wise graphs of dependency relations). The claim is considered obvious over Che as modified by Che_2 and Fei as addressed in the base claim as it would have been obvious to apply the further teaching of Che, Che_2, and/or Fei to the modified device of Che, Che_2, and Fei; one of ordinary skill in the art would have expected only predictable results therefrom.
Regarding claim 3
Che in view of Che_2 in view of Fei teaches or suggests:
The apparatus of claim 1, wherein the embedding module includes:
a natural language embedding unit that embeds the words included in the natural language sentence to generate a natural language embedding vector (Che: ¶ 45-50: system uses bert to determine embedding layer data including embeddings, word vector data input thereto, vectors output thereby to an embedding layer which subsequently outputs vectors to an encoding layer for a GNN operative upon and outputting matrices of vectors; adjacency matrices, etc.); (Fei: ¶ 51-62; Fig 4, 6: embeddings of spans input to coreference module to generate conference scores ;
a dependency relation embedding unit that embeds the dependency relation data to generate a dependency relation embedding vector that integrates the natural language embedding vector and the dependency relation embedding vector to generate the integrated embedding vector (Che: 45-50, 53, 54; Fig 5; and an integrated embedding unit provides vectorized data to the GNN which propagates the information for combining with the vectorized embedding and encoding outputs to generate output recognition data); (Fei: ¶ 51-62; Fig 4, 6). The claim is considered obvious over Che as modified by Che_2 and Fei as addressed in the base claim as it would have been obvious to apply the further teaching of Che, Che_2, and/or Fei to the modified device of Che, Che_2, and Fei; one of ordinary skill in the art would have expected only predictable results therefrom.
Regarding claim 5
Che in view of Che_2 in view of Fei teaches or suggests:
The apparatus of claim 1, wherein the deep learning neural network is a neural network based on a long short-term memory (LSTM) (Che: ¶ 38, 40; Fig 5: GNN comprises, based on, etc. LSTM layer); (Fei: ¶ 6, 42, etc.: neural system integrated with LSTM layer). The claim is considered obvious over Che as modified by Che_2 and Fei as addressed in the base claim as it would have been obvious to apply the further teaching of Che, Che_2, and/or Fei to the modified device of Che, Che_2, and Fei; one of ordinary skill in the art would have expected only predictable results therefrom.
Regarding claim 6
Che in view of Che_2 in view of Fei teaches or suggests:
The apparatus of claim 1, wherein the deep learning neural network is a neural network based on bidirectional encoder representations from transformers (BERT) (Che: ¶ 45, 46, etc.: system based on BERT). The claim is considered obvious over Che as modified by Che_2 and Fei as addressed in the base claim as it would have been obvious to apply the further teaching of Che, Che_2, and/or Fei to the modified device of Che, Che_2, and Fei; one of ordinary skill in the art would have expected only predictable results therefrom.
Regarding claim 8
Che in view of Che_2 in view of Fei teaches or suggests:
The apparatus of claim 1, wherein the training unit calculates a coreference recognition error between the first coreference mention and the second coreference mention, and trains a mention detection parameter of the mention detection unit based on the error (see claim 1 supra; Fei: ¶ 31, 58; claim 2:a ground truth first coreference mention integrated with coreference mentions and antecedents thereof to thereby compute, resolve, etc. a reward, gradient based thereon and to update, train, etc. the policy network thereby, such as with respect to optimizing epochs, error level, iterations, etc.). The claim is considered obvious over Che as modified by Che_2 and Fei as addressed in the base claim as it would have been obvious to apply the further teaching of Che, Che_2, and/or Fei to the modified device of Che, Che_2, and Fei; one of ordinary skill in the art would have expected only predictable results therefrom.
Regarding claim 9
Che in view of Che_2 in view of Fei teaches or suggests:
The apparatus of claim 1, wherein the training unit calculates a coreference recognition error between the first coreference mention and the second coreference mention, and trains a coreference recognition parameter of the coreference recognition unit based on the error (see claim 1 supra; Fei: ¶ 31, 58; claim 2: a ground truth first coreference mention integrated with coreference mentions and antecedents thereof to thereby compute, resolve, etc. a reward, gradient based thereon and to update, train, etc. the policy network thereby, such as with respect to optimizing epochs, error level, iterations, etc.). The claim is considered obvious over Che as modified by Che_2 and Fei as addressed in the base claim as it would have been obvious to apply the further teaching of Che, Che_2, and/or Fei to the modified device of Che, Che_2, and Fei; one of ordinary skill in the art would have expected only predictable results therefrom.
Regarding claim 10
Che teaches:
An apparatus for learning-based coreference resolution (Che: Abstract: a dependency graph maps entity relations), comprising one or more computer-executable modules being configured and executed by a processor using algorithms associated with at least one non-transitory storage device, the algorithms, when executed, causing the processor to execute the one or more computer-executable modules (Chen: Abstract; ¶ 16-18, 27-29; Fig 1-3: such as the disclosed computer bearing processor, instructions borne upon non transitory memory, etc.), the one or more computer-executable modules comprising:
a first data storage that stores a natural language paragraph and a first coreference mention preset for the natural language paragraph (Che: ¶ 23, 24, 27, 45, 46, 54, 63; Fig 2, 3, 5: first storage devices 16 provide data to dialog computer; dialog computer processes input documents and stores same in second memory 32, 34 and/or in cache memory associated with processor 30; dialog computer stores received text and encodes same for processing and buffering by a graph learning architecture as depicted in figure 5; golden labels and structured, labelled, etc. data are additionally stored and retrieved for training of the figure 5 graph learning architecture; the system thus operates to generate contextual word embeddings by building a coreference relation graph based on all the words of received sentences in an input document and determining a sentence level dependency graph by parsing the document sentence wise; paragraphs are thus considered a set of sentences which comprise a subset of each/any document); a second data storage (id.: the memory of the dialog computer);
a data generation module configured to extract one or more natural language sentences from a natural language paragraph from the first data storage (Che: Abstract; ¶ 15, 45, 48, 49: for each input document unstructured natural-language sentences are provided as input to a machine-learning model to generate contextual word embeddings by building a coreference relation graph based on all the words of received sentences in an input document and determining a sentence level dependency graph by parsing the document sentence wise) and perform dependency parsing on the natural language sentences to generate dependency relation data of the natural language sentences (Che: Abstract; ¶ 2, 15, 34, 37, 45, 48, 49: sentence level dependency relations or parse sentences operable to generate a dependency tree; as such Che constructs a relation graph using coreference and dependency relations to form a coreference graph and dependency graph by extracting sentences from the input and performing dependency parsing thereon to generate dependency relation data of the sentences) and store the natural language paragraph, the first coreference mention, the natural language sentence, and the dependency relation data in the second data storage (Che: Abstract; ¶ 45-49, 65; Fig 5: such as by buffering in memory 32, 34 and/or memory associated with processor 30);
an embedding module that configured to extract the natural language sentence and the dependency relation data from the second data storage (Che: Abstract; ¶ 45-49, 65; Fig 5: system generates a graphs based on sentences, words, etc. of input documents, generates a graph of dependencies a graph of coreferences in concert with determined embeddings and provides same to a graph neural network wherein the embedding layer of the system receives a matrix of word vectors, values etc. as input and at least the embedding and encoding layers of the system output vectors); and generate an integrated embedding vector for the natural language paragraph based on the natural language sentence and the dependency relation data (Che: Abstract; ¶ 5-7, 45-49, 65; Fig 5; Claim 1: system generates a graphs based on sentences, words, etc. of input documents, generates a graph of dependencies, a graph of coreferences, etc. in concert with determined embeddings and provides same to a graph neural network wherein the embedding layer of the system receives a matrix of word vectors, values etc. as input and at least the embedding and encoding layers of the system output vectors and thereby generates an entity relation graph based thereon), the integrated embedding vector including semantic information of the natural language paragraph and semantic structure information of the natural language sentence (Che: Abstract; ¶ 5-7, 45-49, 65; Fig 5; Claim 1: semantic information encoded within the word embeddings of the overall document; paragraphs, sentences, etc. therein; semantic sentence structure encoded within, upon, etc. the dependency and coreference graphs which operate in concert with the graph neural network (GNN) to update embeddings that integrate both semantics and structure thereby operating in a manner similar to the integrated embedding vector); and
a coreference resolution module comprising a deep learning neural network calculation unit configured to input the integrated embedding vector to the deep learning neural network to generate a calculation result of the deep learning neural network (Che: Abstract; ¶ 5-7, 45-54, 58, 65; Fig 5; Claim 1, 2, 6, 14, 17: integrated embedding vector such as the word embeddings input to the to the BiLSTM encoding layer, an RNN) which produces contextualized encodings and provides to the GNN which fuses the input word embedding within, upon, etc. the coreference and dependency graphs to produce updated word embeddings as output to the conditional random field decoding layer such that deep learning calculation results are generated at each step, layer, etc. of the processing pipeline to produce calculation, decoding, etc. results such as for prediction such as of entities, coreference entity mentions, etc. ). Thus Che operates to perform named entity recognition based in an entity relation, coreference and dependency graph to obtain updated word embedding which decode to provide enriched entity predictions.
Che strongly suggests but does not explicitly name a training data generation module Che merely performs the recited steps by similarly deriving dependency relations; Che additionally strongly suggests but does not explicitly teach the performance of dependency parsing on the natural language sentences to generate relation data thereof; the integrated embedding vector comprising semantic information of paragraphs and semantic structure information of the sentences; a mention detection unit configured to detect a mention in the natural language paragraph based on the calculation result of the deep learning neural network, a coreference recognition unit configured to generate a second coreference mention of the natural language paragraph based on a result of the mention detection, a training unit configured to calculate a coreference recognition error between the first coreference mention and the second coreference mention, and train the deep learning neural network based on the error to generate a coreference resolution model.
In a related field of endeavor and describing the same work Che_2 teaches a system and method for “Capturing Relations between Entity Mentions via Graph Neural Networks for Domain-specific Named Entity Recognition,” comprising: a training data generation module that extracts configured to extract one or more natural language sentences from a natural language document (Che_2: Abstract): system determines, connects, etc. entity mentions based on coreference and dependency relations to improve entity mention representation); and to perform dependency parsing on the natural language sentences to generate dependency relation data of the natural language sentences (Che_2: § 3: coreference and dependent relations between entity mentions connect mentions in syntactic dependency relations);
an embedding module configured to generate an integrated embedding vector for the natural language paragraph based on the natural language sentence and the dependency relation data, the integrated embedding vector including semantic information of the natural language paragraph and semantic structure information of the natural language sentence (Che_2: § 3; Fig 2: system uses a contextual encoder such as BERT or the pictured BiLSTM to obtain contextual word embeddings comprising semantic information of a natural language input; outputs the embeddings to build an entity relation graph based on coreference and dependency relations of the input word embeddings and applies a graph attention network/graph neural network over the constructed entity relation graph wherein nodes of the GNN comprise semantic embeddings which communicate along dependency and coreference edges to output enhanced word embeddings such as in an integrated embedding vector inclusive of sematic information and semantic structure information of sentences);
a deep learning neural network calculation unit configured to input the integrated embedding vector to the deep learning neural network to generate a calculation result of the deep learning neural network, (Che_2: an input to the GNN is decoded; subsequent to the construction of the entity relation graph, mention representations and entity relations are input to the GAN to output refined mention representations based on neighboring data; said refined mention representations input to a sequence tagging decoder which outputs a set of refined representations in concert with label scores and ultimately entity tags which are used for improved named entity recognition). It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to utilized the modules described in Che_2 as an enhancement to the Che system, method, etc. for at least the purpose of improving named entity recognition based on the particular operations described therein; one of ordinary skill in the art would have expected only predictable results therefrom.
Che in view of Ch_2 does not explicitly teach a mention detection unit configured to detect a mention in the natural language paragraph based on the calculation result of the deep learning neural network, a coreference recognition unit configured to generate a second coreference mention of the natural language paragraph based on a result of the mention detection, a training unit configured to calculate a coreference recognition error between the first coreference mention and the second coreference mention, and train the deep learning neural network based on the error to generate a coreference resolution model.
In a related field of endeavor Fei teaches a deep reinforcement learning system and method (Fei: Title; ¶ 27, 29) for coreference resolution (Fei: ¶ 29, 30) such as by iteratively generating training data such as co-reference graphs for input documents, paragraphs, sentences thereof used over a plurality of epochs (Fei: ¶ 29, 30; Fig 1, 2; claims 2, 16) and a first coreference mention preset (Fei: ¶ 5, 6, etc.; claim 1, 2: such as the recited ground truth coreference information for a document and operable for training a coreference resolution model) and comprising:
a mention detection unit configured to detect a mention in the natural language paragraph based on the calculation result of the deep learning neural network (Fei: ¶ 5, 6, 41-49; Fig 4, 6: system generates attention spans, refines and prunes same, to thereby obtain mention scores using a mention neural network module by applying self-attention on the generated span representations; the mention scores used to generate coreference scores based on mention scores and antecedent scores; the generated coreference scores used to generate a probability distribution for each mention in such a way as to generate contextualized representations of words upon a document and a document with respect to words therein the recited paragraphs are considered encompassed in this way),
a coreference recognition unit configured to generate a second coreference mention of the natural language paragraph based on a result of the mention detection (Fei: ¶ 5, 6; Fig 4, 6: system determines linking actions among/between mentions and updates a coreference graph based thereon; a mention detected in an unpruned or high scored span used in a linking step that selects prior mention antecedents and creates an edge connection thereby in the coreference graph in a manner functional to generate a second coreference based on the linked prior mention resultant from subsequent mention detection), and
a training unit configured to calculate a coreference recognition error between the first coreference mention and the second coreference mention, and train the deep learning neural network based on the error to generate a coreference resolution model (Fei: ¶ 5, 6, 31, 39, 68-72; Fig 4, 6; Table 1; Claim 1, 2: a training reward based on coreference evaluation metrics comparing predicted and ground truth coreference information used to train, update, refine the policy network by generating and updating a gradient based thereon; such as by utilizing, optimizing, etc. one or more coreference metrics such as based on iterative error measurements using the coreference metrics).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to utilize the Fei taught mention detection, coreference linking and training based coreference resolution architecture; such as by applying the Fei teachings at the paragraph level to thereby improve the Che in view of Che_2 dependency and coreference resolution capabilities based on the Che in view of Che_2 integrated embeddings and annotated labelled mentions for at least the purpose of better providing improved inputs to the Fei coreference model to thereby better train or refine the Pei coreference model based thereon; one of ordinary skill in the art would have expected only predictable results therefrom.
Regarding claim 11, 18—the claims are considered to recite substantially similar subject matter to that of claim 2 and are similarly rejected.
Regarding claim 12—the claim is considered to recite substantially similar subject matter to that of claim 3 and is similarly rejected.
Regarding claim 14—the claim is considered to recite substantially similar subject matter to that of claim 5 and is similarly rejected.
Regarding claim 15—the claim is considered to recite substantially similar subject matter to that of claim 6 and is similarly rejected.
Regarding claim 17—the claim is considered to recite substantially similar subject matter to that of claims 1, 10 and is similarly rejected.
Regarding claim 20
Che in view of Che_2 in view of Fei teaches or suggests:
The method of claim 17, further comprising inferring a third coreference mention for the natural language paragraph input by a user using the coreference resolution model (Fei: ¶ 31, 58; claim 2: a ground truth first coreference mention integrated with coreference mentions and antecedents thereof to thereby compute, resolve, etc. a reward, gradient based thereon and to update, train, etc. the policy network thereby which when employed by a user operates to detect or resolve coreferences, scores thereof). The claim is considered obvious over Che as modified by Che_2 and Fei as addressed in the base claim as it would have been obvious to apply the further teaching of Che, Che_2, and/or Fei to the modified device of Che, Che_2, and Fei; one of ordinary skill in the art would have expected only predictable results therefrom.
Response to Arguments
Applicant’s arguments in concert with claim amendments, see Remarks and Claims, filed 4/16/26, with respect to the rejection(s) of claim(s) 1-3, 5, 6, 8-12, 14, 15, 17, 18, 20 under 35 USC 103 over Che in view of Fei have been fully considered and are persuasive. Therefore, the rejection has been withdrawn. However, upon further consideration, a new ground(s) of rejection is made under 35 USC 103 over Che in view of Che_2 in view of Fei. Applicant’s arguments are considered addressed inline in the above art rejection as Che in view of Ch_2 teaches generating dependency based relation data from parsed sentences and fusing encoded word embeddings with dependency and coreference graphs using a GNN to produce integrated embedding vectors in the form of the updated word, entity, etc. embeddings which include semantic and sentence structure information for sentences, sets of sentences, paragraphs, etc. as these comprise the documents and are borne within the recited graphs. These are subsequently input to a deep neural network for further calculation. Fei supplies the remaining coreference pipeline flow; a mention detection neural network operates to score spans and retain appropriate spans to process with a policy network which calculates coreference recognition error between first and second mentions thereof by operating over detected mentions to select actions linking the mention to prior mentions and thereby update a coreference graph, this informs a training procedure which compares the updated coreference graph with ground truth coreferences graphs evaluating any error therein using coreference evaluation metrics and updating, improving, etc. the overall model using rewards based thereon (please see claim 1). As such the arguments are considered addressed supra and no claims currently stand allowable.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PAUL C MCCORD whose telephone number is (571)270-3701. The examiner can normally be reached 730-630 M-F.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, CAROLYN EDWARDS can be reached at (571) 270-7136. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/PAUL C MCCORD/Primary Examiner, Art Unit 2692
Prosecution Insights