Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103 is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Claim Rejections - 35 USC § 103
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 103 that form the basis for the rejections under this section made in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 6, 8, 10, 12, 17, and 19 are rejected under 35 USC 103(a) as being unpatentable over Dong et al. (US 2024/0153296 A1) in view of Nikumb et al. (US 2021/0397418 A1).
Regarding Claims 1, 12, and 19, Dong discloses a computing system (Fig. 1) comprising memory and one or more processors communicatively coupled to the memory (¶17, processor 112 and non-transitory, computer readable memory 114 storing instructions for execution by processor 112), the one or more processors configured to:
identify a plurality of data entity tokens from a target section of a multi-section natural language document (¶20, identify text portions of a document and define bounding boxes around respective text portions of the document to identify individual characters of text in each text portion; e.g., ¶23 and Fig. 2, define bounding box around a contiguous text portion comprising a continuous set of characters);
generate, using an embedding layer of a semantic chunking model, a text span embedding for a text span of the target section (¶24, input embeddings generator module 118 / sentence transformer generates an embeddings vector representative of a series of characters for each text portion / bounding box; ¶51, an end-to-end model comprising the input embeddings generator module 118, transformer-based model 122 + attention score determination module 124 and label generator module 126);
generate, using a span attention layer of the semantic chunking model, a plurality of token-level span attention vectors for the plurality of data entity tokens based on the text span (¶28, transformer based model module 122 receives embedding vectors of text portions and relationship graphs representative of bound boxes / text portions (per ¶25) to determine (1) attention scores for node connections with the graph (see equation (1) in ¶30) and (2) output embedding vectors (see equation (3) in ¶36); ¶51, end-to-end model comprising the input embeddings generator module 118, transformer based model 122 + attention score determination module 124 and label generator module 126);
generate an attended span representation for the text span based on the text span embedding and the plurality of token-level span attention vectors (¶75, the transformer-based model module being a machine learn model with a plurality of layers to calculate, for each text bounding box at each layer, a weighted sum of respective attention value weight of other text bounding boxes to determine respective output embeddings for each respective text bounding box; in view of ¶12, use multiple vectors to encode different forms of pair-wise spatial information to better capture an entity’s importance based on entity’s own absolute positional and content information and relative spatial positions);
identify, using a span classification layer of the semantic chunking model, an entity topic that corresponds to the text span based on the attended span representation (¶33, attention scores enable more accurate label predictions and classifications of nodes, sub-graphs and graph itself (per ¶12, entity (e.g., text portion) classification outperforms known methods); ¶37, a label generator module 126 receives output embeddings calculated by the transformer based model module 124 and outputs labels respective of the graph representing the document; ¶39, perform combinatorial matching where one field category corresponds to only one entity and predict labels for each text portion; ¶51, end-to-end model comprising the input embeddings generator module 118, transformer-based model 122 + attention score determination module 124 and label generator module 126); and
responsive to an identification of the entity topic, generate a subgraph data object for a knowledge graph using the text span (¶37, the label generator module 126 outputs a respective label for each of one or more sub-graphs within the graph).
Dong does not disclose identify, using a natural language understanding (NLU) model, the plurality of data entity tokens from the target section of the multi-section natural language document.
Nikumb discloses using NLU model to identify a plurality of data entity tokens from target sections of multi-section natural language document (¶20, using NLU model to extract entity data for requirements documents (¶14, requirements document is a document such as a request for proposal document, software requirement document, user case document etc.)).
It would’ve been obvious to one ordinarily skilled in the art before the effective filing date of the invention to use a NLU model to identify data entity tokens from the target section of the multi-section natural language document in order to extract entity data (Nikumb, ¶20) corresponding to entity content information (Dong, ¶12).
Further regarding Claim 19, Dong discloses one or more non-transitory computer-readable storage media including instructions that, when executed by one or more processors, cause the one or more processors to implement the method of claim 1 and computing system of claim 12 (¶17, processor 112 and non-transitory, computer readable memory 114 storing instructions for execution by processor 112).
Regarding Claims 6 and 17, Dong discloses wherein:
(i) the knowledge graph comprises a plurality of section topic subgraphs respectively corresponding to a plurality of section topic pairs from a plurality of a multi-section natural language documents (¶60 and Fig. 6, a plurality of sub-graphs 602-606 of entity to entity connections from which categorization labels are generated),
(ii) the subgraph data object comprises a span portion of a section topic subgraph of the plurality of section topic subgraphs that corresponds to the target section and the entity topic (¶60, graph 600 representing a network of transacting entities with each sub-graph representing an entity, where a node labelling system classifies one or more individual entities in the sub-graph and to classify the entire sub-graph), and
(iii) the section topic subgraph comprises one or more span portions respectively corresponding to one or more text spans of the target section (Fig. 6, sub-graphs 602, 604, 606 comprising edges denoting respective entities that have entered into a transaction together).
Regarding Claim 8, Dong discloses wherein the text span comprises a subset of the plurality of data entity tokens and one or more entity relationship tokens associated with the subset of data entity tokens (¶60, subgraphs of entity to entity connections with an edge denoting two entities that have entered into a transaction together; in view of ¶18, reducing input data set to a connected relationship graph) and generating the subgraph data object for the knowledge graph using the text span comprises:
generating one or more entity factor nodes respectively corresponding to the subset of data entity tokens (¶60, each sub-graph is representative of an entity where data respective of the entities represented by the nodes of that graph), and
generating one or more entity factor edges between the one or more entity factor nodes and respectively corresponding to the one or more entity relationship tokens (¶60, with an edge denoting two entities that have entered into a transaction together).
Regarding Claim 10, Dong discloses the computer-implemented method of claim 1, further comprising:
generating, using the span classification layer of the semantic chunking model, a type classification prediction for the text span that identifies a mutually exclusive topic type from one or more mutually exclusive topic types associated with the entity topic (Fig. 4 and see ¶50, label generator module 126 generates label 416 for respective node in graph 408 based on the respective output embeddings vector); and
in response to the type classification prediction satisfying a classification threshold, identifying the text span as a valid span (¶39, label generator module performs combinatorial matching base on a matching cost of a possible label and a predicted label; i.e., determine an assigned label based on the cost or classification threshold), and
generating the subgraph data object for the mutually exclusive topic type (Fig. 4, see mutually exclusion labels 416).
Claims 2-4 and 13-15 are rejected under 35 USC 103(a) as being unpatentable over Dong et al. (US 2024/0153296 A1) and Nikumb et al. (US 2021/0397418 A1) as applied to claims 1 and 12, in view of Yang et al. (Hierarchical Attention Networks for Document Classification).
Regarding Claims 2 and 13, Dong discloses wherein the target section is selected from a plurality of candidate sections of the multi-section natural language document (Dong, ¶12 and Fig. 2, perform entity (e.g., text portion) classification) and the computer-implemented method further comprises:
identifying a plurality of text attributes of the multi-section natural language document based on a formatting of the multi-section natural language document (Dong, ¶20, identify text portions of the document and identify individual characters of text in each text portion; see document format in Fig. 2);
identifying the plurality of candidate sections based on the plurality of text attributes (Dong ¶20, define bounding boxes around respective text portions of the document);
as modified by Nikumb, identifying, using the NLU model (Nikumb, ¶28, using NLU model to parse natural language included in the document to identify an entity), a plurality of entity topics based on the plurality of candidate sections and the plurality of text attributes (Dong ¶22, determine features for each of the bounding boxes such as characteristics of the texts enclosed within the bounding boxes; in view of Dong ¶12, each text portion corresponds to an entity; i.e., apply the NLU model on the text portion (text content comprising a string of characters per Dong ¶48) to identify the entity for each text portion); and
identifying the plurality of data entity tokens based on the plurality of entity topics (Dong ¶23, define bounding box around a contiguous text portion; in view of Dong ¶12, each text portion corresponds to an entity; i.e., identify each text portion as defined by respective bounding box as an entity based on NLU parsing).
Dong does not disclose that the text attributes are hierarchical text attributes.
Yang discloses that documents have hierarchical text attributes (1. Introduction, “First, since documents have a hierarchical structure (words form sentences, sentences form a document), we likewise construct a document representation by first building representations of sentences and then aggregating those into a document representation”).
It would’ve been obvious to one ordinarily skilled in the art before the effective filing date of the invention to identify hierarchical text attributes of the multi-section natural language document such as words and sentences in the respective text portion / text content and using the NLU model to parse the text portion / text content comprising words and sentences for entity topic in order to build a document representation for document classification (Yang, Abstract).
Regarding Claims 3 and 14, Dong discloses wherein generating the plurality of token-level span attention vectors comprises:
generating a plurality of section vectors from the multi-section natural language document based on the plurality of candidate sections, wherein the plurality of section vectors comprises a target vector for the target section that encodes one or more of the plurality of hierarchical text attributes associated with the target section (¶24, input embeddings generator module 118 of the end to end model (¶51) includes a sentence transformer to generate m-dimensional vector that is representative of a text portion; for example the sentence transformer generates a first embedding vector representative of the text content of a text portion; i.e., if a text portion comprises a word, then the embedding vector represents a word; if a text portion comprises a sentence, then the embedding vector represents a sentence);
extracting a plurality of token vectors from the target vector based on the plurality of data entity tokens (in view of ¶12, each text portion is an entity; i.e., ¶24, the sentence transformer generates first embedding vectors representative of the text content of text portion corresponding to entities); and
generating, using a transformer layer of the semantic chunking model, a token-level span attention vector for each of the plurality of token vectors (¶28, transformer based model module 122 of the end to end model (¶51) receives input embeddings respective of each text portion to determine attention scores for determining output embedding vectors).
Regarding Claims 4 and 15, Dong discloses wherein:
(i) the transformer layer is configured to output a plurality of token-level span embeddings and (¶¶30-31, equations (1)-(2) showing xi being embeddings vector associated with node I being used to calculate vectors from node i to node j: ei, j),
(ii) the plurality of token-level span attention vectors is generated, using the span attention layer of the semantic chunking model, based on the plurality of token-level span embeddings (¶36, equation (3) calculate output embedding vectors zi based on attention ai, j from equation (1), which was calculated from ei, j), and
(iii) the entity topic is identified based on the attended span representation and (¶37, generate a respective label for each node of the graph (each node corresponds to a respective text portion or entity per ¶12) and a label for the graph in its entirety (i.e., the graph being the document in its entirety); see Fig. 4).
Dong does not disclose the transformer layer outputs a document classification vector and to identify the entity topic based on the document classification vector.
Yang teaches outputting a document classification vector and to identify the entity topic based on the document classification vector (2.3 Document classification, use document vector v as features for document classification).
It would’ve been obvious to one ordinarily skilled in the art before the effective filing date of the invention to modify the transformer layer / transformer based model module 122 of the end to end model to output a document classification vector by aggregating embeddings of the sentence transformer (Yang, 1 Introduction “…we likewise construct a document representation by first building representations of the sentences and then aggregating those into a document representation”; compare Dong, ¶24 and ¶28; i.e., aggregating the sentence embeddings of sentence transformer in module 118 of the end to end model to generate embedding vector of a graph / document in its entirety) and to identify the entity topic based on the document classification vector in order to output a label for the graph / document in its entirety (Dong, ¶37).
Claims 5 and 16 are rejected under 35 USC 103(a) as being unpatentable over Dong et al. (US 2024/0153296 A1) and Nikumb et al. (US 2021/0397418 A1) as applied to claims 6 and 17, in view of Shailabh et al. (US 2023/0169271 A1).
Regarding Claims 5 and 16, Dong as modified by Nikumb discloses wherein identifying the entity topic comprises: identifying, using the NLU model, a plurality of entity topics from the multi-section natural language document (Nikumb, ¶28, apply NLU model to process requirements document by parsing natural language in the requirements document to obtain data identifying a description of the application (i.e., topic)).
The combination does not teach receiving a plurality of topic embeddings respectively corresponding to the plurality of entity topics.
Shailabh discloses receiving a plurality of topic embeddings respectively corresponding to the plurality of entity topics (¶78, equation (2)) and identifying entity topic based on a comparison between the plurality of topic embeddings and attended span representation (¶83, topic attention network 500 computes document embedding 520 according to equation (8) (i.e., attended span representation comprising aggregated word embeddings of the document) and multiplying the document embedding with topic embedding according to equation (9); this is a similarity function per ¶77).
It would’ve been obvious to one ordinarily skilled in the art before the effective filing date of the invention to use the span classification layer of the semantic chunking model (Dong, ¶28, transformer based model module 122) to identify the entity topic based on a comparison between the plurality of topic embeddings and the attended span representation in order to identify topics included in a document (Shailabh, ¶86).
Claims 7 and 18 are rejected under 35 USC 103(a) as being unpatentable over Dong et al. (US 2024/0153296 A1) and Nikumb et al. (US 2021/0397418 A1) as applied to claims 6 and 17, in view of Zhong et al. (US 2023/0139397 A1).
Regarding Claims 7 and 18, Dong does not disclose receiving a query comprising a text sequence.
Zhong discloses a system for generating a structured rule set for query based on natural language document (¶132, system 600 for utilizing raw text document data to generate question and answer pair data; ¶133, Figs. 5-6) comprising:
receiving a query comprising a text sequence (¶114 and ¶121, receiving raw text document comprising frequently asked questions (i.e., queries));
identifying one or more data entity tokens from the text sequence (¶122 and ¶127, parse raw text document data to extract sentence data including text data (i.e., entities) representing semantic aspects of the sentences and generate sentence embedding data);
identifying section topic subgraph based on the one or more data entity tokens (¶127, apply contextual model’s weights in a nodal hierarchy to the sentence embedding data to generate or predict context values (per ¶137, contextual meanings)); and
generating a structured rule set for the query based on the section topic subgraph (¶128, classification model takes context aware value from contextual model and apply a set of weights in a nodal hierarchy to generate one or more labels for one or more sentences extracted from the raw text document data; ¶129, labels indicate whether a sentence related to a question or an answer; in view of ¶114, the prediction labels corresponding to question and answers in the FAQ document are used as training data (i.e., structured rule set) to train a QA chatbot model to answer questions similar to those in the FAQ document).
It would’ve been obvious to one ordinarily skilled in the art before the effective filing date of the invention to receiving a query comprising a text sequence, identifying one or more data entity tokens from the text sequence, identifying section topic subgraph based on the one or more data entity tokens and generating a structured rule set for the query based on the section topic subgraph to implement a question and answer chatbot to receive queries / questions from humans and send a predicted answer (Zhong, ¶114).
Claim 9 is rejected under 35 USC 103(a) as being unpatentable over Dong et al. (US 2024/0153296 A1) and Nikumb et al. (US 2021/0397418 A1) as applied to claim 8, in view of Scheideler et al. (US 2019/0155926 A1).
Regarding Claim 9, Dong does not disclose removing an entity factor edge from the one or more entity factor edges based on one or more edge pruning criteria.
Scheideler teaches pruning a knowledge graph (Abstract) by removing an entity factor edge from the one or more entity factor edges based on one or more edge pruning criteria (¶31, nodes in the knowledge graph denote facts or entities; ¶70, edges in knowledge graph have a defined weight which indicates significance of the edge or level of proximity of adjacent node; per ¶32 and ¶41, delete nodes with low relevance degree values and surrounding edges, the relevance degree value denote the importance of a node in the knowledge graph).
It would’ve been obvious to one ordinarily skilled in the art before the effective filing date of the invention to remove an entity factor edge from one or more entity factor edges based on one or more edge pruning criteria to delete nodes and edges in a knowledge graph to resolve the problem of exponentially growing knowledge graphs that are over-complex with outdated sub-trees / sub-graphs (Scheideler, ¶¶1-2).
Claims 11 and 20 are rejected under 35 USC 103(a) as being unpatentable over Dong et al. (US 2024/0153296 A1) and Nikumb et al. (US 2021/0397418 A1) as applied to claims 1 and 19, in view of Venkateshwaran et al. (US 2023/0252239 A1).
Regarding Claims 11 and 20, Dong does not disclose generating, using a supervised machine learning model, a plurality of sentence relevancy predictions for a plurality of section sentences of the target section.
Venkateshwaran discloses processing natural language documents to identify data entity tokens from target section of the natural language documents (Abstract, extracting domain specific insights from large documents by breaking down large chunks of text into smaller sentences, identify sentence intents, tagging sentences with domain specific attributes, and defining a semantic ontology using a graph database based on sentence intent) comprising:
generating, using a supervised machine learning model (¶107, supervised machine learning module 916 performing predictions per Fig. 9), a plurality of sentence relevancy predictions for a plurality of section sentences of the target section (¶141 and ¶173, using supervised learning based classifier to make predictions by classifying each sentence based on its intent);
identifying one or more relevant sentences from the plurality of section sentences based on the plurality of sentence relevancy predictions (¶¶142-146); and
identifying the plurality of data entity tokens from the one or more relevant sentences (¶147, tag each sentence with domain relevant categories; e.g., “Auto”, “Homeowners” are domain relevant categories of the sentence based on various aspects of sentence text).
It would’ve been obvious to one ordinarily skilled in the art before the effective filing date of the invention to use supervised machine learning model to identify relevant sentences and identify data entity tokens from the one or more relevant sentences in order to break large documents down into smaller sentences (i.e., chunks), identify sentence intents (i.e., relevant sentences), and tag sentences with domain specific attributes (i.e., data entity tokens) in order to define a semantic ontology (i.e., a graph) for extracting domain specific insights from large documents (Venkateshwaran, Abstract).
Conclusion
Prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
US 2024/0202454 A1 discloses pre-trained language model for semantic chunking.
US 2025/0131289 A1 discloses obtaining aggregated summaries and related knowledge graph relating to a database.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to examiner Richard Z. Zhu whose telephone number is 571-270-1587 or examiner’s supervisor Hai Phan whose telephone number is 571-272-6338. Examiner Richard Zhu can normally be reached on M-Th, 0730:1700.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/RICHARD Z ZHU/Primary Examiner, Art Unit 2654 04/04/2026
Prosecution Insights