DETAILED ACTION
This action is responsive to the claims filed on 01/12/2026. Claims 1-6 and 8-20 are pending for examination.
Response to Arguments
Applicant’s arguments with respect to the 35 U.S.C. 103 rejection of the claims have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or non-obviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1-3, 4, 5, 6, 8-9, 11-13, 15, 16-18, and 20 are rejected under 35 U.S.C. 103 as being unpatentable by Yin et al., ((2020). TaBERT: Pretraining for joint understanding of textual and tabular data. arXiv preprint arXiv:2005.08314.), hereafter referred to as Yin, in view of Riemers et al., (Reimers, N., & Gurevych, I. (2019, November). Sentence-bert: Sentence embeddings using siamese bert-networks. In Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (EMNLP-IJCNLP) (pp. 3982-3992).), hereafter referred to as Reimers, and in further view of Cvitkovic (Cvitkovic, M. (2020). Supervised learning on relational databases with graph neural networks. arXiv preprint arXiv:2002.02046.), hereafter referred to as Cvitkovic, and Bouyarmane et al., (US 11797530), hereafter referred to as Bouyarmane.
Claim 1: Yin teaches the following limitations:
receiving a corpus of training data including a plurality of data tables, wherein each data table defines a respective set of attributes for a respective set of data entities corresponding to each data table entity schema, wherein a first data entity of a first set of data entities of first data table entity schema is associated with a topic characteristic based on a first set of attributes defined by the first data table, and wherein a first attribute of the first set of attributes is associated with a structural characteristic that is common across each of the first set of data entities of the first data table; (Yin, page 3, figure 1, “A content snapshot of the table is created based on the input NL utterance.”, an example training data table is shown in figure 1, data entities represented by each row are associated to a topic characteristic found by the vertical self-attention layer outlined in section 3.1. Attributes are represented by columns and their identifier is the name of the column. Attributes shares structural characteristic from each other that is common across data entities.)
identifying a respective column name for each attribute of the first set of attributes of the first data table; (Yin, page 6, col. 2, paragraph 2, “Its column representation cj is defined by mean-pooling over the Transformer’s output encodings that correspond to the column name (e.g., the representation for the Year column in Fig. 1 is derived from the vector of the Year token in Eq. (1)).”, the column representation or column attribute is represented by the column name.)
at a pre-trained transformer (Yin, page 2, col. 1, paragraph 2, “In this paper we present TABERT, a pretraining approach for joint understanding of NL text and (semi-)structured tabular data (§ 3). TABERT is built on top of BERT, and jointly learns contextual representations for utterances and the structured schema of DB tables (e.g., a vector for each utterance token and table column). Specifically, TABERT linearizes the structure of tables to be compatible with a Transformer-based BERT model.”, Yin expressly introduces TABERT as “a pretrained language model that jointly learns representations for NL sentences and (semi-)structured tables,” and describes it as built on a Transformer-based language model (BERT) that is pretrained on large corpora of text and table/context pairs before being used as a feature extractor for downstream tasks. Accordingly, the Transformer encoder in TABERT is a pretrained Transformer-based machine-learning model.)
a respective concatenation comprising the respective column name and an attribute value for each attribute of the first set of attributes corresponding to the first data entity; (Yin, page 4, col. 1, section “Row Linearization,” “TABERT creates a linearized sequence for each row in the content snapshot as input to the Transformer model. Fig. 1(B) depicts the linearization for R2, which consists of a concatenation of the utterance, columns, and their cell values. Specifically, each cell is represented by the name and data type of the column, together with its actual value, separated by a vertical bar. As an example, the cell … valued 2005 in R2 in Fig. 1 is encoded as Year | real | 2005.” Yin’s Figure 1 identifies R2 as a particular row, corresponding to the claimed first data entity, and depicts R2 as containing respective cells for the Year, Venue, Position, and Event attributes. Yin therefore forms, for each attribute corresponding to the particular data entity R2, a respective cell string containing the applicable column name and attribute value, such as “Year | real | 2005,” “Venue | text | Erfurt,” and “Position | text | 1st.”)
a respective attribute embedding for each attribute of the first set of attributes corresponding to the first data entity; (Yin, page 4, col. 2, section “Vertical Self-Attention Mechanism,” “To generate aligned inputs for vertical attention, we first compute a fixed-length initial vector for each cell at position ⟨i, j⟩, which is given by mean-pooling over the sequence of the Transformer’s output vectors that correspond to its variable-length linearization as in Eq. (1).”; section “Utterance and Column Representations,” “A representation c_j is computed for each column c_j by mean-pooling over its vertically aligned cell vectors, {s_i,j: R_i in content snapshot}, from the last vertical layer.” Each cell position ⟨i, j⟩ identifies a particular row i and column or attribute j. Yin therefore generates a respective fixed-length cell vector for each attribute j of the particular data entity represented by row R2 and uses the respective cell vectors to generate representations for the corresponding columns. The fixed-length cell vector for each attribute of R2 constitutes the claimed respective attribute embedding.)
Reimers, in the same field of generating embeddings using pretrained transformer models, teaches the following limitations that Yin fails to explicitly teach:
inputting, into a respective transformer encoder block of a pretrained transformer, a respective concatenation (Reimers, page 3983, col. 1, “BERT … is a pre-trained transformer network … The input for BERT for sentence-pair regression consists of the two sentences, separated by a special [SEP] token. Multi-head attention over 12 (base-model) or 24 layers (large-model) is applied”; page 3984, Figures 1 and 2, respectively depicting Sentence A and Sentence B being provided to corresponding BERT networks, wherein “[t]he two BERT networks have tied weights (siamese network structure).” Reimers therefore teaches a pretrained transformer arrangement having respective BERT transformer-encoder branches that separately receive respective input sequences. In the proposed combination, each respective cell string generated by Yin, such as “Year | real | 2005,” is supplied as an individual input to a corresponding BERT transformer-encoder branch in the manner taught by Reimers. Accordingly, each respective concatenation comprising a column name and its attribute value is input into a respective transformer encoder block of the pretrained transformer.)
generating a respective attribute embedding… based at least in part on the inputting, (Reimers, page 3984, section 3, “SBERT adds a pooling operation to the output of BERT / RoBERTa to derive a fixed sized sentence embedding. We experiment with three pooling strategies: Using the output of the CLS-token, computing the mean of all output vectors (MEAN-strategy), and computing a max-over-time of the output vectors (MAX-strategy).”; see also Figures 1 and 2, depicting respective outputs u and v generated by pooling the output of the respective BERT branches. Reimers therefore generates a respective fixed-size embedding from the output of each corresponding pretrained BERT transformer-encoder branch. Because each output u or v is generated by pooling the output of the BERT branch that received the corresponding respective input, Reimers expressly teaches that each respective embedding is generated based on the inputting of its corresponding respective input sequence.)
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to have modified Yin’s method by processing each of Yin’s respective column-name/value cell strings using a respective tied-weight pretrained BERT transformer-encoder branch, as taught by Reimers, and pooling the output of each branch to generate the respective attribute embedding. Reimers explains that a conventional BERT cross-encoder requires the inputs to be processed together and causes substantial computational overhead, whereas its Siamese architecture independently generates fixed-size embeddings for respective inputs and reduces the computation required for comparison and retrieval while maintaining BERT accuracy. (Reimers, page 3982, Abstract and section 1, “This reduces the effort for finding the most similar pair from 65 hours with BERT / RoBERTa to about 5 seconds with SBERT, while maintaining the accuracy from BERT.”) A person of ordinary skill would therefore have been motivated to apply Reimers’s independently operating, tied-weight BERT branches to Yin’s respective cell strings to obtain independently reusable attribute embeddings, reduce the computational overhead associated with jointly reprocessing larger concatenated row sequences, and retain the representational accuracy of the pretrained BERT model.
Cvitkovic, in the same field of machine learning embedding, teaches the following limitations which Yin and Reimers fails to explicitly teach:
PNG
media_image1.png
264
546
media_image1.png
Greyscale
Figure 1 of Cvitkovic, Step 2a-2c of Appendix A
wherein generating a first attribute embedding for the first attribute of the first set of attributes comprises: generating a referenced entity embedding for a second data entity that is referenced by the first attribute, the referenced entity embedding generated using data of a second data table of the plurality of data tables; (Cvitkovic, page 2, paragraph 1, “What makes RDBs “relational” is that the values in a column in one table can refer to rows of another table. For example, in Figure 1 the column T Visit :,patient_id refers to rows in T Patient based on the values in T Patient :,patient_id. The value in T Visit i,patient_id indicates which patient came for Visit i. A column like this that refers to another table is called a foreign key.”, These quotes show that a column (i.e., the “first attribute”) in one table can refer to a row in another table (i.e., the “second data entity”), and that this is a foreign-key relationship. In Cvitkovic’s system, each row becomes a node in a graph, and each foreign key induces an edge to another node. The referenced entity embedding corresponds to the hidden state of that target node (hᵗ⁻¹ʷ), which is generated using that second table’s data and passed via message-passing to the first node.)
generating an entity embedding based on the respective attribute embedding for each attribute of the first set of attributes associated with the first data entity and the referenced entity embedding; (Cvitkovic, page 11, paragraph 2, “For each iteration t from 1 to T: (a) Each vertex v sends a “message”
PNG
media_image2.png
30
234
media_image2.png
Greyscale
to each of its neighbors w, where Nv is the set of all neighbors of v.
(b) Each vertex v aggregates the messages it received using a function At, where At takes a variable number of arguments and is invariant to permutations of its arguments:
PNG
media_image3.png
30
170
media_image3.png
Greyscale
(c) Each vertex v updates its hidden state as a function of its current hidden state and the aggregated messages it received:
PNG
media_image4.png
26
122
media_image4.png
Greyscale
”, Cvitkovic’s message-passing GNN takes a graph where each vertex v (a database row/entity) has features x_v and initializes a hidden state “h⁰_v = S(x_v)”; then, at each iteration, each vertex sends messages mᵗ_{vw} to neighbors, aggregates them mᵗ_v, and “updates its hidden state as a function of its current hidden state and the aggregated messages: hᵗ_v = Uᵗ(hᵗ⁻¹_v, mᵗ_v).” In the combination, x_v is formed from the per-attribute embeddings (in this combination it is provided by Yin), and neighbor states hᵗ⁻¹_w correspond to referenced entities linked by foreign keys. The resulting hᵗ_v is therefore an entity embedding based on (i) the attribute embeddings for each attribute of the entity and (ii) embeddings of referenced entities, as recited in the amended claim.)
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings disclosed by Yin and Reimers with the teachings disclosed by Cvitkovic (i.e., generating an entity level embedding using relational embeddings referenced by attributes). A motivation for the combination is to provide a method to combine relational structures of databases to embeddings without the need of manual processing. (Cvitkovic, page 1, section 1, paragraph 3, “we introduce a method based on Graph Neural Networks that operates on RDB data in its relational form without the need for manual feature engineering or flattening.”).
Bouyarmane, in the same field of machine learning embedding, teaches the following limitations which Yin, Reimers, or Cvitkovic fails to explicitly teach:
and parameterizing the topic characteristic for each data entity of the first set of data entities and the structural characteristic for each attribute of the first set of attributes in the machine learning model by generating the attribute embeddings and the entity embedding for each data entity of the first set of data entities (Bouyarmane, col. 10, lines 55-59, “ A given similarity analysis request may indicate a pair of entity records expressed not just in different languages, but also in different schemas in some embodiments, for example with some attributes of the first schema of the pair missing entirely from the second schema (and/or vice versa)”, the examiner interprets topic characteristics as representing the main topic or characteristic of that table of entities. In this case, an analysis request which comprises generating embeddings for attributes and entities, has a way to differentiate the topic characteristics of different schemas, such as having differing attributes.
Col. 2, lines 51-53, “Generally speaking, an entity record may be intended to be used to represent or capture various characteristics of a real-world object”, each entity table represents a certain characteristic of a real-world object.
Col. 16, lines 11-15, “For example, supported character lists containing 100 or 200 characters may be used in some embodiments. In at least some embodiments, the length of the supported character list 655 may be implemented as a tunable hyper-parameter.”, it is interpreted by the examiner that a structural characteristic represents a predefine structure or rule to determine the semantic order of values. In this case, a character list defines a structural characteristic of values.).
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings disclosed by Yin/Reimers/Cvitkovic with the teachings disclosed by Bouyarmane (i.e., generating an entity level embedding and parameterizing the characteristic throughout the dataset). A motivation for the combination is to provide further attention to similar attributes, thus allowing for more accurate identification of entity-attribute relations. (Bouyarmane, col. 4, lines 13-17, ” Because information contained in the entity records is encoded starting at the character level, problems such as misspellings of words, placing of words in the wrong attributes (e.g., including the color of an object in the “title” attribute), or even disparities in entity record schemas, may be overcome in the proposed embedding methodology.”).
Claim 2: Yin, Reimers, Cvitkovic, and Bouyarmane teaches the limitations of claim 1, Yin further teaches:
receiving an input that corresponds to a data entity and an indication of an attribute type identifier, wherein the attribute type identifier corresponds to a column name of at least one data table of the plurality of data tables; (Yin, page 3, figure 1, “A content snapshot of the table is created based on the input NL utterance.”, an example training data table is shown in figure 1, data entities represented by each row are associated to a topic characteristic found by the vertical self-attention layer outlined in section 3.1. Attributes are represented by columns and their identifier is the name of the column. Attributes shares structural characteristic from each other that is common across data entities.
Yin, page 6, col. 2, paragraph 2, “Its column representation cj is defined by mean-pooling over the Transformer’s output encodings that correspond to the column name (e.g., the representation for the Year column in Fig. 1 is derived from the vector of the Year token in Eq. (1)).”, the column representation or column attribute is represented by the column name.);
and generating, by the machine learning model, an output that includes a value corresponding to the attribute type identifier based at least in part on the input (Bouyarmane, col. 17, lines 4-11, “Similarly, output generated for all of the tokens of a given attribute, produced by token-level embedding model 754, may be combined and used as input to the attribute-level embedding model 756. In addition, output generated for all of the attributes of a given entity record, produced by attribute-level embedding model 756, may be combined and used as input to the entity-level embedding model 758.”, an output is generated which contains the attribute type identifier, which was prepended to the sequence of tokens as previously mentioned above).
Claim 3: Yin, Reimers, Cvitkovic, and Bouyarmane teaches the limitations of claim 1, Bouyarmane teaches the following limitations:
generating, by the machine learning model, an input embedding based at least in part on the input, wherein the output is generated based at least in part on the input embedding and the indication of the attribute type identifier (Bouyarmane, col. 17, lines 4-8, “Similarly, output generated for all of the tokens of a given attribute, produced by token-level embedding model 754, may be combined and used as input to the attribute-level embedding model 756.”, an input (tokens), which contains an attribute type identifier represented by a prepended attribute name, is embedded and is resultantly an output to an attribute-level embedding model).
Claim 4: Yin, Reimers, Cvitkovic, and Bouyarmane teaches the limitations of claim 1, Yin further teaches:
wherein each row of the first data table corresponds to a respective data entity of the first set of data entities (Yin, page 3, figure 1, “A content snapshot of the table is created based on the input NL utterance.”, an example training data table is shown in figure 1, data entities are represented by each row.).
Claim 5: Yin, Reimers, Cvitkovic, and Bouyarmane teaches the limitations of claim 1, Yin further teaches:
using a transformer based machine learning model as the pretrained transformer to generate the respective attribute embeddings and the entity embedding, wherein the transformer based machine learning model comprises one or more transformer encoder blocks that implements an attention mechanism that is configured to sample one or more token embeddings of tokens related to the topic characteristic and included in the attribute value for each attribute, wherein sampling the one or more token embeddings to generate the entity embeddings results in the parameterizing of the topic characteristic for each data entity of the first set of data entities. (Yin, page 4, col. 2, paragraph 1, “The base Transformer model in TABERT outputs vector encodings of utterance and cell tokens for each row. These row-level vectors are computed separately and therefore independent of each other. To allow for information flow across cell representations of different rows, we propose vertical self-attention, a self-attention mechanism that operates over vertically aligned vectors from different rows”, the transformer model used in Yin uses self-attention layers to sample token level embeddings.).
Claim 6: Yin, Reimers, Cvitkovic, and Bouyarmane teaches the limitations of claim 1, Yin further teaches:
wherein generating the entity embedding comprises: generating a sampling distribution using an attention layer that receives the respective attribute embedding for each attribute of the first set of attributes as input and sampling the sampling distribution to generate the entity embedding (Yin, page 4, col. 2, paragraph 1, “The base Transformer model in TABERT outputs vector encodings of utterance and cell tokens for each row. These row-level vectors are computed separately and therefore independent of each other. To allow for information flow across cell representations of different rows, we propose vertical self-attention, a self-attention mechanism that operates over vertically aligned vectors from different rows”, the transformer model used in Yin uses self-attention layers to sample token level embeddings.
Page 6, col. 2, paragraph 1, “During pretraining, for each table and its associated NL context in the corpus, we create a series of training instances of paired NL sentences (as synthetically generated utterances) and tables (as content snapshots) by (1) sliding a (non-overlapping) context window of sentences with a maximum length of 128 tokens, and (2) using the NL tokens in the window as the utterance, and pairing it with randomly sampled rows from the table as content snapshots. TABERT is implemented in PyTorch using distributed training”
Page 3, col. 1, paragraph 1, “A masked LM defines a distribution pθ(xm|xe) over the target tokens xm given the masked context xe.”, the distribution being sampled.).
Claim 8: Yin, Reimers, Cvitkovic, and Bouyarmane teaches the limitations of claim 1, Bouyarmane further teaches:
generating, for each token of the attribute value for the first attribute, a token embedding, wherein the respective attribute embedding for the attribute value is generated based at least in part on each token embedding for the attribute value. (Bouyarmane, col. 17, lines 34-41, “In token-level BiLSTM analysis 804, respective vector or tensor embeddings created for each token of each attribute from the character-level analysis may then be used to generate attribute-level representations”, tensor embeddings are generated for each token, which is later used to generate attribute-level representations (the attribute embeddings)).
Claim 9: Yin, Reimers, Cvitkovic, and Bouyarmane teaches the limitations of claim 1, Cvitkovic further teaches:
The method of claim 1, further comprising: identifying that a second attribute value for a second attribute corresponding to a column of the first data table references the second data entity of the second data table of the plurality of data tables; (Cvitkovic, page 2, paragraph 1, “What makes RDBs “relational” is that the values in a column in one table can refer to rows of another table. For example, in Figure 1 the column T Visit :,patient_id refers to rows in T Patient based on the values in T Patient :,patient_id. The value in T Visit i,patient_id indicates which patient came for Visit i. A column like this that refers to another table is called a foreign key.”, This explicitly describes a column in one table (here, patient_id in Visit) referencing rows in another table (Patient). The column is thus acting as a foreign key, which directly corresponds to the “second attribute” recited in Claim 9. It is interpreted that recognizing this foreign key relationship is analogous to identifying that a value in the second attribute (e.g., bbb in Visit[i, patient_id]) refers to a specific second data entity (row in Patient) of figure 1 above.)
and using, for the respective attribute embedding for the second attribute, the referenced entity embedding that is generated for the second data entity using the data of the second data table, (Cvitkovic, page 11, paragraph 2, “For each iteration t from 1 to T: (a) Each vertex v sends a “message”
PNG
media_image2.png
30
234
media_image2.png
Greyscale
to each of its neighbors w, where Nv is the set of all neighbors of v.”, In the GNN formulation, a message is computed and passed from one node (data entity) to another via a foreign key edge. The term hᵗ⁻¹ʷ in this formula is the referenced entity embedding of the second data entity (neighbor node w), and it is computed using that entity’s own table data. This embedding becomes part of the message used to enrich other nodes’ embeddings—including the one corresponding to the second attribute in the first table. Therefore, the referenced entity embedding (hᵗ⁻¹ʷ) is used in the embedding logic for the second attribute.)
wherein the entity embedding for the first data entity is generated based at least in part on the referenced entity embedding for the second data entity, (Cvitkovic, page 11, paragraph 2, “(c) Each vertex v updates its hidden state as a function of its current hidden state and the aggregated messages it received:
PNG
media_image4.png
26
122
media_image4.png
Greyscale
”, The entity embedding for node v (the first data entity) is computed through a neural update function Uₜ, which takes its previous embedding and a set of aggregated messages mᵗᵥ. Those messages contain values like hᵗ⁻¹ʷ from the referenced entity node (of the second data entity), meaning that the embedding hᵗᵥ is explicitly based in part on the referenced entity embedding of the second entity.)
and wherein the first attribute embedding comprises a vectorized representation and the referenced entity embedding and the entity embedding comprise fixed -length vectors. (Cvitkovic, page 11, paragraph 2, “A function S is used to initialize a hidden state vector h 0 v ∈ R d for each vertex:
PNG
media_image5.png
30
138
media_image5.png
Greyscale
… Each vertex v updates its hidden state … hᵗᵥ = Uₜ(hᵗ⁻¹ᵥ, mᵗᵥ).”, Each attribute is vectorized into a fixed-length hidden state using function S, and that vector (h⁰ᵥ) is used as the initial node embedding, satisfying the claim’s “first attribute embedding comprises a vectorized representation.” The final entity embedding (hᵗᵥ) and referenced entity embedding (hᵗ⁻¹ʷ) are both real-valued vectors of fixed dimensionality ℝᵈ, confirming that all vectors involved are fixed-length, as recited by the claim.).
The rationale for motivation to combine Yin with Cvitkovic is similar to that of claim 1 above.
Claims 11-13 and 16-18 recite limitations substantially similar to claims 1-3. Therefore, the rejection of claims 1-3 similarly applies to claims 11-13 and 16-18 respectively.
Claims 14 and 19 recite limitations substantially similar to claim 4. Therefore, the rejection of claim 4 similarly applies to claims 14 and 19.
Claims 15 and 20 recite limitations substantially similar to claim 5. Therefore, the rejection of claim 5 similarly applies to claims 15 and 20.
Claim 10 is rejected under 35 U.S.C. 103 as being unpatentable over Yin, Reimers, Cvitkovic, Bouyarmane, and Green et al. (EP 3316185 A1), hereinafter referred to as Green.
Claim 10: Yin, Cvitkovic, and Bouyarmane teaches the limitations of claim 1, Green, in the same field of entity embedding, teaches the following limitations which Bouyarmane fails to teach:
identifying that the attribute value for a second attribute references set of related attribute values (Green, paragraph 62, “the system may determine the initial embedding for each entity attribute in the second set of entity attributes by taking an average of the sum of the embeddings of pages that are associated with the respective topic”, the initial embedding for an entity in a second set references other embeddings with attributes associated to the respective topic);
and generating a respective attribute embedding for each related attribute value of the set of related attribute values, wherein each of the attribute embeddings for each related attribute value is based on the identified attribute type identifier and the entity embedding is generated based on the each of the attribute embeddings for each related attribute value (Green, paragraph 61, “when the embeddings of entities are updated in step 440 of FIG. 4, all of the embeddings of entities may be updated, not just the embeddings in the particular set associated with that iteration (i.e., the first, second, third, and fourth sets of entity embeddings may be updated, not just the fourth set of entity embeddings)”, when an embedding is updated, other embeddings from other sets associated with it are also updated).
It would have been obvious to a person of ordinary skill in the art to have incorporated the teachings disclosed by Yin/Reimers/Cvitkovic/Bouyarmane with the teachings disclosed by Green (i.e., identifying a second entity embedding and using it to generate the first entity embedding). A motivation for the combination is to further refine the embeddings used to train the machine learning model. (Green, paragraph 60, “At step 440, the system uses a pooling algorithm to determine updated embeddings for the set of entities from step 410 based on the refined embeddings of the set of entity attributes determined at step 430.”).
Conclusion
The prior art made of record on the PTO-892 and not relied upon is considered pertinent to applicant's disclosure:
Schlichtkrull, M., Kipf, T. N., Bloem, P., Van Den Berg, R., Titov, I., & Welling, M. (2018). Modeling relational data with graph convolutional networks. In The semantic web: 15th international conference, ESWC 2018, Heraklion, Crete, Greece, June 3–7, 2018, proceedings 15 (pp. 593-607). Springer International Publishing.
Graham, D., Wang, J., & Ravanbakhsh, S. (2019). Equivariant entity-relationship networks. arXiv preprint arXiv:1903.09033.
Wu, Y., Liu, X., Feng, Y., Wang, Z., & Zhao, D. (2019). Jointly learning entity and relation representations for entity alignment. arXiv preprint arXiv:1909.09317.
Kilias, T., Löser, A., Gers, F. A., Koopmanschap, R., Zhang, Y., & Kersten, M. (2018). Idel: In-database entity linking with neural embeddings. arXiv preprint arXiv:1803.04884.
Yu, T., Wu, C. S., Lin, X. V., Wang, B., Tan, Y. C., Yang, X., ... & Xiong, C. (2020). Grappa: Grammar-augmented pre-training for table semantic parsing. arXiv preprint arXiv:2009.13845.
Mudgal Sunil Kumar, S. (2018). Deep Learning for Entity Matching: A Design Space Exploration.
Nie, H., Han, X., He, B., Sun, L., Chen, B., Zhang, W., ... & Kong, H. (2019, November). Deep sequence-to-sequence entity matching for heterogeneous entity resolution. In Proceedings of the 28th ACM International Conference on Information and Knowledge Management (pp. 629-638).
Lu, W., Jiao, J., & Zhang, R. (2020, October). Twinbert: Distilling knowledge to twin-structured compressed bert models for large-scale retrieval. In Proceedings of the 29th ACM International Conference on Information & Knowledge Management (pp. 2645-2652).
Deng, X., Sun, H., Lees, A., Wu, Y., & Yu, C. (2020). Turl: Table understanding through representation learning. ACM SIGMOD Record, 51(1), 33-40.
Hu, Z., Dong, Y., Wang, K., & Sun, Y. (2020, April). Heterogeneous graph transformer. In Proceedings of the web conference 2020 (pp. 2704-2710).
Herzig, J., Nowak, P. K., Müller, T., Piccinno, F., & Eisenschlos, J. (2020, July). TaPas: Weakly supervised table parsing via pre-training. In Proceedings of the 58th annual meeting of the association for computational linguistics (pp. 4320-4333).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to HYUNGJUN B YI whose telephone number is (703)756-4799. The examiner can normally be reached M-F 9-5.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Saeed Usmaan can be reached on (571) 272-4046. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/H.B.Y./Examiner, Art Unit 2124
/USMAAN SAEED/Supervisory Patent Examiner, Art Unit 2146
