DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 02/04/2026 has been entered.
Response to Arguments
Applicant's arguments filed 04/04/2026 have been fully considered but they are partially persuasive.
Regarding applicant’s remarks directed to the rejection of claims under 35 USC § 103, the arguments are directed to newly amended limitations that were not previously examined by the examiner. Therefore, applicants arguments are rendered moot. The examiner refers to the rejection under 35 USC § 103 in the current office action for more details.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1-2, 5-6, 9, 11-12, 15-16 and 20-21 is/are rejected under 35 U.S.C. 103 as being unpatentable over US Pub No. US20210034977A1 Arik et al. (“Arik”) in view of Shankaranarayana, Sharath M., and Davor Runje. "Attention Augmented Convolutional Transformer for Tabular Time-series." arXiv preprint arXiv:2110.01825 (2021). (“Shankaranarayana”) in further view of US Pub No. US20140279739A1 Elkington et al. (“Elkington”).
In regards to claim 1,
Arik teaches A computer-implemented method comprising: generating, by one or more processors
(Arik, “[0045] The computing device 400 includes a processor 410, memory 420, a storage device 430, a high-speed interface/controller 440 connecting to the memory 420 and high-speed expansion ports 450, and a low speed interface/controller 460 connecting to a low speed bus 470 and a storage device 430. Each of the components 410, 420, 430, 440, 450, and 460, are interconnected using various busses, and may be mounted on a common motherboard or in other manners as appropriate. The processor 410 can process instructions for execution within the computing device 400, including instructions stored in the memory 420 or on the storage device 430 to display graphical information for a graphical user interface (GUI) on an external input/output device, such as display 480 coupled to high speed interface 440. In other implementations, multiple processors and/or multiple buses may be used, as appropriate, along with multiple memories and types of memory. Also, multiple computing devices 400 may be connected, with each device providing portions of the necessary operations (e.g., as a server bank, a group of blade servers, or a multi-processor system).”)
Arik teaches a column-wise vector for a column value of a first table row of a database,
(Arik, “Tabular data refers to data in a table format that generally has a plurality of features 206 a-n more commonly referred to as table entries. Generally speaking, the table format includes a series of columns and a number of row entries for each column. Each row entry for a particular column typically relates to a column heading. In other words, a column with a heading (e.g., a categorical heading) of “Salary” will have row entries of numerical values representing salaries [a first table row of a database]. Because a table entry may be numerical or categorical in nature, likewise a feature 206 may refer to a numerical feature 206 or a categorical feature 206. For instance, FIG. 2A depicts the plurality of features 206 [a column-wise vector], 206 a-n corresponding to the table of user data 12 where the headings of “Age,” “Marital Status,” “Degree,” “Job Role,” Department (“Dept.”), “Current Salary.” Additional Compensation (“Add'l Comp”), Previous Salary (“Prev. Salary”), “Accrued Leave,” and “Date Employed” are categorical features 206 and some of the row entry data corresponding to “Age,” “Current Salary,” Previous Year Additional Compensation (“PrevAdd'l Comp”), Previous Salary (“Prev. Salary”), “Accrued Leave,” and “Date Employed” are numerical features 206. When the feature 206 is a numerical feature 206. TabNet 200 may be configured to use this numerical feature 206 in its raw form. When the feature 206 is a categorical feature 206, TabNet 200 may be configured to map categorical features 206 to trainable embeddings. Here, TabNet 200 does not typically perform global normalization for feature(s) 206, but rather performs batch normalization (BN) during processing (e.g., at the feature transformer 222 and/or at the attentive transformer 212). Each decision step S may receive features 206 of the same dimensional size.”)
Arik teaches and using an encoder sub-model of an attention-based encoder-decoder machine learning model,
(Arik, “[0030] Referring to FIGS. 2A-2G, TabNet 200 [an attention-based (see Attentive Transformer 212 in figs. 2B and 2C) encoder-decoder machine learning model] generally includes an encoder 202 [an encoder sub-model; wherein the encoder sub-model is further used.] and a decoder 204. TabNet 200 is configured to receive a set of features 206, 206 a-n at the encoder 202, to process a subset 216 of the features 206 a-n at each processing decision step S into a decision step output 208, and to aggregate the decision step outputs 208 (i.e., a plurality of processed feature representations) into a final decision output 208 F [a column-wise vector ie some decision (prediction) for a column value].”)
Arik teaches wherein the attention-based encoder-decoder machine learning model comprises the encoder sub-model, a vertical self-attention sub-model, and a decoder sub-model,
(Arik, “[0030] Referring to FIGS. 2A-2G, TabNet 200 [wherein the attention-based encoder-decoder machine learning model comprises the encoder sub-model, a vertical self-attention sub-model (see Attentive Transformer 212 in figs. 2B and 2C), and a decoder sub-model] generally includes an encoder 202 and a decoder 204. TabNet 200 is configured to receive a set of features 206, 206 a-n at the encoder 202, to process a subset 216 of the features 206 a-n at each processing decision step S into a decision step output 208, and to aggregate the decision step outputs 208 (i.e., a plurality of processed feature representations) into a final decision output 208 F.
PNG
media_image1.png
592
806
media_image1.png
Greyscale
”)
Arik teaches and the attention-based encoder-decoder machine learning model is trained by:(i)determining, using the encoder sub-model, an inferred column-wise representation for a training column value of a training table row
(Arik, “[0031] Here, the encoder 202 [using the encoder sub-model] is capable of performing either classification or regression. The encoder 202 generally includes a feature selector 210 and a feature processor 220 at each decision step S. The encoder 202 performs multi-step processing (e.g., shown in FIG. 2C) with N number of decision steps S. With multiple decision steps S, an input for a current step Si is processing information from the previous step Si−1. For instance, in FIG. 2A, the feature selector 210 is initially accounting for processing information (e.g., regarding the subset 216 of features 206 from the previous step Si−1) and the feature processor 220 is generating processing information (e.g., regarding the subset 216 of features 206 for the current step Si).”)
(Arik, “[0038] In some configurations, to determine a decision embedding 242 for each decision step S, the feature processor 220 include a rectified linear unit (RELU) 240. The RELU is a type of activation function that generates a linear relationship between all positive values and zero for all negative values. Here, the RELU 240 functions well for sparse learning because it is sparsely activated (i.e., only activated on positive values). In these configurations, the RELU 240 receives the decision step output 208 for the processed feature(s) 222 [an inferred column-wise representation ie decision (prediction) for a training column value of a training table row ie processed features 222] and generates an embedding (e.g., the decision step embedding 242) to represent the decision step output 208. For instance, the decision embedding 242 can be constructed as dout=Σi=1 N steps ReLU(d[i]), where Nsteps is the total number of decision steps. In some examples, the REI. 240 outputs the decision step embedding 242 from a current step Si to be combined (e.g., concatenated) with a decision step embedding 242 from subsequent and/or prior decision steps S in order to form an overall decision embedding 242 for all steps S that is communicated to the decoder 204 for decoding the decision embedding(s) 242) into a recognizable feature 206.”)
Arik teaches (ii) determine, using a vertical self-attention sub-model, an attenuated representation for a training column based at least in part on the inferred column-wise representation;
(Arik, [0035], “This allows the attention towards sparsely masked features to be tunable. In some implementations, as shown by equation (1), the attentive transformer 212 [using a vertical self-attention sub-model] obtains the trainable mask by using a trainable function. For example, the attentive transformer 212 receives the features 206 and processes the features 206 through a single layer L, L1 mapping that is modulated with prior scale information to understand how much attention has been previously focused on particular features 206 in prior decision steps S [an attenuated representation ie attention for a training column ie particular feature 206]. In other words, the prior scale determines (e.g., based on aggregation) for all prior steps Si−1 how much each feature 206 has been used by the encoder 202 as a relevant feature 206R for feature processing prior to the current decision step Si [based at least in part on the inferred column-wise representation ie prior decisions].”)
Arik teaches (iii) determine, using the decoder sub-model, an inferred column value for the training column based at least in part on the attenuated representation;
(Arik, “[0040] The decoder 204 [using the decoder sub-model] of TabNet 200 is configured to receive an encoded representation of the decision output 208 and to decode the encoded representation into an interpretable decision output 208 [an inferred column value for the training column]. For example, the decoder 204 enables the encoded representation to transform from a computer-readable encoded representation that is efficient for encoder functions into a human-readable decoded representation (i.e., a reconstructed feature) of the decision output 208. Referring to FIGS. 2B, 2C, and 2E, the decoder 204 receives a decision embedding 242 [based at least in part on the attenuated representation; wherein Fig. 2B, 2C, and 2E teaches the decision embedding 242 is obtained in part from the Attentive Transformer 212] as the encoded representation of the decision output from the encoder 202 and uses a feature transformer 224 to transform the decision embedding 242 into the decision output 208. For instance, the decoder 204 includes one or more feature transformers 224 followed by fully-connected (FC) layers to decode the encoded representation (e.g., the decision embedding 242).”)
Arik teaches and(iv) updating the attention-based encoder-decoder machine learning model based at least in part on the inferred column value for the training column;
(Arik, [0044], “In some examples, the encoder 202 communicates the decision embedding 242 at each step S to the decoder 204 such that the decoder 204 is configured to combine each decision step output 208 into the final decision output 208 F [updating the attention-based encoder-decoder machine learning model ie final decision output 208F based at least in part on the inferred column value for the training column ie decision step output 208]. For example, the decoder 204 may apply a linear mapping to obtain the final decision 208 F. As a multiple step example, FIG. 2E depicts the encoder 202 concatenating two decision step outputs 208 from a first step S1 and a second step S2 into the final decision output 208 F.”)
Arik teaches generating, by the one or more processors, a first fixed-size representation of the first table row based at least in part on the column-wise vector;
(Arik, “[0033] In some examples, the feature selector 210 receives a plurality of features 206 and selects the salient features 206 from the plurality of features 206 that corresponds to the desired decision output to form a subset 216 [generating, by the one or more processors, a first fixed-size representation of the first table row based at least in part on the column-wise vector; wherein a subset is a set with some determined length (fixed-size)] of relevant features 206R.”)
However, Arik does not explicitly teach by replacing the training column value with a masked training column value, wherein (a) the masked training column value comprises a predefined value or character based on a column format type, of a plurality of column format types, associated with a training column of the training column value, (b) the plurality of column format types comprises a categorical column format type, a continuous column format type, and a sequential column format type, and (c) each of the plurality of column format types is associated with a different predefined value or character,… generating, by the one or more processors, a cross-row similarity measure between the first table row and a second table row based on a distance between the first fixed-size representation and a second fixed-size representation of the second table row; and merging, by the one or more processors, the first table row with the second table row based on the cross-row similarity measure to improve a storage efficiency of the database.
Shankaranarayana teaches by replacing the training column value with a masked training column value, wherein (a) the masked training column value comprises a predefined value or character based on a column format type, of a plurality of column format types, associated with a training column of the training column value, (b) the plurality of column format types comprises a categorical column format type, a continuous column format type, and a sequential column format type, and (c) each of the plurality of column format types is associated with a different predefined value or character,
(Shankaranaryana, Section II C., “The masking of entire row is similar to masking a word in MLM since, in tabular data, the analogue of a ”word” is an entire row of features. The figure Fig. 4 gives a clear picture of two types of masking employed in our work. Additionally, different from other previous works, our framework has the ability to handle continuous type of inputs as is, without resorting to binning. Although for masking the categorical data we can simply have a specific integer token similar to MASK token employed in MLM [by replacing the training column value with a masked training column value, wherein (a) the masked training column value comprises a predefined value or character based on a column format type], for continuous type of inputs, we mask the data by replacing the original values with the mean value of the particular continuous feature.”; wherein Shankaranaryana teaches handling different types of methods to mask different types of data)
PNG
media_image2.png
48
649
media_image2.png
Greyscale
(Shankaranaryana, Section I, “The current state-of-the-art techniques in NLP learn vector representations of words using transformers and attention mechanism [5] on large datasets with the task of reconstructing an input text [a sequential column format type] with some of the words in it randomly masked [6].”)
Elkington teaches generating, by the one or more processors, a cross-row similarity measure between the first table row and a second table row based on a distance between the first fixed-size representation and a second fixed-size representation of the second table row;
(Elkington, “[0056] A centroid record can be derived from duplicate records. The centroid record is a record that minimizes the overall distance to all of the duplicate records.
[0057] If dist(i,j) is the distance between records i [first table row] and j [second table row], a centroid can be defined as centroid=ArgMin(dist(i,j)) (where i, j=1, 2, . . . N). For example, if five duplicate records are identified, containing the first names “John”, “John”, “Johnathan”, “Jonathan”, and “Jeff”, then “John” is selected as the centroid record since it has minimum distance between all pairs among those values.
[0058] In at least one embodiment, the distance metric dist(i, j) is calculated using a hybrid of both Euclidean distance and edit/keyboard distances. Euclidean distance can be measured as a straight-line distance, in n-dimensional space; given two vectors p [the first fixed-size representation] and q [a second fixed-size representation] it can be described as the square-root of (p1−q1)2+(p2−q2)2+ . . . +(pn−qn)2. Edit/keyboard distance is a measure of how many characters are changed from one value to another, and can also take into account the distance between keys corresponding to those changed characters on a (real or virtual) QWERTY keyboard….
[0065] For each record i, let dist(i, c) be the distance between record i and the centroid record. In at least one embodiment, dist(i, c) can be normalized to a real value in the range [0,1]. For example, a scale parameter can be set, based on which distance metrics are being used. Dist (i, c) can then be normalized by calculating dist(i, c)/scale if dist(i, c)<=scale, or setting dist(i, c) to 1.0 if dist(i, c)>scale.
[0066] A similarity feature value can then be calculated by feat(Similarity)=(1.0−dist(i, c)). [generating a cross-row similarity measure between the first table row and a second table row based on a distance between the first fixed-size representation and a second fixed-size representation of the second table row; wherein the features are provided by Arik]”)
Elkington teaches and merging, by the one or more processors, the first table row with the second table row
(Elkington, “[0147] In step 204 of FIG. 2, ML model 112 generates resolved record(s) with confidence scores. These resolved record(s) form a recommended merging solution [merging, by the one or more processors, the first table row with the second table row]. In at least one embodiment, a user can select one of a plurality of these generated records; in another embodiment, the system itself can make the selection.
[0148] In at least one embodiment, a threshold value can be set, either by the user or by some other entity. When the confidence score for a resolved record exceeds this threshold value, the field is automatically merged using the recommended solution specified by that resolved record, without user intervention. When the confidence score does not exceed the threshold value, the user can be prompted to manually merge the fields and/or to select among a plurality of generated records representing different solutions.”)
Elkington teaches based on the cross-row similarity measure to improve a storage efficiency of the database.
(Elkington, “[0115] In at least one embodiment, a labeling confidence score vector Label_Conf_Score={lcs1, lcs2, . . . , lcsM} is generated and associated with the resolved record sr. When a centroid method is used, the confidence score vector can be calculated based on ranking score among all dist(i, j) other than the one with minimum distance. [based on the cross-row similarity measure to improve a storage efficiency of the database]”)
Arik and Shankaranaryana are both considered to be analogous to the claimed invention because they are in the same field of generating predictions on tabular data. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Arik to incorporate the teachings of Shankaranaryana in order to provide Shankaranaryana’s novel masking techniques as doing so showed improved results over baseline models. (Shankaranaryana, Section IV., “In this work, we proposed a novel end-to-end BERT based architecture for time-series tasks. For the first time, we proposed the use of attention augmented convolutions for tabular time-series data and also proposed major modifications to the masking methodology for tabular time-series data. From our experiments, we showed that each of the individual modifications lead to improved results.”
PNG
media_image3.png
180
576
media_image3.png
Greyscale
)
Elkington is considered to be analogous to the claimed invention because they are in the same field of tabular data processing using machine learning. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Arik to incorporate the teachings of Elkington in order to provide an accurate and reliable technique for automatically resolving and merging duplicate records (Elkington, Abstract, “According to various embodiments of the present invention, an automated technique is implemented for resolving and merging fields accurately and reliably, given a set of duplicated records that represents a same entity. In at least one embodiment, a system is implemented that uses a machine learning (ML) method, to train a model from training data, and to learn from users how to efficiently resolve and merge fields. In at least one embodiment, the method of the present invention builds feature vectors as input for its ML method. In at least one embodiment, the system and method of the present invention apply Hierarchical Based Sequencing (HBS) and/or Multiple Output Relaxation (MOR) models in resolving and merging fields. Training data for the ML method can come from any suitable source or combination of sources.”)
In regards to claim 2,
Arik in view of Shankaranaryana and Elkington teaches The computer-implemented method of claim 1, wherein:
Arik teaches the encoder sub-model is further configured to determine the column-wise vector based at least in part on a column value numerical representation of the column value, and the column value numerical representation for the column value is generated based at least in part on a column format type of a column associated with the column value.
(Arik, “[0030] Referring to FIGS. 2A-2G, TabNet 200 generally includes an encoder 202 and a decoder 204. TabNet 200 is configured to receive a set of features 206, 206 a-n at the encoder 202 [the encoder sub-model is further configured to], to process a subset 216 of the features 206 a-n at each processing decision step S into a decision step output 208 [determine the column-wise vector], and to aggregate the decision step outputs 208 (i.e., a plurality of processed feature representations) into a final decision output 208 F. Tabular data refers to data in a table format that generally has a plurality of features 206 a-n more commonly referred to as table entries. Generally speaking, the table format includes a series of columns and a number of row entries for each column. Each row entry for a particular column typically relates to a column heading. In other words, a column with a heading (e.g., a categorical heading) of “Salary” will have row entries of numerical values representing salaries. Because a table entry may be numerical or categorical in nature, likewise a feature 206 [based at least in part on a column value numerical representation of the column value] may refer to a numerical feature 206 or a categorical feature 206 [is generated based at least in part on a column format type of a column associated with the column value]. For instance, FIG. 2A depicts the plurality of features 206, 206 a-n corresponding to the table of user data 12 where the headings of “Age,” “Marital Status,” “Degree,” “Job Role,” Department (“Dept.”), “Current Salary.” Additional Compensation (“Add'l Comp”), Previous Salary (“Prev. Salary”), “Accrued Leave,” and “Date Employed” are categorical features 206 and some of the row entry data corresponding to “Age,” “Current Salary,” Previous Year Additional Compensation (“PrevAdd'l Comp”), Previous Salary (“Prev. Salary”), “Accrued Leave,” and “Date Employed” are numerical features 206. When the feature 206 is a numerical feature 206. TabNet 200 may be configured to use this numerical feature 206 in its raw form. When the feature 206 is a categorical feature 206, TabNet 200 may be configured to map categorical features 206 to trainable embeddings. Here, TabNet 200 does not typically perform global normalization for feature(s) 206, but rather performs batch normalization (BN) during processing (e.g., at the feature transformer 222 and/or at the attentive transformer 212). Each decision step S may receive features 206 of the same dimensional size.”)
In regards to claim 5,
Arik in view of Shankaranaryana and Elkington teaches The computer-implemented method of claim 1,
Arik teaches wherein generating the masked training column value for the training table row comprises: identifying a designated column of a plurality of columns for the training table row;
(Arik, [0042], “For instance, TabNet 200 may be used for the task of predicting missing features 206, 206M in tabular data. This may be advantageous because often real-world tabular datasets have interdependent feature columns that TabNet 200 can learn to understand in order to predict a missing feature 206M based on other related features 206 in the same column or related interdependent columns.”; wherein the ‘missing data’ is the masked data to predict/reconstruct)
Arik teaches and in response to determining that an initial column value of the training table row for the designated column comprises the categorical column format type, generating the masked training column value based at least in part on a zero-hot encoding value.
(Arik, [0042], “Similarly, a missing feature 206M for gender [and in response to determining that an initial column value of the training table row for the designated column comprises the categorical column format type] may be predicted from other gender entries in combination with relationship entries. To perform unsupervised representation learning, TabNet 200 uses a self-supervised objective with a pretrainer 260. For example, with a binary mask of S∈ {0, 1}B×D [generating the masked training column value based at least in part on a zero-hot encoding value; wherein zero and one are zero-hot encoding values], the encoder 202 of TabNet 200 inputs (1−S)·f and the decoder 204 outputs the reconstructed features S·f. Here, the encoder 202 may initialize the prior scale term to be P[0]=(1−S) so that TabNet 200 emphasizes known features 206 and the decoder's fully-connected layers (FC) is multiplied by S to merely output the unknown or missing features 206M.”)
In regards to claim 6,
Arik in view of Shankaranaryana and Elkington teaches The computer-implemented method of claim 1,
Arik teaches wherein generating the masked training column value for the training table row comprises: identifying a designated column of a plurality of columns for the training table row;
(Arik, [0042], “For instance, TabNet 200 may be used for the task of predicting missing features 206, 206M in tabular data. This may be advantageous because often real-world tabular datasets have interdependent feature columns that TabNet 200 can learn to understand in order to predict a missing feature 206M based on other related features 206 in the same column or related interdependent columns.”; wherein the ‘missing data’ is the masked data to predict/reconstruct)
Shankaranaryana teaches and in response to determining that an initial column value of the training table row for the designated column comprises the continuous column format type, generating the masked training column value based at least in part on a designated extreme numeric value.
(Shankaranaryana, Section II C., “The masking of entire row is similar to masking a word in MLM since, in tabular data, the analogue of a ”word” is an entire row of features. The figure Fig. 4 gives a clear picture of two types of masking employed in our work. Additionally, different from other previous works, our framework has the ability to handle continuous type of inputs as is, without resorting to binning. Although for masking the categorical data we can simply have a specific integer token similar to MASK token employed in MLM, for continuous type of inputs, we mask the data by replacing the original values with the mean value of the particular continuous feature [generating the masked training column value based at least in part on a designated extreme numeric value; wherein a designated extreme value can be the highest value; thus, the mean is generated in part from the highest value/designated extreme value].”)
In regards to claim 9,
Arik in view of Shankaranaryana and Elkington teaches The computer-implemented method of claim 1,
Arik teaches further comprising: generating, by the one or more processors, the second fixed-size representation of the second table row;
(Arik, “[0033] In some examples, the feature selector 210 receives a plurality of features 206 and selects the salient features 206 from the plurality of features 206 that corresponds to the desired decision output to form a subset 216 [generating, by the one or more processors, the second fixed-size representation of the second table row; wherein a subset is a set with some determined length (fixed-size)] of relevant features 206R.”)
Elkington teaches generating, by the one or more processors and based at least in part on the first fixed-size representation and the second fixed-size representation, a cross-row linking determination for the first table row and the second table row;
(Elkington, “[0056] A centroid record can be derived from duplicate records. The centroid record is a record that minimizes the overall distance to all of the duplicate records.
[0057] If dist(i,j) is the distance between records i and j, a centroid can be defined as centroid=ArgMin(dist(i,j)) (where i, j=1, 2, . . . N). For example, if five duplicate records are identified, containing the first names “John”, “John”, “Johnathan”, “Jonathan”, and “Jeff”, then “John” is selected as the centroid record since it has minimum distance between all pairs among those values.
[0058] In at least one embodiment, the distance metric dist(i, j) is calculated using a hybrid of both Euclidean distance and edit/keyboard distances. Euclidean distance can be measured as a straight-line distance, in n-dimensional space; given two vectors p and q it can be described as the square-root of (p1−q1)2+(p2−q2)2+ . . . +(pn−qn)2. Edit/keyboard distance is a measure of how many characters are changed from one value to another, and can also take into account the distance between keys corresponding to those changed characters on a (real or virtual) QWERTY keyboard….
[0065] For each record i, let dist(i, c) be the distance between record i and the centroid record. In at least one embodiment, dist(i, c) can be normalized to a real value in the range [0,1]. For example, a scale parameter can be set, based on which distance metrics are being used. Dist (i, c) can then be normalized by calculating dist(i, c)/scale if dist(i, c)<=scale, or setting dist(i, c) to 1.0 if dist(i, c)>scale.
[0066] A similarity feature value can then be calculated by feat(Similarity)=(1.0−dist(i, c)). [generating, by the one or more processors and based at least in part on the first fixed-size representation and the second fixed-size representation, a cross-row linking determination for the first table row and the second table row; wherein the features are provided by Arik]”)
Elkington teaches and provide, by the one or more processors, an indication that the first table row and the second table row should be linked
(Elkington, “[0147] In step 204 of FIG. 2, ML model 112 generates resolved record(s) with confidence scores. These resolved record(s) form a recommended merging solution [provide, by the one or more processors, an indication that the first table row and the second table row should be linked]. In at least one embodiment, a user can select one of a plurality of these generated records; in another embodiment, the system itself can make the selection.
[0148] In at least one embodiment, a threshold value can be set, either by the user or by some other entity. When the confidence score for a resolved record exceeds this threshold value, the field is automatically merged using the recommended solution specified by that resolved record, without user intervention. When the confidence score does not exceed the threshold value, the user can be prompted to manually merge the fields and/or to select among a plurality of generated records representing different solutions.”)
Elkington teaches based at least in part on the cross-row linking determination.
(Elkington, “[0115] In at least one embodiment, a labeling confidence score vector Label_Conf_Score={lcs1, lcs2, . . . , lcsM} is generated and associated with the resolved record sr. When a centroid method is used, the confidence score vector can be calculated based on ranking score among all dist(i, j) other than the one with minimum distance. [based at least in part on the cross-row linking determination]”)
Claim 11 and 20 are rejected on the same rationale under 35 U.S.C. 103 as claim 1 and claim 10 (with the recitation of a processor and memory) as they are substantially similar.
Claim 12 is rejected on the same rationale under 35 U.S.C. 103 as claim 2 as they are substantially similar.
Claim 15 is rejected on the same rationale under 35 U.S.C. 103 as claim 5 as they are substantially similar.
Claim 16 is rejected on the same rationale under 35 U.S.C. 103 as claim 6 as they are substantially similar.
In regards to claim 21,
Arik in view of Shankaranaryana and Elkington teaches The computer-implemented method of claim 1,
Elkington teaches wherein the cross-row similarity measure comprises a Euclidean distance between the first fixed-size representation and the second fixed-size representation.
(Elkington, “[0056] A centroid record can be derived from duplicate records. The centroid record is a record that minimizes the overall distance to all of the duplicate records.
[0057] If dist(i,j) is the distance between records i and j, a centroid can be defined as centroid=ArgMin(dist(i,j)) (where i, j=1, 2, . . . N). For example, if five duplicate records are identified, containing the first names “John”, “John”, “Johnathan”, “Jonathan”, and “Jeff”, then “John” is selected as the centroid record since it has minimum distance between all pairs among those values.
[0058] In at least one embodiment, the distance metric dist(i, j) is calculated using a hybrid of both Euclidean distance and edit/keyboard distances. Euclidean distance can be measured as a straight-line distance, in n-dimensional space; given two vectors p and q it can be described as the square-root of (p1−q1)2+(p2−q2)2+ . . . +(pn−qn)2. Edit/keyboard distance is a measure of how many characters are changed from one value to another, and can also take into account the distance between keys corresponding to those changed characters on a (real or virtual) QWERTY keyboard….
[0065] For each record i, let dist(i, c) be the distance between record i and the centroid record. In at least one embodiment, dist(i, c) can be normalized to a real value in the range [0,1]. For example, a scale parameter can be set, based on which distance metrics are being used. Dist (i, c) can then be normalized by calculating dist(i, c)/scale if dist(i, c)<=scale, or setting dist(i, c) to 1.0 if dist(i, c)>scale.
[0066] A similarity feature value can then be calculated by feat(Similarity)=(1.0−dist(i, c)). [wherein the cross-row similarity measure comprises a Euclidean distance between the first fixed-size representation and the second fixed-size representation; wherein the features are provided by Arik]”)
Claim(s) 3-4 and 13-14 is/are rejected under 35 U.S.C. 103 as being unpatentable over Arik in view of Shankaranaryana and Elkington in further view of U.S. Pub. No. US20210216862A1 Liu et al. (“Liu”)
In regards to claim 3,
Arik in view of Shankaranaryana and Elkington teaches The computer-implemented method of claim 2,
Liu teaches wherein generating the column value numerical representation for the column value comprises: in response to determining that the column value comprises a categorical column format type, generating the column value numerical representation based at least in part on a one hot encoding representation of the column value.
(Liu, Fig. 2A 121, “[0124]
The context type module 141 is configured to, upon receiving the context (or target) related to the task, determine whether the context is a text context or an image context, if the context is an image, send the image context to the CNN module 142, and if the context is text [in response to determining that the column value has a categorical column format type], send the text context to the text one-hot encoder 143.
[0117] The text one-hot encoder 121 is configured to, upon receiving the text of the task description, convert each word in the textual task description into a one-hot vector, and send the one-hot vectors of the text to the word embedding module 122. In certain embodiments, the text one-hot encoder 121 processes the text by filtering out punctuations, changing the cases of the words into lowercases, splitting the words by spaces to obtain T number of words {w1, w2, . . . , wi . . . , wT}, and converts the T number of words into their corresponding one-hot vectors {v1, v2, . . . , vi . . . , vT} using a fixed-sized vocabulary [generating the column value numerical representation based at least in part on a one-hot encoding representation of the column value]. The one-hot vector vi has a dimension of the fixed-size, the i-th dimension of the one-hot vector vi is set to be 1, and all the other dimensions of the one-hot vector vi are set to 0.”)
Liu is considered to be analogous to the claimed invention because they are in the same field of natural language processing. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Arik in view of Shankaranaryana and Elkington to incorporate the teachings of Liu in order to solve the problem of inefficient text handling and improve natural language processing inference efficiency (Liu, “[0004] For a text question on the target post or the target product, a variety of semantic analyses are available to predict the attribute of the target object based on these texts and images. Some methods answer a text question by focusing on the target text, some methods answers the text question by focusing on the target images. However, none of the methods use both the text information and image information efficiently. [0092] In certain aspects, to improve question-answer efficiency and accuracy, the present disclosure provides a method utilizing inference from the question and from both the text and image of the target object context. Specifically, the disclosure solves the problem based on an end-to-end neural network, coupled with attention-based fusion mechanism.”)
In regards to claim 4,
Arik in view of Shankaranaryana and Elkington teaches The computer-implemented method of claim 2,
Liu teaches wherein generating the column value numerical representation for the column value comprises: in response to determining that the column value comprises a sequential column format type, generating the column value numerical representation based at least in part on an output of processing the column value using an embedding machine learning model that comprises a long short term memory (LSTM) sub-model.
(Liu, Fig. 2A 123, “[0119] The BiLSTM module 123 is configured to, upon receiving the sequential embedding vectors {e1, e2, . . . , ei, . . . , eT} [in response to determining that the column has a sequential column format type], feed the embedding vectors to the BiLSTM, perform BiLSTM on the embedding vectors to obtain task feature vectors {u1, u2, . . . , ui, . . . , uT}, and send the task feature vectors to the task concatenating module 125 and the dual attention fusion module 150. The feature vector ui encodes the semantics of the i-th word in the textual description of the task. In the BiLSTM, the hidden states at time t from the forward and backward LSTMs are respectively ht (f)=LSTM(f)(xt,ht−1 (f)) and ht (b)=LSTM(b)(xt,ht−1 (b)), and the feature vectors are obtained by concatenating the two hidden states at each time step [generating the column value numerical representation based at least in part on an output of processing the column value using an embedding machine learning model that comprises a long short term memory (LSTM) sub-model.].”)
Claim 13 is rejected on the same rationale under 35 U.S.C. 103 as claim 3 as they are substantially similar.
Claim 14 are rejected on the same rationale under 35 U.S.C. 103 as claim 4 as they are substantially similar.
Claim(s) 7 and 17 are rejected under 35 U.S.C. 103 as being unpatentable over Arik in view of Shankaranaryana and Elkington in further view of US Pub No. US20200258498A1 Reisswig et al. (“Reisswig”)
In regards to claim 7,
Arik in view of Shankaranaryana and Elkington teaches The computer-implemented method of claim 1,
Arik teaches wherein generating the masked training column value for the training table row comprises: identifying a designated column of a plurality of columns for the training table row;
(Arik, [0042], “For instance, TabNet 200 may be used for the task of predicting missing features 206, 206M in tabular data. This may be advantageous because often real-world tabular datasets have interdependent feature columns that TabNet 200 can learn to understand in order to predict a missing feature 206M based on other related features 206 in the same column or related interdependent columns.”; wherein the ‘missing data’ is the masked data to predict/reconstruct)
Shankaranaryana teaches and in response to determining that an initial column value of the training table row for the designated column comprises the sequential column format type,
(Shankaranaryana, Section I, “The current state-of-the-art techniques in NLP learn vector representations of words using transformers and attention mechanism [5] on large datasets with the task of reconstructing an input text [in response to determining that an initial column value of the training table row for the designated column comprises the sequential column format type] with some of the words in it randomly masked [6].”)
However, Shankaranaryana does not explicitly teach generating the masked training column value by replacing each character of the initial column value with the predefined value or character.
Reisswig teaches generating the masked training column value by replacing each character of the initial column value with the predefined value or character.
(Reisswig, “[0043] At operation 404, the language model training service 304 masks one or more strings of the language model training sample to generate a masked language model training sample. Masking can include replacing a string with a random string and/or replacing the characters of the string with randomly-selected characters. In some examples, each character of the string is replaced individually [generating the masked training column value by replacing each character of the initial column value with the predefined value or character]. In the example of FIG. 3, the example language model training sample 308 has strings 310C, 310D, and 310F scrambled and/or masked to produce corresponding masked strings 310X, 310Y, 310Z. The example masked language model training sample 308A, then, includes the ordered set of strings 310A, 310B, 310X, 310Y, 310E, 310Z, 310G, 310H.”)
Reisswig is considered to be analogous to the claimed invention because they are in the same field of natural language processing and training machine learning models using masking. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Arik in view of Shankaranaryana and Elkington to incorporate the teachings of Reisswig in order to provide a particular method of masking strings as doing so allows the model to be trained with unlabeled data in an unsupervised manner (Reisswig, “[0024] The resulting string can be referred to as a masked string or scrambled string. The convolutional autoencoder language model can be trained to reconstruct at least one masked string from the training samples based on the other, unmasked strings in the training samples. In this way, the convolutional autoencoder language model can be trained with unlabeled data in an unsupervised manner. Because of the masked strings, the model may be trained to infer from the context of neighboring strings how to reconstruct or fill in the masked strings. As a result, the feature vectors generated by the layers of the model may reflect the context of the input (e.g., forward and reverse context).”)
Claim 17 is rejected on the same rationale under 35 U.S.C. 103 as claim 7 as they are substantially similar.
Claim(s) 8 and 18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Arik in view of Shankaranaryana and Elkington in further view of K. Cho, A. Courville and Y. Bengio, "Describing Multimedia Content Using Attention-Based Encoder-Decoder Networks," (“Cho”).
In regards to claim 8,
Arik in view of Shankaranaryana and Elkington teach The computer-implemented method of claim 1, further comprising:
Arik teaches generating, by the one or more processors, a masked table row by replacing a designated column value of the training table row that is associated with a designated column of a plurality of columns with a masked column value;
(Arik, “[0034] In some implementations, to perform feature selection, the feature selector 210 includes an attentive transformer 212 and a feature mask 214. For instance, the feature mask 214 is a learnable mask for soft selection of salient features 206. In some examples, the feature mask 214 uses sparse selection (referred to as a sparse mask) to select the most salient features 206 (e.g., shown as relevant features 206R, 206Ra-g in FIG. 2A) such that a learning capacity of a decision step S is not wasted on irrelevant features 206 [a masked table row ie feature mask 214 by replacing a designated column value ie irrelevant features of the training table row ie soft selection of salient features 206]. This allows TabNet 200 to become parameter efficient since each feature 206 is meaningfully selected. For instance, the first and second subset 216 a-b of “Occupation Related” and “Accounting Related” may meaningfully impact the value of an employee's year-end bonus compensation when compared to other features 206 such as “Age,” “Marital Status,” or “Date Employed.””)
Arik teaches that is associated with a designated column of a plurality of columns with a masked column value
(Arik, “[0008] The decision step output generated by processing the subset of relevant features may pass through a rectified linear unit (ReLU) of the TabNet. Additionally, the subset of features may correspond to interdependent feature columns of tabular data. In some implementations, the method also includes receiving, at the data processing hardware, tabular data including one or more masked features [that is associated with a designated column of a plurality of columns with a masked column value], and predicting, by the data processing hardware, the one or more masked features using unsupervised pre-training.”)
However, Arik does not teach determining, by the one or more processors and a vertical self-attention sub-model that is associated with the designated column based at least in part on the masked table row, an attention score for a column pair comprising a first column of the plurality of columns and the designated column value; and determining, by the one or more processors and based at least in part on the attention score, a cross-column linking determination for the column pair based at least in part on the attention score for the column pair.
Cho teaches determining, by the one or more processors and a vertical self-attention sub-model that is associated with the designated column based at least in part on the masked table row, an attention score for a column pair comprising a first column of the plurality of columns and the designated column value; and
(Cho, Section III B., “The attention mechanism controls the input actually seen by the decoder and requires another neural network, to which refer as the attention model [vertical self-attention sub-model]. The main job of the attention model is to score each context vector ci [comprising a first column of the plurality of columns] with respect to the current hidden state zt−1 of the decoder3
PNG
media_image4.png
43
260
media_image4.png
Greyscale
(10)
where αt−1j represents the attention weights computed at the previous time step, from the scores et−1i [the designated column value], through a softmax that makes them sum to 1
PNG
media_image5.png
83
188
media_image5.png
Greyscale
(11)
This type of scoring can be viewed as assigning a probability of being attended by the decoder to each context, hence the name of the attention model.
Once the attention weights [an attention score for each column pair comprising a first column of the plurality of columns and the designated column value] are computed, we use them to compute the new context vector ct
PNG
media_image6.png
55
232
media_image6.png
Greyscale
(12)
where φ returns a vector summarizing the whole context set c according to the attention weights.”)
Cho teaches determining, by the one or more processors and based at least in part on the attention score, a cross-column linking determination for the column pair based at least in part on the attention score for the column pair.
Examiner interprets in light of figure 12 and the specifications, (“[0092] As depicted in FIG. 12, the similarity matrix visualization 1201 is associated with sixteen similarity matrix visualization regions because, in this example, n=4. Each similarity matrix visualization region is associated with a table row pair, with one table row being defined by each dimension of the similarity matrix visualization region. For example, the similarity matrix visualization region 1211 is associated with table row Row3 as defined by the vertical dimension of the similarity matrix visualization 1201 and with table row Row4 as defined by the horizontal dimension of the similarity matrix visualization 1201. Moreover, as further depicted in FIG. 12, the coloring scheme of the similarity matrix visualization region 1211 denotes that the table row pair Row3-Row4 has a higher similarity matrix visualization region 1211 than the table row pairs for other neighboring similarity matrix visualization regions, except for the table row pair that is associated with the similarity matrix visualization region 1212, which is also the table row pair Row3-Row4.
[0093] As indicated by similarity matrix visualizations described above, cross-row similarity measures and/or cross-row linking determinations for a set of table row pairs can be combined to generate/display predictive inferences about internal duplication ratio of a set of table rows ( e.g., a set of table rows of a particular relational table) or to generate/display predictive inferences about similarities across two or more relational tables and/or two or more table data objects.”)
(Cho, Fig. 2, Section III B., “Once the attention weights [based at least in part on each attention score] are computed, we use them to compute the new context vector ct
ct=φ({ci}Mi=1,{αti}Mi=1)(12)
where φ returns a vector summarizing the whole context set c according to the attention weights [a cross-column linking determination for each column pair based at least in part on the attention score for the column pair; wherein Fig. 2 shows the similarity matrix].”)
Cho is considered to be analogous to the claimed invention because they are in the same field of natural language processing. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Arik in view of Shankaranaryana and Elkington to incorporate the teachings of Cho in order reduce computational burden and improve quality of outputs (Cho, Section I Introduction, “Attention mechanisms are employed for two purposes. The first is to reduce the computational burden of processing high dimensional inputs by selecting to only process subsets of the input. The second is to allow the system to focus on distinct aspects of the input and thus improve its ability to extract the most relevant information for each piece of the output, thus yielding improvements in the quality of the generated outputs. As the name suggests, soft attention mechanisms avoid a hard selection of which subsets of the input to attend and instead uses a soft weighting of the different subsets. Since all subset are processed, these mechanisms offer no computation advantage. Instead, the advantage brought by the soft-weighting is that it is readily amenable to efficient learning via gradient backpropagation.”)
Claim 18 is rejected on the same rationale under 35 U.S.C. 103 as claim 8 as they are substantially similar.
Claim(s) 10 is rejected under 35 U.S.C. 103 as being unpatentable over Arik in view of Shankaranaryana and Elkington in further view of U.S. Pub. No. US20210049236 Nguyen et al. (“Nguyen”)
In regards to claim 10,
Arik in view of Shankaranaryana and Elkington teaches The computer-implemented method of claim 9,
Elkington teaches the first table row and the second table row are selected from a plurality of table rows,
(Elkington, “[0056] A centroid record can be derived from duplicate records. The centroid record is a record that minimizes the overall distance to all of the duplicate records.
[0057] If dist(i,j) is the distance between records i and j [the first table row and the second table row are selected from a plurality of table rows], a centroid can be defined as centroid=ArgMin(dist(i,j)) (where i, j=1, 2, . . . N). For example, if five duplicate records are identified, containing the first names “John”, “John”, “Johnathan”, “Jonathan”, and “Jeff”, then “John” is selected as the centroid record since it has minimum distance between all pairs among those values.”)
Nguyen teaches and wherein the computer-implemented method further comprises: determining whether to map a table row selected from the plurality of table rows pair to k-dimensional tree data object based at least in part on the cross-row linking determination for the k-dimensional tree data object
(Nguyen, Fig. 4B and 5B, “[0063] As shown in FIG. 5B, at step 503, affinity score matrices are computed based on a target-side query matrix, the output representations for the set of terminal nodes and the output representations for the set of nonterminal nodes. Specifically, the network 402 receives the target-side query matrix
PNG
media_image7.png
31
89
media_image7.png
Greyscale
(407) and the source-side leaf and node matrices L (403) and N (405), and computes the affinity scores AQN∈
PNG
media_image8.png
27
48
media_image8.png
Greyscale
and AQL∈
PNG
media_image9.png
30
45
media_image9.png
Greyscale
through the linear layers 413 and 415:
A QN=(Q t W Q)(NW K)T/√{square root over (d)} (20)
A QL=(Q t W Q)(LW K)T/√{square root over (d)} (21)
At step 505, value representations for the set of terminal nodes are computed based on the output representations. At step 507, value representations for the set of nonterminal nodes are encoded using hierarchical accumulation based on the output representations for the terminal and nonterminal nodes:
PNG
media_image10.png
35
305
media_image10.png
Greyscale
(22)
where w=Luc with uc ∈
PNG
media_image11.png
26
33
media_image11.png
Greyscale
being a trainable vector.
[0064] At step 509, an attention output AttQ (425 in FIG. 4B) [k-dimensional tree data object] of decoder cross-attention is computed based on the affinity score matrices AQN, AQL and the value representations N′, L:
PNG
media_image12.png
35
179
media_image12.png
Greyscale
[determining whether to map the table row pair to a k-dimensional tree data object based at least in part on the cross-row linking determination for the k-dimensional tree data object; wherein computing based on the table row pair ie affinity (attention) score matrices is the determination of mapping it to a tree data object]”)
Nguyen teaches enabling access to a stored version of the k-dimensional tree data object, wherein the k-dimensional tree data object can be used to perform one or more data ingestion operations.
(Nguyen, “[0027] FIGS. 1A-1B are simplified diagrams of alternative configurations of a computing device 100 for implementing a tree transformer system [stored version of the k-dimensional tree data object] according to some embodiments. As shown in FIG. 1 , computing device 100 includes a processor 110 coupled to memory 120. Operation of computing device 100 is controlled by processor 110. And although computing device 100 is shown with only one processor 110, it is understood that processor 110 may be representative of one or more central processing units, multi-core processors, microprocessors, microcontrollers, digital signal processors, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), graphics processing units (GPUs) and/or the like in computing device 100. Computing device 100 may be implemented as a stand-alone subsystem, as a board added to a computing device, and/or as a virtual machine.
[0028] Memory 120 may be used to store software executed by computing device 100 and/or one or more data structures used during operation of computing device 100 [enabling access].”)
Arik in view of Shankaranaryana and Elkington and Nguyen are both considered to be analogous to the claimed invention because they are in the same field of natural language processing. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Arik in view of Shankaranaryana and Elkington to incorporate the teachings of Nguyen in order to fulfill the need for an efficient encoder/decoder in attention-based NLP neural architectures that will execute in constant time (Nguyen, “[0021] In view of the need for an efficient encoder or decoder for hierarchical structures, embodiments described herein provide an attention-based mechanism that encodes trees in a bottom-up manner and executes competitively with the transformer framework at constant parallel time complexity.”)
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
US Pub No. US20220164626A1 Microsoft teaches Automated merge conflict resolution with transformers
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JASMINE THAI whose telephone number is (703)756-5904. The examiner can normally be reached M-F 8-4.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Michael Huntley can be reached at (303) 297-4307. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/J.T.T./Examiner, Art Unit 2129
/MICHAEL J HUNTLEY/Supervisory Patent Examiner, Art Unit 2129
Prosecution Insights