Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This Office Action is in response to amendment filed on October 30, 2025.
Claims 1, 3-4, 6, 9-10, 13, 15-16, and 19-20 have been amended.
The objections and rejections from the prior correspondence that are not restated herein are withdrawn.
Response to Arguments
Applicant's arguments filed on October 30, 2025 have been fully considered but are not persuasive. With regards to the 101 rejection, Applicant argues “as the claim clearly contains multiple limitations that cannot possibly be performed in the human mind, the claim does not recite a mental process. For example, the cited limitation is a step of ‘training a Restricted Boltzmann Machine (RBM) neural network model having hidden nodes and visible nodes using the first dataset and the second dataset.’ A RBM neural network model cannot be possibly trained in the human mind.”
The Examiner respectfully disagrees. Examiner notes that the “training a Restricted Boltzmann Machine (RBM) neural network model…” limitation is not treated as a mental process, but rather as an additional element (in Step 2A prong 2 and 2B) that does not integrate the judicial exception into a practical application because it is simply instruction to use the computer as a tool (MPEP 2106.05(f)).
Applicant also argues that “the Office only provides conclusory statements in alleging the above quoted language without any reasoning or analysis provided”, “the Office further alleges that other steps of the claim are deemed insufficient in integrating the judicial exception into a practical application because it simply links the judicial exception to a field of use (MPEP 2106.05(h)),” and “no reasoning, explanation or analysis is provided by the Office besides the conclusory statements.”
Examiner clarifies that claim limitations that are treated as additional elements are analyzed as:
Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea - see MPEP 2106.05(f);
Adding insignificant extra-solution activity to the judicial exception - see MPEP 2106.05(g), i.e. well-known, understood, routine, and conventional activity as recognized by the courts (MPEP 2106.05(d)(II)(i)); or
Generally linking the use of the judicial exception to a particular technological environment or field of use – see MPEP 2106.05(h).
As indicated in the 101 rejection below, limitations indicated as one of the above do not integrate the judicial exception into a practical application and do not amount to significantly more than the judicial exception for reasons outlined below.
With regards to the prior art rejection, Applicant argues that Sakai fails to teach the amended limitations and that Wu, Hinton, and other cited prior art fail to cure the deficiencies.
The Examiner respectfully disagrees. Sakai teaches generating a one-hot dataset from a third dataset comprising the first dataset appended to the second dataset (Sakai, Pg. 1, Col. 1, Para. 3 teaches data fusion: “consider a situation where we observe two datasets: a purchase dataset and a media-viewing dataset of consumers. Both of them include demographic information related to consumers, such as gender, age, or income, which can be thought of as common variables. Using data fusion we can determine ‘the media that are viewed by consumers who purchase specific types of items.’” See also Pg. 3, Col. 1, Para. 1 “to complement the missing parts of xNobs with the predictors fA: XB × XC → DA and fB: XA × XC → DB, and to generate a complemented data matrix xˆN ∈ [0,1]N×R.”);
dividing the one-hot dataset into a plurality of batches (Sakai, Pg. 5, Col. 2, Para. 2 teaches “To accelerate the learning process and to prevent algorithms from being trapped by local optimal solutions, we used the “mini-batch” method and ‘momentum’ method [9]. In the ‘mini-batch’ method, we divided a training dataset into a number of small datasets and updated parameters for each small dataset.”).
Wu in combination with Sakai also teaches for each batch of the plurality of batches, estimating, for a missing entry in the respective batch of the plurality of batches, a first value for the visible nodes corresponding to the missing entry based on current values of the hidden nodes … given a set of current values for the visible nodes and a set of the current values for the hidden nodes [Wu, Pg. 11, Paragraph 2, "Let vmis and vobs denote the missing and observed parts of the visible data, respectively. Let vmis(t) denote the imputed value of vmis at iteration t. [...] Draw vmis(t+1) (for each batch of the plurality of batches, estimating... in the respective batch… a first value for the visible nodes corresponding to the missing entry) conditioned on h(t) (based on current values of the hidden nodes) and the current parameter estimate θ(t)." See also Pg. 5 for partitioning θ into a number of blocks and then finding a consistent estimate for each block conditional on the current estimates of other blocks, and Sakai, Pg. 5, Col. 2, Para. 2 also teaches dividing the dataset into mini-batches as shown above].
Therefore, Sakai in view of Wu and Hinton teaches the amended limitations of claim 1 as outlined in the rejection below.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because they are directed to abstract ideas without significantly more.
Claim 1
Step 1: This claim is a method, one of the four statutory categories.
Step 2A Prong 1:
generating a one-hot dataset from a third dataset comprising the first dataset appended to the second dataset;
for each batch of the plurality of batches, estimating, for a missing entry in the respective batch of the plurality of batches, a first value for the visible nodes corresponding to the missing entry based on current values of the hidden nodes
Mental process: estimation, judgment (i.e. concepts performed in the human mind)
Step 2A Prong 2:
receiving heterogenous survey data
This limitation is deemed insufficient in integrating the judicial exception into a practical application because it is insignificant extra-solution activity (MPEP 2106.05(g)).
comprising at least a first dataset having a first set of attributes and a second dataset having a second set of attributes, the first set of attributes and the second set of attributes having at least one common attribute and at least one attribute that is not in common between the first set of attributes and the second set of attributes, the first dataset and the second dataset having at least one missing entry
This limitation is deemed insufficient in integrating the judicial exception into a practical application because it simply links the judicial exception to a field of use (MPEP 2106.05(h)).
training a Restricted Boltzmann Machine (RBM) neural network model having hidden nodes and visible nodes using one-hot dataset
This limitation is deemed insufficient in integrating the judicial exception into a practical application because it is simply instruction to use the computer as a tool (MPEP 2106.05(f)).
dividing the combined dataset into a plurality of batches
This limitation is deemed insufficient in integrating the judicial exception into a practical application because it simply links the judicial exception to a field of use (MPEP 2106.05(h)).
comprising a first randomly selected sample made according to a first joint probability distribution of a value for the missing entry given a set of current values for the visible nodes and a set of the current values for the hidden nodes
This limitation is deemed insufficient in integrating the judicial exception into a practical application because it simply links the judicial exception to a field of use (MPEP 2106.05(h)).
the neural network model includes a visible layer corresponding to the visible nodes and a hidden layer corresponding to the hidden nodes that are configured as a fully connected bipartite graph
This limitation is deemed insufficient in integrating the judicial exception into a practical application because it simply links the judicial exception to a field of use (MPEP 2106.05(h)).
The additional elements as disclosed above, alone or in combination, do not integrate the judicial exception into a practical application as per the reasons discussed above. Therefore, the claim is directed to an abstract idea.
Step 2B:
receiving heterogenous survey data
This limitation is deemed to not amount to significantly more than the judicial exception because it is well-known, understood and routine activity, as recognized by the courts (MPEP 2106.05(d)(II)(i)).
comprising at least a first dataset having a first set of attributes and a second dataset having a second set of attributes, the first set of attributes and the second set of attributes having at least one common attribute and at least one attribute that is not in common between the first set of attributes and the second set of attributes, the first dataset and the second dataset having at least one missing entry
This limitation is deemed to not amount to significantly more than the judicial exception because it simply links the judicial exception to a field of use (MPEP 2106.05(h)).
training a Restricted Boltzmann Machine (RBM) neural network model having hidden nodes and visible nodes using the one-hot dataset
This limitation is deemed to not amount to significantly more than the judicial exception because it is simply instruction to use the computer as a tool (MPEP 2106.05(f)).
comprising a first randomly selected sample made according to a first joint probability distribution of a value for the missing entry given a set of current values for the visible nodes and a set of the current values for the hidden nodes
This limitation is deemed to not amount to significantly more than the judicial exception because it simply links the judicial exception to a field of use (MPEP 2106.05(h)).
the neural network model includes a visible layer corresponding to the visible nodes and a hidden layer corresponding to the hidden nodes that are configured as a fully connected bipartite graph
This limitation is deemed to not amount to significantly more than the judicial exception because it simply links the judicial exception to a field of use (MPEP 2106.05(h)).
The additional elements as disclosed above, in combination with the abstract ideas, are not sufficient to amount to significantly more than the judicial exception, as per the reasons discussed above. Therefore, the claim is not patent eligible.
Claim 2 inherits the rejections of claim 1.
Step 2A Prong 1:
imputing substitute values for the at least one missing entry to create an output dataset that does not include the at least one missing entry
Mental process: estimation, evaluation
Step 2A Prong 2:
The additional elements as disclosed above, alone or in combination, do not integrate the judicial exception into a practical application as per the reasons discussed above. Therefore, the claim is directed to an abstract idea.
Step 2B:
The additional elements as disclosed above, in combination with the abstract ideas, are not sufficient to amount to significantly more than the judicial exception, as per the reasons discussed above. Therefore, the claim is not patent eligible.
Claim 3 inherits the rejections of claim 2.
Step 2A Prong 1:
estimating, based on the current values of the hidden nodes obtained from the trained RBM, second values for the visible nodes corresponding to the at least one missing entry
Mental process: evaluation, estimation
Step 2A Prong 2:
The additional elements as disclosed above, alone or in combination, do not integrate the judicial exception into a practical application as per the reasons discussed above. Therefore, the claim is directed to an abstract idea.
Step 2B:
The additional elements as disclosed above, in combination with the abstract ideas, are not sufficient to amount to significantly more than the judicial exception, as per the reasons discussed above. Therefore, the claim is not patent eligible.
Claim 4 inherits the rejections of claim 3.
Step 2A Prong 1:
…according to a second probability distribution function of p(VmisslVpart,h), where Vmiss are current values of the visible nodes corresponding to the at least one missing entry, vpart are current values of the visible nodes not corresponding to the at least one missing entry, and h are the current values of the hidden nodes
Mathematical concept: mathematical equation
Step 2A Prong 2:
wherein the estimating second values is based on random sampling of the current values of the hidden nodes obtained from the trained neural network model…
This limitation is deemed insufficient in integrating the judicial exception into a practical application because it simply links the judicial exception to a field of use (MPEP 2106.05(h)).
The additional elements as disclosed above, alone or in combination, do not integrate the judicial exception into a practical application as per the reasons discussed above. Therefore, the claim is directed to an abstract idea.
Step 2B:
wherein the estimating second values is based on random sampling of the current values of the hidden nodes obtained from the trained neural network model…
This limitation is deemed to not amount to significantly more than the judicial exception because it simply links the judicial exception to a field of use (MPEP 2106.05(h)).
The additional elements as disclosed above, in combination with the abstract ideas, are not sufficient to amount to significantly more than the judicial exception, as per the reasons discussed above. Therefore, the claim is not patent eligible.
Claim 5 inherits the rejections of claim 2.
Step 2A Prong 2:
outputting an imputed dataset including the substitute values.
This limitation is deemed insufficient in integrating the judicial exception into a practical application because it is insignificant extra-solution activity (MPEP 2106.05(g)).
The additional elements as disclosed above, alone or in combination, do not integrate the judicial exception into a practical application as per the reasons discussed above. Therefore, the claim is directed to an abstract idea.
Step 2B:
outputting an imputed dataset including the substitute values.
This limitation is deemed to not amount to significantly more than the judicial exception because it is well-known, understood and routine activity, as recognized by the courts (MPEP 2106.05(d)(II)(i)).
The additional elements as disclosed above, in combination with the abstract ideas, are not sufficient to amount to significantly more than the judicial exception, as per the reasons discussed above. Therefore, the claim is not patent eligible.
Claim 6 inherits the rejections of claim 1.
Step 2A Prong 1:
wherein the first joint probability distribution is p(VmisslVpart,h), where Vmiss are current values of the visible nodes corresponding to the at least one missing entry, Vpart are current values of the visible nodes no corresponding to the at least one missing entry, and h are the current values of the hidden nodes.
Mathematical concept: mathematical equation
Step 2A Prong 2:
The additional elements as disclosed above, alone or in combination, do not integrate the judicial exception into a practical application as per the reasons discussed above. Therefore, the claim is directed to an abstract idea.
Step 2B:
The additional elements as disclosed above, in combination with the abstract ideas, are not sufficient to amount to significantly more than the judicial exception, as per the reasons discussed above. Therefore, the claim is not patent eligible.
Claim 7 inherits the rejections of claim 1.
Step 2A Prong 1:
alternately Gibbs sampling the visible layer and the hidden layer fork iterations, where k>1
Mathematical concept: mathematical calculation
Step 2A Prong 2:
The additional elements as disclosed above, alone or in combination, do not integrate the judicial exception into a practical application as per the reasons discussed above. Therefore, the claim is directed to an abstract idea.
Step 2B:
The additional elements as disclosed above, in combination with the abstract ideas, are not sufficient to amount to significantly more than the judicial exception, as per the reasons discussed above. Therefore, the claim is not patent eligible.
Claim 8 inherits the rejections of claim 1.
Step 2A Prong 2:
wherein the first data set has at least ten times the number of entries as the second data set
This limitation is deemed insufficient in integrating the judicial exception into a practical application because it simply links the judicial exception to a field of use (MPEP 2106.05(h)).
The additional elements as disclosed above, alone or in combination, do not integrate the judicial exception into a practical application as per the reasons discussed above. Therefore, the claim is directed to an abstract idea.
Step 2B:
wherein the first data set has at least ten times the number of entries as the second data set
This limitation is deemed to not amount to significantly more than the judicial exception because it simply links the judicial exception to a field of use (MPEP 2106.05(h)).
The additional elements as disclosed above, in combination with the abstract ideas, are not sufficient to amount to significantly more than the judicial exception, as per the reasons discussed above. Therefore, the claim is not patent eligible.
Claim 9 inherits the rejections of claim 1.
Step 2A Prong 2:
wherein the first joint probability distribution corresponds to the one-hot dataset
This limitation is deemed insufficient in integrating the judicial exception into a practical application because it simply links the judicial exception to a field of use (MPEP 2106.05(h)).
The additional elements as disclosed above, alone or in combination, do not integrate the judicial exception into a practical application as per the reasons discussed above. Therefore, the claim is directed to an abstract idea.
Step 2B:
wherein the first joint probability distribution corresponds to the one-hot dataset
This limitation is deemed to not amount to significantly more than the judicial exception because it simply links the judicial exception to a field of use (MPEP 2106.05(h)).
The additional elements as disclosed above, in combination with the abstract ideas, are not sufficient to amount to significantly more than the judicial exception, as per the reasons discussed above. Therefore, the claim is not patent eligible.
Claim 10 inherits the rejections of claim 9.
Step 2A Prong 1:
…k-fold contrastive divergence algorithm…
Mathematical concept: mathematical equation
Step 2A Prong 2:
applying a k-fold contrastive divergence algorithm to each of the plurality of batches
This limitation is deemed insufficient in integrating the judicial exception into a practical application because it is simply instruction to apply exception (MPEP 2106.05(f)).
The additional elements as disclosed above, alone or in combination, do not integrate the judicial exception into a practical application as per the reasons discussed above. Therefore, the claim is directed to an abstract idea.
Step 2B:
applying a k-fold contrastive divergence algorithm to each of the plurality of batches
This limitation is deemed to not amount to significantly more than the judicial exception because it is simply instruction to apply the exception (MPEP 2106.05(f)).
The additional elements as disclosed above, in combination with the abstract ideas, are not sufficient to amount to significantly more than the judicial exception, as per the reasons discussed above. Therefore, the claim is not patent eligible.
Claim 11 inherits the rejections of claim 1.
Step 2A Prong 2:
wherein the first dataset corresponds to a first survey data having a first scale and the second dataset corresponds to a second survey data having a second scale.
This limitation is deemed insufficient in integrating the judicial exception into a practical application because it simply links the judicial exception to a field of use (MPEP 2106.05(h)).
The additional elements as disclosed above, alone or in combination, do not integrate the judicial exception into a practical application as per the reasons discussed above. Therefore, the claim is directed to an abstract idea.
Step 2B:
wherein the first dataset corresponds to a first survey data having a first scale and the second dataset corresponds to a second survey data having a second scale.
This limitation is deemed to not amount to significantly more than the judicial exception because it simply links the judicial exception to a field of use (MPEP 2106.05(h)).
The additional elements as disclosed above, in combination with the abstract ideas, are not sufficient to amount to significantly more than the judicial exception, as per the reasons discussed above. Therefore, the claim is not patent eligible.
Claim 12 inherits the rejections of claim 11.
Step 2A Prong 2:
wherein the first scale is larger than the second scale.
This limitation is deemed insufficient in integrating the judicial exception into a practical application because it simply links the judicial exception to a field of use (MPEP 2106.05(h)).
The additional elements as disclosed above, alone or in combination, do not integrate the judicial exception into a practical application as per the reasons discussed above. Therefore, the claim is directed to an abstract idea.
Step 2B:
wherein the first scale is larger than the second scale.
This limitation is deemed to not amount to significantly more than the judicial exception because it simply links the judicial exception to a field of use (MPEP 2106.05(h)).
The additional elements as disclosed above, in combination with the abstract ideas, are not sufficient to amount to significantly more than the judicial exception, as per the reasons discussed above. Therefore, the claim is not patent eligible.
Claim 13 recites similar limitations to claim 1. Therefore, it is rejected under the same rationale.
Claim 14 recites similar limitations to claim 2. Therefore, it is rejected under the same rationale.
Claim 15 recites similar limitations to claim 3. Therefore, it is rejected under the same rationale.
Claim 16 recites similar limitations to claim 4. Therefore, it is rejected under the same rationale.
Claim 17 recites similar limitations to claim 7. Therefore, it is rejected under the same rationale.
Claim 18 recites similar limitations to claim 8. Therefore, it is rejected under the same rationale.
Claim 19 recites similar limitations to claim 9. Therefore, it is rejected under the same rationale.
Claim 20 recites similar limitations to claim 10. Therefore, it is rejected under the same rationale.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-7, 9 13-17 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Sakai et al. ("Data Fusion Using Restricted Boltzmann Machines"), hereafter referred to as Sakai in view of Wu et al. ("Accelerate Training of Restricted Boltzmann Machines via Iterative Conditional Maximum Likelihood Estimation"), hereafter referred to as Wu, and Hinton (“A Practical Guide to Training Restricted Boltzmann Machines”).
In regards to claim 1, Sakai teaches
A method, comprising:
receiving heterogenous survey data comprising at least a first dataset having a first set of attributes and a second dataset having a second set of attributes, the first set of attributes and the second set of attributes having at least one common attribute and at least one attribute that is not in common between the first set of attributes and the second set of attributes, the first dataset and the second dataset having at least one missing entry [Sakai, Abstract, "Suppose that we are given two datasets, where some variables are different each other and others are the same. The goal of data fusion is to complement the missing unique variables in each dataset using the common variables."; See also pg. 1, Col. 1, Paragraph 2];
generating a one-hot dataset from a third dataset comprising the first dataset appended to the second dataset (Sakai, Pg. 1, Col. 1, Para. 3 teaches data fusion: “consider a situation where we observe two datasets: a purchase dataset and a media-viewing dataset of consumers. Both of them include demographic information related to consumers, such as gender, age, or income, which can be thought of as common variables. Using data fusion we can determine ‘the media that are viewed by consumers who purchase specific types of items.’” See also Pg. 3, Col. 1, Para. 1 “to complement the missing parts of xNobs with the predictors fA: XB × XC → DA and fB: XA × XC → DB, and to generate a complemented data matrix xˆN ∈ [0,1]N×R.”);
training a Restricted Boltzmann Machine (RBM) neural network model having hidden nodes and visible nodes using the one-hot dataset [Sakai, Pg. 2, Col. 1, Paragraph 3, "RBMs have the following characteristics: [...] Secondly, there is restriction on the bipartite graph, which makes the hidden units and visible units mutually independent (having hidden nodes and visible nodes)."; "Problem Setting of Data Fusion", Fig. 1, "As shown in Figure 1, we are given a data matrix that is divided into two parts: One is specified by variable vectors XA and XC while the other is specified by XB and XC where XA and XB are disjoint from each other."; "Framework of RBM-Based Data Fusion", "Let xNobs ∈ {0,1}N×R − denote the observed part of the data matrix. [...] In the procedure of data fusion using RBMS, it is most important how we learn RBMs from xNobsZ which includes missing values (training a Restricted Boltzmann Machine (RBM) neural network model...using the one-hot dataset)."]…
dividing the one-hot dataset into a plurality of batches (Sakai, Pg. 5, Col. 2, Para. 2 teaches “To accelerate the learning process and to prevent algorithms from being trapped by local optimal solutions, we used the “mini-batch” method and ‘momentum’ method [9]. In the ‘mini-batch’ method, we divided a training dataset into a number of small datasets and updated parameters for each small dataset.”).
Sakai does not teach
for each batch of the plurality of batches, estimating, for a missing entry in the respective batch of the plurality of batches, a first value for the visible nodes corresponding to the missing entry based on current values of the hidden nodes comprising a first randomly selected sample made according to a first joint probability distribution of a value for the missing entry given a set of current values for the visible nodes and a set of the current values for the hidden nodes, wherein
the neural network model includes a visible layer corresponding to the visible nodes and a hidden layer corresponding to the hidden nodes that are configured as a fully connected bipartite graph.
However, Wu does teach
for each batch of the plurality of batches, estimating, for a missing entry in the respective batch of the plurality of batches, a first value for the visible nodes corresponding to the missing entry based on current values of the hidden nodes … given a set of current values for the visible nodes and a set of the current values for the hidden nodes [Wu, Pg. 11, Paragraph 2, "Let vmis and vobs denote the missing and observed parts of the visible data, respectively. Let vmis(t) denote the imputed value of vmis at iteration t. [...] Draw vmis(t+1) (for each batch of the plurality of batches, estimating... in the respective batch… a first value for the visible nodes corresponding to the missing entry) conditioned on h(t) (based on current values of the hidden nodes) and the current parameter estimate θ(t)." See also Pg. 5 for partitioning θ into a number of blocks and then finding a consistent estimate for each block conditional on the current estimates of other blocks, and Sakai, Pg. 5, Col. 2, Para. 2 also teaches dividing the dataset into mini-batches as shown above], wherein
the neural network model includes a visible layer corresponding to the visible nodes and a hidden layer corresponding to the hidden nodes that are configured as a fully connected bipartite graph [Wu, Fig 1.; Note: This figure, as shown below, shows an RBM with 2 layers of nodes. These nodes are fully connected in a bipartite graph.; Wu, 2.1, "A RBM is a bipartite undirected graphical model…"].
PNG
media_image1.png
350
600
media_image1.png
Greyscale
The Sakai and Wu references would have been recognized by those of ordinary skill in the art as useful for applicant’s purpose in combining multiple datasets with missing entries using restricted Boltzmann machines.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of the prior art for estimating the values of the visible nodes associated with a missing entry by Wu with the method for training an RBM for data fusion as disclosed by Sakai.
One of ordinary skill in the arts would have been motivated to combine the disclosed methods in order to create a fast and efficient algorithm for training RBMs (Wu, pg. 2, 2nd paragraph)
Sakai in view of Wu does not teach
…comprising a first randomly selected sample made according to a first joint probability distribution of a value for the missing entry…
However, Hinton does teach
…comprising a first randomly selected sample made according to a first joint probability distribution of a value for the missing entry [Hinton, pg. 5, Paragraph 6, "Because there are no direct connections between visible units in an RBM, it is also very easy to get an unbiased sample of the state of a visible unit, given a hidden vector; p(vi = 1 | h) (a first joint probability distribution) (8) [...] Getting an unbiased sample of ⟨vihj⟩model model, however, is much more difficult. It can be done by starting at any random state of the visible units (a first randomly selected sample) and performing alternating Gibbs sampling for a very long time. One iteration of alternating Gibbs sampling consists of updating all of the hidden units in parallel using equation 7 followed by updating all of the visible units in parallel using equation 8."; Hinton, pg. 18, Paragraph 2, "If the label is missing from a subset of the cases, it can be Gibbs sampled from its exact conditional distribution (comprising a first randomly selected sample made according to a first joint probability distribution of a value for the missing entry)."]…
The Sakai, Wu, and Hinton references would have been recognized by those of ordinary skill in the art as useful for applicant’s purpose in combining multiple datasets with missing entries using restricted Boltzmann machines.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of the prior art for selecting a sample of hidden units using a joint probability distribution by Hinton with the method for training an RBM on a combined dataset with missing values as disclosed by Sakai in view of Wu.
One of ordinary skill in the arts would have been motivated to combine the disclosed methods in order to get an unbiased sample of hidden units (Hinton, Pg. 5, Paragraph 6).
In regards to claim 2, Sakai in view of Wu and Hinton teach [t]he method of claim 1. Furthermore, Sakai teaches
imputing substitute values for the at least one missing entry to create an output dataset that does not include the at least one missing entry [Sakai, pg. 3, Paragraph 1, "Our objective is to complement the missing parts of xNobs (imputing substitute values for the at least one missing entry) with the predictors […], and to generate a complemented data matrix x̂N∈[0,1]N×R (create an output dataset that does not include the at least one missing entry)."].
In regards to claim 3, Sakai in view of Wu and Hinton teach [t]he method of claim 2. Furthermore, Wu and Sakai teaches
estimating, based on the current values of the hidden nodes obtained from the trained RBM, second values for the visible nodes corresponding to the at least one missing entry [Wu, Pg. 11, Paragraph 2, "Let vmis and vobs denote the missing and observed parts of the visible data, respectively. Let vmis(t) denote the imputed value of vmis at iteration t. [...] Draw vmis(t+1) (estimating...second values for the visible layer nodes corresponding to the at least one missing entry) conditioned on h(t) (based on the current values of the hidden layer nodes) and the current parameter estimate θ(t)."; Sakai, pg. 1, col. 2, Paragraph 3, "According to [1], a finite mixture model is first learned from the observed data using the EM (Expectation and Maximization) algorithm, and then the missing values are complemented by sampling values from the learned model (obtained from the trained RBM)."].
In regards to claim 4, Sakai in view of Wu and Hinton teach [t]he method of claim 3. Furthermore, Hinton teaches
wherein the estimating second values is based on random sampling of the current values of the hidden nodes obtained from the trained neural network model according to a second probability distribution function of p(vmiss|vpart, h), where vmiss are current values of the visible nodes corresponding to the at least one missing entry, vpart are current values of the visible nodes not corresponding to the at least one missing entry, and h are the current values of the hidden nodes [Hinton, pg. 4, Paragraph 6, "Because there are no direct connections between visible units in an RBM, it is also very easy to get an unbiased sample of the state of a visible unit, given a hidden vector; p(vi = 1 | h) (8) (a second probability distribution function of p(vmiss|vpart, h)... h are the current values of the hidden nodes ; Note: Since all of the visible nodes are conditionally independent, p(vmiss|vpart,h) = p(vmiss|h))" [...]; One iteration of alternating Gibbs sampling consists of updating all of the hidden units in parallel using equation 7 followed by updating all of the visible units in parallel using equation 8."; Hinton, pg. 18, Paragraph 2, "If the label is missing from a subset of the cases, it can be Gibbs sampled from its exact conditional distribution (wherein the estimating second values is based on random sampling of the current values of the hidden nodes obtained from the trained neural network model...vmiss are current values of the visible layer nodes corresponding to the at least one missing entry)"].
In regards to claim 5, Sakai in view of Wu and Hinton teach [t]he method of claim 2. Furthermore, Sakai teaches
outputting an imputed dataset including the substitute values [Sakai, Pg. 3, Col. 1, 1st Paragraph, "... to generate a complemented data matrix..."].
In regards to claim 6, Sakai in view of Wu and Hinton teach [t]he method of claim 1. Furthermore, Hinton teaches
wherein the first joint probability distribution is p(vmiss|vpart, h), where vmiss are current values of the visible nodes corresponding to the at least one missing entry, vpart are current values of the visible nodes not corresponding to the at least one missing entry, and h are the current values of the hidden nodes [Hinton, pg. 4, Paragraph 6, "Because there are no direct connections between visible units in an RBM, it is also very easy to get an unbiased sample of the state of a visible unit, given a hidden vector; p(vi = 1 | h) (8) (wherein the first joint probability distribution is p(vmiss|vpart, h)... h are the current values of the hidden nodes ; Note: Since all of the visible nodes are conditionally independent, p(vmiss|vpart,h) = p(vmiss|h))" [...]; One iteration of alternating Gibbs sampling consists of updating all of the hidden units in parallel using equation 7 followed by updating all of the visible units in parallel using equation 8."; Hinton, pg. 18, Paragraph 2, "If the label is missing from a subset of the cases, it can be Gibbs sampled from its exact conditional distribution (vmiss are current values of the visible layer nodes corresponding to the at least one missing entry)"].
In regards to claim 7, Sakai in view of Wu and Hinton teach [t]he method of claim 1. Furthermore, Hinton teaches
alternately Gibbs sampling the visible layer and the hidden layer for k iterations, where k > 1 [Hinton, pg. 5, Paragraph 4, "RBMs typically learn better models if more steps of alternating Gibbs sampling are used before collecting the statistics for the second term in the learning rule, which will be called the negative statistics. CDn will be used to denote learning using n full steps of alternating Gibbs sampling."].
In regards to claim 9, Sakai in view of Wu and Hinton teach [t]he method of claim 1. Furthermore, Sakai teaches
wherein the first joint probability distribution corresponds to the one-hot dataset [Sakai, Pg. 1, Col. 1, 1st Paragraph, "As shown in Figure 1, we are given a data matrix that is divided into two parts: One is specified by variable vectors XA and XC while the other is specified by XB and XC, where XA and XB are disjoint from each other. [...] Suppose here that the values of (XA,XB,XC) are independently and identically distributed according to a joint probability distribution P∗. [...] In the LV approach, data fusion is reduced to an unsupervised learning task, i.e., estimation of the joint distribution P* having latent-variables from the observed data."].
In regards to claim 13, the claim recites similar limitations to claim 1. Therefore, it is rejected under the same rationale.
In regards to claim 14, the claim recites similar limitations to claim 2. Therefore, it is rejected under the same rationale.
In regards to claim 15, the claim recites similar limitations to claim 3. Therefore, it is rejected under the same rationale.
In regards to claim 16, the claim recites similar limitations to claim 4. Therefore, it is rejected under the same rationale
In regards to claim 17, the claim recites similar limitations to claim 7. Therefore, it is rejected under the same rationale.
In regards to claim 19, the claim recites similar limitations to claim 9. Therefore, it is rejected under the same rationale.
Claims 8 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Sakai in view of Wu and Hinton as applied to claims 1 and 13 above, and further in view of Wilderjans et al. ("Simultaneous analysis of coupled data blocks differing in size: A comparison of two weighting schemes"), hereafter referred to as Wilderjans.
In regards to claim 8, Sakai in view of Wu and Hinton teach [t]he method of claim 1.
Sakai in view of Wu and Hinton does not teach
wherein the first data set has at least ten times the number of entries as the second data set.
However, Wilderjans does teach
wherein the first data set has at least ten times the number of entries as the second data set [Wilderjans, Pg. 3, 2.1.1. Model, "The specific multiway multiblock component model under study (for more information about the family of multiway multiblock component models, see Smilde et al. (2000)), approximates an I×J×K object by attribute by source real-valued data block D1 and an I×L object by covariate real-valued data block D2..."; Wilderjans, Pg. 4, 2.2.2 Model, "the Array/total ratio, r, of the size of T1 (D1, M1), to the total size T1 + T2...: r=(I×J×K)/(I×J×K)+(I×L). This factor was manipulated at four levels: .50, .90, .95, .99." (Note: When the size of D1 (I×J×K) is 10 times larger than the size of D2 (I×L), r = 0.90.)].
The Sakai, Wu, Hinton, and Wilderjans references would have been recognized by those of ordinary skill in the art as useful for applicant’s purpose in combining datasets containing missing values using an RBM and imputing those missing values.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of the prior art for combining datasets of different sizes by Wilderjans with the method for fusing datasets with missing values using an RBM as disclosed by Sakai in view of Wu and Hinton.
One of ordinary skill in the arts would have been motivated to combine the disclosed methods in order to achieve better performance when analyzing the combined datasets (Wilderjans, pg. 13, 2nd paragraph).
In regards to claim 18, the claim recites similar limitations to claim 8. Therefore, it is rejected under the same rationale.
Claims 10 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Sakai in view of Wu and Hinton as applied to claims 9 and 19 above, and further in view of Ning et al ("LCD: A Fast Contrastive Divergence Based Algorithm for Restricted Boltzmann Machine"), hereafter referred to as Ning.
In regards to claim 10, Sakai in view of Wu and Hinton teach [t]he method of claim 9.
Sakai in view of Wu and Hinton does not teach
applying a k-fold contrastive divergence algorithm to each of the plurality of batches.
However, Ning does teach
applying a k-fold contrastive divergence algorithm to each of the plurality of batches [Ning, Pg. 3, 2nd Paragraph, "For every input, CD starts a Markov Chain by assigning an input vector to the visible units and performs k steps of Gibbs Sampling (applying a k-fold contrastive divergence algorithm to each of the plurality of batches)... The corresponding algorithm is denoted as CD-k. As illustrated in Alg 1, it consists of a number of epochs. In each epoch, the algorithm goes through the inputs in batches (dividing the combined dataset into a plurality of batches)."].
The Sakai, Wu, Hinton, and Ning references would have been recognized by those of ordinary skill in the art as useful for applicant’s purpose in combining datasets containing missing values using an RBM and imputing those missing values.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of the prior art for dividing the training dataset and applying contrastive divergence to them by Ning with the method for training an RBM on a combined dataset containing missing values as disclosed by Sakai in view of Wu and Hinton.
One of ordinary skill in the arts would have been motivated to combine the disclosed methods in order to speed up the training of the RBM (Ning, Pg. 11, 1st Paragraph).
In regards to claim 20, the claim recites similar limitations to claim 10. Therefore, it is rejected under the same rationale.
Claims 11 and 12 are rejected under 35 U.S.C. 103 as being unpatentable over Sakai in view of Wu and Hinton as applied to claim 1 above, and further in view of Samson et al (US 7698345 B2), hereafter referred to as Samson, and Wilderjans.
In regards to claim 11, Sakai in view of Wu and Hinton teach [t]he method of claim 1.
Sakai in view of Wu and Hinton does not teach
wherein the first dataset corresponds to a first survey data having a first scale and the second dataset corresponds to a second survey data having a second scale.
However, Samson does teach
wherein the first dataset corresponds to a first survey data… and the second dataset corresponds to a second survey data [Samson, Col.2, Lines 10-15, "...market researchers have employed database fusion techniques to efficiently merge or fuse database information from multiple research panels or surveys (typically two at a time) into a single database representing a single virtual population group or respondent-level panel. (wherein the first dataset corresponds to a first survey data...and the second dataset corresponds to a second survey data)"]…
The Sakai, Wu, Hinton, and Samson references would have been recognized by those of ordinary skill in the art as useful for applicant’s purpose in combining datasets containing missing values using an RBM and imputing those missing values.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of the prior art for fusing survey datasets by Samson with the method for training an RBM on a combined dataset containing missing values as disclosed by Sakai in view of Wu and Hinton.
One of ordinary skill in the arts would have been motivated to combine the disclosed methods in order to “enable the development of a database that reveals correlations between the consumption activities, preferences, etc. associated with two datasets or databases in a manner that the individual datasets could not.” (Samson, Col. 2, Lines 16-20).
Sakai in view of Wu, Hinton, and Samson does not teach
…having a first scale...having a second scale.
However, Wilderjans does teach
a first survey data having a first scale...a second survey data having a second scale [Wilderjans, Abstract, "When dealing with such coupled data blocks (i.e., different N-way N-mode data blocks that have one or more modes in common) it often happens that one data block is much larger in size than the other(s) (a first survey data having a first scale...a second survey data having a second scale).”, see also pg. 1 – pg. 2, I. Introduction].
In regards to claim 12, Sakai in view of Wu, Hinton, Samson, and Wilderjans teach [t]he method of claim 11. Further, Wilderjans teaches
wherein the first scale is larger than the second scale [Wilderjans, Abstract, "When dealing with such coupled data blocks (i.e., different N-way N-mode data blocks that have one or more modes in common) it often happens that one data block is much larger in size than the other(s).", see also pg. 1 – pg. 2, I. Introduction].
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Le Roux et al. (US 20100228694 A1) teaches processing data using RBMs.
THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ANDREW J JUNG whose telephone number is (571)270-3779. The examiner can normally be reached on Monday through Friday from 9am to 5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, DAVID WILEY can be reached on 571-272-4150. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ANDREW J JUNG/Supervisory Patent Examiner, Art Unit 2175