DETAILED ACTION
This office action is in response to submission of application on 11/30/2023.
Claims 1-20 are presented for examination.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 02/29/2024 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 9-10, and 12-14 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 8 recites “The method of claim 1, further comprising at least one of…” followed by three limitations. The phrase “at least one of” suggests that these additional limitations make up a group of alternatives, and that the claim encompasses embodiments where only some of these features are included. However, claims 9-10 and 12-14 are dependent on claim 8 and include elements which rely on claim 8’s optional features. For example, claim 8 includes the limitation “constructing a plurality of GMM distributions…” within its optional features. Claim 10, which depends on claim 8, includes the limitation “storing the constructed plurality of GMM distributions…” In the case of an embodiment of claim 8 in which the plurality of GMM distributions is not constructed, it is unclear how the plurality of GMM distributions could be stored in claim 10, and thus it is unclear how claim 10 would modify claim 8. Claims 9 and 12-14 are also rejected for similar reasons – it is unclear how they modify claim 8 in light of its optional features.
Additionally, Claim 13 recites “probabilistic similarity based analysis of the representational numeric values against the constructed plurality of GMM distributions to find the plurality of smaller sets.” However, per claim 8, the plurality of GMM distributions are constructed using the plurality of smaller sets. It is unclear how a similarity based analysis against the constructed plurality of GMM distributions can be used to find the smaller sets, when the smaller sets were already required to construct the plurality of GMM distributions. For examination purposes, the claim limitation above will be interpreted as “probabilistic similarity based analysis of the representational numeric values against the constructed plurality of GMM distributions to find the subset of complex objects.”
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Claim 1:
Step 1: The claim is directed to a method, which falls within the statutory category of a process.
Step 2A Prong 1: The claim is directed to an abstract idea. Specifically, the claim recites:
iteratively deriving parameters of constituent Gaussian distributions of the GMM distribution based on executing an Expectation-Maximization (EM) algorithm on the processor communicatively coupled to the memory, the EM algorithm incorporating the data set as an input thereinto; (Abstract idea – mathematical concept. Deriving GMM parameters using the Expectation-Maximization (EM) algorithm is a mathematical calculation. See MPEP 2106.04(a)(2)(I).)
modifying the data set by replacing, for each constituent Gaussian distribution of the GMM distribution, at least one of: numeric values and vectors of the numeric values of the data set that differ in magnitude from a center of the each constituent Gaussian distribution by less than a threshold with a mean value of the data set, with a weight of the mean value being indicative of a cardinality thereof within the modified data set; (Abstract idea – mental process. Replacing elements of a data set with weighted mean values when the elements differ from GMM component centers by less than a threshold can practically be performed in the human mind or with the aid of pen and paper, for example, by viewing the data set and GMM components on a sheet of paper, mentally calculating the distance between data points and GMM component centers, mentally comparing said distance to a threshold, and when the difference is below the threshold, replacing the data points by hand with their mean and a weight indicating their cardinality. The courts have recognized that claims can recite a mental process even if they are claimed as being performed on a computer. See MPEP 2106.04(a)(2)(III).)
continuing the iterative derivation of the parameters of the constituent Gaussian distributions of the GMM distribution based on the execution of the EM algorithm that now incorporates the modified data set thereinto along with the weight of the mean value; (Abstract idea – mathematical concept. Deriving GMM parameters using the Expectation-Maximization (EM) algorithm is a mathematical calculation. See MPEP 2106.04(a)(2)(I).)
reducing a data footprint of the data set through the GMM distribution based on the continued iterative derivation of the parameters of the constituent Gaussian distributions thereof. (Abstract idea – mental process. Reducing the footprint of the data set can practically be performed in the human mind or with the aid of pen and paper, for example, by iteratively executing the mathematical and mental steps claimed above, which replace individual elements in the data set with summarizing weighted means, thereby reducing the number of elements in the data set. See MPEP 2106.04(a)(2)(III).)
Step 2A Prong 2: The additional elements recited in the claim do not integrate the abstract idea into a practical application, individually or in combination. Specifically, the claim recites the additional elements:
using a processor communicatively coupled to a memory (This limitation is interpreted as implementation of the disclosed method in a generic computing environment, and thus amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely using a computer as a tool to perform an abstract idea – see MPEP 2106.05(f).)
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. Specifically, the claim recites the additional elements:
using a processor communicatively coupled to a memory (This limitation is interpreted as implementation of the disclosed method in a generic computing environment, and thus amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely using a computer as a tool to perform an abstract idea – see MPEP 2106.05(f).)
Claims 2-20:
Claim 2 recites The method of claim 1, comprising determining the threshold in accordance with at least one of: estimating an entropy of a probability distribution characterizing an extent to which the at least one of: the numeric values and the vectors of the numeric values are generated from the each constituent Gaussian distribution; and the at least one of: the numeric values and the vectors of the numeric values being different in magnitude from the center of the each constituent Gaussian distribution by less than a numeric distance and from another center of at least one other constituent Gaussian distribution of the GMM distribution by more than another numeric distance. Determining the threshold by estimating the entropy of a probability distribution is an evaluation/judgement which can practically be performed in the human mind or with the aid of pen and paper, and thus amounts to a mental process. See MPEP 2106.04(a)(2)(III). Therefore, the claim merges with the abstract idea recited in claim 1, and does not recite additional elements that are sufficient to amount to significantly more than the abstract idea.
Claim 3 recites The method of claim 1, wherein in accordance with the at least one of: the numeric values and the vectors of the numeric values that differ in magnitude from the center of the each constituent Gaussian distribution by less than the threshold being the same as one another, the method further comprises: adding a standard deviation of the same at least one of: the numeric values and the vectors of the numeric values as equal to 0 to the GMM distribution; and performing a subsequent phase of the continued iterative derivation of the parameters of the constituent Gaussian distributions without the same at least one of: the numeric values and the vectors of the numeric values. Determining that two data elements are the same, adding a component with standard deviation equal to 0 to the GMM distribution, and removing the data elements from the data set for subsequent iterations are evaluations/judgements which can practically be performed in the human mind or with the aid of pen and paper, and thus amount to a mental process. See MPEP 2106.04(a)(2)(III). Therefore, the claim merges with the abstract idea recited in claim 1, and does not recite additional elements that are sufficient to amount to significantly more than the abstract idea.
Claim 4 recites The method of claim 1, further comprising: identifying at least one outlier element in the data set in accordance with the execution of the EM algorithm; adding the identified at least one outlier element to the GMM distribution with a standard deviation thereof equal to 0; and continuing an iterative process of the EM algorithm after removing the at least one outlier element from the modified data set. Identifying an outlier in the data set, adding a component with standard deviation equal to 0 to the GMM distribution, and removing the data element from the data set for subsequent iterations are evaluations/judgements which can practically be performed in the human mind or with the aid of pen and paper, and thus amount to a mental process. See MPEP 2106.04(a)(2)(III). Therefore, the claim merges with the abstract idea recited in claim 1, and does not recite additional elements that are sufficient to amount to significantly more than the abstract idea.
Claim 5 recites The method of claim 1, further comprising: specifying a first number of the constituent Gaussian distributions of the GMM distribution as an input parameter to the EM algorithm; and the EM algorithm working with a second number of the constituent Gaussian distributions of the GMM distribution that is less than the first number of the constituent Gaussian distributions in accordance with the GMM distribution with the first number of the constituent Gaussian distributions inadequately approximating the data set. Specifying a first number of GMM components and then reducing the number of components if the first number of components is inadequate is a judgement/evaluation which can practically be performed in the human mind or with the aid of pen and paper (i.e. mental process). See MPEP 2106.04(a)(2)(III). Executing the EM algorithm is a mathematical concept. See MPEP 2106.04(a)(2)(I). Therefore, the claim merges with the abstract idea recited in claim 1, and does not recite additional elements that are sufficient to amount to significantly more than the abstract idea.
Claim 6 recites The method of claim 1, further comprising one of: replacing at least one of: the data set and the modified data set with the GMM distribution for an operation to be performed using the processor communicatively coupled to the memory; and adding the GMM distribution as metadata to the memory for availability thereof together with the at least one of: the data set and the modified data set for the operation to be performed using the processor communicatively coupled to the memory. Replacing the data set with the GMM distribution for performing a downstream operation is a judgement/evaluation which can practically be performed in the human mind or with the aid of pen and paper (i.e. mental process). See MPEP 2106.04(a)(2)(III). Performing the operation using the processor and memory amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely using a computer as a tool to perform an abstract idea – see MPEP 2106.05(f). Therefore, the claim merges with the abstract idea recited in claim 1, and does not recite additional elements that are sufficient to amount to significantly more than the abstract idea.
Claim 7 recites The method of claim 1, comprising at least one of: the numeric values of the data set being taken by consecutive rows of a data column of a data table that one of: constitutes and at least is part of the data set; and the vectors of the numeric values of the data set being taken by consecutive rows of one of: the data table and another data table for at least a subset of data columns thereof. Obtaining the data points from a data table is an evaluation/judgement that can practically be performed in the human mind or with the aid of pen and paper (i.e. mental process), for example, by viewing a data table on a sheet of paper and mentally selecting data points to be used as input to the EM algorithm. Therefore, the claim merges with the abstract idea recited in claim 1, and does not recite additional elements that are sufficient to amount to significantly more than the abstract idea.
Claim 8 recites The method of claim 1, further comprising at least one of: the data set being a smaller set of a larger data set; constructing a plurality of GMM distributions comprising the generated GMM distribution for a corresponding plurality of smaller sets of the larger data set comprising the data set; and merging the GMM distributions of the constructed plurality of GMM distributions together to form another GMM distribution. Constructing a plurality of GMM distributions by the method of claim 1 and then merging them together are evaluations/judgements which can practically be performed in the human mind or with the aid of pen and paper (i.e. mental process). Therefore, the claim merges with the abstract idea recited in claim 1, and does not recite additional elements that are sufficient to amount to significantly more than the abstract idea.
Claim 9 recites The method of claim 8, further comprising at least one of: merging the GMM distributions of the constructed plurality of GMM distributions together in accordance with modification of the EM algorithm to account for all parameters of the constructed plurality of GMM distributions; and optimizing splitting of the larger data set into the plurality of smaller sets comprising the data set in accordance with maximizing approximation of the constructed plurality of GMM distributions to the corresponding plurality of smaller sets. Merging the GMM distributions in accordance with the EM algorithm and splitting the data set in order to maximize GMM approximation of the smaller sets are evaluations/judgements which can practically be performed in the human mind or with the aid of pen and paper (i.e. mental process). Therefore, the claim merges with the abstract idea recited in claim 1, and does not recite additional elements that are sufficient to amount to significantly more than the abstract idea.
Claim 10 recites The method of claim 8, further comprising at least one of: the data set taking a form of an output of transformation of a complex object; the complex object being at least one of: an image, video data, text and a time series data of sensor measurements; the transformation of the complex object being at least one of: a feature extraction operation, an embedding operation and an internal layer of an autoencoder; storing at least one of: the complex object and the output of the transformation in the memory along with the constructed plurality of GMM distributions; and storing the constructed plurality of GMM distributions without storing the at least one of: the complex object and the output of the transformation. The data set being obtained via a transformation (such as feature extraction, embedding, or encoding) of a complex object (such as an image, video, text, or time series) amounts to adding insignificant extra-solution activity (necessary data gathering) to the judicial exception – see MPEP2106.05(g). Storing the constructed GMM distributions and possibly storing the complex object and output of the transformation also amounts to adding insignificant extra-solution activity (post-solution data storage) to the judicial exception – see MPEP2106.05(g). Therefore, the claim merges with the abstract idea recited in claim 8, and does not recite additional elements that are sufficient to amount to significantly more than the abstract idea.
Claim 11 recites The method of claim 1, further comprising the processor utilizing at least one of: the data set and the GMM distribution in at least one of: learning a Machine Learning (ML) model, preparing an intelligence report and data clustering. Utilizing the data set and GMM distribution for machine learning, intelligence reporting, or data clustering amounts to generally linking the use of a judicial exception to a particular technological environment or field of use – see MPEP 2106.05(h). Therefore, the claim does not recite additional elements that are sufficient to amount to significantly more than the abstract idea.
Claim 12 recites The method of claim 8, further comprising at least one of: the processor utilizing at least one of: the data set and the constructed plurality of GMM distributions in at least one of: learning an ML model, preparing a business intelligence report and data clustering; generating at least one data sample based on selecting a subset of the plurality of smaller sets in accordance with finding the another GMM distribution that is representative of the constructed plurality of GMM distributions followed by generating an artificial at least one of: at least one numeric value and at least one vector of numeric values based on the constructed plurality of GMM distributions; choosing the another GMM distribution as representative of the constructed plurality of GMM distributions based on analysis of distances between the GMM distributions of the constructed plurality of GMM distributions; and the analysis of the distances between the GMM distributions of the constructed plurality of GMM distributions being based on at least one of: a Wasserstein distance and a Kullback-Leibler divergence. Utilizing the data set and GMM distribution for machine learning, intelligence reporting, or data clustering amounts to generally linking the use of a judicial exception to a particular technological environment or field of use – see MPEP 2106.05(h). Selecting a subset of the plurality of smaller data sets and generating an artificial sample based on the constructed GMM distributions are evaluations/judgements which can practically be performed in the human mind or with the aid of pen and paper (i.e. mental process). Choosing the representative GMM distribution of the constructed GMM distributions based on analysis of distances between the constructed GMM distributions, where the analysis is based on Wasserstein distance or Kullback-Leibler divergence, is an evaluation/judgement which can practically be performed in the human mind or with the aid of pen and paper (i.e. mental process). Therefore, the claim merges with the abstract idea recited in claim 8, and does not recite additional elements that are sufficient to amount to significantly more than the abstract idea.
Claim 13 recites The method of claim 10, further comprising finding a subset of complex objects in data associated with the processor communicatively coupled to the memory that is most similar to the complex object based on execution of the EM algorithm in accordance with the transformation of the complex object into representational numeric values thereof and probabilistic similarity based analysis of the representational numeric values against the constructed plurality of GMM distributions to find the plurality of smaller sets. Finding a subset of similar complex objects using a similarity based analysis of the numeric values against the constructed GMM distributions is an evaluation/judgement which can practically be performed in the human mind or with the aid of pen and paper (i.e. mental process). Therefore, the claim merges with the abstract idea recited in claim 10, and does not recite additional elements that are sufficient to amount to significantly more than the abstract idea.
Claim 14 recites The method of claim 8, further comprising at least one of: determining that pairs of data columns belonging to different data tables of the data set have corresponding GMM representations thereof in the constructed plurality of GMM distributions closest to one another; and measuring closeness of the corresponding GMM representations based on at least one of: a Wasserstein distance and a Kullback-Leibler divergence. Determining that pairs of data columns from different data tables have corresponding GMM representations closest to one another is an evaluation/judgement that can practically be performed in the human mind or with the aid of pen and paper (i.e. mental process), and measuring the closeness based on Wasserstein distance or Kullback-Leibler divergence is a mathematical concept. Therefore, the claim merges with the abstract idea recited in claim 8, and does not recite additional elements that are sufficient to amount to significantly more than the abstract idea.
Claims 15-17 are system claims containing substantially the same elements as method claims 1-2 and 6, respectively, and are rejected on the same grounds under 35 U.S.C. 101 as claims 1-2 and 6, respectively, mutatis mutandis.
The additional components of A data processing device to generate a GMM distribution that approximates a data set, comprising: a memory; and a processor communicatively coupled to the memory, the processor executing instructions stored in the memory are interpreted as a general-purpose computer and mere instructions to apply the judicial exception on the computer. Therefore, the claims do not recite additional elements that are sufficient to amount to significantly more than the abstract idea.
Claims 18-20 are system claims containing substantially the same elements as method claims 1-2, and are rejected on the same grounds under 35 U.S.C. 101 as claims 1-2, respectively, mutatis mutandis.
The additional components of A data processing device to generate a GMM distribution that approximates a data set, comprising: a memory; and a processor communicatively coupled to the memory, the processor executing instructions stored in the memory are interpreted as a general-purpose computer and mere instructions to apply the judicial exception on the computer.
The additional components of utilize the generated GMM distribution one of: along with and instead of at least one of: the data set and the modified data set for computation using a Machine Learning (ML) algorithm also executing on the processor amount to generally linking the use of a judicial exception to a particular technological environment or field of use. Therefore, the claims do not recite additional elements that are sufficient to amount to significantly more than the abstract idea.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1, 3, 7, 11, and 15 are rejected under 35 U.S.C. 102(a)(2) as being anticipated by
Bradley et al. (hereinafter Bradley) “Scaling EM (Expectation-Maximization) Clustering to Large Databases” (published Nov. 1998).
Regarding Claim 1,
Bradley teaches A method of generating a Gaussian Mixture Model (GMM) distribution that approximates a data set (Pg. 2-3, section 1: “An efficient representation of the probability density function is the mixture model, which asserts that the data is a combination of k individual component densities, corresponding to the k clusters… We specifically address the problem of computing mixture models over large databases.” Pg. 9, section 2: “We elaborate on the SEM algorithm by first by describing the sufficient statistics for the Gaussian mixture model.” A Gaussian Mixture Model distribution is generated to represent (i.e. approximate) the dataset.)
using a processor communicatively coupled to a memory, (Examiner notes that this limitation is interpreted as implementation of the disclosed method in a generic computing environment. Pg. 1, Abstract: “The approach operates within the confines of a limited main memory buffer and requires at most a single database scan. Data resolution is preserved to the extent possible based upon the size of the main memory buffer and the fit of the current clustering model to the data. We extend the method to efficiently update multiple models simultaneously. Computational tests indicate that this scalable scheme outperforms sampling-based approaches…” The use of a “main memory buffer”, “database scan”, and “computational tests” necessitate implementation on a computer.)
comprising:
iteratively deriving parameters of constituent Gaussian distributions of the GMM distribution based on executing an Expectation-Maximization (EM) algorithm on the processor communicatively coupled to the memory, the EM algorithm incorporating the data set as an input thereinto; (Pg. 2, section 1: “The Expectation-Maximization (EM) algorithm [DLR77, CS96] is an effective and popular technique for estimating the mixture model parameters or fitting the model to the database. The EM algorithm iteratively refines an initial cluster model to better fit the data” Pg. 7, section 2: “The allocated main memory buffer is initially filled with a random sample from the database. The given initial mixture model is updated over the contents of the buffer via the standard EM algorithm.” The parameters of the mixture model are iteratively refined (i.e. derived) by executing the Expectation-Maximization algorithm using sample data from the database (i.e. the data set) as input.)
modifying the data set by replacing, for each constituent Gaussian distribution of the GMM distribution, at least one of: numeric values and vectors of the numeric values of the data set that differ in magnitude from a center of the each constituent Gaussian distribution by less than a threshold with a mean value of the data set, with a weight of the mean value being indicative of a cardinality thereof within the modified data set; (Pg. 7-8, section 2: “Then each data point in the buffer is classified, depending upon the degree to which the datapoint is summarized by the current mixture model, as belonging to one of three sets… The retained set
R
… The discard set
B
… The compressed set
C
…” Pg. 9, section 2.1: “A data record entering either of the sets
B
or
C
is summarized with other elements of the set via sufficient statistics, which are then used to update the Gaussian mixture model. After the data records are summarized, their sufficient statistics remain in the main memory buffer and the individual data records used to generate the statistics are purged. We next describe the specific sufficient statistics used to update the Gaussian mixture model. Let
S
=
{
x
1
,
x
2
,
…
,
x
N
}
⊂
D
, be a subset of data to be summarized by the sufficient statistics triple
(
θ
,
Γ
,
N
)
:
θ
=
∑
i
=
1
N
x
i
… Computing the mean… of the set S,
µ
S
… from triple
(
θ
,
Γ
,
N
)
, is as follows:
µ
S
=
1
N
θ
…” Pg. 12, section 2.3.1: “If the current mixture model is accurate, areas near the component Gaussian means
µ
h
will have the greatest density. The full mixture model can be sufficiently updated by locally modeling these regions with the sufficient statistics triples
(
θ
,
Γ
,
N
)
. The basic intuition is to discard items that are not likely to change degree of membership in clusters… Data modeled well under a cluster (probability close to 1.0 w.r.t. this cluster) is unlikely to change membership or influence other model components. We identify these regions by thresholding the Mahalanobis radius [DH73],
M
D
i
s
t
x
,
µ
,
∑
=
(
x
-
µ
)
T
∑
-
1
(
x
-
µ
)
near the
k
component Gaussian means and summarizing data points within this radius.” Pg. 10, section 2.2: “Updates over sufficient statistics treat the set represented by
(
θ
,
Γ
,
N
)
as a single data record weighted by
N
.” Data points which fall within a threshold radius of the mean of a Gaussian component (i.e. differ in magnitude from the center of a Gaussian component by less than a threshold) are purged from memory and replaced by sufficient statistics which represent the mean of the data, weighted by the number of data points
N
(i.e. cardinality).)
continuing the iterative derivation of the parameters of the constituent Gaussian distributions of the GMM distribution based on the execution of the EM algorithm that now incorporates the modified data set thereinto along with the weight of the mean value; and (Pg. 9-10, section 2.1-2.2: “A data record entering either of the sets
B
or
C
is summarized with other elements of the set via sufficient statistics, which are then used to update the Gaussian mixture model… Step 2 of Algorithm 2 requires updating the mixture model parameters over the contents of the buffer:
R
j
+
1
∪
C
j
∪
B
j
consisting of singleton data records
x
∈
R
j
+
1
and sets of sufficient statistics
θ
,
Γ
,
N
∈
C
j
∪
B
j
. The Extended EM (ExEM) Algorithm performs this operation. ExEM updates the model parameters exactly as the standard EM algorithm (Algorithm 1) over singleton data records. Updates over sufficient statistics treat the set represented by
(
θ
,
Γ
,
N
)
as a single data record weighted by
N
.” The parameters of the Gaussian mixture model are updated based on the EM algorithm operating on the contents of the buffer, which includes the weighted sufficient statistics that replaced the data points in sets
B
and
C
(i.e. the modified data set).)
reducing a data footprint of the data set through the GMM distribution based on the continued iterative derivation of the parameters of the constituent Gaussian distributions thereof. (Pg. 8, section 2: “Note that admitting data into the set
B
and subsequently only storing the data’s sufficient statistics frees space in the buffer allowing the method to load more data… representing sets of records in
C
by their sufficient statistics frees main memory allowing the method to load more data.” As data is iteratively replaced by sufficient statistics during the derivation of GMM parameters, space is freed in the memory buffer (i.e. the data footprint is reduced).)
Regarding Claim 3, Bradley teaches The method of claim 1, as shown above.
Bradley also teaches wherein in accordance with the at least one of: the numeric values and the vectors of the numeric values that differ in magnitude from the center of the each constituent Gaussian distribution by less than the threshold being the same as one another, the method further comprises:
adding a standard deviation of the same at least one of: the numeric values and the vectors of the numeric values as equal to 0 to the GMM distribution; and (Pg. 8, section 2: “The discard set
B
consists of records which can be safely discarded and modeled effectively by the sufficient statistics associated with one of the clusters. Elements of the set
B
are typically records whose cluster membership is nearly ‘certain’. Since they will not change cluster membership, it suffices to summarize them and update the corresponding cluster only through sufficient statistics. Note that admitting data into the set B and subsequently only storing the data’s sufficient statistics frees space in the buffer allowing the method to load more data.” Pg. 9, section 2.1: “Let
S
=
{
x
1
,
x
2
,
…
,
x
N
}
⊂
D
, be a subset of data to be summarized by the sufficient statistics triple
(
θ
,
Γ
,
N
)
…
Γ
=
∑
i
=
1
N
(
x
i
)
⋅
(
x
i
)
T
is an
n
×
n
matrix. Computing the… covariance of the set
S
… from triple
(
θ
,
Γ
,
N
)
, is as follows…
∑
S
=
1
N
(
Γ
-
1
N
θ
⋅
θ
T
)
.” Data records in discard set
B
(i.e. that differ in magnitude from the center of a Gaussian component by less than the threshold, as explained in regard to claim 1) are summarized by sufficient statistics triple
(
θ
,
Γ
,
N
)
, from which variance and thus standard deviation can be derived. In the case that the data records to be summarized are the same as one another, the standard deviation will be equal to 0. These sufficient statistics are then used to update the GMM clusters (i.e. the standard deviation equal to 0 is added to the GMM distribution).)
performing a subsequent phase of the continued iterative derivation of the parameters of the constituent Gaussian distributions without the same at least one of: the numeric values and the vectors of the numeric values. (See the portion of section 2 cited above. Data records in discard set
B
(i.e. that differ in magnitude from the center of a Gaussian component by less than the threshold), including those that are the same as one another, are removed from the data set for subsequent iterations.)
Regarding Claim 7, Bradley teaches The method of claim 1, as shown above.
Bradley also teaches comprising at least one of:
the numeric values of the data set being taken by consecutive rows of a data column of a data table that one of: constitutes and at least is part of the data set; and
the vectors of the numeric values of the data set being taken by consecutive rows of one of: the data table and another data table for at least a subset of data columns thereof. (Pg. 1, Abstract: “Practical statistical clustering algorithms… typically require many database scans to converge, and within each scan they require the access to every record in the data table. For large databases, the scans become prohibitively expensive.” Pg. 3-4, section 1: “The algorithm has the ability to operate with a forward-only cursor over a view of the database. The single forward only data scan addresses the fact that the individual data records provided to the clustering algorithm may be the result of an expensive join query over a potentially distributed data warehouse.” Pg. 15, section 4.1.1: “Scalable EM (Algorithm 2) was applied to data sets where the numbers of rows (points) were varied from 10,000 to 1 million.” The data points (i.e. vectors of numeric values of the data set) are taken by a single forward-only scan (i.e. consecutive rows) of the data table.)
Regarding Claim 11, Bradley teaches The method of claim 1, as shown above.
Bradley also teaches further comprising the processor utilizing at least one of: the data set and the GMM distribution in at least one of: learning a Machine Learning (ML) model, preparing an intelligence report and data clustering. (Pg. 1-2, section 1: “Data clustering is important in many fields, including data mining [FPSU96], statistical data analysis [KR89,BR93], compression [ZRL97], and vector quantization [DH73]. Applications include data analysis and modeling [FDW97,FHS96], image segmentation, marketing, fraud detection, predictive modeling, data summarization, general data reporting tasks, data cleaning and exploratory data analysis [B*96]. Clustering is a crucial data mining step and performing this task over large databases is essential. A general view of clustering places it in the framework of density estimation [S86, S92, A73]. Clustering can be viewed as identifying the dense regions of the data source. An efficient representation of the probability density function is the mixture model…” The data set and mixture model distribution are utilized for data clustering, as well as data reporting (i.e. preparing an intelligence report).)
Claim 15 is a system claim containing substantially the same elements as method claim 1. Bradley teaches the elements of claim 1, as shown above.
Bradley also teaches A data processing device to generate a GMM distribution that approximates a data set, comprising: a memory; and a processor communicatively coupled to the memory, the processor executing instructions stored in the memory (Examiner notes that this limitation is interpreted as implementation of the disclosed method in a generic computing environment. Pg. 1, Abstract: “The approach operates within the confines of a limited main memory buffer and requires at most a single database scan. Data resolution is preserved to the extent possible based upon the size of the main memory buffer and the fit of the current clustering model to the data. We extend the method to efficiently update multiple models simultaneously. Computational tests indicate that this scalable scheme outperforms sampling-based approaches…” The use of a “main memory buffer”, “database scan”, and “computational tests” necessitate implementation on a computer.)
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 2 and 16 are rejected under 35 U.S.C. 103 as being unpatentable over Bradley in view of
Scrucca, “Entropy-Based Anomaly Detection for Gaussian Mixture Modeling” (published 04/03/2023) and
Vaver, U.S. Patent US-8676799-B1 (published 08/25/2010).
Regarding Claim 2, Bradley teaches The method of claim 1, as shown above.
Bradley does not appear to explicitly disclose the remaining features of claim 2.
However, Scrucca teaches comprising determining the threshold in accordance with at least one of:
estimating an entropy of a probability distribution characterizing an extent to which the at least one of: the numeric values and the vectors of the numeric values are generated from the each constituent Gaussian distribution; and (Pg. 4-5, section 2.4: “Entropy is a measure of average uncertainty or information content in a random variable that plays a central role in information theory… Assuming that the distribution of the multivariate random variable
X
can be expressed as a finite mixture of Gaussian components… an estimate of the entropy is obtained by summing over the contribution
h
i
=
-
1
n
log
f
^
(
x
i
;
θ
^
)
from each data point. This can also be interpreted as a measure of the degree of anomaly or outlierness of each observation.” Pg. 7, section 3.1: “Figure 2a shows a plot of the entropy contribution values hi against the probability points… The dashed line refers to the entropy contribution from the uniform noise, so observations above that line can be included in the initial set of anomalies.” The determination of whether a point is anomalous/uncertain (i.e. the threshold) is based on the entropy of the probability distribution represented by the mixture of Gaussian components.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Bradley and Scrucca. Bradley teaches generating a Gaussian Mixture Model using a modified version of the Expectation-Maximization (EM) algorithm in which less important data points are iteratively replaced by summarizing statistics to reduce the memory footprint of the data set. Scrucca teaches measuring the entropy of a Gaussian Mixture Model to identify anomalous or uncertain data points. One of ordinary skill would have motivation to combine Bradley and Scrucca because Bradley seeks to “discard items that are not likely to change degree of membership in clusters” (Bradley, pg. 12, section 2.3.1), and Scrucca teaches a way to determine such items by measuring their certainty or uncertainty via entropy: “an estimate of the entropy… can also be interpreted as a measure of the degree of anomaly or outlierness of each observation” (Scrucca, Pg. 4-5, section 2.4).
Bradley and Scrucca do not appear to explicitly disclose the remaining features of claim 2.
However, Vaver teaches the at least one of: the numeric values and the vectors of the numeric values being different in magnitude from the center of the each constituent Gaussian distribution by less than a numeric distance and from another center of at least one other constituent Gaussian distribution of the GMM distribution by more than another numeric distance. (Col. 1, lines 47-57: “identifying whether each geographic entity in the plurality of geographic entities is an ambiguously classified entity or a definitively classified entity, wherein an ambiguously classified entity is a geographic entity that is clustered into one of the clusters in the first set of clusters and is within a threshold distance of another geographic entity in one of the other clusters in the first set of clusters, and wherein a definitively classified entity is a geographic entity that is clustered into one of the clusters in the first set of clusters and is more than a threshold distance from each geographic entity in each of the other clusters in the first set of clusters…” The determination of whether a data point is ambiguous or definitive (i.e. the threshold) is based on the data point being closest to one cluster (i.e. differing in magnitude from one Gaussian component by less than a numeric distance) and more than a threshold distance from each of the other clusters (i.e. differing in magnitude from another Gaussian component by more than another numeric distance).)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Bradley, Scrucca, and Vaver. Bradley teaches generating a Gaussian Mixture Model using a modified version of the Expectation-Maximization (EM) algorithm in which less important data points are iteratively replaced by summarizing statistics to reduce the memory footprint of the data set. Scrucca teaches measuring the entropy of a Gaussian Mixture Model to identify anomalous or uncertain data points. Vaver teaches determining whether an entity is ambiguously or definitively classified based on its distance from each cluster. One of ordinary skill would have motivation to combine Bradley, Scrucca, and Vaver because Bradley seeks to “discard items that are not likely to change degree of membership in clusters” (Bradley, pg. 12, section 2.3.1), and Vaver teaches a way to determine such items by measuring their definitiveness: “identifying whether each geographic entity in the plurality of geographic entities is an ambiguously classified entity or a definitively classified entity” (Vaver, col. 1, lines 47-57).
Claim 16 is a system claim containing substantially the same elements as method claim 2. Bradley, Scrucca, and Vaver teach the elements of claim 2, as shown above.
Claim 4 is rejected under 35 U.S.C. 103 as being unpatentable over Bradley in view of
Makantasis et al. (hereinafter Makantasis), “Data-Driven Background Subtraction Algorithm for in-Camera Acceleration in Thermal Imagery” (published 10/15/2017) and
Kristan et al. (hereinafter Kristan), “Multivariate online kernel density estimation with Gaussian kernels” (published 04/08/2011).
Regarding Claim 4, Bradley teaches The method of claim 1, as shown above.
Bradley does not appear to explicitly disclose the remaining features of claim 4.
However, Makantasis teaches further comprising:
identifying at least one outlier element in the data set in accordance with the execution of the EM algorithm; (Pg. 3, section I.B: “Our method exploits GMMs with unknown number of components, which are dynamically estimated directly from the data… the Expectation Maximization (EM) algorithm is adopted to estimate model parameters.” Pg. 6, section V.B: “When the new observed sample cannot be modeled by any existing component, i.e. the value of the new sample will not be close to what the model has already learnt [see Eq.(25)], a new component is created with mixing coefficient
ω
-
n
e
w
, mean value
µ
n
e
w
and standard deviation
σ
n
e
w
, defined as…
µ
n
e
w
=
x
n
e
w
” During the EM algorithm, an observed sample which cannot be modeled by any existing component (i.e. an outlier element in the data set) is identified.)
adding the identified at least one outlier element to the GMM distribution [with a standard deviation thereof equal to 0]; and (See the portion of section V.B cited above. A new component is added to the GMM with mean value equal to the identified sample (i.e. the outlier element is added to the GMM distribution).)
Bradley teaches continuing an iterative process of the EM algorithm after removing the at least one outlier element from the modified data set. (Pg. 8, section 2: “The discard set
B
consists of records which can be safely discarded and modeled effectively by the sufficient statistics associated with one of the clusters. Elements of the set
B
are typically records whose cluster membership is nearly ‘certain’. Since they will not change cluster membership, it suffices to summarize them and update the corresponding cluster only through sufficient statistics. Note that admitting data into the set B and subsequently only storing the data’s sufficient statistics frees space in the buffer allowing the method to load more data.” Bradley teaches discarding data records whose cluster membership is certain. One of ordinary skill in the art will recognize that the cluster membership of an outlier identified by Makantasis above is certain since a component cluster is added to the GMM specifically for that outlier data record. Thus, in combination, Bradley and Makantasis teach removing the outlier from the data set.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Bradley and Makantasis. Bradley teaches generating a Gaussian Mixture Model using a modified version of the Expectation-Maximization (EM) algorithm in which less important data points are iteratively replaced by summarizing statistics to reduce the memory footprint of the data set. Makantasis teaches a dynamic Gaussian Mixture Model which adds a new GMM component for data records which are not sufficiently close to existing components. One of ordinary skill would have motivation to combine Bradley and Makantasis in order to “allow dynamic model adaptation” (Makantasis, pg. 1, section I.B) and enable the GMM to handle outlier data records which cannot be adequately represented by the existing components.
Bradley and Makantasis do not appear to explicitly disclose adding a GMM component with a standard deviation thereof equal to 0.
However, Kristan teaches adding a GMM component with a standard deviation thereof equal to 0. (Pg. 2631, section 1.1: “The second key idea is that we treat each new observation as a distribution in the form of a Dirac-delta function and we model the sample distribution by the mixture of Gaussian and Dirac-delta functions.” Pg. 2631, section 2: “Each separate data-point can be presented in a distribution as a single Dirac-delta function, with its probability mass concentrated at that data-point. Noting that the Dirac-delta can be generally written as a Gaussian with zero covariance…” For a new separate data point (i.e. the outlier), a Dirac-delta function is added to the GMM, which is a Gaussian component with zero covariance (i.e. standard deviation equal to 0).)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Bradley, Makantasis, and Kristan. Bradley teaches generating a Gaussian Mixture Model using a modified version of the Expectation-Maximization (EM) algorithm in which less important data points are iteratively replaced by summarizing statistics to reduce the memory footprint of the data set. Makantasis teaches a dynamic Gaussian Mixture Model which adds a new GMM component for data records which are not sufficiently close to existing components. Kristan teaches adding a new data point to a GMM as a gaussian component with zero covariance. One of ordinary skill would have motivation to combine Bradley, Makantasis, and Kristan because the new component corresponding to the outlier identified by Makantasis “models only one sample” (Makantasis, pg. 7, section V.B), and thus can be most accurately represented by “a single Dirac-delta function, with its probability mass concentrated at that data-point” (Kristan, pg. 2631, section 2).
Claim 5 is rejected under 35 U.S.C. 103 as being unpatentable over Bradley in view of
Figueiredo et al. (hereinafter Figueiredo), “Unsupervised learning of finite mixture models” (published 03/31/2002).
Regarding Claim 5, Bradley teaches The method of claim 1, as shown above.
Bradley does not appear to explicitly disclose the remaining features of claim 5.
However, Figueiredo teaches further comprising:
specifying a first number of the constituent Gaussian distributions of the GMM distribution as an input parameter to the EM algorithm; and (Section 5.1.2: “By starting with
k
n
z
=
k
, where
k
is much larger than the true/optimal number of mixture components, this algorithm is robust with respect to initialization.” The number of mixture components
k
n
z
(i.e. number of constituent Gaussian distributions of the GMM distribution) is initialized to a first number
k
.)
the EM algorithm working with a second number of the constituent Gaussian distributions of the GMM distribution that is less than the first number of the constituent Gaussian distributions in accordance with the GMM distribution with the first number of the constituent Gaussian distributions inadequately approximating the data set. (Section 5.1.1: “An important feature of the M-step defined by (17) is that it performs component annihilation, thus being an explicit rule for moving from the current value of
k
n
z
to a smaller one. Notice that this prevents the algorithm from approaching the boundary of the parameter space: When one of the components becomes ‘too weak,’ meaning that it is not supported by the data, it is simply annihilated.” A smaller number of mixture components
k
n
z
(i.e. number of constituent Gaussian distributions of the GMM distribution) is used by the EM algorithm when one of the components is not supported by the data (i.e. in accordance with the first number of components inadequately approximating the data set).)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Bradley and Figueiredo. Bradley teaches generating a Gaussian Mixture Model using a modified version of the Expectation-Maximization (EM) algorithm in which less important data points are iteratively replaced by summarizing statistics to reduce the memory footprint of the data set. Figueiredo teaches generating a Gaussian Mixture Model while dynamically reducing the number of components. One of ordinary skill would have motivation to combine Bradley and Figueiredo because “EM may converge to the boundary of the parameter space. For example, when fitting a Gaussian mixture with unconstrained covariance matrices, one of the
α
m
's may approach zero and the corresponding covariance matrix may become arbitrarily close to singular. When the number of components assumed is larger than the optimal/true one, this tends to happen frequently, thus being a serious problem for methods that require mixture estimates for various values of k” (Figueiredo, section 3.2.2). By dynamic “component annihilation… [o]ne of the drawbacks of standard EM for mixtures is thus avoided” (Figueiredo, 5.1.1).
Claims 6, 17-18, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Bradley in view of
Li et al. (hereinafter Li), “Mixture distribution modeling for scalable graph-based semi-supervised learning” (published 07/20/2020).
Regarding Claim 6, Bradley teaches The method of claim 1, as shown above.
Bradley does not appear to explicitly disclose the remaining features of claim 6.
However, Li teaches further comprising one of: replacing at least one of: the data set and the modified data set with the GMM distribution for an operation to be performed using the processor communicatively coupled to the memory; and adding the GMM distribution as metadata to the memory for availability thereof together with the at least one of: the data set and the modified data set for the operation to be performed using the processor communicatively coupled to the memory. (Pg. 1-2, section 1: “In order to overcome the limitations of existing scalable graph-based SSL given above, a novel scalable graph-based SSL method is proposed in this paper. We use mixture model components as anchors instead of single data points, to give a better estimation of data distributions with a smaller number of anchors than existing anchor-based methods… We choose the Gaussian Mixture Model (GMM) to simplify the computation of similarities and give a satisfying estimation of data distribution.” Single data points (i.e. the data set) are replaced by mixture model components (i.e. the GMM distribution) for a downstream graph-based semi-supervised learning task (i.e. an operation to be performed).)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Bradley and Li. Bradley teaches generating a Gaussian Mixture Model using a modified version of the Expectation-Maximization (EM) algorithm in which less important data points are iteratively replaced by summarizing statistics to reduce the memory footprint of the data set. Li teaches using a Gaussian Mixture Model to reduce the size of a large dataset in order to efficiently perform downstream machine learning tasks such as graph-based semi-supervised learning. One of ordinary skill would have motivation to combine Bradley and Li because “[t]he smaller anchor set and less complex graph construction help to achieve better efficiency than existing works, while the good estimation of data distribution by the mixture model guarantees the effectiveness of our method” (Li, pg. 2, section 1).
Claim 17 is a system claim containing substantially the same elements as method claim 6. Bradley and Li teach the elements of claim 6, as shown above.
Claim 18 is a system claim containing substantially the same elements as method claim 1. Bradley teaches the elements of claim 1, as shown above.
Bradley also teaches A data processing device to generate a GMM distribution that approximates a data set, comprising: a memory; and a processor communicatively coupled to the memory, the processor executing instructions stored in the memory (Examiner notes that this limitation is interpreted as implementation of the disclosed method in a generic computing environment. Pg. 1, Abstract: “The approach operates within the confines of a limited main memory buffer and requires at most a single database scan. Data resolution is preserved to the extent possible based upon the size of the main memory buffer and the fit of the current clustering model to the data. We extend the method to efficiently update multiple models simultaneously. Computational tests indicate that this scalable scheme outperforms sampling-based approaches…” The use of a “main memory buffer”, “database scan”, and “computational tests” necessitate implementation on a computer.)
Bradley does not appear to explicitly disclose the final limitation of claim 18.
However, Li teaches utilize the generated GMM distribution one of: along with and instead of at least one of: the data set and the modified data set for computation using a Machine Learning (ML) algorithm also executing on the processor. (Pg. 1-2, section 1: “In order to overcome the limitations of existing scalable graph-based SSL given above, a novel scalable graph-based SSL method is proposed in this paper. We use mixture model components as anchors instead of single data points, to give a better estimation of data distributions with a smaller number of anchors than existing anchor-based methods… We choose the Gaussian Mixture Model (GMM) to simplify the computation of similarities and give a satisfying estimation of data distribution.” Single data points (i.e. the data set) are replaced by mixture model components (i.e. the GMM distribution) for a downstream graph-based semi-supervised learning task (i.e. computation using a machine learning algorithm).)
Regarding Claim 20, Bradley and Li teach The data processing device of claim 18, as shown above.
Bradley also teaches wherein the processor executes instructions to reduce a data footprint of the data set through the GMM distribution based on the continued iterative derivation of the parameters of the constituent Gaussian distributions thereof. (Pg. 8, section 2: “Note that admitting data into the set
B
and subsequently only storing the data’s sufficient statistics frees space in the buffer allowing the method to load more data… representing sets of records in
C
by their sufficient statistics frees main memory allowing the method to load more data.” As data is iteratively replaced by sufficient statistics during the derivation of GMM parameters, space is freed in the memory buffer (i.e. the data footprint is reduced).)
Claims 8-10 and 12 are rejected under 35 U.S.C. 103 as being unpatentable over Bradley in view of
Zhang et al. (hereinafter Zhang), “Distributed Learning of Finite Gaussian Mixtures” (published 11/10/2021).
Regarding Claim 8, Bradley teaches The method of claim 1, as shown above.
Bradley does not appear to explicitly disclose the remaining features of claim 8.
However, Zhang teaches further comprising at least one of:
the data set being a smaller set of a larger data set; (Pg. 7, section 3.1.1: “Suppose we have a random sample
X
=
{
x
1
,
x
2
,
.
.
.
,
x
N
}
, and it is either randomly partitioned into
M
subsets stored on
M
local machines or can be treated as such. Let
X
m
of size
M
m
be the data set on the
m
th local machine.” Each subset
X
m
is a smaller set of larger data set
X
.)
constructing a plurality of GMM distributions comprising the generated GMM distribution for a corresponding plurality of smaller sets of the larger data set comprising the data set; and (Pg. 7, section 3.1.1: “The local inference learns the mixture via the pMLE
G
^
m
=
… under the GMM. We call
G
^
m
a local estimate.” Each local machine learns a local estimate GMM distribution
G
^
m
for its corresponding smaller set
X
m
of the larger data set
X
.)
merging the GMM distributions of the constructed plurality of GMM distributions together to form another GMM distribution. (Pg. 9, section 4: “In the context of finite mixtures, let
G
^
1
,
…
,
G
^
m
be the local estimators. A natural aggregated estimator of the mixing distribution is the weighted average
G
-
=
∑
m
=
1
M
λ
m
G
^
m
… The mixing distribution
G
-
is likely close to the true mixing distribution
G
*
, except for the incorrect number of support points. This problem can be solved by approximating
G
-
by some
G
∈
G
K
. This suggests another aggregation approach. Let
ρ
(
·
,
·
)
be a divergence in the space of mixing distributions. We can aggregate the local estimates via the reduction estimator, given by
G
-
R
=
a
r
g
i
n
f
G
∈
G
K
p
(
G
-
,
G
)
.” The local estimates
G
^
1
,
…
,
G
^
m
(i.e. GMM distributions) are aggregated (i.e. merged) to form mixing distribution
G
-
(i.e. another GMM distribution).)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Bradley and Zhang. Bradley teaches generating a Gaussian Mixture Model using a modified version of the Expectation-Maximization (EM) algorithm in which less important data points are iteratively replaced by summarizing statistics to reduce the memory footprint of the data set. Zhang teaches a split-and-conquer approach for distributed learning of Gaussian Mixture Models. One of ordinary skill would have motivation to combine Bradley and Zhang because “In the era of big data, the sizes of the datasets for various applications may be so large that they cannot be stored on a single machine… Data analysis methods should therefore be designed so that they can work with subsets of the dataset, in parallel or sequentially… In this paper, we develop a novel split-and-conquer procedure for a finite Gaussian mixture” (Zhang, pg. 1-2, section 1). “Experiments based on simulated and real-world data show that the proposed split-and-conquer approach has comparable statistical performance with the global estimator based on the full dataset… It also has better statistical and computational performance than some existing methods” (Zhang, pg. 1, Abstract).
Regarding Claim 9, Bradley and Zhang teach The method of claim 8, as shown above.
Zhang also teaches further comprising at least one of:
merging the GMM distributions of the constructed plurality of GMM distributions together in accordance with modification of the EM algorithm to account for all parameters of the constructed plurality of GMM distributions; and (Pg. 12-13, section 5: “[T]he optimization reduces to searching for
K
subpopulations
Φ
γ
for
γ
∈
[
K
]
to make up
G
… An iterative algorithm quickly emerges following the well-known majorization–minimization (MM) idea (Hunter and Lange, 2004)… The subpopulations
Φ
γ
are separated in the majorization function (15). This allows us to update the subpopulation parameters, one
Φ
γ
at a time and possibly in parallel… The MM algorithm then iterates between the majorization step (15) and the maximization step (16) until some user-selected convergence criterion is met.” The aggregation (i.e. merging) to form mixing distribution
G
is executed by a majorization-maximization (MM) algorithm (i.e. a modified EM algorithm) which updates all subpopulation parameters (i.e. accounts for all parameters of the constructed GMM distributions).)
optimizing splitting of the larger data set into the plurality of smaller sets comprising the data set in accordance with maximizing approximation of the constructed plurality of GMM distributions to the corresponding plurality of smaller sets.
Regarding Claim 10, Bradley and Zhang teach The method of claim 8, as shown above.
Zhang also teaches further comprising at least one of:
the data set taking a form of an output of transformation of a complex object; (Pg. 22, section 7.3.2: “In this section, we demonstrate the use of the GMR [Gaussian mixture reduction] method on the famous NIST dataset for character recognition… It consists of around 4M images of hand written digits and characters… Following the common practice, we first trained a 5-layer convolutional neural network and reduced each image to a
d
=
50
feature vector of real values.” The data set provided to the Gaussian mixture reduction method consists of the output of a CNN (i.e. a transformation) operating on an image (i.e. a complex object).)
the complex object being at least one of: an image, video data, text and a time series data of sensor measurements; (See the portion of section 7.3.2 cited above. The complex object is an image.)
the transformation of the complex object being at least one of: a feature extraction operation, an embedding operation and an internal layer of an autoencoder; (See the portion of section 7.3.2 cited above. The transformation of the complex object is a CNN-based feature extraction/embedding operation.)
storing at least one of: the complex object and the output of the transformation in the memory along with the constructed plurality of GMM distributions; and (Pg. 7, section 3.1.1: “Suppose we have a random sample
X
=
{
x
1
,
x
2
,
.
.
.
,
x
N
}
, and it is either randomly partitioned into
M
subsets stored on
M
local machines or can be treated as such. Let
X
m
of size
M
m
be the data set on the
m
th local machine. The local inference learns the mixture via the pMLE
G
^
m
=
… under the GMM. We call
G
^
m
a local estimate.” The data sets (i.e. the output of the transformation) and local estimates (i.e. the constructed GMM distributions) are stored in local memory.)
Bradley also teaches storing the constructed plurality of GMM distributions without storing the at least one of: the complex object and the output of the transformation. (Pg. 9, section 2.1: “A data record entering either of the sets B or C is summarized with other elements of the set via sufficient statistics, which are then used to update the Gaussian mixture model. After the data records are summarized, their sufficient statistics remain in the main memory buffer and the individual data records used to generate the statistics are purged.” The GMM is updated and stored while the summarized data points (i.e. the output of the transformation) are purged (i.e. not stored).)
Regarding Claim 12, Bradley and Zhang teach The method of claim 8, as shown above.
Bradley also teaches further comprising at least one of:
the processor utilizing at least one of: the data set and the constructed plurality of GMM distributions in at least one of: learning an ML model, preparing a business intelligence report and data clustering; (Pg. 1-2, section 1: “Data clustering is important in many fields, including data mining [FPSU96], statistical data analysis [KR89,BR93], compression [ZRL97], and vector quantization [DH73]. Applications include data analysis and modeling [FDW97,FHS96], image segmentation, marketing, fraud detection, predictive modeling, data summarization, general data reporting tasks, data cleaning and exploratory data analysis [B*96]. Clustering is a crucial data mining step and performing this task over large databases is essential. A general view of clustering places it in the framework of density estimation [S86, S92, A73]. Clustering can be viewed as identifying the dense regions of the data source. An efficient representation of the probability density function is the mixture model…” The data set and mixture model distribution are utilized for data clustering.)
generating at least one data sample based on selecting a subset of the plurality of smaller sets in accordance with finding the another GMM distribution that is representative of the constructed plurality of GMM distributions followed by generating an artificial at least one of: at least one numeric value and at least one vector of numeric values based on the constructed plurality of GMM distributions;
choosing the another GMM distribution as representative of the constructed plurality of GMM distributions based on analysis of distances between the GMM distributions of the constructed plurality of GMM distributions; and
the analysis of the distances between the GMM distributions of the constructed plurality of GMM distributions being based on at least one of: a Wasserstein distance and a Kullback-Leibler divergence.
Claims 13-14 are rejected under 35 U.S.C. 103 as being unpatentable over Bradley in view of Zhang, and further in view of
Przyborowski et al. (hereinafter Przyborowski), “Schema matching using Gaussian mixture models with Wasserstein distance” (published 03/31/2022).
Regarding Claim 13, Bradley and Zhang teach The method of claim 10, as shown above.
Bradley and Zhang do not appear to explicitly disclose the remaining features of claim 13.
However, Przyborowski teaches further comprising finding a subset of complex objects in data associated with the processor communicatively coupled to the memory that is most similar to the complex object based on execution of the EM algorithm in accordance with the transformation of the complex object into representational numeric values thereof and probabilistic similarity based analysis of the representational numeric values against the constructed plurality of GMM distributions to find the plurality of smaller sets. (Pg. 6, section 6.3: “We present the algorithm for classification problem using Gaussian mixture models and Wasserstein distance.” The algorithm is included for reference below. The objects in training dataset
Z
j
0
(i.e. a subset of complex objects) are found to be most similar to the objects in test dataset
U
i
(i.e. the complex object) based on fitting a Gaussian mixture model
q
i
to
U
i
(i.e. execution of the EM algorithm) and comparing the test GMM
q
i
against all generated training GMMs
p
j
(i.e. probabilistic similarity based analysis against the constructed GMM distributions).)
PNG
media_image1.png
144
637
media_image1.png
Greyscale
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Bradley, Zhang, and Przyborowski. Bradley teaches generating a Gaussian Mixture Model using a modified version of the Expectation-Maximization (EM) algorithm in which less important data points are iteratively replaced by summarizing statistics to reduce the memory footprint of the data set. Zhang teaches a split-and-conquer approach for distributed learning of Gaussian Mixture Models. Przyborowski teaches calculating the Wasserstein distance between Gaussian mixture models. One of ordinary skill would have motivation to combine Bradley, Zhang, and Przyborowski because, as is illustrated in Przyborowski’s algorithm 2 (pg. 6), measuring the Wasserstein distance between GMMs allows GMMs to be used for classifying data and identifying similarity between data.
Regarding Claim 14, Bradley and Zhang teach The method of claim 8, as shown above.
Bradley and Zhang do not appear to explicitly disclose the remaining features of claim 14.
However, Przyborowski teaches further comprising at least one of:
determining that pairs of data columns belonging to different data tables of the data set have corresponding GMM representations thereof in the constructed plurality of GMM distributions closest to one another; and (Pg. 6, section 6.3: “We present the algorithm for classification problem using Gaussian mixture models and Wasserstein distance.” The algorithm is included for reference below. The classification of
U
i
is based on determining that the GMM representations
p
j
0
and
q
i
of data sets
Z
j
0
and
U
i
(i.e. pairs of data columns belonging to different data tables) are closest to one another by minimizing the Wasserstein distance.)
PNG
media_image1.png
144
637
media_image1.png
Greyscale
measuring closeness of the corresponding GMM representations based on at least one of: a Wasserstein distance and a Kullback-Leibler divergence. (See the portions of section 6.3 and algorithm 2 cited above. The closeness of GMM representations
p
j
and
q
i
is measured based on a Wasserstein distance.)
Claim 19 is rejected under 35 U.S.C. 103 as being unpatentable over Bradley in view of Li, and further in view of Scrucca and Vaver.
Claim 19 is a system claim containing substantially the same elements as method claim 2. Bradley, Scrucca and Vaver teach the elements of claim 2, and Bradley and Li teach the elements of parent claim 18, as shown above.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to BENJAMIN M ROHD whose telephone number is (571)272-6445. The examiner can normally be reached Mon-Thurs 8:00-6:00 EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Viker Lamardo can be reached at (571) 270-5871. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/B.M.R./Examiner, Art Unit 2147
/MICHAEL J HUNTLEY/Supervisory Patent Examiner, Art Unit 2129
Prosecution Insights