DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Amendments
This Office Action is in response to the amendment filed on March 11, 2026.
Claims 1, 8, and 15 have been amended.
No claims have been cancelled.
No new claims have been added.
The objections and rejections from the prior correspondence that are not restated herein are withdrawn.
Response to Arguments
Applicant's arguments filed on March 11, 2026 have been fully considered.
Applicant’s arguments regarding the 35 U.S.C. 103 rejections of the previous office action have been fully considered but are moot because the new reference of CARRIZOSA, in combination with ZHOU, BORGWARDT, BERTSIMAS, LIU, FROST, and GAMBELLA, teaches the newly added limitation, as shown in the rejections below.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-2, 4-9, 11-16, and 18-20 are rejected under 35 U.S.C. 103 as being unpatentable over ZHOU (“Hierarchical Maximum-Margin Clustering”) in view of BORGWARDT ("An Algorithm for the Separation-Preserving Transition of Clusterings"), BERTSIMAS (“Interpretable clustering: an optimization approach”), LIU ("Clustering Through Decision Tree Construction"), FROST (“ExKMC: Expanding Explainable k-Means Clustering”), CARRIZOSA (“Strongly agree or strongly disagree?” Rating features in Support Vector Machines”), and GAMBELLA (“Optimization Problems for Machine Learning: A Survey:”), hereafter ZHOU, BORGWARDT, BERTSIMAS, LIU, FROST, CARRIZOSA, and GAMBELLA respectively.
Regarding Claim 1:
ZHOU teaches:
A system for training a machine to perform unsupervised interpretable machine learning, comprising: at least one processor; and a memory device coupled with the at least one processor; (ZHOU [pg. 9, section 6. Conclusion] teaches: “We have presented a hierarchical clustering method for unsupervised construction of taxonomies.” Furthermore, clustering is an unsupervised learning method.” ZHOU [pg. 7, Table 1] teaches: "The runtime (in seconds) is measured on a machine with Intel Xeon 2.8GHz CPU and 16GB memory.")
receive a dataset having multiple features; (ZHOU [pg. 6, section 5. Experiments] teaches: "We use two datasets collected from ImageNet: VEHICLE contains 20 vehicle classes (e.g., cab and canoe) and 26,624 images (Hwang et al., 2011), and IMAGENET consists of 28,957 images spanning 20 non-animal, nonvehicle classes (e.g., lamp and drum) (Hwang et al., 2012). The raw image features are the provided bag-of-words histograms obtained by SIFT (Deng et al., 2010; 2009). We also project them down to 100 dimensions with PCA. The semantic hierarchies of VEHICLE and IMAGENET are given in Fig. 3 of (Hwang et al., 2011) and Fig. 2(e) of (Hwang et al., 2012), respectively.")
train to jointly cluster and interpret resulting clusters of the dataset by at least: clustering the dataset into clusters by using an unsupervised clustering algorithm; (ZHOU [pg. 1, Abstract] teaches: "We present a hierarchical maximum-margin clustering method for unsupervised data analysis." ZHOU [pg. 1, section 1. Introduction] teaches: “Recent progress in maximum-margin methods has led to the development of maximum-margin clustering (MMC) techniques (Xu et al., 2004), which aim to learn both the separating hyperplanes that separate clusters of data, and the label assignments of instances to the clusters.” ZHOU [pg. 3, section 3. Hierarchical Maximum-Margin Clustering] teaches: “The proposed method builds on the standard flat MMC clustering […].” ZHOU [pg. 3, section 2 Related Work] teaches: "our method utilizes the grouping and exclusive regularizers for unsupervised clustering." ZHOU [pg. 9, Conclusion] teaches: "We test our method on four standard datasets, showing the efficacy of our method in clustering, and the ability to capture semantics via the hierarchies." Examiner’s note: ZHOU teaches building on the MMC method, which learns both the hyperplanes that separate clusters of data and label assignments of instances to the clusters (i.e., train to jointly cluster and interpret resulting clusters) by using the hierarchical maximum-margin clustering method, which is an unsupervised clustering method (i.e., by using an unsupervised clustering algorithm), on various datasets.)
generating hyperplanes in a multi-dimensional feature space of the dataset, […] (ZHOU [pg. 1, Abstract] teaches: "Recent progress in maximum-margin methods has led to the development of maximum-margin clustering (MMC) techniques (Xu et al., 2004), which aim to learn both the separating hyperplanes that separate clusters of data, and the label assignments of instances to the clusters (i.e., clustering)." ZHOU [pg. 3, section 3. Hierarchical Maximum-Margin Clustering] teaches: “The proposed method builds on the standard flat MMC clustering […].” ZHOU [pg. 3, section 3. Hierarchical Maximum-Margin Clustering] teaches: "We propose a hierarchical clustering method based on the maximum-margin criterion." ZHOU [pg. 6, section 5. Experiments] teaches: “We evaluate the performance of HMMC on four datasets from two public image collections: Animal With Attributes (AWA) (Lampert et al., 2009) and ImageNet (Deng et al., 2009).” Furthermore, ZHOU [pg. 6, section 5. Experiments] describes various datasets, such as the AWA-ATTR containing 85 features, and the AWA-PCA containing100 dimensions. Therefore, under broadest reasonable interpretation, the multi-dimensional feature space of the dataset can be interpreted as the number of features in the datasets used for performing the hierarchical maximum-margin clustering for learning the hyperplanes that separate the clusters of data.)
repeating the clustering and generating until convergence, wherein the clustering in a subsequent iteration uses the generated hyperplanes from a previous iteration to optimize performance of the clustering; (ZHOU [pg. 3, section 3. Hierarchical Maximum-Margin Clustering] teaches: "We use
D
t
to denote the data on
n
t
, and HMMC splits
D
t
into
K
t
clusters by learning a linear model
w
t
k
for each cluster
k
. We collect the
K
t
cluster models in
w
t
=
w
t
k
k
=
1
K
t
. We split the data
D
t
on node
n
t
using the MMC idea – finding a clustering assignment such that the resultant margin between clusters is maximal over all possible assignments." Additionally, ZHOU [pg. 4, section 4.2. Splitting A Node] teaches: "We use an alternating descent algorithm to reach a solution. In each iteration we fix the model parameters
w
t
and optimize
y
t
by solving a clustering assignment problem, and then we update
w
t
while keeping
y
t
fixed using a proximal quasi-Newton algorithm (Lee et al., 2012; Schmidt, 2010). The algorithm stops when the objective converges to a local optimum with respect to these steps." Examiner's note: ZHOU defines
w
t
as the linear models (i.e., hyperplanes) that separate the clusters and
y
t
as the label assignments of instances to the clusters (i.e., clustering). Therefore, under BRI, using the clustering in a subsequent iteration can be interpreted as fixing
w
t
to optimize
y
t
and then
y
t
is fixed to optimize
w
t
and so on until the objective converges (i.e., repeating the clustering and generating).)
ZHOU is not relied upon for teaching:
[…] the hyperplanes separating pairs of the clusters that have been clustered, wherein a hyperplane separates a pair of clusters that have been clustered, each of the hyperplanes being associated with a description;
adjusting the hyperplanes to further improve the performance of the clustering;
interacting via a user interface to present the clusters and interpretation of the clusters, wherein a cluster’s interpretation is provided based on the description of the hyperplanes that construct a polytope containing the cluster,
wherein a polytope is constructed around each cluster, and wherein flexibility in the cluster's interpretation and trade-off between quality of clusters and quality of interpretability are controlled via configurable parameters, the configurable parameters including a grid of potential hyperplanes and sparsity of the hyperplanes.
However, BORGWARDT teaches: […] the hyperplanes separating pairs of the clusters that have been clustered, wherein a hyperplane separates a pair of clusters that have been clustered, […] (BORGWARDT [pg. 2, section 1.1 Separable Clusterings] teaches: "Two clusters
C
t
,
C
2
are called separable if there exists a (separating) hyperplane that partitions the underlying space into two halfspaces, each of which contains one of the clusters. A separable clustering
C
=
(
C
1
,
…
,
C
k
)
requires separability of all pairs of clusters, as well as a special positioning of the corresponding hyperplanes: they have to create a partition of the underlying space into a cell complex
P
=
P
1
,
…
,
P
k
of polyhedral cells, one cell
P
i
for each cluster
C
i
formed through the intersection of all halfspaces that contain
C
i
." Examiner's note: BORGWARDT explicitly teaches creating partitions of the clusters using hyperplanes (i.e., generating hyperplanes). Additionally, under BRI, the hyperplanes generated for a dataset can be interpreted as the boundaries of the cells within a Power Diagram for that dataset (see BORGWARDT Fig. 2). The Power Diagram uses the center of the clusters as the site vector in order to calculate the corresponding hyperplanes that separate the clusters. Thus, the step of clustering must occur prior to generating the hyperplanes (i.e., that have been clustered).)
wherein a cluster’s interpretation is provided based on the description of the hyperplanes that construct a polytope containing the cluster, (BORGWARDT [pg. 1, Abstract] teaches: "[...] separable clusterings lie on the boundary of partition polytopes, form a subset of the vertices of the corresponding transportation polytopes, and circuits of both polytopes are readily interpreted as sequential or cyclical exchanges of items between clusters." BORGWARDT [pg. 2, section 1.1 Separable Clusterings] teaches: " A separable clustering
C
=
(
C
1
,
…
,
C
k
)
requires separability of all pairs of clusters, as well as a special positioning of the corresponding hyperplanes: they have to create a partition of the underlying space into a cell complex
P
=
P
1
,
…
,
P
k
of polyhedral cells, one cell
P
i
for each cluster
C
i
formed through the intersection of all halfspaces that contain
C
i
." Examiner's note: paragraph [0021] of the present application defines polytopes as "the intersection of half-spaces". Therefore, under BRI, "a cluster’s interpretation is provided based on description of the hyperplanes that construct a polytope containing the cluster" can be interpreted as the polyhedral cells that partition the underlying space.”)
[…] wherein a polytope is constructed around each cluster, […] (BORGWARDT [pg. 2, section 1.1 Separable Clusterings] teaches: "Two clusters
C
t
,
C
2
are called separable if there exists a (separating) hyperplane that partitions the underlying space into two halfspaces, each of which contains one of the clusters. A separable clustering
C
=
(
C
1
,
…
,
C
k
)
requires separability of all pairs of clusters, as well as a special positioning of the corresponding hyperplanes: they have to create a partition of the underlying space into a cell complex
P
=
P
1
,
…
,
P
k
of polyhedral cells, one cell
P
i
for each cluster
C
i
formed through the intersection of all halfspaces that contain
C
i
." Furthermore, paragraph [0021] of the specification recites in part “For example, a system and/or method can describe each cluster by constructing a polytope around it. A polytope can be considered the intersection of half-spaces. Fig. 1 shows an example of a polytope in a 2-dimensional space in an embodiment.” Under broadest reasonable interpretation, BORGWARDT’s Figures 6, 8, 9, and 10 on pages 9, 18, 21, and 28 respectively, along with [pg. 2, section 1.1 Separable Clusterings] as cited above, teach constructing polytopes around each cluster.)
Accordingly, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention, having the teachings of ZHOU and BORGWARDT before them, to include BORGWARDT’s Power Diagram created by hyperplanes in ZHOU’s clustering method. One would have been motivated to make such a combination in order to “explicitly represent how the separation is realized” for the clusters and use the Power Diagram as a clear transition to separate clusters (BORGWARDT [pg. 12, section 3 The Overall Transition]).
ZHOU in view of BORGWARDT is not relied upon for teaching:
[…] each of the hyperplanes being associated with a description;
adjusting the hyperplanes to further improve the performance of the clustering;
interacting via a user interface to present the clusters and interpretation of the clusters, wherein a cluster’s interpretation is provided based on description of the hyperplanes that construct a polytope containing the cluster.
[…] and wherein flexibility in the cluster's interpretation and trade-off between quality of clusters and quality of interpretability are controlled via configurable parameters, the configurable parameters including a grid of potential hyperplanes and sparsity of the hyperplanes.
However, BERTSIMAS teaches: […] each of the hyperplanes being associated with a description; (BERTSIMAS [pg. 23-24, Figures 4, 5, and 6] shows feature based rules on the branch nodes of the decision trees. For example, the splits are labeled with conditions like Diabetes < 0.5, Smoking < 0.5, and Gender = "Female" followed by the respective clusters. Furthermore, paragraph [0023] of the instant application states "In an embodiment, the system and/or method can allow for constraints on the hyperplanes that construct each polytope allowing for a wider range of cluster explanations including axis parallel partitions (e.g., similar to decision trees)." A person having ordinary skill in the art would recognize that decision tree splits, as use in BERTSIMAS, function equivalently to hyperplanes in BORGWARDT for defining and separating a dataset. In BERTSIMAS, each split (e.g., Diabetes < 0.5, Smoking < 0.5, Gender = "Female", etc.) creates a mathematically or feature based boundary (i.e., a hyperplane) that partitions data into branches, forming clusters. This complements BORGWARDT's hyperplane-based separation by adding clear, interpretable descriptions to explain how these clusters are formed.)
adjusting the hyperplanes to further improve the performance of the clustering; (BERTSIMAS [pg. 4, section 1.1 Contributions] teaches: "We provide an implementation of our method with an iterative coordinate-descent approach that scales to larger problems, well-approximating the globally optimal solution. We use widely two established validation criteria, the Silhouette Metric (Rousseeuw 1987) and the Dunn Index (Dunn 1974), as the algorithm's objective function." Additionally, BERTSIMAS [pg. 9, section 2.2 Loss functions for cluster quality] teaches: "However, empirical results suggest that the Dunn Index has superior performance in returning intuitive partitions of the data when they are well-separated." BERTSIMAS [pg. 13, section 3.1 Coordinate-descent Implementation] teaches: "The user can specify to optimize either the Silhouette Metric or Dunn Index described in Sect. 2.2. These metrics penalize low separation, which naturally limits the depth of the tree." Examiner's note: under BRI, "adjusting the hyperplanes to further improve the performance of the clustering" can be interpreted as the coordinate descent using validation criteria such as Silhouette metric or Dunn Index which penalizes low separation for the clusters. Furthermore, the "hyperplanes" are taught by the combination of ZHOU and BORGWARDT.)
[…] and quality of interpretability are controlled via configurable parameters. (BERTSIMAS [pg. 4, section 1.1 Contributions] teaches: "We use widely two established validation criteria, the Silhouette Metric (Rousseeuw 1987) and the Dunn Index (Dunn 1974)". BERTSIMAS [pg. 13, section 3.1 Coordinate Descent Implementation] teaches: "The user can specify to optimize either the Silhouette Metric or Dunn Index described in Sect. 2.2. These metrics penalize low separation, which naturally limits the depth of the tree." BERTSIMAS [pg. 9, section 2.2 Loss functions for cluster quality] teaches: "However, empirical results suggest that the Dunn Index has superior performance in returning intuitive partitions of the data when they are well-separated." BERTSIMAS [pg. 28, section 5.2.4 Results on quantitative performance] teaches: “Although interpretability is our primary objective in cluster development, we also want to ensure that our resultant groupings are reasonable from the perspective of the internal validation criteria which provide a quantitative evaluation. Table 5 shows the metric scores obtained for both the Silhouette Metric and the Dunn Index.”)
Accordingly, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention, having the teachings of ZHOU, BORGWARDT, and BERTSIMAS before them to include BERTSIMAS’ feature based rule explanations and coordinate descent using validation criteria such as the Silhouette metric and Dunn Index in ZHOU and BORGWARDT’s clustering method. One would have been motivated to make such a combination in order to account “for mixed densities and identifying meaningful separation in less structured data settings.”, (BERTSIMAS [pg. 37, section 7 discussion].)
ZHOU in view of BORGWARDT and BERTSIMAS is not relied upon for teaching, but LIU teaches: interacting via a user interface to present the clusters and interpretation of the clusters, […] (LIU [pg. 12, section 3. User-Oriented Pruning of Cluster Trees] teaches: "We propose two interactive approaches to allow the user to explore the cluster tree to find meaningful/useful clusters. Browsing: The user simply explores the tree him/herself to find meaningful clusters (prune the rest). A user interface has been built to facilitate this exploration. This is not a difficult task because the major clusters are identified at the top levels of the tree.")
Accordingly, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention, having the teachings of ZHOU, BORGWARDT, BERTSIMAS, and LIU before them, to include LIU's interface in ZHOU, BORGWARDT, and BERTSIMAS' clustering and hyperplane generation method. One would have been motivated to make such a combination in order to facilitate users the interactive exploration of clusters to find meaningful/useful clusters (LIU [pg. 12, section 3. User-Oriented Pruning of Cluster Trees]).
ZHOU in view of BORGWARDT, BERTSIMAS, and LIU is not relied upon for teaching:
wherein flexibility in the cluster's interpretation and trade-off between quality of clusters […] are controlled via configurable parameters, the configurable parameters including a grid of potential hyperplanes and sparsity of the hyperplanes.
However, FROST teaches: wherein flexibility in the cluster's interpretation and trade-off between quality of clusters […] are controlled via configurable parameters, (FROST [pg. 2, section 1. Introduction] teaches: "Then, given a budget of k' > k leaves, it greedily expands the tree to reduce the clustering cost. At each step, the clusters form a refinement of the previous clustering. By adding more thresholds, we gain more flexibility in the data partition, and we also allow multiple leaves to correspond to the same cluster (with k clusters total)." FROST [pg. 1, Abstract] We study algorithms for k-means clustering, focusing on a trade-off between explainability and accuracy. [...] We prove that as k' increases, the surrogate cost is non-increasing, and hence, we trade explainability for accuracy.” FRONT [pg. 4, section 2 Preliminaries] We use the IMM algorithm to build a tree with k leaves as the initialization for our algorithm. Then, we smoothly trade explainability for accuracy by expanding the tree to have k' > k leaves, for a user-specified parameter k'.”)
Accordingly, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention, having the teachings of ZHOU, BORGWARDT, BERTSIMAS, LIU, and FROST before them, to include FROST’s user-specified parameter for providing trade-off between explainability and accuracy in ZHOU, BORGWARDT, BERTSIMAS, and LIU’s clustering and hyperplane generation method. One would have been motivated to make such a combination in order to provide a flexible and smooth trade-off between explainability and accuracy of clusters based on a user-specified parameter (FROST [pg. 1, Abstract], [pg. 4, section 2 Preliminaries] , and [pg. 11, section 4.2 Discussion]).
ZHOU in view of BORGWARDT, BERTSIMAS, LIU, and FROST is not relied upon for teaching:
[…] the configurable parameters including a grid of potential hyperplanes and sparsity of the hyperplanes.
However, CARRIZOSA teaches: […] the configurable parameters including a grid of potential hyperplanes […] (CARRIZOSA [pages 257, section 1. Introduction] teaches: “The SVM aims at separating both classes by means of a hyperplane,
ω
T
x
+
b
=
0
with
ω
=
ω
1
,
ω
2
,
…
,
ω
d
”. CARRIZOSA [pages 258, section 2.1. The idea] teaches: “The Discrete Level Support Vector Machine (DILSVM) is a variant of the SVM classifier where for each feature
j
, the score
ω
j
can only take on a discrete number of values. Let
A
⊂
R
be a finite set that includes the value 0, which models the marginal impact of the feature on the classifier […] For adequate choices of
A
, this model gains interpretability and visualization. For instance, let us consider
A
=
-
a
,
0
,
a
>
0
. […] for a given rating level
a
, the DILSVM detects those features which strongly agree with the positive class, those which strongly disagree (and therefore strongly agree with the negative class), and those which are irrelevant to the classifier. This DILSVM classifier can be represented as a collection of three-point Likert scales, one for each feature, measuring the extent to which the feature is in agreement with the positive class. When looking for more granularity of the scale, we can increase the size of
A
. For
A
=
-
a
1
,
-
a
2
,
0
,
a
2
,
a
1
,
a
1
>
a
2
>
0
, the DILSVM classifier can be seen as a collection of five-point Likert scales where features
j
with
ω
j
=
a
1
are seen to strongly agree with the positive class, those with
ω
j
=
a
2
agree (but not so strongly), while
ω
j
=
-
a
1
-
a
2
strongly disagree (disagree).” CARRIZOSA [page 259, section 3. Constructing the DILSVM classifier] teaches: “Inspired by Likert scales, we assume that the set
A
is symmetric and defined as
A
=
-
a
1
,
…
,
-
a
K
,
0
,
a
K
,
…
,
a
1
, where
a
1
>
…
>
a
K
>
0
are the so-called rating levels telling us about the extent to which each feature is in agreement with the positive class. We denote this model by
D
I
L
S
V
M
K
. Please note that
A
could be considered asymmetric without loss of generality. Our model involves
K
+
1
parameters, namely
K
rating levels as well as the tradeoff parameter
C
. In the following, we formulate
D
I
L
S
V
M
K
, when
K
+
1
parameters are fixed, as an MILP problem.” CARRIZOSA [page 263, section 4. Numerical experiments] teaches: “The tuning procedure used to choose the tradeoff parameter C and the rating levels
a
k
,
k
=
1
,
…
,
K
, is given in Section 4.1.” CARRIZOSA [page 263, section 4.1. Parameter settings] teaches: “Following the usual approach, for the
D
I
L
S
V
M
1
, parameters
C
and
a
1
are tuned y inspecting a grid of the form
C
n
∈
10
-
6
,
…
,
10
6
and of the form
a
1
∈
2
0
,
…
,
2
10
. For
D
I
L
S
V
M
2
,
C
and
a
1
are tuned with the same grid, and
a
2
∈
a
1
2
,
a
1
2
2
.” Examiner’s note: Specification paragraph [0037] states “the terms
w
i
,
j
represent the coefficients of the hyperplane” and “For providing interpretability, in embodiment, a constraint can be set that can require that all of the coefficients to be integers, and do not allow decimal values. Another constraint sets an upper bound on the maximum integer size, represented by the term 'M'. For instance, an upper bound can be set such that the integer coefficients can be no larger than 10. Setting the maximum allowable coefficient integer size provides several benefits. For example, it allows for enumerating all possible hyperplanes, so that exhaustive local search can be performed. As another example, the maximum coefficient integer size can be considered as defining a grid of potential hyperplanes that are considered.” Under BRI, a grid of potential hyperplanes can be interpreted as CARRIZOSA’s set
A
, which defines the finite set of values the hyperplane coefficient
ω
j
can take for the hyperplane
ω
T
x
+
b
. Additionally, CARRIZOSA [page 258, Equation (4)] discloses that
ω
j
∈
A
∀
j
=
1
,
…
,
d
. Furthermore, the configurable parameters including a grid of potential hyperplanes can be interpreted as tuning the rating levels
a
K
that define the set of finite values the coefficient of the hyperplane can take.)
Accordingly, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention, having the teachings of ZHOU, BORGWARDT, BERTSIMAS, LIU, FROST, and CARRIZOSA before them, to include CARRIZOSA’s set
A
defining finite values for hyperplane coefficients in ZHOU, BORGWARDT, BERTSIMAS, LIU, and FROST’s clustering and hyperplane generation method. One would have been motivated to make such a combination because for adequate choices of
A
the model gains interpretability and visualization (CARRIZOSA [page 258, section 2.1. The idea]).
ZHOU in view of BORGWARDT, BERTSIMAS, LIU, FROST, and CARRIZOSA is not relied upon for teaching, but GAMBELLA teaches: […] the configurable parameters including […] sparsity of the hyperplanes. (GAMBELLA [pg. 10, section 3.4.1 Hard Margin SVM] teaches: “The training of the SVM model involves finding the hyperplane that separates the data and whose distance to the closest data point in either of the classes, i.e., margin, is maximized.” GAMBELLA [pg. 11, section 3.4.3 Sparse SVM] teaches: "Using the 1-norm is also an approach to sparsify w, i.e., reduce the number of features that are involved in the classification model. [...] The number of features can be explicitly modeled in (42)-(45) by using binary variables
z
∈
0,1
p
where
z
j
=
1
indicates that feature
j
is selected and otherwise
z
j
=
0
.” GAMBELLA [pg. 17, section 5.1 Principal Components] teaches: "To promote the sparsity of the projected components, and thus make them more interpretable, [55] formulates a Mixed Integer Nonlinear Programming (MINLP) problem and shows that the level of sparsity can be imposed in the model.")
Accordingly, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention, having the teachings of ZHOU, BORGWARDT, BERTSIMAS, LIU, FROST, CARRIZOSA and GAMBELLA before them, to include GAMBELLA’s imposing of sparsity in the model in ZHOU, BORGWARDT, BERTSIMAS, LIU, FROST, and CARRIZOSA’s clustering and hyperplane generation method. One would have been motivated to make such a combination in order to “promote the sparsity of the projected components, and thus make them more interpretable” (GAMBELLA [pg. 17, section 5.1 Principal Components]).
Regarding Claim 2:
ZHOU in view of BORGWARDT, BERTSIMAS, LIU, FROST, CARRIZOSA, and GAMBELLA teaches the elements of claim 1 as outlined above. ZHOU further teaches:
The system of claim 1, wherein the clustering and the generating of the hyperplanes are performed as a single mixed integer non-linear programming that solves alternating minimization between the clustering and the hyperplane generating. (ZHOU [pg. 2, section 2. Related Work] teaches: “It is a maximum-margin method for clustering […] that learns both the maximum-margin hyperplane for each cluster and the clustering assignment of instances to clusters. Since this joint learning results in a non-convex formulation, unlike SVMs, it is often solved by a semidefinite relaxation (Xu et al., 2004; Valizadegan & Jin, 2006) or alternating optimization (Zhang et al., 2007).” Additionally, ZHOU [pg. 3, Equation (1)] teaches: “We use
D
t
to denote the data on
n
t
, and HMMC splits
D
t
into
K
t
clusters by learning a linear model
w
t
k
for each cluster
k
.” (Examiner’s note: a linear model for each cluster can be interpreted as the hyperplane that separates the clusters of data (ZHOU [pg. 1, Abstract]). Additionally, ZHOU [pg. 3, 3. Hierarchical Maximum-Margin Clustering] teaches: “a hierarchical maximum margin clustering formulated as:
PNG
media_image1.png
251
596
media_image1.png
Greyscale
Additionally, ZHOU [pg. 4, Convergence Analysis] teaches: “We now show that this alternating descent algorithm converges to a local optimum. The optimization consists of two alternating steps: updating the discrete
y
t
and the continuous
w
t
. In the
w
t
update, we fix the clustering
y
t
and use a method that is guaranteed to find a global optimum (Lee et al., 2012; Schmidt, 2010).” Furthermore, ZHOU [pg. 4, section 4.2. Splitting A Node] teaches: “The algorithm stops when the objective converges to a local optimum with respect to these steps.” Examiner’s note: with
y
t
i
∈
{
1
,
…
,
K
t
}
being variable integers (i.e., clusters),
w
t
being a continuous variable that defines the linear model (i.e., hyperplanes as taught by BORGWARDT) for each cluster, and the slack variables
ξ
t
i
y
2
being squared, Equation (1) becomes mixed-integer nonlinear and uses alternating optimization (i.e., alternating minimization) to solve the equation.)
Regarding Claim 4:
ZHOU in view of BORGWARDT, BERTSIMAS, LIU, FROST, CARRIZOSA, and GAMBELLA teaches the elements of claim 1 as outlined above. GAMBELLA further teaches:
wherein the hyperplanes are generated based on configurable parameters that control sparsity of the hyperplanes for interpretability. (GAMBELLA [pg. 10, section 3.4.1 Hard Margin SVM] teaches: “The training of the SVM model involves finding the hyperplane that separates the data and whose distance to the closest data point in either of the classes, i.e., margin, is maximized.” GAMBELLA [pg. 11, section 3.4.3 Sparse SVM] teaches: "Using the 1-norm is also an approach to sparsify w, i.e., reduce the number of features that are involved in the classification model. [...] The number of features can be explicitly modeled in (42)-(45) by using binary variables
z
∈
0,1
p
where
z
j
=
1
indicates that feature
j
is selected and otherwise
z
j
=
0
.” GAMBELLA [pg. 17, section 5.1 Principal Components] teaches: "To promote the sparsity of the projected components, and thus make them more interpretable, [55] formulates a Mixed Integer Nonlinear Programming (MINLP) problem and shows that the level of sparsity can be imposed in the model.")
Regarding Claim 5:
ZHOU in view of BORGWARDT, BERTSIMAS, LIU, FROST, CARRIZOSA, and GAMBELLA teaches the elements of claim 1 as outlined above. BERTSIMAS further teaches:
The system of claim 1, wherein the adjusting of the hyperplanes is performed based on a selected clustering metric. (BERTSIMAS [pg. 4, section 1.1 Contributions] teaches: "We provide an implementation of our method with an iterative coordinate-descent approach that scales to larger problems, well-approximating the globally optimal solution. We use widely two established validation criteria, the Silhouette Metric (Rousseeuw 1987) and the Dunn Index (Dunn 1974), as the algorithm's objective function." Additionally, BERTSIMAS [pg. 9, section 2.2 Loss functions for cluster quality] teaches: "However, empirical results suggest that the Dunn Index has superior performance in returning intuitive partitions of the data when they are well-separated." BERTSIMAS [pg. 13, section 3.1 Coordinate-descent Implementation] teaches: "The user can specify to optimize either the Silhouette Metric or Dunn Index described in Sect. 2.2. These metrics penalize low separation, which naturally limits the depth of the tree." Examiner's note: under BRI, "adjusting the hyperplanes to further improve the performance of the clustering" can be interpreted as the coordinate descent using validation criteria such as Silhouette metric or Dunn Index which penalizes low separation for the clusters. Furthermore, the "hyperplanes" are taught by the combination of ZHOU and BORGWARDT.)
Regarding Claim 6:
ZHOU in view of BORGWARDT, BERTSIMAS, LIU, FROST, CARRIZOSA, and GAMBELLA teaches the elements of claim 5 as outlined above. BERTSIMAS further teaches:
The system of claim 5, wherein the selected clustering metric includes Silhouette index. (BERTSIMAS [pg. 4, section 1.1 Contributions] teaches: "We provide an implementation of our method with an iterative coordinate-descent approach that scales to larger problems, well-approximating the globally optimal solution. We use widely two established validation criteria, the Silhouette Metric (Rousseeuw 1987) and the Dunn Index (Dunn 1974)”. BERTSIMAS [pg. 13, section 3.1 Coordinate-descent Implementation] teaches: "The user can specify to optimize either the Silhouette Metric or Dunn Index described in Sect. 2.2.”)
Regarding Claim 7:
ZHOU in view of BORGWARDT, BERTSIMAS, LIU, FROST, CARRIZOSA, and GAMBELLA teaches the elements of claim 5 as outlined above. BERTSIMAS further teaches:
The system of claim 5, wherein the selected clustering metric includes Dunn index. (BERTSIMAS [pg. 4, section 1.1 Contributions] teaches: "We provide an implementation of our method with an iterative coordinate-descent approach that scales to larger problems, well-approximating the globally optimal solution. We use widely two established validation criteria, the Silhouette Metric (Rousseeuw 1987) and the Dunn Index (Dunn 1974)”. BERTSIMAS [pg. 13, section 3.1 Coordinate-descent Implementation] teaches: "The user can specify to optimize either the Silhouette Metric or Dunn Index described in Sect. 2.2.”)
Regarding Claim 8:
The claim recites similar limitations as corresponding claim 1 and is rejected for similar reasons as claim 1 using similar teachings and rationale.
Regarding Claim 9:
ZHOU in view of BORGWARDT, BERTSIMAS, LIU, FROST, CARRIZOSA, and GAMBELLA teaches the elements of claim 8 as outlined above. Additionally, the claim recites similar limitation as corresponding claim 2 and is rejected for similar reasons as claim 2 using similar teachings and rationale.
Regarding Claim 11:
ZHOU in view of BORGWARDT, BERTSIMAS, LIU, FROST, CARRIZOSA, and GAMBELLA teaches the elements of claim 8 as outlined above. Additionally, the claim recites similar limitation as corresponding claim 4 and is rejected for similar reasons as claim 4 using similar teachings and rationale.
Regarding Claim 12:
ZHOU in view of BORGWARDT, BERTSIMAS, LIU, FROST, CARRIZOSA, and GAMBELLA teaches the elements of claim 8 as outlined above. Additionally, the claim recites similar limitation as corresponding claim 5 and is rejected for similar reasons as claim 5 using similar teachings and rationale.
Regarding Claim 13:
ZHOU in view of BORGWARDT, BERTSIMAS, LIU, FROST, CARRIZOSA, and GAMBELLA teaches the elements of claim 12 as outlined above. Additionally, the claim recites similar limitation as corresponding claim 6 and is rejected for similar reasons as claim 6 using similar teachings and rationale.
Regarding Claim 14:
ZHOU in view of BORGWARDT, BERTSIMAS, LIU, FROST, CARRIZOSA, and GAMBELLA teaches the elements of claim 12 as outlined above. Additionally, the claim recites similar limitation as corresponding claim 7 and is rejected for similar reasons as claim 7 using similar teachings and rationale.
Regarding Claim 15:
The claim recites similar limitations as corresponding claim 1 and is rejected for similar reasons as claim 1 using similar teachings and rationale. ZHOU further teaches:
A computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions readable by a device to cause the device to: (ZHOU [pg. 7, Table 1] teaches: "Clustering performance on the four datasets. […]The runtime (in seconds) is measured on a machine with Intel Xeon 2.8GHz CPU and 16GB memory." Examiner’s note: ZHOU explicitly teaches evaluating the performance of the clustering using a processor and a memory. The specific details of the clustering must be programmed with the required instructions in order to execute the clustering of datasets.)
Regarding Claim 16:
ZHOU in view of BORGWARDT, BERTSIMAS, LIU, FROST, CARRIZOSA, and GAMBELLA teaches the elements of claim 15 as outlined above. Additionally, the claim recites similar limitation as corresponding claim 2 and is rejected for similar reasons as claim 2 using similar teachings and rationale.
Regarding Claim 18:
ZHOU in view of BORGWARDT, BERTSIMAS, LIU, FROST, CARRIZOSA, and GAMBELLA teaches the elements of claim 15 as outlined above. Additionally, the claim recites similar limitation as corresponding claim 4 and is rejected for similar reasons as claim 4 using similar teachings and rationale.
Regarding Claim 19:
ZHOU in view of BORGWARDT, BERTSIMAS, LIU, FROST, CARRIZOSA, and GAMBELLA teaches the elements of claim 15 as outlined above. Additionally, the claim recites similar limitation as corresponding claim 5 and is rejected for similar reasons as claim 5 using similar teachings and rationale.
Regarding Claim 20:
ZHOU in view of BORGWARDT, BERTSIMAS, LIU, FROST, CARRIZOSA, and GAMBELLA teaches the elements of claim 19 as outlined above. BERTSIMAS further teaches:
The computer program product of claim 19, wherein the selected clustering metric includes at least one selected from the group of Silhouette index and Dunn index. (BERTSIMAS [pg. 4, section 1.1 Contributions] teaches: "We provide an implementation of our method with an iterative coordinate-descent approach that scales to larger problems, well-approximating the globally optimal solution. We use widely two established validation criteria, the Silhouette Metric (Rousseeuw 1987) and the Dunn Index (Dunn 1974)”. BERTSIMAS [pg. 13, section 3.1 Coordinate-descent Implementation] teaches: "The user can specify to optimize either the Silhouette Metric or Dunn Index described in Sect. 2.2.)
Claims 3, 10, and 17 are rejected under 35 U.S.C. 103 as being unpatentable over ZHOU in view of BORGWARDT, BERTSIMAS, LIU, FROST, CARRIZOSA, and GAMBELLA as applied respectively above to claims 1 and 8, and further in view of ABBASI (“Fair Clustering via Equitable Group Representations”), hereafter ABBASI.
Regarding Claim 3:
ZHOU in view of BORGWARDT, BERTSIMAS, LIU, FROST, CARRIZOSA, and GAMBELLA teaches the elements of claim 1 as outlined above. ZHOU further teaches:
[…] k-means clustering […] (ZHOU [pg. 2] teaches: “Our method is a top-down clustering method, and the canonical example of such a method is hierarchical k-means clustering, which performs k-means recursively in a top-down manner”.)
However, ZHOU in view of BORGWARDT, BERTSIMAS, LIU, FROST, CARRIZOSA, and GAMBELLA is not relied upon for teaching, but ABBASI teaches: wherein the clustering is implemented using a representation aware […] clustering that clusters with awareness of representation error using a clustering metric. (ABBASI [pg. 3, section 2.1 Quality of group representation] teaches: "Definition 2 (AbsError). The absolute (representation) error of a clustering is defined as
A
b
s
E
r
r
o
r
C
X
=
∑
x
∈
X
d
x
,
C
,
where 𝑋 is a set of points, 𝐶 is a set of centers and 𝑑 (𝑥,𝐶) is an arbitrary distance function between 𝑥 and nearest center to it in 𝐶.” Additionally, ABBASI [pg. 3, section 2.1 Quality of group representation] teaches “Definition 3”, which is a relative (representation) error of a clustering. Examiner’s note: under BRI, the absolute or relative (representation) error can be interpreted as the awareness of representation error and the arbitrary distance function can be interpreted as the clustering metric.)
Accordingly, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention, having the teachings of ZHOU, BORGWARDT, BERTSIMAS, LIU, FROST, CARRIZOSA, GAMBELLA, and ABBASI before them to include ABBASI’s representation errors in ZHOU, BORGWARDT, BERTSIMAS, LIU, FROST, CARRIZOSA, and GAMBELLA’s clustering method. One would have been motivated to make such a combination in order to achieve fair clustering that leads to an optimal representation, (ABBASI [pg. 3, section 2.1 Quality of group representation; Figure 2]).
Regarding Claim 10:
ZHOU in view of BORGWARDT, BERTSIMAS, LIU, FROST, CARRIZOSA, and GAMBELLA teaches the elements of claim 8 as outlined above. Additionally, the claim recites similar limitation as corresponding claim 3 and is rejected for similar reasons as claim 3 using similar teachings and rationale.
Regarding Claim 17:
ZHOU in view of BORGWARDT, BERTSIMAS, LIU, FROST, CARRIZOSA, and GAMBELLA teaches the elements of claim 15 as outlined above. Additionally, the claim recites similar limitation as corresponding claim 3 and is rejected for similar reasons as claim 3 using similar teachings and rationale.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Alvaro S Laham Bauzo whose telephone number is (571)272-5650. The examiner can normally be reached Mon-Fri 7:30 AM - 11:00 AM | 1:00 PM - 5:30 PM ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Usmaan Saeed can be reached on (571) 272-4046. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/A.S.L./Examiner, Art Unit 2146
/USMAAN SAEED/Supervisory Patent Examiner, Art Unit 2146
Prosecution Insights