Detailed Action
This Office Action is in response to the remarks entered on 04/21/2026. Amended claims 1, 7 and 17 have been entered. Claims 1-17 and 19-21 are currently pending.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim 1 and 3-10, and 12-16 are rejected under 35 U.S.C. 103 as being unpatentable over Zemouri et al. (Zemouri et al, “Recurrent radial basis function network for time-series prediction”, 2003, hereinafter ‘Zemouri’) in view of Gutiérrez et al. (Gutiérrez et al, “Logistic Regression by Means of Evolutionary Radial Basis Function Neural Networks”, 2011, hereinafter ‘Gutiérrez’) and further in view of Ehrman (US 20180018566 A1, hereinafter ‘Ehrman’).
Regarding claim 1, Zemouri teaches:
A method for a recurrent neural network that processes a sequence of items, of which a new item at an iteration t is an unclassified item and previous items are classified items, the method comprising: ([Zemouri, page 454, right col, line 3-10] discloses that the network can be used in two kinds of applications: regression and classification. [Zemouri, page 456, left col, line 1-11] discloses the recurrent radial basis neural network function. [Zemouri, page 456, 3.1. Looped neuron, line 1-17] discloses that the output of the function is generated iteratively for each instant t. The input
I
i
is the unclassified item and the output
x
i
(
t
)
are the classified items. The equation (13) indicates that the previous items
x
(
t
-
1
)
are classified, wherein the
x
(
t
-
1
)
is a classification output of
x
i
(
t
)
at time
t
-
1
)
for each iteration t of said recurrent neural network, transforming an output vector of a hidden layer of said recurrent neural network to an output feature vector [Zemouri, page 456, left col, line 1-11] discloses the recurrent radial basis neural network function. [Zemouri, page 456, 3.1. Looped neuron, line 1-17] discloses transforming an output vector of a hidden layer of the recurrent neural network to an output feature vector. The equation (13) indicates that the previous items
x
(
t
-
1
)
are classified at t-1 (previous neuron, which is a hidden layer), wherein the
x
(
t
-
1
)
is an output of
x
i
(
t
)
at time
t
-
1
)
computing an output embedding vector based on said output vector of the hidden layer; ([Zemouri, page 457, right col, lines 1-10] discloses calculating output embedding vector
y
(
t
)
based on the hidden layer output
x
j
(
t
)
and the prototype vector
μ
i
j
)
by subtracting said output embedding vector from one of said plurality of qualified feature vectors; ([Zemouri, page 454, left col, 2.1. RBF networks definition, line 4 – right col, line 4] The Euclidian distance is calculated for a given input vector x (qualified feature vectors) and the prototype vector
μ
j
. [Zemouri, page 457, last para, line 1-5] The prototypes are extracted from the output of the looped neurons (recurrent neural network) );
However, Zemouri does not specifically disclose:
storing a plurality of qualified feature vectors in a plurality of computation columns of an associated memory device having rows and computation columns, wherein each of said plurality of qualified feature vectors describes one of said classified items;
transforming an output vector … to an output feature vector using said associative memory device
and said output feature vector describing the probabilities of said unclassified item having a kth class
concurrently activating at least two rows of said associative memory device to concurrently in each of said plurality of computation columns, calculate a distance vector by subtracting said output embedding vector from one of said plurality of qualified feature vectors;
concurrently activating at least two rows of said associative memory device to concurrently in each of said plurality of computation columns, compute a similarity score for each distance vector; and
creating said output feature vector as a vector of said plurality of computed similarity scores.
Gutiérrez teaches:
said output feature vector describing the probabilities of said unclassified item having a kth class ([Gutiérrez, page 249, left col, line 1-18] discloses calculating the probability that x belongs to class l (kth class). Additionally, [Gutiérrez, page 249, left col, last para, line 8-12] discloses applying the softmax transformation (softmax layer converts output to a probability) to the outputs of the network)
Before the effective filing date of the invention to a person of ordinary skill in the art, it would have been obvious, having the teachings of Zemouri and Gutiérrez to use the method of using probabilities of unclassified item having a kth class to generate the output of Gutiérrez to implement the recurrent neural network system of Zemouri. The suggestion and/or motivation for doing so is to improve the performance of the system, as probabilistic classifiers help recurrent neural network systems handle ambiguous data.
However, Zemouri in view of Gutiérrez does not specifically disclose:
storing a plurality of qualified feature vectors in a plurality of computation columns of an associated memory device having rows and computation columns, wherein each of said plurality of qualified feature vectors describes one of said classified items;
transforming an output vector … to an output feature vector using said associative memory device
concurrently activating at least two rows of said associative memory device to concurrently in each of said plurality of computation columns;
concurrently activating at least two rows of said associative memory device to concurrently in each of said plurality of computation columns, compute a similarity score for each distance vector; and
creating said output feature vector as a vector of said plurality of computed similarity scores.
Ehrman teaches:
storing a plurality of qualified feature vectors in a plurality of computation columns of an associated memory device having rows and computation columns, wherein each of said plurality of qualified feature vectors describes one of said classified items; ([Ehrman, 0047] The calculated dataset C are stored in memory array 110. The MSB of all binary numbers in dataset C may be in the same row, and the LSB of all binary numbers in dataset C may be on the same row and so are all the bits in between. The rows where the dataset, MSB, and LSB are being stored are the first computation columns. As described in [Ehrman, 0048] “It may be appreciated that the vectors, as the whole dataset, are physically stored in rows in memory array 110, but for clarity drown as columns.”, the rows can be interpreted as columns)
transforming an output vector … to an output feature vector using said associative memory device ([Ehrman, 0021 and 0039] discloses that the k-Mins processor and store 130 are in associative memory array 140. [Ehrman, 0084] discloses generating RSP signal 1245 performing OR operation to value X (output vector). [Ehrman, 0085-0086] discloses that the k-Mins algorithms are used to calculate the distance between the A and D, where A and D are defined by vectors, and the vector
C
P
of m bits are generated (output feature vector). The distance is calculated using the cosine distance formula)
concurrently activating at least two rows of said associative memory device to concurrently in each of said plurality of computation columns, calculate a distance vector by subtracting said output embedding vector from one of said plurality of qualified feature vectors; ([Ehrman, 0086-0087] The distance vector C^P is calculated between object A and each object D^P in the dataset (subtracting) and is stored as a binary number in a large dataset C. The object A corresponds to the unclassified item. [Ehrman, 0022] The classified object features are the output feature vectors (activations) from the neural network. [Ehrman, 0044-0045] The rows are selected concurrently to be compared to the target rows k (associative memory) to select the vector
C
P
);
concurrently activating at least two rows of said associative memory device to concurrently in each of said plurality of computation columns, compute a similarity score for each distance vector; and ([Ehrman, 0044-0045] The rows are selected concurrently to be compared to the target rows k. The distance calculated based on the K-NN algorithm described in [Ehrman, 0005] is the similarity score for the vectors)
creating said output feature vector as a vector of said plurality of computed similarity scores. ([Ehrman, 0054] The Vector V which is computed based on the dataset C and k-Mins corresponds to the similarity score vector)
Before the effective filing date of the invention to a person of ordinary skill in the art, it would have been obvious, having the teachings of Zemouri, Gutiérrez and Ehrman to use the associative memory having rows and columns of Ehrman to implement the recurrent neural network system of Zemouri. The suggestion and/or motivation for doing so is to improve the efficiency of the system as associative memory can locate data in a single clock cycle [Ehrman, 0035].
Regarding claim 3, Zemouri in view of Gutiérrez teaches:
[Gutiérrez, page 249, left col, line 1-18] discloses calculating the probability that x belongs to class l (kth class). Additionally, [Gutiérrez, page 249, left col, last para, line 8-12] discloses applying the softmax transformation (nonlinear function) to the outputs of the network)
Zemouri in view of Gutiérrez does not specifically disclose:
concurrently activating at least two rows of said associative memory device to concurrently activate a
Ehrman teaches:
concurrently activating at least two rows of said associative memory device to concurrently activate a [Ehrman, 0044-0045] The rows are selected concurrently to be compared to the target rows k).
Regarding claim 4, Zemouri in view of Gutiérrez teaches:
wherein said nonlinear function is the SoftMax function. ([Gutiérrez, page 249, left col, line 1-18] discloses calculating the probability that x belongs to class l. Additionally, [Gutiérrez, page 249, left col, last para, line 8-12] discloses applying the softmax transformation (softmax layer converts output to a probability) to the outputs of the network)
Regarding claim 5, Zemouri in view of Gutiérrez teaches:
finding an extreme value in said probability distribution vector ([Gutiérrez, page 249, left col, line 14-25] discloses finding a maximum probability (extreme value) in the probability of x)
However, Zemouri in view of Gutiérrez does not specifically disclose:
finding an extreme value
Ehrman teaches:
finding an extreme value [Ehrman, 0005] The K-nearest neighbors algorithm calculates the similarity between an introduced object X (unclassified) and each and every objects in the dataset (classified items). [Ehrman, 0044] The k-Mins set, which are the smallest numbers in the dataset C, is determined. [Ehrman, 0091] The similarity calculation of Ehrman provides superior computation complexity of O(1).).
Regarding claim 6, Zemouri in view of Gutiérrez and further in view of Ehrman teaches concurrently activating at least two rows of said associative memory device to activate a K-nearest neighbors (KNN) function on said similarity score vector to provide k classified items most similar to said unclassified item ([Ehrman, 0005] The K-nearest neighbors algorithm calculates the similarity between an introduced object X (unclassified) and each and every object in the dataset (classified items). [Ehrman, 0004] shows that the objects in the dataset are classified. [Ehrman, 0044-0045] The rows are selected concurrently to be compared to the target rows k).
Regarding claim 7, Zemouri teaches:
A system for a recurrent neural network that processes a sequence of items, of which a new item at iteration t is an unclassified item and previous items are classified items, the system comprising: ([Zemouri, page 454, right col, line 3-10] discloses that the network can be used in two kind of application: regression and classification. [Zemouri, page 456, left col, line 1-11] discloses the recurrent radial basis neural network function. [Zemouri, page 456, 3.1. Looped neuron, line 1-17] discloses that the output of the function is generated iteratively for each instant t. The input
I
i
is the unclassified item and the output
x
i
(
t
)
are the classified items. The equation (13) indicates that the previous items
x
(
t
-
1
)
are classified, wherein the
x
(
t
-
1
)
is a classification output of
x
i
(
t
)
at time
t
-
1
)
a hidden layer computer to receive said input and to run said input in said recurrent neural network to compute a hidden layer vector; ([Zemouri, page 456, left col, line 1-11] discloses the recurrent radial basis neural network function. [Zemouri, page 456, 3.1. Looped neuron, line 1-17] discloses transforming an output vector of a hidden layer of the recurrent neural network to an output feature vector. The equation (13) indicates that the previous items
x
(
t
-
1
)
are classified at t-1 (previous neuron, which is a hidden layer), wherein the
x
(
t
-
1
)
is an output of
x
i
(
t
)
at time
t
-
1
. [Zemouri, page 457, right col, line 30-36] The hidden layer is the hidden layer computer that computes outputs based on the weights)
an output handler to transform an output vector of said hidden layer vector to an output feature vector, said output vector describing an unclassified item [Zemouri, page 456, left col, line 1-11] discloses the recurrent radial basis neural network function. [Zemouri, page 456, 3.1. Looped neuron, line 1-17] discloses transforming an output vector of a hidden layer of the recurrent neural network to an output feature vector. The equation (13) indicates that the previous items
x
(
t
-
1
)
are classified at t-1 (previous neuron, which is a hidden layer), wherein the
x
(
t
-
1
)
is an output of
x
i
(
t
)
at time
t
-
1
. [Zemouri, page 457, right col, line 30-36] The output layer is the output handler that computes outputs based on the weights between the hidden and the output layer)
computing an output embedding vector based on said output vector of the hidden layer; ([Zemouri, page 457, right col, lines 1-10] discloses calculating output embedding vector
y
(
t
)
based on the hidden layer output
x
j
(
t
)
and the prototype vector
μ
i
j
)
by subtracting said output embedding vector from one of said plurality of qualified feature vectors, ([Zemouri, page 454, left col, 2.1. RBF networks definition, line 4 – right col, line 4] The Euclidian distance is calculated for a given input vector x (qualified feature vectors) and the prototype vector
μ
j
. [Zemouri, page 457, last para, line 1-5] The prototypes are extracted from the output (
y
(
t
)
) of the looped neurons (recurrent neural network), therefore interpreted as the output embedding vector)
said input arranger, said hidden layer computer and said output handler operating for each iteration t of said recurrent neural network. ([Zemouri, page 456, 3.1. Looped neuron, line 1-17] discloses that the output of the function is generated iteratively for each instant t. Each neuron of the input layer is the input arranger. [Zemouri, page 457, right col, line 30-36] The hidden layer is the hidden layer computer that computes outputs based on the weights and the output layer is the output handler that computes outputs based on the weights between the hidden and the output layer)
However, Zemouri does not specifically disclose:
an associative memory device comprised of rows and computation columns
an input arranger to store information regarding said unclassified item in said computation columns of said associative memory device, to manipulate said information within said computation columns
said output feature vector describing the probabilities of said unclassified item having a kth class, said output handler to store a plurality of qualified feature vectors in a plurality of said computation columns
concurrently activating at least two rows of said associative memory device to concurrently in each of said plurality of computation columns calculate a distance vector by subtracting said output embedding vectors from one of said plurality of qualified feature vectors,
concurrently activating at least two rows of said associative memory device to concurrently in each of said plurality of computation columns compute a similarity score for one of said distance vectors, and
creating said output feature vector as a vector of said plurality of computed similarity scores
Gutiérrez teaches:
said output feature vector describing the probabilities of said unclassified item having a kth class ([Gutiérrez, page 249, left col, line 1-18] discloses calculating the probability that x belongs to class l (kth class). Additionally, [Gutiérrez, page 249, left col, last para, line 8-12] discloses applying the softmax transformation (softmax layer converts output to a probability) to the outputs of the network)
Before the effective filing date of the invention to a person of ordinary skill in the art, it would have been obvious, having the teachings of Zemouri and Gutiérrez to use the method of using probabilities of unclassified item having a kth class to generate the output of Gutiérrez to implement the recurrent neural network system of Zemouri. The suggestion and/or motivation for doing so is to improve the performance of the system, as probabilistic classifiers help recurrent neural network systems handle ambiguous data.
However, Zemouri in view of Gutiérrez does not specifically disclose:
an associative memory device comprised of rows and computation columns
an input arranger to store information regarding said unclassified item in said computation columns of said associative memory device, to manipulate said information within said computation columns
said output handler to store a plurality of qualified feature vectors in a plurality of said computation columns, wherein each of said plurality of qualified feature vectors describes one of said classified items, and to perform the following operations;
concurrently activating at least two rows of said associative memory device to concurrently in each of said plurality of computation columns calculate a distance vector by subtracting said output embedding vectors from one of said plurality of qualified feature vectors,
concurrently activating at least two rows of said associative memory device to concurrently in each of said plurality of computation columns compute a similarity score for one of said distance vectors, and
creating said output feature vector as a vector of said plurality of computed similarity scores
Ehrman teaches:
an associative memory device comprised of rows and computation columns ([Ehrman, 0041] The dataset is stored in the memory device comprised of rows and computation columns);
an input arranger to store information regarding said unclassified item in said computation columns of said associative memory device, to manipulate said information within said computation columns ([Ehrman, 0005] The K-nearest neighbors algorithm calculates the similarity between an introduced object X (unclassified) and each and every object in the dataset (classified items). [Ehrman, 0022] indicates that the output classified item features and output unclassified activations are stored in the rows and the columns)
said output handler to store a plurality of qualified feature vectors in a plurality of said computation columns, wherein each of said plurality of qualified feature vectors describes one of said classified items, and to perform the following operations; ([Ehrman, 0047] The calculated dataset C are stored in memory array 110. The MSB of all binary numbers in dataset C may be in the same row, and the LSB of all binary numbers in dataset C may be on the same row and so are all the bits in between. The rows where the dataset, MSB, and LSB are being stored are the first computation columns. As described in [Ehrman, 0048] “It may be appreciated that the vectors, as the whole dataset, are physically stored in rows in memory array 110, but for clarity drown as columns.”, the rows can be interpreted as columns)
concurrently activating at least two rows of said associative memory device to concurrently in each of said plurality of computation columns calculate a distance vector by subtracting said output embedding vectors from one of said plurality of qualified feature vectors, ([Ehrman, 0086-0087] The distance vector C^P is calculated between object A and each object D^P in the dataset (subtracting) and is stored as a binary number in a large dataset C. The object A corresponds to the unclassified item. [Ehrman, 0022] The classified object features are the output feature vectors (activations) from the neural network. [Ehrman, 0044-0045] The rows are selected concurrently to be compared to the target rows k (associative memory) to select the vector
C
P
)
concurrently activating at least two rows of said associative memory device to concurrently in each of said plurality of computation columns compute a similarity score for one of said distance vectors, and ([Ehrman, 0044-0045] The rows are selected concurrently to be compared to the target rows k. The distance calculated based on the K-NN algorithm described in [Ehrman, 0005] is the similarity score for the vectors);
creating said output feature vector as a vector of said plurality of computed similarity scores, ([Ehrman, 0054] The Vector V which is computed based on the dataset C and k-Mins corresponds to the similarity score vector)
Before the effective filing date of the invention to a person of ordinary skill in the art, it would have been obvious, having the teachings of Zemouri, Gutiérrez and Ehrman to use the associative memory having rows and columns of Ehrman to implement the recurrent neural network system of Zemouri. The suggestion and/or motivation for doing so is to improve the efficiency of the system as associative memory can locate data in a single clock cycle [Ehrman, 0035].
Regarding claim 8, Zemouri in view of Gutiérrez and further in view of Ehrman teaches:
comprising said input arranger to reduce the dimension of said information ([Zemouri, page 455, left col, Fig. 1, Fig. 2, and line 2-3] discloses reducing conflicting zone between prototype vectors which reduces the dimension of the prototype).
Regarding claim 9, Zemouri teaches:
wherein said output handler also comprises a linear module and a nonlinear module ([Zemouri, page 460, left col, line 4-12] The network is composed of a linear neuron (linear module) and a looped neuron (a nonlinear module) ).
Regarding claim 10, Zemouri in view of Gutiérrez and further in view of Ehrman teaches:
nonlinear module implements a SoftMax function to create a probability distribution vector from a vector of said similarity scores. ([Gutiérrez, page 249, left col, line 1-18] discloses calculating the probability that x belongs to class l. Additionally, [Gutiérrez, page 249, left col, last para, line 8-12] discloses applying the softmax transformation (softmax layer converts output to a probability) to the outputs of the network)
Regarding claim 12, Zemouri in view of Gutiérrez and further in view of Ehrman teaches:
wherein said nonlinear module is a k-nearest neighbors module to provide k classified items most similar to said unclassified item ([Ehrman, 0087] The distance vector C^P is calculated between object A and each object D^P in the dataset and is stored as a binary number in a large dataset C. The object A corresponds to the unclassified item. [Ehrman, 0044-0045] The rows are selected concurrently to be compared to the target rows k. The distance calculated based on the K-NN algorithm described in [Ehrman, 0005] is the similarity score for the vectors.).
Regarding claim 13, Zemouri in view of Gutiérrez and further in view of Ehrman teaches:
wherein said linear module is a distance transformer to generate said similarity scores ([Ehrman, 0085] The cosine distance which is used to generate the distance dataset C is the distance calculator. [Ehrman, 0054] The Vector V which contains the scalar is computed based on the dataset C and k-Mins corresponds to the similarity score vector).
Regarding claim 14, Zemouri in view of Gutiérrez and further in view of Ehrman teaches:
wherein said distance transformer comprises a vector adjuster and a distance calculator ([Ehrman, 0085] The cosine distance which is used to generate the distance dataset C is the distance calculator. [Ehrman, 0047] The vector adjuster is merely an output handler which stores the calculated distance.).
Regarding claim 15, Zemouri in view of Gutiérrez and further in view of Ehrman teaches:
said distance transformer to store columns of an adjustment matrix in first computation columns of said memory array, and to distribute said hidden layer vector to each computation column, and said vector adjuster to compute an output feature vector within said first computation columns ([Ehrman, 0047] The calculated dataset C are stored in memory array 110. The MSB of all binary numbers in dataset C may be in the same row, and the LSB of all binary numbers in dataset C may be on the same row and so are all the bits in between. The rows where the dataset, MSB, and LSB are being stored are the first computation columns. As described in [Ehrman, 0048] “It may be appreciated that the vectors, as the whole dataset, are physically stored in rows in memory array 110, but for clarity drown as columns.”, the rows can be interpreted as columns.).
Regarding claim 16, Zemouri in view of Gutiérrez and further in view of Ehrman teaches:
said distance transformer to initially store columns of an output embedding matrix in second computation columns of said associative memory array and to distribute said output feature vector to all said second computation columns, and said distance calculator to compute a distance vector within said second computation columns ([Ehrman, 0049] Vector D is the inverse value of the dataset C that is stored in the column C, and processed by K-Mins algorithm 120. After the MSB and LSB are processed, the K-Min processor processes another row i in memory array to generate the Vector D. The row i is interpreted as the second computation columns. As described in [Ehrman, 0048] “It may be appreciated that the vectors, as the whole dataset, are physically stored in rows in memory array 110, but for clarity drown as columns.”, the rows can be interpreted as columns.).
Claim 2 is rejected under 35 U.S.C. 103 over Zemouri in view of Gutiérrez in view of Ehrman and further in view of Tran (US 10748630 B2, hereinafter ‘Tran’).
Regarding claim 2, Zemouri in view of Gutiérrez and further in view of Ehrman teaches the method of claim 1.
Zemouri in view of Gutiérrez and further in view of Ehrman does not specifically disclose:
reducing a size of an input vector of said recurrent neural network by concurrently multiplying said input vector by a plurality of columns of an input embedding matrix.
Tran teaches:
reducing a size of an input vector of said recurrent neural network by concurrently multiplying said input vector by a plurality of columns of an input embedding matrix ([Tran, column 15, line 51-65] “(99) Efficiency can be increased, and the total number of inputs reduced, by reconfiguring the memory arrays as shown in FIG. 31. Specifically, the input lines of the memory array are shifted periodically to another row or column, thus reducing the unused portions of the array, and therefore reducing the number of repeated input lines over the array needed to perform the scan. Specifically, in the case of the present example where the shift X=2, the arrows indicate that each input line periodically shifts over by two rows or two columns, transforming the widely spaced apart memory cell utilization trapezoidal shapes to closely spaced memory cell utilization rectangular shapes. While extra space between memory cell portions are needed for wire bundles to implement this shift, the number of inputs needed in the memory cell array is greatly reduced (only 5n+6)”).
Before the effective filing date of the invention to a person of ordinary skill in the art, it would have been obvious, having the teachings of Ehrman, Jaech and Tran to use the process of reducing the dimension of input matrices of Tran to implement the system for a neural network of Ehrman and Jaech. The suggestion and/or motivation for doing so is to efficiently store and process the neural network as smaller input data takes less time to process.
Claim 11 is rejected under 35 U.S.C. 103 over Zemouri in view of Gutiérrez in view of Ehrman and further in view of Xiao (US 20170323636 A1, hereinafter ‘Xiao’).
Regarding claim 11, Zemouri in view of Gutiérrez and further in view of Ehrman teaches the system of claim 10.
Zemouri in view of Gutiérrez and further in view of Ehrman does not specifically teach:
The system comprising an extreme value finder to find an extreme value in said probability distribution vector.
Xiao teaches:
comprising an extreme value finder to find an extreme value in said probability distribution vector. ([Xiao, 0058] The softmax function is an operation that maps the vector W′.sub.2(s′(W′.sub.1(z))) to a vector of probabilities between 0 and 1 by taking the exponential of each coordinate and dividing it by a normalizing factor equal to the sum of these exponentials. In practice, a probability distribution is obtained and the term giving the maximum probability is chosen. The maximum probability is the extreme value.).
Before the effective filing date of the invention to a person of ordinary skill in the art, it would have been obvious, having the teachings of Ehrman, Jaech, and Xiao, to use the process of finding a value from probability distribution vector of Xiao to implement the system for a neural network of Ehrman and Jaech. The suggestion and/or motivation for doing so is to improve the accuracy of the system, as the largest value of the probability function corresponds to the best matching result.
Claim 17, 19-21 are rejected under 35 U.S.C. 103 as being unpatentable over Zemouri in view of Gutiérrez in view of Ehrman and further in view of Xiao.
Regarding claim 17, Ehrman teaches:
A method for comparing an unclassified language item described by an unclassified vector of features to a plurality of classified language items, each described by a classified vector of features, the method comprising ([Ehrman, 0091] The K-NN process may be utilized to process language items. [Ehrman, 0005] The K-nearest neighbors algorithm calculates the similarity between an introduced object X (unclassified) and each and every object in the dataset (classified items) ):
storing each classified vector of features in a computation column of an associative memory device, said associative memory device having a plurality of computation columns ([Ehrman, 0005] The K-nearest neighbors algorithm calculates the similarity between an introduced object X (unclassified) and each and every object in the dataset (classified items). [Ehrman, 0048] indicates that the dataset are stored in the rows and the columns.);
concurrently activating at least two rows of said associative memory device to concurrently in each of said plurality of computation columns, calculate a distance vector by subtracting said unclassified vector from each said classified vector ([Ehrman, 0087] The distance vector C^P is calculated between object A and each object D^P in the dataset (subtracting) and is stored as a binary number in a large dataset C. The object A corresponds to the unclassified item. [Ehrman, 0022] The classified object features are the output feature vectors (activations) from the neural network. [Ehrman, 0044-0045] The rows are selected concurrently to be compared to the target rows k);
concurrently activating at least two rows of said associative memory device to concurrently in each of said plurality of computation columns, compute a distance scalar for each distance vector, each distance scalar providing a similarity score between said unclassified item and one of said plurality of classified items thereby creating a similarity score vector comprising a plurality of distance scalars ([Ehrman, 0044-0045] The rows are selected concurrently to be compared to the target rows k. The distance calculated based on the K-NN algorithm described in [Ehrman, 0005] is the similarity score for the vectors. [Ehrman, 0054] The Vector V which contains the scalar is computed based on the dataset C and k-Mins corresponds to the similarity score vector.); and
concurrently activating at least two rows of said associative memory device to concurrently in each of said plurality of computation columns, activate a Ehrman, 0054] The Vector V which contains the scalar is computed based on the dataset C and k-Mins corresponds to the similarity score vector. The calculation of vector V is performed by a function.).
Ehrman does not specifically disclose activate a nonlinear function on an element of said similarity score vector to create a probability distribution element for one of said plurality of classified language items thereby creating a probability distribution vector comprising a plurality of probability distribution elements.
Xiao teaches activate a nonlinear function on an element of said similarity score vector to create a probability distribution element for one of said plurality of classified language items thereby creating a probability distribution vector comprising a plurality of probability distribution elements ([Xiao, 0058] “The softmax function is an operation that maps the vector W′.sub.2(s′(W′.sub.1(z))) to a vector of probabilities between 0 and 1 by taking the exponential of each coordinate and dividing it by a normalizing factor equal to the sum of these exponentials. In practice, a probability distribution is obtained and the term giving the maximum probability is chosen”. According to the ABSTRACT of Xiao, the system of Xiao processes input text sequence which is language items. Performing the operation in the computation columns is being taught by the Ehrman above.).
Before the effective filing date of the invention to a person of ordinary skill in the art, it would have been obvious, having the teachings of Ehrman, and Xiao to use the process of comparing similarity scores of classified and unclassified data of Xiao to implement the system for a neural network of Ehrman. The suggestion and/or motivation for doing so is to generate more accurate classification result as probability given for each vector is the probability that the classification is correct.
Regarding claim 19, Ehrman in view of Xiao teaches:
wherein said nonlinear function is the SoftMax function ([Xiao, 0058] “The softmax function is an operation that maps the vector W′.sub.2(s′(W′.sub.1(z))) to a vector of probabilities between 0 and 1 by taking the exponential of each coordinate and dividing it by a normalizing factor equal to the sum of these exponentials. In practice, a probability distribution is obtained and the term giving the maximum probability is chosen”, the SoftMax function is non-linear function.).
Regarding claim 20, Ehrman teaches:
comprising finding an extreme value in said probability distribution vector to find a classified item most similar to said unclassified language item ([Ehrman, 0005] The K-nearest neighbors algorithm calculates the similarity between an introduced object X (unclassified) and each and every object in the dataset (classified items). [Ehrman, 0004] shows that the objects in the dataset are classified. [Ehrman, 0044-0045] The rows are selected concurrently to be compared to the target rows k.).
Regarding claim 21, Ehrman teaches:
activating a K-nearest neighbors (KNN) function on said similarity score vector to provide k classified language items most similar to said unclassified language item ([Ehrman, 0005] The K-nearest neighbors algorithm calculates the similarity between an introduced object X (unclassified) and each and every object in the dataset (classified items). [Ehrman, 0004] shows that the objects in the dataset are classified. [Ehrman, 0044-0045] The rows are selected concurrently to be compared to the target rows k.).
Response to Arguments
Response to Arguments under 35 U.S.C. 103
Arguments: Applicant asserts that (a) the fields of the references are distinct because Zemouri and Gutiérrez are concerned with the mathematical models and architectures of neural networks, while Ehrman teaches finding the k smallest or largest values from a static, pre-loaded dataset in constant time [Remarks, page 10], (b) Ehrman is concerned with finding the k smallest or largest values from a static, pre-loaded dataset in constant time, and claims 1 and 7 recite a process of ‘vector subtraction’ which is a mathematical operation, not a Boolean filtering step, (c) Ehrman provides no teaching for how its associative memory could be made to perform parallel vector subtraction on entire numerical vectors [Remarks, pages 10-11] and (d) Xia still does not teach “concurrently activating at least two rows of said associative memory device … classified vector.”
Examiner’s Response: Examiner respectfully disagrees. Regarding (a), all Zemouri, Gutiérrez, and Ehrman disclose the same fields of performing pattern classification (recognition) using a neural network. [Ehrman, 0003] discloses that “Data mining is a computational process of discovering patterns in large datasets” and it teaches the method of performing pattern recognition more efficiently using an associative memory [Ehrman, 0006]. [Gutiérrez, page 246, left col, I. INTRODUCTION] and [Zemouri, page 453, right col, lines 5-8] also discloses that the methods/algorithms are performing pattern classification. Therefore, it is obvious for someone who knows the art to introduce Ehrman’s associative memory method to improve the methods of Gutiérrez and Zemouri.
Regarding (b), The applicant alleges that Ehrman is concerned with finding the k smallest or largest values from a static, pre-loaded dataset in constant time, and claims 1 and 7 recite a process of ‘vector subtraction’ which is a mathematical operation, not a Boolean filtering step. However, Ehrman disclose not only Boolean operations, but also arithmetic operations using the hidden layer representations to generate the final results. Ehrman, paragraph [0086], discloses calculating distance vectors
C
P
of m bits, wherein the cosine distance is calculated using the formula disclosed in the paragraph. Also, 'transformation' can have multiple meanings, including arithmetic transformations and all kinds of modifications to vectors (e.g., addition, multiplication, etc.).
Regarding (c), the [0044] and [0045] specifically disclose that calculation of distance vectors C^P is performed in parallel, by concurrently selecting rows
C
P
where
C
i
P
is 0. Lastly, regarding (d), the arguments regarding Xia is not persuasive because Ehrman discloses the amended subject matter and the arguments (a), (b), and (c) are not persuasive.
Accordingly, arguments regarding claims 1, 7, and 17 are not persuasive.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Dannihelka et al., “Associative Long Short-Term Memory” (This prior art is pertinent because it discloses combining associative memory technology with LSTM to reduce noise)
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JUN KWON whose telephone number is (571)272-2072. The examiner can normally be reached Monday – Friday 7:30AM – 4:30PM ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Abdullah Kawsar can be reached at (571)270-3169. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JUN KWON/Examiner, Art Unit 2127
/ABDULLAH AL KAWSAR/Supervisory Patent Examiner, Art Unit 2127
Prosecution Insights