DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of Claims
Claims 1-18 are pending and examined herein.
Claims 3, 6, 9, 12, 15, and 18 are rejected under 35 U.S.C. 112(b).
Claims 1-18 are rejected under 35 U.S.C. 101.
Claims 1, 5, 7, 11, 13, and 17 are rejected under 35 U.S.C. 102.
Claims 2-4, 6, 8-10, 12, 14-16, and 18 are rejected under 35 U.S.C. 103.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 3, 9, and 15 recite the limitation "the mean-squared approximation error". There is insufficient antecedent basis for this limitation in the claim.
Claims 6, 12, and 18 recite the limitation "the product". There is insufficient antecedent basis for this limitation in the claim.
Claims 6, 12, and 18 recite the limitation "the output product of the vectors". There is insufficient antecedent basis for this limitation in the claim.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-18 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
MPEP § 2109(III) sets out steps for evaluating whether a claim is drawn to patent-eligible subject
matter. The analysis of claims 1-18, in accordance with these steps, follows.
Step 1 Analysis:
Step 1 is to determine whether the claim is directed to a statutory category (process, machine,
manufacture, or composition of matter. Claims 1-6 are directed to a process, claims 7-12 are directed to a machine, and claims 13-18 are directed to an article of manufacture. All claims are directed to statutory categories and analysis proceeds.
Step 2A Prong One, Step 2A Prong Two, and Step 2B Analysis:
Step 2A Prong One asks if the claim recites a judicial exception (abstract idea, law of nature, or natural phenomenon). If the claim recites a judicial exception, analysis proceeds to Step 2A Prong Two, which asks if the claim recites additional elements that integrate the abstract idea into a practical application. If the claim does not integrate the judicial exception, analysis proceeds to Step 2B, which asks if the claim amounts to significantly more than the judicial exception. If the claim does not amount to significantly more than the judicial exception, the claim is not eligible subject matter under 35 U.S.C. 101.
None of the claims represent an improvement to technology.
Regarding claim 1, the following are abstract ideas:
decomposing, by a computer processor, a tensor into a plurality of low-rank tensor factors; and (Decomposing a tensor into tensor factors is a mathematical calculation, which is a mathematical concept.)
generating, … each low-rank tensor factor; and (Generating a tensor factor can be practically performed in the human mind, i.e. solving an equation to generate the tensor factors. This is a mental process.)
The following claim elements are additional elements which, taken alone or in combination with the other additional elements, do not integrate the judicial exception into a practical application nor amount to significantly more than the judicial exception:
A method for low-rank decomposition of a tensor, the method comprising: (This is an intended use statement which does not limit the claim.)
by the computer processor … by a corresponding neural network; and (This recites generic computer and machine learning components. This amounts to mere instructions to apply an exception.)
feeding, by the computer processor, each neural network with a corresponding plurality of input tensors. (This recites generic machine learning processes and components. This amounts to mere instructions to apply an exception.)
Regarding claim 2, the rejection of claim 1 is incorporated herein. The following claim elements are additional elements which, taken alone or in combination with the other additional elements, do not integrate the judicial exception into a practical application nor amount to significantly more than the judicial exception:
further comprising training the neural network is trained in a self-supervised manner. (This recites generic machine learning processes and components. This amounts to mere instructions to apply an exception.)
Regarding claim 3, the rejection of claim 1 is incorporated herein. Further, the following is an abstract idea:
training the neural network to minimize the mean-squared approximation error. (This recites the mathematical calculation of minimizing a function. This is a mathematical concept.)
Regarding claim 4, the rejection of claim 1 is incorporated herein. Further, the following is an abstract idea:
optimizing, by the computer processor, the parameters of each corresponding neural network using a corresponding stochastic gradient descent. (This recites the mathematical calculation of optimizing a function. This is a mathematical concept.)
Regarding claim 5, the rejection of claim 1 is incorporated herein. Further, the following is an abstract idea:
wherein decomposing the tensor comprises decomposing the tensor as the product of the low-rank tensor factors. (This describes a mathematical equation, which is a mathematical concept.)
Regarding claim 6, the rejection of claim 4 is incorporated herein. The following is an abstract idea:
wherein the tensor is a matrix, the low-rank tensor factors are vectors, and the product is the outer product of the vectors. (As in claim 1, the decomposition of the tensor/matrix into low-rank tensor factors is a mathematical concept; specifying a matrix and vectors is a continuation of the abstract idea. A product is a mathematical calculation, which is a mathematical concept.)
Regarding claim 7, the following claim elements are additional elements which, taken alone or in combination with the other additional elements, do not integrate the judicial exception into a practical application nor amount to significantly more than the judicial exception:
A system for low-rank decomposition of a tensor, comprising: (This is an intended use statement which does not limit the claim.)
a database; and (This limitation recites generic computer components; this amounts to mere instructions to apply an exception.)
a computer processor, wherein the computer processor comprises functionality for: a database; and (This limitation recites generic computer components and functions; this amounts to mere instructions to apply an exception.)
The remainder of claim 7 recites substantially similar subject matter to claim 1 and is rejected with the same rationale, mutatis mutandis.
Claims 8-12 recite substantially similar subject matter to claims 2-6 respectively and are rejected with the same rationale, mutatis mutandis.
Regarding claim 13, the following claim elements are additional elements which, taken alone or in combination with the other additional elements, do not integrate the judicial exception into a practical application nor amount to significantly more than the judicial exception:
A non-transitory computer readable medium storing instructions executable by a computer processor, the instructions comprising functionality for: (This limitation recites generic computer components and functions; this amounts to mere instructions to apply an exception.)
The remainder of claim 13 recites substantially similar subject matter to claim 1 and is rejected with the same rationale, mutatis mutandis.
Claims 14-18 recite substantially similar subject matter to claims 2-6 respectively and are rejected with the same rationale, mutatis mutandis.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1, 5, 7, 11, 13, and 17 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Wu (“A Biased Deep Tensor Factorization Network For Tensor Completion”, 2021).
Regarding claim 1, Wu teaches
A method for low-rank decomposition of a tensor, the method comprising: (Page 2 states "In this paper, the deep learning technique is applied to tensor decomposition. Generally, the horizontal and lateral corresponding elements of data tensor are embedded into the latent matrix space respectively. Since the data are usually located on a low dimensional manifold, the dimension of the embedding tensor is generally much smaller than the order of the data tensor." As the embedding is low-dimensional, the tensor decomposition is low-rank.)
decomposing, by a computer processor, a tensor into a plurality of low-rank tensor factors; and (Page 5 states "In the training stage, each observation point is transformed by its corresponding horizontal and lateral tensors to obtain the potential feature tensors, which are then used as the input of the multi-layer perceptron layer, and fused by the horizontal and lateral potential feature tensors of the bilinear pooling layer." The feature tensors, horizonal tensor
U
→
i
N
and lateral tensor
V
→
i
N
are interpreted as the low-rank tensor factors, as page 6 states "After multi-layer perceptron mapping of the potential eigenvalues of the horizontal and lateral tensors, the next step is to fuse the horizontal and lateral network outputs to get the predicted model value, using the
*
M
-product." Page 6 states "After multi-layer perceptron mapping of the potential eigenvalues of the horizontal and lateral tensors, the next step is to fuse the horizontal and lateral network outputs to get the predicted model value, using the
*
M
-product.” Page 6 states
x
i
j
=
X
i
,
j
,
:
=
H
→
*
M
σ
U
→
i
N
⊙
V
→
i
N
,
(
9
)
where the notation
⊙
represents Hadamard product, and
H
is the unknown parameter to be trained. The
*
M
-product, according to page 4 is
C
=
A
^
*
M
B
^
=
A
∆
B
×
3
M
-
1
,
C
∈
R
l
×
m
×
n
. Therefore,
U
→
i
N
and
V
→
i
N
are the tensor factors. Page 8 states "All our methods have been implemented for tensor completion on the Pycharm 2018.2.5 x64 platform equipped with an Intel Core i5-8300H CPU, 8GB of RAM and python 3.7.4." Hereinafter, this is considered to be the explanation for “by the computer processor”.)
generating, by the computer processor, each low-rank tensor factor by a corresponding neural network; and (Figure 1 on page 5 shows the neural network architecture. The horizontal slice layers and the corresponding layer bias is interpreted as the neural network for tensor factor
U
→
i
N
and the lateral slice layers and the corresponding layer bias is interpreted as the neural network for tensor factor
V
→
i
N
.)
feeding, by the computer processor, each neural network with a corresponding plurality of input tensors. (Page 5 states "Secondly, the processed tensor is transformed horizontally (U) and laterally (V). The two transformed tensors are used as inputs." The transformed tensors are interpreted as the plurality of input tensors, which are fed to the neural networks as shown in Figure 1, page 5.)
Regarding claim 5, the rejection of claim 1 is incorporated herein. Wu teaches
wherein decomposing the tensor comprises decomposing the tensor as the product of the low-rank tensor factors. (Page 6 states "After multi-layer perceptron mapping of the potential eigenvalues of the horizontal and lateral tensors, the next step is to fuse the horizontal and lateral network outputs to get the predicted model value, using the
*
M
-product.” Page 6 states "After multi-layer perceptron mapping of the potential eigenvalues of the horizontal and lateral tensors, the next step is to fuse the horizontal and lateral network outputs to get the predicted model value, using the
*
M
-product.” Page 6 states
x
i
j
=
X
i
,
j
,
:
=
H
→
*
M
σ
U
→
i
N
⊙
V
→
i
N
,
(
9
)
where the notation
⊙
represents Hadamard product, and
H
is the unknown parameter to be trained. The
*
M
-product, according to page 4 is
C
=
A
^
*
M
B
^
=
A
∆
B
×
3
M
-
1
,
C
∈
R
l
×
m
×
n
. Therefore,
U
→
i
N
and
V
→
i
N
are the tensor factors, and the tensor is decomposed as the product of the tensor factors.)
Regarding claim 7, Wu teaches
A system for low-rank decomposition of a tensor, comprising: (Page 2 states "In this paper, the deep learning technique is applied to tensor decomposition. Generally, the horizontal and lateral corresponding elements of data tensor are embedded into the latent matrix space respectively. Since the data are usually located on a low dimensional manifold, the dimension of the embedding tensor is generally much smaller than the order of the data tensor.")
a database; and ("To show the advantages of BDTFN for handling with large-scale and high-dimensional traffic data, we particularly choose two publicly available data sets collected from California transportation system (i.e., PeMS) as our benchmark data sets." PeMS is interpreted as the database.)
a computer processor, wherein the computer processor comprises functionality for: (Page 8 states "All our methods have been implemented for tensor completion on the Pycharm 2018.2.5 x64 platform equipped with an Intel Core i5-8300H CPU, 8GB of RAM and python 3.7.4." The CPU is interpreted as the processor.)
The remainder of claim 7 recites substantially similar subject matter to claim 1 and is rejected with the same rationale, mutatis mutandis.
Claim 11 recites substantially similar subject matter to claim 5 and is rejected with the same rationale, mutatis mutandis.
Regarding claim 13, Wu teaches
A non-transitory computer readable medium storing instructions executable by a computer processor, the instructions comprising functionality for: (Page 8 states "All our methods have been implemented for tensor completion on the Pycharm 2018.2.5 x64 platform equipped with an Intel Core i5-8300H CPU, 8GB of RAM and python 3.7.4." The CPU is interpreted as the processor. One of ordinary skill in the art would understand that a non-transitory computer readable medium storing instructions executable by a computer processor with functionality for the method is necessary for the processor to execute the method.)
The remainder of claim 13 recites substantially similar subject matter to claim 1 and is rejected with the same rationale, mutatis mutandis.
Claim 17 recites substantially similar subject matter to claim 1 and is rejected with the same rationale, mutatis mutandis.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 2, 8, and 14 is/are rejected under 35 U.S.C. 103 as being unpatentable over Wu (“A Biased Deep Tensor Factorization Network For Tensor Completion”, 2021) as applied to claim 1 above, and further in view of Yang (“Augmented Tensor Decomposition with Stochastic Alternating Optimization”, 2021).
Regarding claim 2, the rejection of claim 1 is incorporated herein. Wu teaches
the neural network (Figure 1 on page 5 shows the neural network architecture. The horizontal slice layers and the corresponding layer bias is interpreted as the neural network for tensor factor
U
→
i
N
and the lateral slice layers and the corresponding layer bias is interpreted as the neural network for tensor factor
V
→
i
N
.)
However, Yang—directed to analogous art—teaches
training [the model] is trained in a self-supervised manner. (Page 3, Section 3: Augmented Tensor Decomposition, states "We show our model in Figure 1. The design is inspired by the recent popularity of self-supervised learning. To exploit the latent class assignment, we introduce data augmentation into CPD model and design self-supervised loss to constrain the learned low-rank features (i.e., the coefficient vectors).")
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to combine the teachings of Wu and Yang because, as stated by Wu on page 3, "In this paper, we are motivated to improve the CPD model by exploiting the latent classes and learn good bases (i.e., rank-one components) to provide better features for classification." Page 3 also states "We show our model in Figure 1. The design is inspired by the recent popularity of self-supervised learning. To exploit the latent class assignment, we introduce data augmentation into CPD model and design self-supervised loss to constrain the learned low-rank features (i.e., the coefficient vectors)." Additionally, page 10 states, "We show that by explicitly contrasting similar and dissimilar samples, the decomposition results are more aligned with downstream classification. "
Although the model taught by Yang is not a neural network, one of ordinary skill in the art would be able to use the loss function taught by Yang for a neural network as taught by Wu.
Claims 8 and 14 recite substantially similar subject matter to claim 2 and are rejected with the same rationale, mutatis mutandis.
Claim(s) 3, 9, and 15 is/are rejected under 35 U.S.C. 103 as being unpatentable over Wu (“A Biased Deep Tensor Factorization Network For Tensor Completion”, 2021) as applied to claim 1 above, and further in view of Goodfellow (“Chapter 5: Machine Learning Basics”, 2016), hereinafter “Goodfellow-5”.
Regarding claim 3, the rejection of claim 1 is incorporated herein. Wu teaches
the neural network (Figure 1 on page 5 shows the neural network architecture. The horizontal slice layers and the corresponding layer bias is interpreted as the neural network for tensor factor
U
→
i
N
and the lateral slice layers and the corresponding layer bias is interpreted as the neural network for tensor factor
V
→
i
N
.)
Wu does not appear to explicitly teach
training the [model] to minimize the mean-squared approximation error.
However, Goodfellow-5, directed to analogous art—teaches
training the [model] to minimize the mean-squared approximation error. (Page 127, Section 5.4.4 ‘Trading off Bias and Variance to Minimize Mean Squared Error’ states "For example, imagine that we are interested in approximating the function shown in figure 5.2 and we are only offered the choice between a model with large bias and one that suffers from large variance. How do we choose between them? The most common way to negotiate this trade-off is to use cross-validation. Empirically, cross-validation is highly successful on many real-world tasks. Alternatively, we can also compare the mean squared error (MSE) of the estimates:”.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to combine the teachings of Wu and Goodfellow-5 because, as Goodfellow-5 states on page 127, "The MSE measures the overall expected deviation—in a squared error sense— between the estimator and the true value of the parameter θ. As is clear from equation 5.54, evaluating the MSE incorporates both the bias and the variance. Desirable estimators are those with small MSE and these are estimators that manage to keep both their bias and variance somewhat in check."
Although the model taught by Goodfellow-5 is not a neural network, one of ordinary skill in the art would be able to use the optimization taught by Goodfellow-5 for a neural network as taught by Wu.
Claims 9 and 15 recite substantially similar subject matter to claim 3 and are rejected with the same rationale, mutatis mutandis.
Claim(s) 4, 10, and 16 is/are rejected under 35 U.S.C. 103 as being unpatentable over Wu (“A Biased Deep Tensor Factorization Network For Tensor Completion”, 2021) as applied to claim 1 above, and further in view of Goodfellow (“Chapter 8. Optimization for Training Deep Models”, 2016), hereinafter “Goodfellow-8”.
Regarding claim 4, the rejection of claim 1 is incorporated herein. Wu does not appear to explicitly teach
optimizing, by the computer processor, the parameters of each corresponding neural network using a corresponding stochastic gradient descent.
However, Goodfellow—directed to analogous art—teaches
optimizing, by the computer processor, the parameters of each corresponding neural network using a corresponding stochastic gradient descent. (Page 290 states "Stochastic gradient descent (SGD) and its variants are probably the most used optimization algorithms for machine learning in general and for deep learning in particular." As it is for deep learning, SGD is for a neural network. Page 291, Algorithm 8.1 shows that the parameter
θ
is optimized.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to combine the teachings of Wu and Goodfellow-8 because, as Goodfellow-8 states on page 292, "The most important property of SGD and related minibatch or online gradient-based optimization is that computation time per update does not grow with the number of training examples. This allows convergence even when the number of training examples becomes very large. For a large enough dataset, SGD may converge to within some fixed tolerance of its final test set error before it has processed the entire training set."
Claims 10 and 16 recite substantially similar subject matter to claim 4 and are rejected with the same rationale, mutatis mutandis.
Claim(s) 6, 12, and 18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Wu (“A Biased Deep Tensor Factorization Network For Tensor Completion”, 2021) as applied to claim 1 above, and further in view of Ma (“BDMF: A Biased Deep Matrix Factorization Model for Recommendation”, 2019) and Wang (“Nonnegative Matrix Factorization: A Comprehensive Review”, 2013).
Regarding claim 6, the rejection of claim 4 is incorporated herein. Wu does not appear to explicitly teach
wherein the tensor is a matrix, the low-rank tensor factors are vectors, and the product is the outer product of the vectors.
However, Ma—directed to analogous art—teaches
wherein the tensor is a matrix, the low-rank tensor factors are vectors, and (Page 1041 states "The input section of BDMF consists of high-dimensional feature vectors for user u and item v, denoted by u and v, respectively. We first construct a user-item interaction matrix Y according to Equation (1), and then take u from matrix Y and v from matrix
Y
T
" The matrix is interpreted as the tensor. Page 1041 states "The projection section maps user vector and item vector from the input section into two dense vector spaces and trains the two networks separately. Finally,
u
and
v
are mapped to low-dimensional vectors in a latent space, as shown in Equations (9) and (10)."
u
and
v
are interpreted as the low-rank tensor factors.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to combine the teachings of Wu and Ma because, as stated by Ma on Page 1040 "As one of the most effective CF method, matrix factorization has been widely studied." Additionally, Wu states on page 3, "In this paper, a Biased Deep Tensor Factorization Network (BDTFN) is proposed based on deep learning and tensor decomposition, inspired by the idea of a Biased Deep Matrix Factorization (BDMF) model [32]."
The combination of Wu and Ma does not appear to explicitly teach
the product is the outer product of the vectors.
However, Wang—directed to analogous art—teaches
the product is the outer product of the vectors. (Page 1339 states "Using the bilinear model, complete NMF can be rewritten as linear combination of rank-one nonnegative matrices expressed by
X
=
∑
i
=
1
L
U
⋅
i
V
i
⋅
=
∑
i
=
1
L
U
⋅
i
∘
V
i
⋅
T
where
U
⋅
i
is the ith column vector of
U
while
V
i
⋅
is the ith row vector of
V
, and
∘
denotes the outer product of two vectors.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to combine the teachings of Wu and Ma with the factorization of Wang because as stated by Wang on page 1336, "By contrast, a new paradigm of factorization—Nonnegative Matrix Factorization (NMF), which incorporates the nonnegativity constraint and thus obtains the parts-based representation as well as enhancing the interpretability of the issue correspondingly, was initiated by Paatero and Tapper [1], [2] together with Lee and Seung [3], [4]."
Claims 12 and 18 recite substantially similar subject matter to claim 6 and are rejected with the same rationale, mutatis mutandis.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JESSICA THUY PHAM whose telephone number is (571)272-2605. The examiner can normally be reached Monday - Friday, 9 A.M. - 5:00 P.M..
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Li Zhen can be reached at (571) 272-3768. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/J.T.P./Examiner, Art Unit 2121
/Li B. Zhen/Supervisory Patent Examiner, Art Unit 2121