DETAILED ACTION
A summary of this action:
Claims 1-14 have been presented for examination.
This action is non-Final.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea of a mental process or mathematical concept without significantly more.
Step 1: Claims 1-9 are directed to a device or an apparatus, which is a system and is a statutory invention. Claims 10-14 are directed to a method, which is a process and is a statutory category invention. Therefore, claims 1-14 are directed to patent eligible categories of invention.
Claim 1
Step 2A, Prong 1: Independent claims 1, 10 and 15, as drafted, are a process that, under its broadest reasonable interpretation, cover performance of the limitation in the mind but for the recitation of generic computer components. That is, other than reciting “device,” “computer,” and “non-transitory computer readable medium,” nothing in the claim element precludes the step from practically being performed in the mind.
Independent claims 1, 10, and 15 similarly recite determine a function in response to receiving an input relating, the function corresponding to the local model, which is an abstract idea and covers mental processes of assessing a function corresponding to a local model, as described in [0006] of the specification, because the claims are derived from Mental Processes based on concepts performed in the human mind or with the aid of pencil and paper.
Independent claim 1, recites estimation based on a refined regularization term of the local model, which is an abstract idea and covers mental processes of assessing a refined regularization term being refined by shape information, as described in [0006] of the specification, because the claims are derived from Mental Processes based on concepts performed in the human mind or with the aid of pencil and paper.
Independent claims 1, 10, and 15 similarly recite the refined regularization term being refined by shape information of the local model relating to the received input so as to optimize the parameter for model estimation, which is an abstract idea and covers mental processes of assessing a local model based on the shape information of the local model related to the received input, as described in [0006] of the specification, because the claims are derived from Mental Processes based on concepts performed in the human mind or with the aid of pencil and paper.
Independent claims 10 and 15 similarly recite optimizing a parameter for model estimation based on a refined regularization term of the local model, which is an abstract idea and covers mental processes of assessing a method for model estimation pertaining to a local model, a function in response to receiving an input, and the function corresponding to determining the local model, as described in [00015] of the specification, because the claims are derived from Mental Processes based on concepts performed in the human mind or with the aid of pencil and paper.
Thus, the claims recite the abstract idea of a mental process performed in the human mind, or with the aid of pencil and paper.
Dependent claims 2-9 and 11-14 further narrow the abstract ideas, identified in the independent claims. See analysis below.
Step 2A, Prong 2: The judicial exception is not integrated into a practical application. Independent claim 1 and dependent claims 2-9 recite the additional limitation “device,” independent claim 10 recites the additional limitation “computer,” and “independent claim 15 recites “non-transitory computer readable medium,” this limitation does not integrate the judicial exception into a practical application because it is nothing more than generally linking the use of the judicial exception to a particular technological environment. See MPEP 2106.05(h). Alternatively, this additional element merely uses a computer device as a tool to perform the abstract idea. (MPEP 2106.05(f)).
The additional recited claim 1 receive an input relating to a local model, are mere instructions to implement an abstract idea using a computer in its ordinary capacity, or merely uses the computer as a tool to perform the identified abstract idea. See MPEP (2106.05(f)) use of a computer or other machinery in its ordinary capacity for economic or other tasks (e.g., to receive, store, or transmit data) or simply adding a general purpose computer or computer components after the fact to an abstract idea (e.g., a mental process) does not integrate a judicial exception into a practical application. (MPEP 2106.05(f)(2)).
The additional claim 2 and 12 similarly recite the limitation of wherein the device is further configured to compute a variational probability of a latent variable using the refined regularization term, can be viewed as is insignificant extra-solution activity, specifically pertaining to mere data gathering/output necessary to perform the abstract idea (MPEP 2106.05(g)) and is not sufficient to integrate the judicial exception into a practical application. This is akin to selecting information, based on types of information and availability of information in a power-grid environment, for collection, analysis and display, which has been identified as extra solution activity. Therefore, the judicial exception is not integrated into a practical application.
The additional claim 2 and 12 similarly recite the limitation of computing and setting a latent state number based on the variational probability of the latent variable, can be viewed as is insignificant extra-solution activity, specifically pertaining to mere data gathering/output necessary to perform the abstract idea (MPEP 2106.05(g)) and is not sufficient to integrate the judicial exception into a practical application. This is akin to selecting information, based on types of information and availability of information in a power-grid environment, for collection, analysis and display, which has been identified as extra solution activity. Therefore, the judicial exception is not integrated into a practical application.
The additional recited claims 3 and 12 limitation of wherein the device is further configured to compute and set a latent state number based on the variational probability of the latent variable, can be viewed as is insignificant extra-solution activity, specifically pertaining to mere data gathering/output necessary to perform the abstract idea (MPEP 2106.05(g)) and is not sufficient to integrate the judicial exception into a practical application. This is akin to selecting information, based on types of information and availability of information in a power-grid environment, for collection, analysis and display, which has been identified as extra solution activity. Therefore, the judicial exception is not integrated into a practical application.
The additional recited claims 8 and 13 limitation of a loop process is repeatedly performed until the criterion has converged, are mere instructions to implement an abstract idea using a computer in its ordinary capacity, or merely uses the computer as a tool to perform the identified abstract idea. See MPEP (2106.05(f)) use of a computer or other machinery in its ordinary capacity for economic or other tasks (e.g., to receive, store, or transmit data) or simply adding a general purpose computer or computer components after the fact to an abstract idea (e.g., a mental process) does not integrate a judicial exception into a practical application. (MPEP 2106.05(f)(2)).
The additional recited claims 8 and 13 limitation of computing the variational probability of the latent variable, are mere instructions to implement an abstract idea using a computer in its ordinary capacity, or merely uses the computer as a tool to perform the identified abstract idea. See MPEP (2106.05(f)) use of a computer or other machinery in its ordinary capacity for economic or other tasks (e.g., to receive, store, or transmit data) or simply adding a general purpose computer or computer components after the fact to an abstract idea (e.g., a mental process) does not integrate a judicial exception into a practical application. (MPEP 2106.05(f)(2)).
The additional recited claims 8 and 13 limitation computing and setting the latent state number, can be viewed as is insignificant extra-solution activity, specifically pertaining to mere data gathering/output necessary to perform the abstract idea (MPEP 2106.05(g)) and is not sufficient to integrate the judicial exception into a practical application. This is akin to selecting information, based on types of information and availability of information in a power-grid environment, for collection, analysis and display, which has been identified as extra solution activity. Therefore, the judicial exception is not integrated into a practical application.
The additional recited claim 9 limitation of compute the refined regularization term based on the received input relating to the local model, can be viewed as is insignificant extra-solution activity, specifically pertaining to mere data gathering/output necessary to perform the abstract idea (MPEP 2106.05(g)) and is not sufficient to integrate the judicial exception into a practical application. This is akin to selecting information, based on types of information and availability of information in a power-grid environment, for collection, analysis and display, which has been identified as extra solution activity. Therefore, the judicial exception is not integrated into a practical application.
The additional recited claim 14 limitation of computing the refined regularization term based on information of the local model, can be viewed as is insignificant extra-solution activity, specifically pertaining to mere data gathering/output necessary to perform the abstract idea (MPEP 2106.05(g)) and is not sufficient to integrate the judicial exception into a practical application. This is akin to selecting information, based on types of information and availability of information in a power-grid environment, for collection, analysis and display, which has been identified as extra solution activity. Therefore, the judicial exception is not integrated into a practical application.
Dependent claims 2-9 and 11-14 further narrow the abstract ideas, identified in the independent claims, and do not introduce further additional elements for consideration beyond those addressed above. The additional elements have been considered both individually and as an ordered combination in to determine whether they integrate the exception into a practical application. Therefore, the dependent claims do not integrate the claimed invention into a practical application.
Step 2B:
The claims do not amount to significantly more. The judicial exception does not amount to significantly more. Independent claim 1 and dependent claims 2-9 recite the additional limitation “device,” independent claim 10 recites the additional limitation “computer,” and “independent claim 15 recites “non-transitory computer readable medium,” this limitation does not amount to significantly more because it is nothing more than generally linking the use of the judicial exception to a particular technological environment. See MPEP 2106.05(h). Alternatively, this additional element merely uses a computer device as a tool to perform the abstract idea. (MPEP 2106.05(f)).
The additional recited claim 1 receive an input relating to a local model, are mere instructions to implement an abstract idea using a computer in its ordinary capacity, or merely uses the computer as a tool to perform the identified abstract idea. See MPEP (2106.05(f)) use of a computer or other machinery in its ordinary capacity for economic or other tasks (e.g., to receive, store, or transmit data) or simply adding a general purpose computer or computer components after the fact to an abstract idea (e.g., a mental process) does not amount to significantly more. (MPEP 2106.05(f)(2)).
The additional claim 2 and 12 similarly recite the limitation of wherein the device is further configured to compute a variational probability of a latent variable using the refined regularization term, can be viewed as is insignificant extra-solution activity, specifically pertaining to mere data gathering/output necessary to perform the abstract idea (MPEP 2106.05(g)) and does not amount to significantly more. This is akin to selecting information, based on types of information and availability of information in a power-grid environment, for collection, analysis and display, which has been identified as extra solution activity. Therefore, the judicial exception does not amount to significantly more.
The additional claim 2 and 12 similarly recite the limitation of computing and setting a latent state number based on the variational probability of the latent variable, can be viewed as is insignificant extra-solution activity, specifically pertaining to mere data gathering/output necessary to perform the abstract idea (MPEP 2106.05(g)) and does not amount to significantly more. This is akin to selecting information, based on types of information and availability of information in a power-grid environment, for collection, analysis and display, which has been identified as extra solution activity. Therefore, the judicial exception does not amount to significantly more.
The additional recited claims 3 and 12 limitation of wherein the device is further configured to compute and set a latent state number based on the variational probability of the latent variable, can be viewed as is insignificant extra-solution activity, specifically pertaining to mere data gathering/output necessary to perform the abstract idea (MPEP 2106.05(g)) and does not amount to significantly more. This is akin to selecting information, based on types of information and availability of information in a power-grid environment, for collection, analysis and display, which has been identified as extra solution activity. Therefore, the judicial exception does not amount to significantly more.
The additional recited claims 8 and 13 limitation of a loop process is repeatedly performed until the criterion has converged, are mere instructions to implement an abstract idea using a computer in its ordinary capacity, or merely uses the computer as a tool to perform the identified abstract idea. See MPEP (2106.05(f)) use of a computer or other machinery in its ordinary capacity for economic or other tasks (e.g., to receive, store, or transmit data) or simply adding a general purpose computer or computer components after the fact to an abstract idea (e.g., a mental process) does not amount to significantly more. (MPEP 2106.05(f)(2)).
The additional recited claims 8 and 13 limitation of computing the variational probability of the latent variable, are mere instructions to implement an abstract idea using a computer in its ordinary capacity, or merely uses the computer as a tool to perform the identified abstract idea. See MPEP (2106.05(f)) use of a computer or other machinery in its ordinary capacity for economic or other tasks (e.g., to receive, store, or transmit data) or simply adding a general purpose computer or computer components after the fact to an abstract idea (e.g., a mental process) does not amount to significantly more. (MPEP 2106.05(f)(2)).
The additional recited claims 8 and 13 limitation computing and setting the latent state number, can be viewed as is insignificant extra-solution activity, specifically pertaining to mere data gathering/output necessary to perform the abstract idea (MPEP 2106.05(g)) and does not amount to significantly more. This is akin to selecting information, based on types of information and availability of information in a power-grid environment, for collection, analysis and display, which has been identified as extra solution activity. Therefore, the judicial exception does not amount to significantly more.
The additional recited claim 9 limitation of compute the refined regularization term based on the received input relating to the local model, can be viewed as is insignificant extra-solution activity, specifically pertaining to mere data gathering/output necessary to perform the abstract idea (MPEP 2106.05(g)) and does not amount to significantly more. This is akin to selecting information, based on types of information and availability of information in a power-grid environment, for collection, analysis and display, which has been identified as extra solution activity. Therefore, the judicial exception does not amount to significantly more.
The additional recited claim 14 limitation of computing the refined regularization term based on information of the local model, can be viewed as is insignificant extra-solution activity, specifically pertaining to mere data gathering/output necessary to perform the abstract idea (MPEP 2106.05(g)) and does not amount to significantly more. This is akin to selecting information, based on types of information and availability of information in a power-grid environment, for collection, analysis and display, which has been identified as extra solution activity. Therefore, the judicial exception does not amount to significantly more.
Dependent claims 2-9 and 11-14 further narrow the abstract ideas, identified in the independent claims, and do not introduce further additional elements for consideration beyond those addressed above. The additional elements have been considered both individually and as an ordered combination in to determine whether they does not amount to significantly more. Therefore, the dependent claims does not amount to significantly more.
Therefore, the claims as a whole does not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional elements, when considered alone or in combination, do not amount to significantly more than the judicial exception.
As stated in Section I.B. of the December 16, 2014 101 Examination Guidelines, “[t]o be patent-eligible, a claim that is directed to a judicial exception must include additional features to ensure that the claim describes a process or product that applies the exception in a meaningful way, such that it is more than a drafting effort designed to monopolize the exception.”
The dependent claims include the same abstract ideas recited as recited in the independent claims, and merely incorporate additional details that narrow the abstract ideas and fail to add significantly more to the claims.
Dependent claims 4, 8, 11, and 13 similarly recite “wherein the parameter for model estimation includes a criterion value,” which further narrows the abstract idea identified in the independent claim, which is directed to a “Mental Process.”
Dependent claim 5 and 12 similarly recite “wherein the device is further configured to determine whether the criterion value has converged using the refined regularization term of the local model,” which further narrows the abstract idea identified in the independent claim, which is directed to a “Mental Process.”
Dependent claim 6 recites “wherein the device is further configured to determine the local model, which further narrows the abstract idea identified in the independent claim, which is directed to a “Mental Process.”
Dependent claim 7 and 12 similarly recite “wherein the device is further configured to classify the parameter using the refined regularization term of the local model,” which further narrows the abstract idea identified in the independent claim, which is directed to a “Mental Process.”
Dependent claim 8 recites “a loop process is repeatedly performed until the criterion has converged, which further narrows the abstract idea identified in the independent claim, which is directed to a “Mental Process.”
Dependent claims 8 and 12 similarly recite “computing the variational probability of the latent variable using the refined regulation term of the local model, which further narrows the abstract idea identified in the independent claim, which is directed to a “Mental Process.”
Dependent claims 8 and 13 similarly recite “optimizing the parameter,” which further narrows the abstract idea identified in the independent claim, which is directed to a “Mental Process.”
Dependent claims 8 and 13 similarly recite “classifying the parameter,” which further narrows the abstract idea identified in the independent claim, which is directed to a “Mental Process.”
Dependent claims 8 and 13 similarly recite “determining whether the criterion value has converged using the refined regulation term of the local model,” which further narrows the abstract idea identified in the independent claim, which is directed to a “Mental Process.”
Dependent claim 9 recites “determining constraints,” which further narrows the abstract idea identified in the independent claim, which is directed to a “Mental Process.”
Dependent claim 11 recites “wherein the parameter for model estimation includes a
criterion value,” which further narrows the abstract idea identified in the independent claim, which is directed to a “Mental Process.”
Dependent claim 12 recites “classifying the parameter using the refined regulation term of the local model,” which further narrows the abstract idea identified in the independent claim, which is directed to a “Mental Process.”
Dependent claim 12 recites “determining whether the criterion value has converged using the refined regulation term of the local model,” which further narrows the abstract idea identified in the independent claim, which is directed to a “Mental Process.”
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1, 6, 9, 10, 14, and 15 are rejected under are rejected under 35 U.S.C. 103 as being unpatentable over FIRL (Regularization of shape optimization problems using FE-based parametrization), herein FIRL, in view of LIU (Sparse generalized linear model with L0 approximation for feature selection and prediction with big omics data), herein LIU, and in view of STREIB (High-Dimensional LASSO-Based Computational Regression Models: Regularization, Shrinkage, and Selection) , herein STRIEIB.
Claim 1
Claim 1 is rejected because FIRL teaches determine a function in response to receiving an input relating, the function corresponding to the local model FIRL ([Section 2.2 Application as filter function] “For smoothing of discrete response functions (determine a function) of finite element models (corresponding to the local model) the set D is defined by the set of optimization variables at finite element nodes... Example I Figure 4 visualizes a basic property of smoothing operations by convolution namely the enlarged support of the smoothed function. The two dimensional and constant function fc in Fig. 4a has the value 1 in the domain 6 < x < 15, 6 < y < 15 and the value 0 elsewhere. The function fs plotted (in response) in Fig. 4b is obtained by convolution of fc with a linear filter function with radius equal to 3 (receiving an input) (cf. Fig. 3b). The result shows a clear smoothing in the support region of function fc.”)
FIRL does not explicitly teach optimize a parameter for model estimation based on a refined regularization term of the local model.
However, LIU teaches optimize a parameter for model estimation based on a refined regularization term of the local model LIU ([Background] “recent research works including ours show that sparse regression models with L0 penalty (local solution) (refined regularization term of the local model) outperforms L1 (global solution) by a substantial margin [5, 9–11].”) See also LIU ([Results | pdf page 8 of 12] “To overcome such bias in parameter estimation (optimize a parameter for model estimation), some packages re-estimate the parameters with the selected features and standard GLM model.”)
It would have been obvious to one of ordinary skill in the art, before the effective filing date, to combine the teachings of LIU with FIRL as the references deal with devices and methods for model estimation. LIU would modify FIRL wherein the refined regularization term being refined by shape information of the local model relating to the received input so as to optimize the parameter for model estimation. The benefits of doing so allows for selecting a small subset of informative features (biomarkers) to conduct association studies and clinical predictions, which has become an important step toward effective big data mining. (LIU [Background]).
The combination of FIRAL and LIU does not explicitly teach the refined regularization term being refined by shape information of the local model relating to the received input so as to optimize the parameter for model estimation.
However, STREIB also teaches the refined regularization term being refined by shape information of the local model relating to the received input so as to optimize the parameter for model estimation STREIB ([Section 6.2. Explanation of Variable Selection | pdf 8 of 25] “From Figure 3 one can see that decreasing values of l (optimize the parameter for model estimation) lead to the shrinkage (refined regularization term) of the regression coefficients (relating to the received input) and some of these even become zero. To understand this behavior, we depict in Figure 5A, B a two-dimensional LASSO (A) and ridge regression (B) model (local model). The regularization term of each regression model is depicted in blue, corresponding to the diamond shape (refined by shape information) for the L1-norm and the circle for the L2-norm. The solution of the optimization problem is given by the intersection of the ellipsis and the boundary of the penalty shapes.”) See also STREIB ([Figure 3].)
PNG
media_image1.png
628
444
media_image1.png
Greyscale
STREIB Figure 3 Reference
It would have been obvious to one of ordinary skill in the art, before the effective filing date, to combine the teachings of STREIB with FIRL and LIU as the references deal with devices and methods for model estimation. STREIB would modify FIRL and LIU wherein the refined regularization term being refined by shape information of the local model relating to the received input so as to optimize the parameter for model estimation. The benefits of doing so provides modern, computational regression models valuable tools for analyzing high-dimensional problems. (STREIB [Abstract]). Accordingly, claim 1 is rejected based on the combination of these references.
Claim 6
Claim 6 is rejected because the combination of teaches claim 1.
FIRL does not explicitly teach configured to determine the local model.
However, LIU teaches configured to determine the local model LIU ([Background] “recent research works including ours show that sparse regression models with L0 penalty (local solution) outperforms L1 (global solution) by a substantial margin [5, 9–11]…Sparse modeling is one of the important approaches for simultaneous phenotype prediction and biomarker identification. In this paper, we propose a L0 penalized generalized linear regression (GLM) for feature selection and prediction.”)
It would have been obvious to one of ordinary skill in the art, before the effective filing date, to combine the teachings of LIU with FIRL as the references deal with devices and methods for model estimation. LIU would modify FIRL wherein configured to determine the local model. The benefits of doing so allows for selecting a small subset of informative features (biomarkers) to conduct association studies and clinical predictions, which has become an important step toward effective big data mining. (LIU [Background]). Accordingly, claim 6 is rejected based on the combination of these references.
Claim 9
Claim 9 is rejected because the combination of FIRL, LIU, and STREIB teaches the claim 1 limitations.
LIU teaches the device is further configured to compute the refined regularization term based on the received input relating to the local model LIU ([Section A. One-Dimensional Signal] “Fig. 1. MLS reconstruction of a given set of scattered data points (xi, fi) in 1-D. The local MLS approximation _ for the point gx is shown in blue. Its computation and evaluation at every point of the domain yields the complete MLS reconstruction shown in red.”) See also LIU ([Figure 1].)
PNG
media_image2.png
433
808
media_image2.png
Greyscale
LIU Figure 1 Reference
It would have been obvious to one of ordinary skill in the art, before the effective filing date, to combine the teachings of LIU with FIRL as the references deal with devices and methods for model estimation. LIU would modify FIRL device is further configured to compute the refined regularization term based on the received input relating to the local model. The benefits of doing so allows for selecting a small subset of informative features (biomarkers) to conduct association studies and clinical predictions, which has become an important step toward effective big data mining. (LIU [Background]). Accordingly, claim 9 is rejected based on the combination of these references.
Claim 10
Claim 10 is rejected because it is the method embodiment of claim 1 with similar limitations to claim 1, and is such rejected using the same reasoning found in claim 1.
Claim 14
Claim 14 is rejected because the combination of FIRAL, LIU and STREIB teaches the claim 1 limitations.
FIRAL does not explicitly teach computing the refined regularization term based on information of the local model.
However, LIU teaches the computing the refined regularization term based on information of the local model LIU ([Section A. One-Dimensional Signal] “Fig. 1. MLS reconstruction of a given set of scattered data points (xi, fi) in 1-D. The local MLS approximation _ for the point gx is shown in blue. Its computation and evaluation at every point of the domain yields the complete MLS reconstruction shown in red.”) See also LIU ([Figure 1].)
PNG
media_image2.png
433
808
media_image2.png
Greyscale
LIU Figure 1 Reference
It would have been obvious to one of ordinary skill in the art, before the effective filing date, to combine the teachings of LIU with FIRL as the references deal with devices and methods for model estimation. LIU would modify FIRL wherein computing the refined regularization term based on information of the local model. The benefits of doing so allows for selecting a small subset of informative features (biomarkers) to conduct association studies and clinical predictions, which has become an important step toward effective big data mining. (LIU [Background]). Accordingly, claim 14 is rejected based on the combination of these references.
Claim 15
Claim 15 is rejected because it is the non-transitory computer-readable embodiment of claim 1 with similar limitations to claim 1, and is such rejected using the same reasoning found in claim 1.
Claim(s) 2-5, 7-8, and 11-13 are rejected under are rejected under 35 U.S.C. 103 as being unpatentable over FIRL, in view of LIU, in view of STREIB, and in view of GANCHEV (Posterior Regularization for Structured Latent Variable Models), herein GANCHEV.
Claim 2
Claim 2 is rejected because the combination of FIRL, LIU, and STREIB teaches the claim 1 limitations.
The combination of FIRL, LIU, and STREIB does not explicitly teach configured to compute a variational probability of a latent variable using the refined regularization term.
However, GANCHEV teaches configured to compute a variational probability of a latent variable using the refined regularization term GANCHEV ([Section 4.2 Generalized Expectation Criteria] “In order to avoid the costly optimization procedure described above, Bellare et al. (2009) propose a variational approximation. Recall that at a high level, the difficulty in optimizing Equation 15 is because the last term couples the constraint features f with the model parameters q. In order to separate out these quantities, Bellare et al. (2009) introduce an auxiliary distribution q(Y) _ pq(Y|X), which is used to approximate the last term in Equation 15. The variational objective contains three terms instead of two:
PNG
media_image3.png
85
998
media_image3.png
Greyscale
”)
It would have been obvious to one of ordinary skill in the art, before the effective filing date, to combine the teachings of GANCHEV with FIRL, LIU, and STREIB as the references deal with devices and methods for model estimation. GANCHEV would modify FIRL, LIU, and STREIB wherein the configured to compute a variational probability of a latent variable using the refined regularization term. The benefits of doing so provides efficient algorithm for learning with posterior regularization and illustrate its versatility on a diverse set of structural constraints such as bijectivity, symmetry and group sparsity in several large scale experiments, including multi-view learning, cross-lingual dependency grammar induction, unsupervised part-of-speech induction, and bitext word alignment. (GANCHEV [Abstract]). Accordingly, claim 2 is rejected based on the combination of these references.
Claim 3
Claim 3 is rejected because the combination of FIRL, LIU, and STREIB teaches the claim 2 limitations.
The combination of FIRL, LIU, and STREIB does not explicitly teach configured to compute and set a latent state number based on the variational probability of the latent variable.
However, GANCHEV teaches configured to compute and set a latent state number based on the variational probability of the latent variable.
GANCHEV ([Section 5.1 Models] “We consider two models below: IBM Model 1 proposed by Brown et al. (1994) and the HMM model proposed by Vogel et al. (1996). Both models can be expressed as:
PNG
media_image4.png
64
602
media_image4.png
Greyscale
where y is the alignment and yj is the index of the hidden state (source language index) generating the target language word at index j. The models differ in their definition of the distortion probability pd(yj | j, yj−1). Model 1 assumes that the target words are generated independently and assigns uniform distortion probability. The HMM model assumes that only the distance between the current and previous source word index is important pd(yj | j, yj−1) = pd(yj | yj −yj−1). Both models are augmented by adding a special “null” word to the source sentence.”
It would have been obvious to one of ordinary skill in the art, before the effective filing date, to combine the teachings of GANCHEV with FIRL, LIU, and STREIB as the references deal with devices and methods for model estimation. GANCHEV would modify FIRL, LIU, and STREIB wherein configured to compute and set a latent state number based on the variational probability of the latent variable. The benefits of doing so provides efficient algorithm for learning with posterior regularization and illustrate its versatility on a diverse set of structural constraints such as bijectivity, symmetry and group sparsity in several large scale experiments, including multi-view learning, cross-lingual dependency grammar induction, unsupervised part-of-speech induction, and bitext word alignment. (GANCHEV [Abstract]). Accordingly, claim 3 is rejected based on the combination of these references.
Claim 4
Claim 4 is rejected because the combination of teaches claim 1.
The combination of FIRAL, LIU, and STREIB does not explicitly teach wherein the parameter for model estimation includes a criterion value.
However, GANCHEV teaches wherein the parameter for model estimation includes a criterion value GANCHEV ([Section 2.6 Generative Posterior Regularization via Expectation Maximization] “Using this interpretation, we can view EM as performing coordinate ascent on F(q,q). Starting from an initial parameter estimate q0, the algorithm iterates two block-coordinate ascent steps until a convergence criterion is reached
PNG
media_image5.png
156
853
media_image5.png
Greyscale
It would have been obvious to one of ordinary skill in the art, before the effective filing date, to combine the teachings of GANCHEV with FIRL, LIU, and STREIB as the references deal with devices and methods for model estimation. GANCHEV would modify FIRL, LIU, and STREIB wherein the parameter for model estimation includes a criterion value. The benefits of doing so provides efficient algorithm for learning with posterior regularization and illustrate its versatility on a diverse set of structural constraints such as bijectivity, symmetry and group sparsity in several large scale experiments, including multi-view learning, cross-lingual dependency grammar induction, unsupervised part-of-speech induction, and bitext word alignment. (GANCHEV [Abstract]). Accordingly, claim 4 is rejected based on the combination of these references.
Claim 5
Claim 5 is rejected because the combination of teaches claim 4.
The combination of FIRAL, LIU, and STREIB does not explicitly teach configured to determine whether the criterion value has converged using the refined regularization term of the local model.
However, GANCHEV teaches configured to determine whether the criterion value has converged using the refined regularization term of the local model Expectation Maximization GANCHEV ([Section 2.6 Generative Posterior Regularization via Expectation Maximization] “Proof: The proof is analogous to the proof of monotonic increase of the standard EM objective. Essentially, JQ (qt+1) = F(qt+2,qt+1) _ F(qt+1,qt+1) _ F(qt+1,qt) = JQ (qt). The two inequalities are ensured by the E0-step andM-step. E0-step sets qt+1 =argmaxq2Q F(q,qt), hence JQ (qt) = F(qt+1,qt). The M-step sets qt+1 = argmaxq F(qt+1,q), hence F(qt+1,qt+1) _F(qt+1,qt). Finally, JQ (qt+1) = maxq2Q F(q,qt+1) _ F(qt+1,qt+1) _ Note that the proposition is only meaningful when Q is non-empty and JQ is well-defined. As for standard EM, to prove that coordinate ascent on F(q,q) converges to stationary points of JQ (q), we need to make additional assumptions on the regularity of the likelihood function and boundedness of the parameter space as in Tseng (2004).”)
It would have been obvious to one of ordinary skill in the art, before the effective filing date, to combine the teachings of GANCHEV with FIRL, LIU, and STREIB as the references deal with devices and methods for model estimation. GANCHEV would modify FIRL, LIU, and STREIB configured to determine whether the criterion value has converged using the refined regularization term of the local model. The benefits of doing so provides efficient algorithm for learning with posterior regularization and illustrate its versatility on a diverse set of structural constraints such as bijectivity, symmetry and group sparsity in several large scale experiments, including multi-view learning, cross-lingual dependency grammar induction, unsupervised part-of-speech induction, and bitext word alignment. (GANCHEV [Abstract]). Accordingly, claim 5 is rejected based on the combination of these references.
Claim 7
Claim 7 is rejected because the combination of teaches claim 1.
The combination of FIRL, LIU, and STREIB does not explicitly teach configured to classify the parameter using the refined regularization term of the local model.
However, GANCHEV teaches configured to classify the parameter using the refined regularization term of the local model GANCHEV ([Section 6 Multiview Learning] “In addition, this framework allows us to use different labeled training sets for the two classifiers, in the case where they have different label sets. That is, we don’t require that our two views are both on the same labeled corpus. In that case, we can reduce the hypothesis space by preferring pairs of models that agree on compatible labeling of some additional unlabeled data rather than on identical labeling, while still minimizing KL in closed form. When the two views come from models that differ not only in the label set but also in the model structure of the output space, our framework can still encourage agreement, but the KL minimization cannot be computed in closed form. Finally, this method uses soft assignments to latent variables resulting in a more stable optimization procedure.”)
It would have been obvious to one of ordinary skill in the art, before the effective filing date, to combine the teachings of GANCHEV with FIRL, LIU, and STREIB as the references deal with devices and methods for model estimation. GANCHEV would modify FIRL, LIU, and STREIB configured to determine whether configured to classify the parameter using the refined regularization term of the local model. The benefits of doing so provides efficient algorithm for learning with posterior regularization and illustrate its versatility on a diverse set of structural constraints such as bijectivity, symmetry and group sparsity in several large scale experiments, including multi-view learning, cross-lingual dependency grammar induction, unsupervised part-of-speech induction, and bitext word alignment. (GANCHEV [Abstract]). Accordingly, claim 7 is rejected based on the combination of these references.
Claim 8
Claim 8 is rejected because the combination of teaches claim 3.
FIRL does not explicitly teach does not explicitly teach the parameter for model estimation includes a criterion value.
However, LIU teaches the parameter for model estimation includes a criterion value LIU ([Section II. ORDINARY LEAST SQUARES AND MOVING LEAST SQUARES | pdf page 3 of 15] “A common criterion for approximation error is OLS, which can be expressed as
PNG
media_image6.png
79
337
media_image6.png
Greyscale
Setting the derivative to 0, the optimal can be represented by
PNG
media_image7.png
83
766
media_image7.png
Greyscale
OLS obtains a globally defined function that approximates the given scalar value at point in the least squares sense. It considers all samples in the training set with equal importance in the process of minimization. As a consequence, OLS is sensitive to outliers, a small amount of outliers within the data can severely bias the least squares estimation.”)
LIU also teaches the loop process includes computing the variational probability of the latent variable LIU ([Background] “Recall the iterative system in Eq. (3), note that each feature is penalized by a different penalty, which is inversely proportional to the squared magnitude of that parameter estimator ηj. i.e., λj = λ 2η2 j , and ηj = βj. Smaller βj will lead to larger λj. A tiny βj, will become smaller and λj will be getting larger in each iteration of L0ADRIDGE algorithm. βj → 0, and λj → ∞. On the other hand, a larger βj will lead a finite λj, and nonzero βj, when the number of iteration goes to ∞. The solution of L0ADRIDGE will converge to that of Eq. (7), because the effect of nonzero ηj will be canceled out in Eq. (5). Note that our proposed methods will find a sparse solution with a large number of iterations and small ε, even though the solution of L2 regularized modeling is not sparse. Small parameters (βjs) become smaller at each iteration and will eventually go to zero (below the machine ).We can also set a parameter to 0 if it is below predefined ε = 1e − 6 to speed up the convergence of the algorithm.” See also LIU ([Results] “Logistic regression: The logistic regression data was generated with the coefficients of [ β1, β5, β10]=[ 0.5, 0.5,−0.4], respectively, and the remaining coefficients were set to zero. The score z = Xβ+ε, where ε is the random noise with the signal to noise ratio of 4. Then, the probability y is generated from the logistic function y = 1/(1 + e−z). Note that y is the true probability instead of binary (1/0) in this simulation. Unlike the previous example, the optimal values of λ in this simulation were selected with the standard 5-fold cross-validation. We divided the λ from λmin = 1e−4, to λmax into 100 equal intervals in log-scale, then chose the optimal λ with the smallest test error. The simulation was also repeated 100 times. The computational results were reported in Table 3. The values in the parenthesis are the positive/negative standard deviation.”)
It would have been obvious to one of ordinary skill in the art, before the effective filing date, to combine the teachings of LIU with FIRL as the references deal with devices and methods for model estimation. LIU would modify FIRL wherein the parameter for model estimation includes a criterion value. The benefits of doing so allows for selecting a small subset of informative features (biomarkers) to conduct association studies and clinical predictions, which has become an important step toward effective big data mining. (LIU [Background]).
The combination of FIRL and LIU does not explicitly teach a loop process is repeatedly performed until the criterion value has converged.
However, STREIB teaches a loop process is repeatedly performed until the criterion value has converged STREIB ([Section 9. Adaptive LASSO] “For the above results we used g = 1, however, g is a tuning parameter that can be estimated from the data. Specifically, in Figure 6C,D we repeated our analysis for different values of g. From Figure 6C one can see that the minimal mean-squared error is obtained for g = 0.25 but g = 1.0 also gives good results. In Figure 6D we show the same results as in Figure 6C but for the mean-squared error in dependence on lmin. There, one sees that the lmin for large values of g are quite large.”)
It would have been obvious to one of ordinary skill in the art, before the effective filing date, to combine the teachings of STREIB with FIRL and LIU as the references deal with devices and methods for model estimation. STREIB would modify FIRL and LIU wherein a loop process is repeatedly performed until the criterion value has converged. The benefits of doing so provides modern, computational regression models valuable tools for analyzing high-dimensional problems. (STREIB [Abstract]).
The combination of FIRAL, LIU, and STREIB does not explicitly teach computing and setting the latent state number, optimizing the parameter, classifying the parameter, or determining whether the criterion value has converged.
However, GANCHEV teaches computing and setting the latent state number GANCHEV ([Abstract] “We present posterior regularization, a probabilistic framework for structured, weakly supervised learning. Our framework efficiently incorporates indirect supervision via constraints on posterior distributions of probabilistic models with latent variables. Posterior regularization separates model complexity from the complexity of structural constraints it is desired to satisfy. By directly imposing decomposable regularization on the posterior moments of latent variables during learning, we retain the computational efficiency of the unconstrained model while ensuring desired constraints hold in expectation. We present an efficient algorithm for learning with posterior regularization and illustrate its versatility on a diverse set of structural constraints such as bijectivity, symmetry and group sparsity in several large scale experiments, including multi-view learning, cross-lingual dependency grammar induction, unsupervised part-of-speech induction, and bitext word alignment.”) See also GANCHEV ([Introduction] “Generative models (probabilistic grammars, graphical models, etc.) are usually estimated by maximizing the likelihood of the observed data by marginalizing over the hidden variables, typically via the Expectation Maximization (EM) algorithm.”)
GANCHEV also teaches optimizing the parameter GANCHEV ([Section 2.1 Assumptions] “Note that the algorithm we will present in Section 2.6 will not allow us to optimize an objective with a > 1, and this leads us to have both a KL-penalty term in Equation 2 and also to potentially have slack in the definition of the constraint set Q.”)
GANCHEV also teaches classifying the parameter GANCHEV ([Section 6 Multiview Learning] “In addition, this framework allows us to use different labeled training sets for the two classifiers, in the case where they have different label sets. That is, we don’t require that our two views are both on the same labeled corpus. In that case, we can reduce the hypothesis space by preferring pairs of models that agree on compatible labeling of some additional unlabeled data rather than on identical labeling, while still minimizing KL in closed form. When the two views come from models that differ not only in the label set but also in the model structure of the output space, our framework can still encourage agreement, but the KL minimization cannot be computed in closed form. Finally, this method uses soft assignments to latent variables resulting in a more stable optimization procedure.”)
GANCHEV also teaches determining whether the criterion value has converged GANCHEV ([Section 2.6 Generative Posterior Regularization via Expectation Maximization] “Proof: The proof is analogous to the proof of monotonic increase of the standard EM objective. Essentially, JQ (qt+1) = F(qt+2,qt+1) _ F(qt+1,qt+1) _ F(qt+1,qt) = JQ (qt). The two inequalities are ensured by the E0-step andM-step. E0-step sets qt+1 =argmaxq2Q F(q,qt), hence JQ (qt) = F(qt+1,qt). The M-step sets qt+1 = argmaxq F(qt+1,q), hence F(qt+1,qt+1) _
F(qt+1,qt). Finally, JQ (qt+1) = maxq2Q F(q,qt+1) _ F(qt+1,qt+1) _ Note that the proposition is only meaningful when Q is non-empty and JQ is well-defined. As for standard EM, to prove that coordinate ascent on F(q,q) converges to stationary points of JQ (q), we need to make additional assumptions on the regularity of the likelihood function and boundedness of the parameter space as in Tseng (2004).”)
It would have been obvious to one of ordinary skill in the art, before the effective filing date, to combine the teachings of GANCHEV with FIRL, LIU, and STREIB as the references deal with devices and methods for model estimation. GANCHEV would modify FIRL, LIU, and STREIB wherein determining whether the criterion value has converged. The benefits of doing so provides efficient algorithm for learning with posterior regularization and illustrate its versatility on a diverse set of structural constraints such as bijectivity, symmetry and group sparsity in several large scale experiments, including multi-view learning, cross-lingual dependency grammar induction, unsupervised part-of-speech induction, and bitext word alignment. (GANCHEV [Abstract]). Accordingly, claim 8 is rejected based on the combination of these references.
Claim 11
Claim 11 is rejected because it is the method embodiment of claim 4 with similar limitations to claim 4, and is such rejected using the same reasoning found in claim 4.
Claim 12
Claim 12 is rejected because it is the method embodiment of claims 2, 3 ,5, and 7 with similar limitations to claims 2, 3 ,5, and 7, and is such rejected using the same reasoning found in claim 2, 3 ,5, and 7.
Claim 13
Claim 13 is rejected because it is the method embodiment of claim 8 with similar limitations to claim 8, and is such rejected using the same reasoning found in claim 8.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MARTIN K VU whose telephone number is (703)756-5944. The examiner can normally be reached 7:30 am to 4:30 pm M-F.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Renee Chavez can be reached on 571-270-1104. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/M.K.V./Examiner, Art Unit 2186
/RENEE D CHAVEZ/Supervisory Patent Examiner, Art Unit 2186