DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
The amendment filed August 7, 2026, has been entered. Claims 1-6, and 9-20 remain pending in the application.
Response to Amendment
Referring to applicant’s arguments: claim rejections under 35 U.S.C. 101
Examiner’s response: rejections under 35 U.S.C. 101 are held.
Further, regarding applicant’s arguments for 35 U.S.C. § 101 from page 7 mentioning “Independent claims 1 and 19 are directed to a practical application. … may significantly reduce the time complexity and space complexity required for estimating the uncertainty of the artificial neural network model and may improve the accuracy of estimation. See paragraphs [0011]-[0014] and [0104] of the originally filed specification,” as well as page 8 “independent claims 1 and 19 are integrated into a practical application,” the examiner did not find this persuasive, since the improvement was not reflected in the amended claims. Looking up uncertainty values by cluster from a table can be performed mentally, please see 35 U.S.C. 101 rejections for details.
Referring to applicant’s arguments: claim rejections under 35 U.S.C. 103
Examiner’s response: rejections under 35 U.S.C. 103 are held.
Regarding Applicant’s arguments from page 9, paragraph 1 of the §103 section, of the remarks stating, “Claims 1, 11, 19, and 20 are not rejected based only on the combination of Shaffer and Shahid.” The examiner did not find this persuasive since Shaffer teaches “an apparatus for estimating uncertainty of an artificial neural network model, the apparatus comprising: a storage configured to store a clustering model having a plurality of clusters;” from paragraph [0051] where Shaffer shows a mass storage device, and in [0011-0012] and [0008] shows storing the model that recognizes clustering of the patterns.
Further, Shaffer additionally teaches “and a controller configured to: extract a feature pattern from a hidden layer of the artificial neural network model;” with paragraph [0005] showing an apparatus that includes a pattern recognition unit also extract pattern vectors, which is synonymous with feature pattern.
The reference Shahid teaches “determine a cluster corresponding to the feature pattern by using the clustering model;” in pages 8-9 of Discussion section of grouping variables into cluster, and then identify classification of those variables according to their groups.
Applicant's arguments with respect to the amended limitations "and a table recording a respective uncertainty value for each of the plurality of clusters" and "look up, in the table, an uncertainty value corresponding to the determined cluster as an uncertainty value of the artificial neural network model at inference time", for claim 1, and similar amended limitations in claim 19, are moot because the arguments do not apply to the combinations of references used in the current rejections. The examiner has brought in reference Shrestha to teach these limitations. Please see the 35 U.S.C. 103 rejections for details.
From the applicant’s arguments on pages 9-12, where applicant stated the references for each of the dependent claims fail to render obvious the respective dependent claim, these are moot because the arguments do not apply to the combinations of references used for the independent claims.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefore, subject to the conditions and requirements of this title.
Claims 1-6, 9-20 are rejected under 35 U.S.C § 101 since the invention is directed to an abstract idea (mental process or math concept) without significantly more.
Claim 1: Regarding claim 1, in step 1 of the 101-analysis set forth in MPEP 2106, the claim recites “An apparatus for estimating uncertainty of an artificial neural network model, the apparatus comprising: a storage configured to store a clustering model having a plurality of clusters and a table recording a respective uncertainty value for each of the plurality of clusters;
and a controller configured to: extract a feature pattern from a hidden layer of the artificial neural network model;
determine a cluster corresponding to the feature pattern by using the clustering model; and look up, in the table, an uncertainty value corresponding to the determined cluster as an uncertainty value of the artificial neural network model at inference time,” and an apparatus is one of the four statutory categories of invention.
In step 2A prong 1 of the 101-analysis set forth in the MPEP 2106, the examiner has determined that the following limitations recite a process that, under the broadest reasonable interpretation, covers a mental process but for recitation of generic computer components:
determine a cluster corresponding to the feature pattern by using the clustering model; (mental process, a person can mentally evaluate and identify a cluster that correspond to a feature pattern, see MPEP 2106.04(a)(2)(III)),
look up, in the table, an uncertainty value corresponding to the determined cluster as an uncertainty value of the artificial neural network model at inference time... (This recites a mental process, a person can mentally evaluate and determine an uncertainty value or quantity that corresponds to the cluster from viewing a table, see MPEP 2106.04(a)(2)(III)),
If claim limitations, under their broadest reasonable interpretation, cover performance of the limitations as a mental process but for the recitation of generic computer components, then it falls within the mental process grouping of abstract ideas. Accordingly, the claim “recites” an abstract idea. In step 2A prong 2 of the 101 analysis set forth in the MPEP 2106, the examiner has determined that the following elements do not integrate this judicial exception into a practical application:
An apparatus for estimating uncertainty of an artificial neural network model, the apparatus comprising: a storage configured to … (is considered mere instructions to apply an exception using generic computer – see MPEP 2106.05(f)),
store a clustering model having a plurality of clusters; (In step 2A, prong 2, this recites mere data storing, which is considered insignificant extra-solution activity – see MPEP 2106.05(g)),
and a table recording a respective uncertainty value for each of the plurality of clusters, (In step 2A, prong 2, this recites mere data gathering, which is considered insignificant extra-solution activity – see MPEP 2106.05(g)),
and a controller configured to… (This is considered mere instructions to apply an exception using generic computer – see MPEP 2106.05(f)),
extract a feature pattern from a hidden layer of the artificial neural network model; (In step 2A, prong 2, this recites mere data gathering, which is considered insignificant extra-solution activity – see MPEP 2106.05(g)),
Since the claim as a whole, looking at the additional elements individually and in combination, does not contain any other additional elements that are indicative of integration into a practical application, the claim is “directed” to an abstract idea.
In step 2B of the 101-analysis set forth in the 2019 PEG, the examiner has determined that the claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
As discussed above, additional elements iii and vi recite mere instructions to apply the judicial exception using generic computer components, which are not indicative of significantly more. The additional element v recites mere data storing, and additional element iv and vii recite mere data gathering, and are considered insignificant extra-solution activities. In step 2B, these insignificant extra-solution activities are well understood routine and conventional activities which includes storing and retrieving information in memory from court case Versata Dev. Group, Inc. v. SAP Am., Inc., 793 F.3d 1306, 1334, 115 USPQ2d 1681, 1701 (Fed. Cir. 2015); OIP Techs., 788 F.3d at 1363, 115 USPQ2d at 1092-93;– see MPEP 2106.05(d) (II)(iv)), as well as receiving or transmitting data over a network, e.g., using the Internet to gather data, see court case Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362; – see MPEP 2106.05(d) (II)(i)),
Considering the additional elements individually and in combination, and the claim as a whole, the additional elements do not provide significantly more than the abstract idea. Therefore, the claim is not patent eligible.
Claim 2:
Regarding claim 2, it is dependent upon claim 1, and thereby incorporates the limitations of, and corresponding analysis applied to claim 1. Further, claim 2 recites the following abstract idea:
i. The apparatus of claim 1, wherein … normalize the feature pattern, (This recites a mathematical relationship, mathematical formula or equation, or mathematical calculation, see in paragraph [0072] from the specification stating “Returning to FIG. 4, the controller 40 may perform normalization on a first feature pattern 420 received through the input device 20. In this case, the controller 40 may divide the first feature pattern 420 in L2 norm. For reference, norm means a scheme for calculating the magnitude of a vector, i.e., a scheme for measuring the distance (difference) between two points. As the number of dimensions (the number of features) of the norm is 2, the L2 norm represents the square root of the result of adding all the square values of differences between the same dimensions”, see MPEP 2106.04(a)(2), subsection I),
Further, claim 2 recites an additional element:
ii. …the controller is configured to… (In step 2A, prong 2, this is considered mere instructions to apply an exception using generic computer, see MPEP 2106.05(f)). (In step 2B, this is also considered mere instructions to apply an exception using generic computer – see MPEP 2106.05(f)).
If claim limitations, under their broadest reasonable interpretation, covers performance of the limitations as a mathematical concept but for the recitation of generic computer components, then it falls within the mathematical concept grouping of abstract ideas. Accordingly, the claim “recites” an abstract idea. Since the claim does not recite additional elements that either integrate the judicial exception into a practical application, nor provide significantly more than the judicial exception, the claim is not patent eligible.
Claim 3:
Regarding claim 3, it is dependent upon claim 1, and thereby incorporates the limitations of, and corresponding analysis applied to claim 1. Further, claim 3 recites the following additional element:
i. The apparatus of claim 1, wherein the clustering model includes a plurality of representative patterns for each section as a clustering model of a section division structure, (In step 2A, prong 2, this is considered mere instructions to apply an exception using generic computer – see MPEP 2106.05(f)), (In step 2B, this is also considered mere instructions to apply an exception using generic computer – see MPEP 2106.05(f)),
Since the claim does not recite additional elements that either integrate the judicial exception into a practical application, nor provide significantly more than the judicial exception, the claim is not patent eligible.
Claim 4:
Regarding claim 4, it is dependent upon claim 3, and thereby incorporates the limitations of, and corresponding analysis applied to claim 3. Further, claim 4 recites the following additional element:
i. The apparatus of claim 3, wherein the controller is configured to, (In step 2A, prong 2, this is considered mere instructions to apply an exception using generic computer – see MPEP 2106.05(f)), (In step 2B, this is also considered mere instructions to apply an exception using generic computer – see MPEP 2106.05(f)),
Further, claim 4 recites the following abstract ideas:
ii. divide the feature pattern into a plurality of sections; (This is considered a mental process, since a person can mentally divide a feature pattern into sections, see MPEP 2106.04(a)(2)(III)),
iii. compare a first section of the feature pattern with representative patterns in a first section of the clustering model to determine a first representative pattern that is most similar; (This is considered a mental process, since a person can mentally compare a first section of feature pattern with representative patterns in a second section to determine the most similar first representative pattern, see MPEP 2106.04(a)(2)(III)),
iv. compare a second section of the feature pattern with representative patterns in a second section of the clustering model to determine a second representative pattern that is most similar; (This is considered a mental process, since a person can mentally compare a second section of feature pattern with representative patterns in a second section to determine the most similar second representative pattern, see MPEP 2106.04(a)(2)(III)),
v. and determine a cluster corresponding to an integrated representative pattern formed of sequential combinations of the first representative pattern and the second representative pattern as a cluster of the feature pattern, (This is considered a mental process, since a person can mentally identify a cluster that corresponds to an integrated representative pattern formed of sequential combinations of the first representative pattern and the second representative pattern as a cluster of the feature pattern, see MPEP 2106.04(a)(2)(III)),
If claim limitations, under their broadest reasonable interpretation, covers performance of the limitations as a mental process but for the recitation of generic computer components, then it falls within the mental process grouping of abstract ideas. Accordingly, the claim “recites” an abstract idea. Since the claim does not recite additional elements that either integrate the judicial exception into a practical application, nor provide significantly more than the judicial exception, the claim is not patent eligible.
Claim 5:
Regarding claim 5, it is dependent upon claim 4, and thereby incorporates the limitations of, and corresponding analysis applied to claim 4. Further, claim 5 recites the following additional element:
i. The apparatus of claim 4, wherein the controller is configured to, (In step 2A, prong 2, this is considered mere instructions to apply an exception using generic computer – see MPEP 2106.05(f)), (In step 2B, this is also considered mere instructions to apply an exception using generic computer – see MPEP 2106.05(f)),
Further, claim 5 recites the following abstract idea:
ii. determine a representative pattern that is most similar to the feature pattern for each of the divided sections, (This is a mental process, since a person can mentally identify and determine a most similar representative pattern to the feature patterns for each divided section, see MPEP 2106.04(a)(2)(III)),
If claim limitations, under their broadest reasonable interpretation, covers performance of the limitations as a mental process but for the recitation of generic computer components, then it falls within the mental process grouping of abstract ideas. Accordingly, the claim “recites” an abstract idea.
Since the claim does not recite additional elements that either integrate the judicial exception into a practical application, nor provide significantly more than the judicial exception, the claim is not patent eligible.
Claim 6:
Regarding claim 6, it is dependent upon claim 4, and thereby incorporates the limitations of, and corresponding analysis applied to claim 4. Further, claim 6 recites the following additional element:
i. The apparatus of claim 4, wherein the controller is configured to … (In step 2A, prong 2, this is considered mere instructions to apply an exception using generic computer – see MPEP 2106.05(f)), (In step 2B, this is also considered mere instructions to apply an exception using generic computer – see MPEP 2106.05(f)),
Further, claim 6 recites the following abstract idea:
ii. select a representative pattern having a shortest dot product or Euclidean distance, least mean square (LMS), or cross correlation (CC) with respect to the feature pattern for each divided section, (This is considered a mental process, since a person can mentally identify and select a representative pattern that has a shortest value of the metrics of Euclidean distance or dot product, LMS, or CC value with respect to the feature pattern, see MPEP 2106.04(a)(2)(III)), (This is also considered a math concept or calculation, cited from specification [0092] which states “As shown in FIG. 7, the controller 40 may calculate a distance between a feature pattern 710 and a representative pattern of a cluster and may determine a calculated distance as an uncertainty value of the feature pattern 710. In this case, the distance between the feature pattern 710 and the representative pattern of the cluster means a dot product, Euclidean distance, LMS, or CC, ” see MPEP 2106.04(a)(2), subsection I),
If claim limitations, under their broadest reasonable interpretation, covers performance of the limitations as a mental process or math concept, but for the recitation of generic computer components, then it falls within the mental process grouping or math concept grouping of abstract ideas. Accordingly, the claim “recites” an abstract idea.
Since the claim does not recite additional elements that either integrate the judicial exception into a practical application, nor provide significantly more than the judicial exception, the claim is not patent eligible.
Claim 9:
Regarding claim 9, it is dependent upon claim 4, and thereby incorporates the limitations of, and corresponding analysis applied to claim 4. Further, claim 9 recites the following additional element:
i. The apparatus of claim 4, wherein the controller is configured to …, (In step 2A, prong 2, this is considered mere instructions to apply an exception using generic computer – see MPEP 2106.05(f)), (In step 2B, this is also considered mere instructions to apply an exception using generic computer – see MPEP 2106.05(f)),
Further, claim 9 recites the following abstract idea:
ii. determine a distance between the integrated representative pattern of the cluster and the feature pattern as an uncertainty value of the feature pattern. (This recites a math calculation – see MPEP 2106.04(a)(2), subsection I), cited from the specification [0013] which states “The apparatus may also estimate a distance between the representative pattern in the cluster and the feature pattern as an uncertainty value of the feature pattern,” where in specification [0066] an equation shows how to find representative patterns “determine representative patterns of each section based on following Equation 1.”
If claim limitations, under their broadest reasonable interpretation, covers performance of the limitations as a mathematical concept but for the recitation of generic computer components, then it falls within the mathematical concept grouping of abstract ideas. Accordingly, the claim “recites” an abstract idea. Since the claim does not recite additional elements that either integrate the judicial exception into a practical application, nor provide significantly more than the judicial exception, the claim is not patent eligible.
Claim 10:
Regarding claim 10, it is dependent upon claim 9, and thereby incorporates the limitations of, and corresponding analysis applied to claim 9. Further, claim 10 recites the following additional element:
i. The apparatus of claim 9, wherein the controller is configured to, (In step 2A, prong 2, this is considered mere instructions to apply an exception using generic computer – see MPEP 2106.05(f)), (In step 2B, this is also considered mere instructions to apply an exception using generic computer – see MPEP 2106.05(f)),
Further, claim 10 recites the following abstract idea:
ii. … determine the feature pattern as an out of distribution (OOD) pattern when a distance between the integrated representative pattern of the cluster and the feature pattern exceeds a threshold distance, (This is a mental process, since a person can mentally evaluate and identify if a feature pattern is an out of distribution pattern if a distance between feature pattern and integrated representative pattern of a cluster is larger than a threshold distance, see MPEP 2106.04(a)(2)(III)),
If claim limitations, under their broadest reasonable interpretation, covers performance of the limitations as a mental process but for the recitation of generic computer components, then it falls within the mental process grouping of abstract ideas. Accordingly, the claim “recites” an abstract idea. Since the claim does not recite additional elements that either integrate the judicial exception into a practical application, nor provide significantly more than the judicial exception, the claim is not patent eligible.
Claim 11:
Regarding claim 11, it is dependent upon claim 1, and thereby incorporates the limitations of, and corresponding analysis applied to claim 1. Further, claim 11 recites the following additional element:
i. The apparatus of claim 1, wherein the clustering model includes a plurality of representative patterns for each layer as a tree-structured clustering model, (In step 2A, prong 2, this is considered mere instructions to apply an exception using generic computer – see MPEP 2106.05(f)), (In step 2B, this is also considered mere instructions to apply an exception using generic computer – see MPEP 2106.05(f)),
Since the claim does not recite additional elements that either integrate the judicial exception into a practical application, nor provide significantly more than the judicial exception, the claim is not patent eligible.
Claim 12:
Regarding claim 12, it is dependent upon claim 11, and thereby incorporates the limitations of, and corresponding analysis applied to claim 11. Further, claim 12 recites the following additional element:
i. The apparatus of claim 11, wherein the controller is configured to, (In step 2A, prong 2, this is considered mere instructions to apply an exception using generic computer – see MPEP 2106.05(f)), (In step 2B, this is also considered mere instructions to apply an exception using generic computer – see MPEP 2106.05(f)),
Further, claim 12 recites the following abstract ideas:
ii. determine a first representative pattern that is most similar to the feature pattern among representative patterns of a first layer in the tree-structured clustering model; (This is a mental process, since a person can mentally evaluate and determine if a first representative pattern is most similar to the feature pattern, see MPEP 2106.04(a)(2)(III)),
iii. determine a second representative pattern that is most similar to the feature pattern among sub-representative patterns of the first representative pattern; (This is a mental process, since a person can mentally evaluate and determine if a second representative pattern is most similar to the feature pattern among sub-representative patterns of the first representative pattern, see MPEP 2106.04(a)(2)(III)),
iv. and determine a cluster corresponding to the second representative pattern as a cluster of the feature pattern, (This is a mental process, since a person can mentally evaluate and determine a cluster that corresponds to a second representative pattern as a cluster of a feature pattern, see MPEP 2106.04(a)(2)(III)),
If claim limitations, under their broadest reasonable interpretation, covers performance of the limitations as a mental process but for the recitation of generic computer components, then it falls within the mental process grouping of abstract ideas. Accordingly, the claim “recites” an abstract idea.
Since the claim does not recite additional elements that either integrate the judicial exception into a practical application, nor provide significantly more than the judicial exception, the claim is not patent eligible.
Claim 13:
Regarding claim 13, it is dependent upon claim 12, and thereby incorporates the limitations of, and corresponding analysis applied to claim 12. Further, claim 13 recites the following additional element:
i. The apparatus of claim 12, wherein the controller is configured to …, (In step 2A, prong 2, this is considered mere instructions to apply an exception using generic computer – see MPEP 2106.05(f)), (In step 2B, this is also considered mere instructions to apply an exception using generic computer – see MPEP 2106.05(f)),
Further, claim 13 recites the following abstract idea:
ii. … determine a representative pattern that is most similar to the feature pattern for each layer, (This is a mental process, since a person can mentally evaluate and determine a representative pattern that is most similar to a feature pattern for each layer, see MPEP 2106.04(a)(2)(III)),
If claim limitations, under their broadest reasonable interpretation, covers performance of the limitations as a mental process but for the recitation of generic computer components, then it falls within the mental process grouping of abstract ideas. Accordingly, the claim “recites” an abstract idea. Since the claim does not recite additional elements that either integrate the judicial exception into a practical application, nor provide significantly more than the judicial exception, the claim is not patent eligible.
Claim 14:
Regarding claim 14, it is dependent upon claim 12, and thereby incorporates the limitations of, and corresponding analysis applied to claim 12. Further, claim 14 recites the following additional element:
i. The apparatus of claim 12, wherein the controller is configured to …, (In step 2A, prong 2, this is considered mere instructions to apply an exception using generic computer – see MPEP 2106.05(f)), (In step 2B, this is also considered mere instructions to apply an exception using generic computer – see MPEP 2106.05(f)),
Further, claim 14 recites the following abstract idea:
ii. … select a representative pattern having a shortest dot product or Euclidean distance, least mean square (LMS), or cross correlation (CC) with respect to the feature pattern for each layer, (This is considered a mental process, since a person can mentally identify and select a representative pattern that has a shortest value of the metrics of Euclidean distance or dot product, LMS, or CC value with respect to the feature pattern, see MPEP 2106.04(a)(2)(III)), (This is also considered a math concept or calculation, cited from specification [0092] which states “As shown in FIG. 7, the controller 40 may calculate a distance between a feature pattern 710 and a representative pattern of a cluster and may determine a calculated distance as
an uncertainty value of the feature pattern 710. In this case, the distance between the feature pattern 710 and the representative pattern of the cluster means a dot product, Euclidean distance, LMS, or CC, ” see MPEP 2106.04(a)(2), subsection I),
If claim limitations, under their broadest reasonable interpretation, covers performance of the limitations as a mental process or math concept, but for the recitation of generic computer components, then it falls within the mental process grouping or math concept grouping of abstract ideas. Accordingly, the claim “recites” an abstract idea. Since the claim does not recite additional elements that either integrate the judicial exception into a practical application, nor provide significantly more than the judicial exception, the claim is not patent eligible.
Claim 15:
Regarding claim 15, it is dependent upon claim 12, and thereby incorporates the limitations of, and corresponding analysis applied to claim 12. Further, claim 15 recites the following additional element:
i. wherein the storage is configured to further store a table in which uncertainty values for each of the plurality of clusters are recorded, (In step 2A, prong 2, this recites mere data storing, which is considered insignificant extra-solution activity – see MPEP 2106.05(g),). In step 2B, this insignificant extra-solution activity is well understood routine and conventional activity which includes storing and retrieving information in memory data from court case Versata Dev. Group, Inc. v. SAP Am., Inc., 793 F.3d 1306, 1334, 115 USPQ2d 1681, 1701 (Fed. Cir. 2015); OIP Techs., 788 F.3d at 1363, 115 USPQ2d at 1092-93; – see MPEP 2106.05(d) (II)(iv)), Since the claim does not recite additional elements that either integrate the judicial exception into a practical application, nor provide significantly more than the judicial exception, the claim is not patent eligible.
Claim 16:
Regarding claim 16, it is dependent upon claim 15, and thereby incorporates the limitations of, and corresponding analysis applied to claim 15. Further, claim 16 recites the following abstract idea:
i. ... to determine an uncertainty value corresponding to the cluster based on the table (This is considered a mental process, a person can mentally evaluate and determine an uncertainty value corresponding to the cluster from viewing a table, see MPEP 2106.04(a)(2)(III)),
Further, claim 16 recites the following additional element:
ii. The apparatus of claim 15, wherein the controller is configured…, (In step 2A, prong 2, this is considered mere instructions to apply an exception using generic computer – see MPEP 2106.05(f)), (In step 2B, this is also considered mere instructions to apply an exception using generic computer – see MPEP 2106.05(f)),
Since the claim does not recite additional elements that either integrate the judicial exception into a practical application, nor provide significantly more than the judicial exception, the claim is not patent eligible.
Claim 17:
Regarding claim 17, it is dependent upon claim 12, and thereby incorporates the limitations of, and corresponding analysis applied to claim 12. Further, claim 17 recites the following additional element:
i. The apparatus of claim 12, wherein the controller is configured to …, (In step 2A, prong 2, this is considered mere instructions to apply an exception using generic computer – see MPEP 2106.05(f)), (In step 2B, this is also considered mere instructions to apply an exception using generic computer – see MPEP 2106.05(f)),
ii. … determine a distance between the second representative pattern of the cluster and the feature pattern as an uncertainty value of the feature pattern, (This recites a math calculation – see MPEP 2106.04(a)(2), subsection I), cited from the specification [0013] which states “The apparatus may also estimate a distance between the representative pattern in the cluster and the feature pattern as an uncertainty value of the feature pattern,” where in specification [0066] an equation shows how to find representative patterns “determine representative patterns of each section based on following Equation 1.”
PNG
media_image1.png
80
968
media_image1.png
Greyscale
If claim limitations, under their broadest reasonable interpretation, covers performance of the limitations as a mathematical concept but for the recitation of generic computer components, then it falls within the mathematical concept grouping of abstract ideas. Accordingly, the claim “recites” an abstract idea. Since the claim does not recite additional elements that either integrate the judicial exception into a practical application, nor provide significantly more than the judicial exception, the claim is not patent eligible.
Claim 18:
Regarding claim 18, it is dependent upon claim 17, and thereby incorporates the limitations of, and corresponding analysis applied to claim 17. Further, claim 18 recites the following abstract idea:
i. The apparatus of claim 17, wherein the controller is configured to…, (In step 2A, prong 2, this is considered mere instructions to apply an exception using generic computer – see MPEP 2106.05(f)), (In step 2B, this is also considered mere instructions to apply an exception using generic computer – see MPEP 2106.05(f)),
ii. … determine the feature pattern as an out of distribution (OOD) pattern when a distance between the second representative pattern of the cluster and the feature pattern exceeds a threshold distance, (This is a mental process, since a person can mentally evaluate and identify if a feature pattern is an out of distribution pattern if a distance between feature pattern and integrated representative pattern of a cluster is larger than a threshold distance, see MPEP 2106.04(a)(2)(III)),
If claim limitations, under their broadest reasonable interpretation, covers performance of the limitations as a mental process but for the recitation of generic computer components, then it falls within the mental process grouping of abstract ideas. Accordingly, the claim “recites” an abstract idea.
Since the claim does not recite additional elements that either integrate the judicial exception into a practical application, nor provide significantly more than the judicial exception, the claim is not patent eligible.
Claim 19: Regarding claim 19, in step 1 of the 101-analysis set forth in MPEP 2106, the claim recites "A method of estimating uncertainty of an artificial neural network model, the method comprising: storing, by storage, a clustering model… ," and a method is one of the four statutory categories of invention.
In step 2A prong 1 of the 101-analysis set forth in the MPEP 2106, the examiner has determined that the following limitations recite a process that, under the broadest reasonable interpretation, covers a mental process or math concept but for recitation of generic computer components:
i. determining, … a cluster corresponding to the feature pattern by using the clustering model; (This is a mental process, since a person can mentally evaluate, identify, and determine a cluster that corresponds to a feature pattern, see MPEP 2106.04(a)(2)(III)),
ii. and looking up, … in the table, an uncertainty value corresponding to the determined cluster as an uncertainty value of the artificial neural network model at inference time, (This recites a mental process, a person can mentally evaluate and determine an uncertainty value or quantity that corresponds to the cluster from viewing a table, see MPEP 2106.04(a)(2)(III)),
If claim limitations, under their broadest reasonable interpretation, covers performance of the limitations as a mental process or mathematical concept, but for the recitation of generic computer components, then it falls within the mental process grouping or mathematical concept grouping of abstract ideas. Accordingly, the claim “recites” an abstract idea.
In step 2A prong 2 of the 101-analysis set forth in MPEP 2106, the examiner has determined that the following additional elements do not integrate this judicial exception into a practical application:
iii. A method of estimating uncertainty of an artificial neural network model, the method comprising: storing, by storage, a clustering model having a plurality of clusters; (In step 2A, prong 2, this recites mere data storing, which is considered insignificant extra-solution activity – see MPEP 2106.05(g)),
iv. and a table recording a respective uncertainty value for each of the plurality of clusters, (In step 2A, prong 2, this recites mere data gathering, which is considered insignificant extra-solution activity – see MPEP 2106.05(g)),
v. …by a controller, or ... by the controller (This is considered mere instructions to apply an exception using generic computer – see MPEP 2106.05(f)),
vi. extracting, … a feature pattern from a hidden layer of the artificial neural network model; (In step 2A, prong 2, this recites mere data gathering, which is considered insignificant extra-solution activity – see MPEP 2106.05(g)),
Since the claim as a whole, looking at the additional elements individually and in combination, does not contain any other additional elements that are indicative of integration into a practical application, the claim is “directed” to an abstract idea.
In step 2B of the 101-analysis set forth in the 2019 PEG, the examiner has determined that the claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
As discussed above, additional element v recites mere instructions to apply the judicial exception using generic computer components, which is not indicative of significantly more. The additional element iii recites mere data storing, and additional elements iv and vi recite data gathering, and are considered to be insignificant extra-solution activities. In step 2B, these insignificant extra-solution activities are well understood routine and conventional activities which include storing and retrieving information in memory data from court case Versata Dev. Group, Inc. v. SAP Am., Inc., 793 F.3d 1306, 1334, 115 USPQ2d 1681, 1701 (Fed. Cir. 2015); OIP Techs., 788 F.3d at 1363, 115 USPQ2d at 1092-93; – see MPEP 2106.05(d) (II)(iv)), as well as receiving or transmitting data over a network, e.g., using the Internet to gather data, see court case Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362; – see MPEP 2106.05(d) (II)(i)),
Considering the additional elements individually and in combination, and the claim as a whole, the additional elements do not provide significantly more than the abstract idea. Therefore, the claim is not patent eligible.
Claim 20:
Regarding claim 20, it is dependent upon claim 19, and thereby incorporates the limitations of, and corresponding analysis applied to claim 19. Further, claim 20 recites the following additional element:
i. The method of claim 19, wherein the clustering model includes a plurality of representative patterns for each section as a clustering model of a section division structure, or wherein the clustering model includes a plurality of representative patterns for each layer as a tree-structured clustering model, (In step 2A, prong 2, this is considered mere instructions to apply an exception using generic computer – see MPEP 2106.05(f)), (In step 2B, this is also considered mere instructions to apply an exception using generic computer – see MPEP 2106.05(f)),
Since the claim does not recite additional elements that either integrate the judicial exception into a practical application, nor provide significantly more than the judicial exception, the claim is not patent eligible.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1, 11, 19, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Shaffer R. (US PG Pub. No. US20010013026A1), published on August 9, 2001, (hereafter, SHAFFER), in view of Shahid, N. et al., in “Comparison of hierarchical clustering and neural network clustering: an analysis on precision dominance”, published on April 6, 2023, available on https://pmc.ncbi.nlm.nih.gov/articles/PMC10079863/pdf/41598_2023_Article_32790.pdf , (hereafter, SHAHID), further in view of Shrestha, D. et al., in “Machine learning approaches for estimation of prediction interval for the model output,” published on March 2006, available at: https://www.sciencedirect.com/science/article/pii/S0893608006000153 , (hereafter, Shrestha).
Claim 1:
Regarding claim 1, SHAFFER teaches “an apparatus for estimating uncertainty of an artificial neural network model, the apparatus comprising: a storage configured to store a clustering model having a plurality of clusters;”
See [0051], where SHAFFER describes “Referring to FIG. 5, once the raw sensor data is in the memory of the PNN training computer system 130, it can then be placed in a sensory data file 200 on a mass storage device as shown in FIG. 4. As shown in FIG. 5 and step S10 of FIG. 6, a pattern extraction unit 210 is provided which is an application specific module supplied by the user comprising software routines for converting raw sensor signals stored in the sensory data file into pattern vectors amenable for data analysis.” Here, SHAFFER describes a mass storage device as a storage for storing data into pattern vectors for data analysis. This storage is part of a larger computer system.
See [0011-0012], where SHAFFER also notes “Supervised pattern recognition algorithms used in pattern recognition unit 20 are known in the art and used to analyze chemical sensor 10 array data. The two most popular pattern recognition approaches are linear discriminant analysis (LDA) and artificial neural networks (ANN). … ANNs have become the de facto standard for chemical sensor pattern recognition due to the increasing power of personal computers and their inherent advantages in modeling complex data spaces. The typical ANN for chemical sensor array pattern recognition uses the back-propagation (BP) method for learning the classification rules. The conventional ANN comprises of an input layer, one or two hidden layers, and an output layer of neurons… For chemical sensor arrays, the neurons, as a group, serve to map the input pattern vectors to the desired outputs (data class).” The term clustering model is construed as any algorithm or model that recognizes patterns in data. Here, SHAFFER teaches that by using an artificial neural network (ANN) to map input pattern vectors to desired outputs or data class, this ANN is also a type of clustering model classifies data into classes or clusters.
Further, see [0008], where SHAFFER mentions “Recognition of the signature of the target compound(s) (analyte(s)) is based on the clustering of the patterns in the m-dimensional space.” Here, SHAFFER teaches that patterns from data can be clustered and pattern recognition models are built from the data, and data is later processed by a pattern recognition unit 20 (also part of the computer system).
Further, SHAFFER teaches “and a controller configured to: extract a feature pattern from a hidden layer of the artificial neural network model;”
See [0005], where SHAFFER shows "FIG. 1 is a diagram showing a configuration of a chemical detection apparatus known in the prior art which includes a sensor 10 and a pattern recognition unit 20. The pattern recognition unit 20 would include a computer system and software to analyze data received from the sensor 10 in order to identify the substance detected." Note the controller is construed by examiner as a computing system that can process or analyze data, extract information from data, or perform calculations or estimates from data. Here, SHAFFER shows a computing system that includes a sensor, a pattern recognition unit that can analyze data.
Further, see [0034], where SHAFFER mentions "Further objects and advantages of the present invention are achieved in accordance with embodiments by a PNN training computer system used to identify chemical analytes. This device uses a series of partially selective sensors to generate a set of sensory data when exposed to the chemical analytes. A pattern extraction unit connected to the partially selective sensors is used to extract a set of pattern vectors from the set of sensory data which form a set of initial hidden layer neurons of an initial probabilistic neural network. Then an LVQ classifier unit, connected to the pattern extraction unit, is used to reduce the number of neurons in the set of hidden layer neurons using learning vector quantization." Here, SHAFFER shows using a pattern extraction unit to extract pattern vectors from data that later forms a hidden layer of a probabilistic neural network model or PNN, (i.e. artificial neural network). SHAFFER shows extracting pattern vectors (a feature pattern) from a generated hidden layer.
However, SHAFFER did not explicitly teach "…and a table recording a respective uncertainty value for each of the plurality of clusters;" or “determine a cluster corresponding to the feature pattern by using the clustering model;” or "and look up, in the table, an uncertainty value corresponding to the determined cluster as an uncertainty value of the artificial neural network model at inference time."
In an analogous system, SHAHID teaches “determine a cluster corresponding to the feature pattern by using the clustering model;”
See SHAHID in Discussion section, pages 8-9, mention “A Self-Organizing Map (SOM)-Neural Network Clustering was adopted to group similar characteristic water variables into clusters. A Batch Learning Algorithm was used to identify classification of variables according to their grouping in an input space, 16 locations with 18 parametric values. Three aspects of SOM; Sample Hits, Neighbouring Weight Distances, Weight Planes were considered to highlight water components’ disposition and consolidation. The visual of sample hits, Fig. 3 demonstrates that two locations’ data is strongly correlated with respect to parametric elements, synonymous with a neuron marked ‘2’. This map also indicates that 16 samples locations can be categorized into 8 groups based on data conformity features.” Here, SHAHID teaches categorizing into groups (i.e. determine clusters) based on data conformity features that help identify classification of variables according to their grouping (i.e. corresponding to the feature pattern). The SOM-neural network clustering is a type of clustering model SHAHID teaches here.
Further, see SHAHID In Statistical Methods' Application section, on page 3 describe “to determine an improved machine learning approach presenting an optimised analysis of pattern based results from a large data, two methods, Neural Network Clustering (NNC) and Hierarchical Clustering (HC), are employed.” Here, SHAHID describes using pattern based results (i.e. feature pattern) in data using neural network clustering methods.
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to combine the base reference of SHAFFER and incorporate into the teachings of SHAHID because both references teach using feature patterns to define clusters and generate estimates for a neural network.
One of ordinary skill in the art would be motivated to do so because incorporating the system of SHAFFER into the methods of SHAHID would bring “the synthesis of hierarchical segmentation and GH-EXIN neural-network was deemed to improve the accuracy of clustering” (page 2, first full paragraph, SHAHID).
However, SHAFFER in view of SHAHID, did not teach “…and a table recording a respective uncertainty value for each of the plurality of clusters;” or “and look up, in the table, an uncertainty value corresponding to the determined cluster as an uncertainty value of the artificial neural network model at inference time.”
In an analogous art, Shrestha teaches “and a table recording a respective uncertainty value for each of the plurality of clusters;” and “and look up, in the table, an uncertainty value corresponding to the determined cluster as an uncertainty value of the artificial neural network model at inference time.”
See Shrestha in page 226, abstract mention “Uncertainty is expressed in the form of the two quantiles (constituting the prediction interval) of the underlying distribution of prediction errors. The idea is to partition the input space into different zones or clusters having similar model errors using fuzzy c-means clustering. The prediction interval is constructed for each cluster on the basis of empirical distributions of the errors associated with all instances belonging to the cluster under consideration and propagated from each cluster to the examples according to their membership grades in each cluster.” Here, Shrestha explicitly notes measuring uncertainty using prediction intervals, and these are measured for each cluster. Also, Shrestha shows that the uncertainty is ‘expressed in the form of the two quantiles (constituting the prediction interval),’ Shrestha shows that the prediction intervals represent uncertainty values and uncertainty is construed to mean a value that measures a model’s probability or confidence in producing accurate predictions. See table 2 for details.
Also, see Shrestha in page 226, Introduction first paragraph note “decision makers require not only accurate forecasting of certain variables but also the uncertainty estimates associated with the forecasts. In weather prediction, the forecasts are typically given together with the associated uncertainty, but in other areas, e.g. in water management the prevailing format of forecasting is deterministic, i.e. in the form of a point forecast.” Here, Shrestha shows that the point forecast is showing at inference time.
Further, see Shrestha in page 226, last paragraph of Introduction, mention “the proposed method is employed to estimate the prediction limit or interval using linear regression and three machine learning techniques—locally weighted regression, artificial neural networks, and model trees—for artificial and hydrological data sets. The performance of the novel method is compared to other methods.” Here, Shrestha describes using this method in artificial neural networks.
Also, see Shrestha in page 231, section 7.2 Procedure mention “ To estimate the effect of models' complexity on the prediction limits, experiments were also conducted using locally weighted regression, model trees and artificial neural networks as both prediction models and LUEMs.” Here, Shrestha mentions using the artificial neural network models in estimating the prediction limits, which here referred to the prediction interval that represented uncertainty values described from page 226.
Later, see Shrestha in page 231, section 8. Results, note “Table 2 shows the computed PIs for clusters for SieveQ1 data set using 95% degree confidence level (i.e. α=0.05).” Here, Shrestha describes a table to view prediction intervals (labeled PIs here) per cluster. Inference for one in the skill of the art, means using data from a small group (sample) to make educated conjectures on facts about a large group (population). Prediction intervals are also used in inference in that these produce educated conjectures about the data.
PNG
media_image2.png
475
878
media_image2.png
Greyscale
Also, see Shrestha in page 232, illustrate in table 2 of a prediction interval, which is viewed as an uncertainty value, since prediction interval estimate the confidence the probability a value is included within a range and probability shows this not being certain. Here, Shrestha also shows one can look up from viewing the table which cluster is associated with which prediction interval (i.e. uncertainty).
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to combine the references of SHAFFER and SHAHID and incorporate into the teachings of Shrestha since Shrestha teaches using of a table recording a respective uncertainty value for each of the plurality of clusters.
One of ordinary skill in the art would be motivated to do so because incorporating the systems of SHAFFER and SHAHID into the methods of Shrestha would achieve “a machine learning technique is an algorithm that estimates (induces) a hitherto unknown mapping (or dependency) between a system’s inputs and its outputs from the available data (Mitchell, 1998). As such a dependency is discovered, it can be used to predict (or effectively deduce) the future system’s output from the known input values,” (see Shrestha in page 227, section 3. Machine Learning Techniques).
Claim 11:
Regarding claim 11, SHAFFER in view of SHAHID, further in view of Shrestha, teach the limitations in claim 1. Further, SHAHID teaches “The apparatus of claim 1, wherein the clustering model includes a plurality of representative patterns for each layer as a tree-structured clustering model,”
See SHAHID in page 9, Discussion section, first full paragraph, describe “an ML algorithm ‘clusterdata’ was used to obtain a cluster tree Fig. 6, displaying different levels containing clusters of water concentration variables. The co-application of dissimilarity function and linkage function facilitated the linking of a pair of clusters/ a pair of variable & a cluster based on the closest proximity distance between them. To verify whether linking of variables into clusters in the [dendrogram] is an accurate representation of variables’ similarity or difference in a real system, a cophenetic coefficient was computed.” Here, SHAHID mentions a cluster tree that displays various levels of clusters and linking of variables into clusters shows (i.e. representative patterns). Further, see Figure 6 in SHAHID for a visual view of a tree-structured clustering model.
PNG
media_image3.png
481
660
media_image3.png
Greyscale
Further, SHAHID in page 2, part of Introduction section, first full paragraph, describe “A neural network, Self-Organizing Tree Algorithm (SOTA) was used by 33 for the analysis of gene expression data.... the algorithm was a hierarchical cluster obtained with accuracy and robustness of a neural network. Moreover, it was clarified that SOTA clustering had an advantage over classical hierarchical clustering, where clustering process is conducted from top to bottom and the highest hierarchical levels are resolved before going to the details of the lowest levels.” Here, SHAHID elaborates on applying tree structure clustering in context of a neural network model.
See page 5, first full paragraph, where SHAHID describes “In each plot, the connection of weights corresponding to a particular input with the layer’s neurons is represented by three prominent colours.” Here, SHAHID mentions that the clustering method can apply to the layer’s neurons (i.e. each neural network layer). See page 4, neural network constitution section for more details in SHAHID.
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to combine the base reference of SHAFFER and incorporate into the teachings of SHAHID because both references teach using feature patterns to define clusters and generate estimates for a neural network.
One of ordinary skill in the art would be motivated to do so because incorporating the system of SHAFFER into the methods of SHAHID would bring “the synthesis of hierarchical segmentation and GH-EXIN neural-network was deemed to improve the accuracy of clustering” (page 2, first full paragraph, SHAHID).
Claim 19:
Regarding claim 19, the claim recites similar limitations as corresponding claim 1 and is rejected for similar reasons as claim 1 using similar teachings and rationale.
Claim 20:
Regarding claim 20, the claim recites similar limitations as corresponding claim 11 and is rejected for similar reasons as claim 11 using similar teachings and rationale.
Claim 2 is rejected under 35 U.S.C. 103 as being unpatentable over SHAFFER in view of SHAHID, further in view of Shrestha, and further in view of Suarez-Alvarez, M. et al., in “Statistical approach to normalization of feature vectors and clustering of mixed datasets,” available on: https://www.jstor.org/stable/pdf/41727100.pdf?refreqid=fastly-default%3A9ff915d71a191f221a267e01132b98be&ab_segments=&initiator=&acceptTC=1 ,
Published on September 8, 2012, (hereafter, SUAREZ-ALVAREZ).
Claim 2:
Regarding claim 2, SHAFFER in view of SHAHID, further in view of Shrestha, teach the limitations in claim 1. However, referring to claim 2, SHAFFER in view of SHAHID, further in view of Shrestha, did not teach “The apparatus of claim 1, wherein the controller is configured to normalize the feature pattern.”
In an analogous system, SUAREZ-ALVAREZ teaches “The apparatus of claim 1, wherein the controller is configured to normalize the feature pattern,”
See page 2637, after equation 2.1, section (ii) Statistical treatment of feature vectors, where SUAREZ-ALVAREZ mentions "these approaches intended to normalize each feature component to the [0, 1] range. It was noted that providing all attributes are normally distributed, the probability of the attribute value normalized by (2.8) is in the [-1, 1] range is equal to 68 per cent." Further, see page 2633, first half paragraph, part of Introduction section, where SUAREZ-ALVAREZ describes “It will be shown that the accuracy usually increases when clustering is performed using normalized metrics”. Overall, SUAREZ-ALVAREZ teaches normalizing the feature or attributes (i.e. normalizing the feature pattern). The term feature pattern is construed by examiner to mean any set of data points or information that measures a feature or variable.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the references of SHAFFER, SHAHID, and Shrestha, with the teachings of SUAREZ-ALVAREZ by using the teachings of SHAFFER, SHAHID, and Shrestha, of a computer system that uses feature patterns to define clusters and generate estimates for a neural network, with SUAREZ-ALVAREZ’s teaching of normalizing the feature patterns.
One of ordinary skill in the art would be motivated to do so because by integrating SUAREZ-ALVAREZ’s framework into the system taught by SHAFFER, SHAHID, and Shrestha, one with ordinary skill in the art would achieve the goal of providing an “accuracy usually increases when clustering is performed using normalized metrics” (page 2633, first half paragraph, part of Introduction section, SUAREZ-ALVAREZ).
Claim 3 is rejected under 35 U.S.C. 103 as being unpatentable over SHAFFER in view of SHAHID, further in view of Shrestha, and further in view of Errico, J. et al., (US Patent No. US5796924A), published on August 18, 1998, (hereafter, ERRICO).
Claim 3:
Regarding claim 3, SHAFFER in view of SHAHID, further in view of Shrestha, teach the limitations in claim 1. Regarding claim 3, SHAFFER in view of SHAHID, further in view of Shrestha, did not teach “the apparatus of claim 1, wherein the clustering model includes a plurality of representative patterns for each section as a clustering model of a section division structure.”
In an analogous system, ERRICO teaches “The apparatus of claim 1, wherein the clustering model includes a plurality of representative patterns for each section as a clustering model of a section division structure”,
See col. 1, lines 28-40, where ERRICO describes " In a broad sense, a pattern recognition system is a device used to classify measurements (also know as features) that represent a set of patterns. The structure of a recognition system typically includes a sensor for gathering pattern data, a feature extraction mechanism, and a classification algorithm." Further, see col. 1, lines 62-67 through col. 2, lines 1-2, where ERRICO describes "Classification assigns pattern data into one or more prespecified classes based on the extracted features. A class is a set of patterns having certain traits or attributes in common. For example, handwriting samples of the letter E could belong to the same class. The task of a classification algorithm is to partition the feature space associated with the recognition system into separate regions for each class. The border of each class region is known as a decision boundary." Here, ERRICO teaches of a classification algorithm (i.e. clustering model) that includes pattern data into one or more prespecified classes based on extracted features and partition the feature associated with the recognition system into separate regions for each class (i.e. a plurality of representative patterns for each section…as a section division structure). See col. 3, lines 39-46, where ERRICO mentions additional details.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the references of SHAFFER, SHAHID, and Shrestha, with the teachings of ERRICO by using teachings of SHAFFER, SHAHID, and Shrestha, of a computer system that uses feature patterns to define clusters and generate estimates for a neural network, with ERRICO’s teaching of a clustering model that includes representative patterns for each section as a clustering model of a section division structure.
One of ordinary skill in the art would be motivated to do so because by integrating ERRICO’s framework into the methods of SHAFFER, SHAHID, and Shrestha, one with ordinary skill in the art would achieve the goal of a method that "enhances the fidelity of a training set and consequently improves the reliability of an associated pattern recognition system. It is also an advantage of the present invention to provide a method and system that improves the accuracy of a recognition system's classifiers without repetitive training," (ERRICO, col. 3, lines 5-9), and "this significantly improves the reliability of pattern classifiers because their training will emphasize distinguishing between classes that are otherwise easily confused," (ERRICO, col. 3, lines 65-67).
Claims 4 and 6 are rejected under 35 U.S.C. 103 as being unpatentable over SHAFFER in view of SHAHID, further in view of Shrestha, further in view of ERRICO, and further in view of Wunsch, II. D. et al., (US PG Pub. No. US20120221573A1), published on August 30, 2012, (hereafter, WUNSCH), and further in view of Ikeda, H. et al., (US PG Pub. No. US20030169919A1), published on September 11, 2023, (hereafter, IKEDA).
Claim 4:
Regarding claim 4, SHAFFER in view of SHAHID, further in view of Shrestha, further in view of ERRICO, teaches the limitations in claim 3.
However, SHAFFER in view of SHAHID, further in view of Shrestha, further in view of ERRICO, did not teach “The apparatus of claim 3, wherein the controller is configured to: divide the feature pattern into a plurality of sections; compare a first section of the feature pattern with representative patterns in a first section of the clustering model to determine a first representative pattern that is most similar; compare a second section of the feature pattern with representative patterns in a second section of the clustering model to determine a second representative pattern that is most similar; and determine a cluster corresponding to an integrated representative pattern formed of sequential combinations of the first representative pattern and the second representative pattern as a cluster of the feature pattern.”
In an analogous art, WUNSCH teaches “The apparatus of claim 3, wherein the controller is configured to: divide the feature pattern into a plurality of sections;”
See WUNSCH in [0075] describe “FIG. 14 illustrates a tree showing how data can be divided using the present methods and systems described herein. A large data set can be fed into the FIG. 2 system and can result in a plurality of clusters, here three clusters. Each of these clusters can be fed back into the FIG. 2 system which can further cluster and fragment the data of any individual cluster. As shown the leftmost cluster is divided into two more clusters at the next level. A leftmost cluster is then broken into two clusters. The leftmost cluster at that level is then divided into three clusters. " When WUNSCH mentions data being divided which results in clusters, and then fragment the data of any individual cluster (i.e. dividing the feature pattern into a plurality of sections). See WUNSCH in [0036] and [0037] for more information.
Further, WUNSCH teaches “compare a first section of the feature pattern with representative patterns in a first section of the clustering model to determine a first representative pattern that is most similar;”
See WUNSCH in [0047-0053] describe “Because the real partitions of the datasets used here are already known, the performance of the BARTMAP can then be evaluated by comparing the resulting clusters with the real structures in terms of external criteria. In this test of the methods described herein, both the Rand index and the adjusted Rand index, which is designed to correct the Rand index for randomness, are used. Assuming that P is a pre-specified partition of dataset X with N data objects, which is also independent from a clustering structure C resulting from the use of the BARTMAP algorithm/methodology for a pair of data objects xi and xj, results in four different cases based on how xi and xj are placed in C and P.
Case 1: xi and xj belong to the same clusters of C and the same category of P.
Case 2: xi and xj belong to the same clusters of C but different categories of P.
Case 3: xi and xj belong to different clusters of C but the same category of P.
Case 4: xi and xj belong to different clusters of C and a different category of P. Correspondingly, the number of pairs of samples for the four cases are denoted as a, b, c, and d, respectively. Because the total number of pairs of samples is M(M−1)/2, denoted a L, we have a+b+c+d=L. The Rand index and the adjusted Rand index can then be defined as follows, with larger values indicating more similarity between C and P,”
PNG
media_image4.png
208
526
media_image4.png
Greyscale
Here, WUNSCH describes comparing a P variable which is a pre-specified partition of dataset X with N data objects, with a clustering structure C, with the Rand index, a metric of similarity (i.e. that is most similar to a representative pattern).
Further, see WUNSCH in [0009] describe " a data interpretation system includes a first module to receive a first subspace of inputs from a data set and to produce first clusters", see further information from [0004] where WUNSCH describes models of clustering. In [0047], WUNSCH compares the resulting clusters (i.e. feature pattern) with the real structures (i.e. representative pattern). When WUNSCH describes clustering structures in [0048], WUNSCH shows using the BARTMAP clustering model to compare similarities between the two types of patterns from a first part of feature data to a corresponding cluster in the model. WUNSCH also talks about in [0009] that a first cluster is created from a first subspace of inputs, and this corresponds to a first section of the BARTMAP clustering model. Since there are various clusters from the partition of dataset X with N data objects, one of them include a first cluster created from a first subspace of inputs, where first subspace relates to first section.
Further, WUNSCH teaches “compare a second section of the feature pattern with representative patterns in a second section of the clustering model to determine a second representative pattern that is most similar;”
See WUNSCH in [0007] note “an example method according to the present disclosure is an unsupervised method for extracting information from a data set, including: …creating second clusters of related data from a second subspace of data in the data set; …The method of above may further include inputting the first cluster into the creating first cluster and creating the second cluster, iteratively.”
Further, WUNSCH in [0047-0048, 0053] describe “because the real partitions of the datasets used here are already known, the performance of the BARTMAP can then be evaluated by comparing the resulting clusters with the real structures in terms of external criteria. In this test of the methods described herein, both the Rand index and the adjusted Rand index, which is designed to correct the Rand index for randomness, are used. Assuming that P is a pre-specified partition of dataset X with N data objects, which is also independent from a clustering structure C resulting from the use of the BARTMAP algorithm/methodology for a pair of data objects xi and xj, results in four different cases based on how xi and xj are placed in C and P … The Rand index and the adjusted Rand index can then be defined as follows, with larger values indicating more similarity between C and P,
PNG
media_image4.png
208
526
media_image4.png
Greyscale
Since WUNSCH mentions this is done iteratively in [0007], the method mentioned from [0047-0053] is performed for subsequent clusters including first, second, and subsequent clusters. See WUNSCH in [0004] and [0009] for more details. When WUNSCH describes clustering structures in [0048], WUNSCH talks about first, second, and possibly subsequent clusters with using the ith and jth data objects. WUNSCH describes comparing the resulting clusters (i.e. feature pattern) with the real structures (i.e. representative patterns) which shows using the BARTMAP clustering model to compare similarities between the two types of patterns from a second subspace of feature data to a corresponding cluster in the model. Since there are various clusters from the partition of dataset X with N data objects, one of them include a second cluster created from a second subspace of inputs, where second subspace relates to second section.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the references of SHAFFER, SHAHID, Shrestha, and ERRICO with the teachings of WUNSCH by using the teachings of SHAFFER, SHAHID, Shrestha, and ERRICO of using feature patterns to define clusters and generate estimates for a neural network, with WUNSCH’s teaching of comparing a section of a feature pattern with representative patterns to determine similarity of a representative pattern.
One of ordinary skill in the art would be motivated to do so because by integrating WUNSCH’s framework into the system of SHAFFER, SHAHID, Shrestha, and ERRICO, one with ordinary skill in the art would achieve the goal of providing an “improved algorithm can achieve clustering structures with higher qualities than or compared to those with other commonly used biclustering or clustering algorithms with significantly improved speed” (WUNSCH, [0006]).
However, SHAFFER in view of SHAHID, further in view of Shrestha, further in view of ERRICO, and further in view of WUNSCH did not teach “and determine a cluster corresponding to an integrated representative pattern formed of sequential combinations of the first representative pattern and the second representative pattern as a cluster of the feature pattern.”
In an analogous field, IKEDA teaches “and determine a cluster corresponding to an integrated representative pattern formed of sequential combinations of the first representative pattern and the second representative pattern as a cluster of the feature pattern,”
See IKEDA in paragraph [0119] describe “the cluster number update process of step S23 will now be described referring to FIG. 11. The CPU 11 sequentially selects each pattern data as observational pattern data (S31). The order of selection may be, for example, the order of assignment of provisional cluster numbers in step S21 (for example, the order of input). The CPU 11 then obtains the cluster number currently assigned to the observational pattern data and determines neighborhood clusters (S32).” Here, IKEDA teaches an order of selection (i.e. sequential combinations) of cluster numbers. The term sequential is construed to mean an order or grouping of any set of values. See [0115] in IKEDA for more information.
Further, IKEDA notes in [0109] “In the above description, clusters are determined by performing a learning process through recursive adjustment of learning condition parameters and using correlations between prototypes, new prototypes are added after the preliminary cluster determination, and secondary cluster determination process is applied. It is also possible to independently apply the prototype adding technique to a learning formation of a prototype map which is already being used or to a clustering technique.” Here, IKEDA mentions the determination of clusters is recursive, using first or second or subsequent combinations of pattern data (i.e. feature pattern) and observational pattern data (S31) (i.e. representative pattern) to form these clusters. IKEDA teaches that the combinations of pattern data and observational pattern data are part of an integrated representative pattern.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the references of SHAFFER, SHAHID, Shrestha, ERRICO, and WUNSCH with the teachings of IKEDA by using the teachings of SHAFFER, SHAHID, Shrestha, ERRICO, and WUNSCH of using feature patterns to define clusters and generate estimates for a neural network, with IKEDA’s teaching of a representative pattern formed of sequential combinations.
One of ordinary skill in the art would be motivated to do so because by integrating IKEDA’s framework into the system of SHAFFER, SHAHID, Shrestha, ERRICO, and WUNSCH, one with ordinary skill in the art would achieve the goal of providing “By adding a new prototype to this portion, it is possible to avoid adding new prototypes over the entirety of the boundary, and the efficiency for the re-learning and re-clustering processes can be improved,” ([0094], IKEDA).
Claim 6:
Regarding claim 6, SHAFFER in view of SHAHID, further in view of Shrestha, further in view of ERRICO, further in view of WUNSCH, and further in view of IKEDA teaches the limitations of claim 4. Further, SHAFFER teaches “The apparatus of claim 4, wherein the controller is configured to select a representative pattern having a shortest dot product or Euclidean distance, least mean square (LMS), or cross correlation (CC) with respect to the feature pattern for each divided section,”
See SHAFFER in [0034] mention "Further objects and advantages of the present invention are achieved … by a PNN training computer system used to identify chemical analytes. This device uses a series of partially selective sensors to generate a set of sensory data ... A pattern extraction unit connected to the partially selective sensors is used to extract a set of pattern vectors from the set of sensory data which form a set of initial hidden layer neurons of an initial probabilistic neural network. Then an LVQ classifier unit, connected to the pattern extraction unit, is used to reduce the number of neurons in the set of hidden layer neurons using learning vector quantization. Once the LVQ classifier completes processing, a… unit connected to the LVQ classifier unit identifies neurons in the set of hidden layer neurons which have the shortest Euclidean distance to the pattern vectors and eliminates neurons in the set of hidden layer neurons which have not been identified as having the shortest Euclidean distance to any pattern vectors of the set of pattern vectors". Here, SHAFFER teaches a unit that is configured to identify a pattern vector (i.e. representative pattern) that has a shortest Euclidean distance from the neurons in the hidden layer (i.e. feature pattern for each divided section).
Claim 5 is rejected under 35 U.S.C. 103 as being unpatentable over SHAFFER in view of SHAHID, further in view of Shrestha, further in view of ERRICO, further in view of WUNSCH, further in view of IKEDA, and further in view of Komkov S., et al., (US PG Pub. No. US20230229897A1), published on July 20, 2023, (hereafter, KOMKOV).
Claim 5:
Regarding claim 5, SHAFFER in view of SHAHID, further in view of Shrestha, further in view of ERRICO, further in view of WUNSCH, and further in view of IKEDA, teaches the limitations of claim 4.
However, SHAFFER in view of SHAHID, further in view of Shrestha, further in view of ERRICO, further in view of WUNSCH, and further in view of IKEDA did not teach “The apparatus of claim 4, wherein the controller is configured to determine a representative pattern that is most similar to the feature pattern for each of the divided sections.”
In an analogous system, KOMKOV teaches “The apparatus of claim 4, wherein the controller is configured to determine a representative pattern that is most similar to the feature pattern for each of the divided sections,”
See KOMKOV in [0079] mention "as will be described below in detail, the distributions obtained for two pieces of input data (their feature tensors) may be used to determine the distance value and used the distance value to modify the result of the normal operation of the neural network. For example, the NN may output some similarities to some predefined classes. By comparing the input data tensor distribution with representative class tensor distributions, a class with highest degree of compliance can be obtained. This may differ from the class determined by the NN... Similarly, for open-set classification, the input data tensor distribution may be compared with data tensor distributions of other data input previously to obtain the distance value and the distance value may be used to modify similarity between NN-output feature vectors before comparing their similarity." Here, KOMKOV shows comparing the input data tensor distribution (i.e. feature pattern) to see if that is similar to the representative class tensor distributions (i.e. representative patterns). Further, see KOMKOV in [0077] describe “each class may be associated with a representative distribution of the tensor... the comparison between the input data tensor distribution and the representative tensor distribution for a class provides a measure (an indication) of compliance with that class.” Here, KOMKOV teaches this method applies for each of the classes of the divided sections. Further, see KOMKOV in [0076] show “ input data is processed by a neural network with at least one intermediate layer. A tensor of a predetermined layer is analyzed for input data to obtain a discrete distribution function of the tensor features.” Here, KOMKOV shows this method applies to each layer of the neural network.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the references of SHAFFER, SHAHID, Shrestha, ERRICO, WUNSCH, and IKEDA along with the teachings of KOMKOV by using the teachings of SHAFFER, SHAHID, Shrestha, ERRICO, WUNSCH, and IKEDA of using feature patterns to define clusters and generate estimates for a neural network, with KOMKOV’s teaching of determining a representative pattern that is most similar to the feature patterns for each divided section.
One of ordinary skill in the art would be motivated to do so because by integrating KOMKOV’s framework into the system of SHAFFER, SHAHID, Shrestha, ERRICO, WUNSCH, and IKEDA, one with ordinary skill in the art would achieve the goal of providing a method that “may improve the main rate of the model, e.g. accuracy. The obtained similarity function may improve testing results on big representative production test sets,” (KOMKOV, [0079]).
Claims 9, and 10 are rejected under 35 U.S.C. 103 as being unpatentable over SHAFFER in view of SHAHID, further in view of Shrestha, further in view of ERRICO, further in view of WUNSCH, further in view of IKEDA, and further in view of Seo, I., et al., (Pub. No. KR101680055B1), published on November 29, 2016, (hereafter, SEO).
Claim 9:
Regarding claim 9, SHAFFER in view of SHAHID, further in view of Shrestha, further in view of ERRICO, further in further in view of WUNSCH, further in view of IKEDA, teaches the limitations in claim 4. However, SHAFFER in view of SHAHID, further in view of ERRICO, further in view of WUNSCH, further in view of IKEDA, did not teach “The apparatus of claim 4, wherein the controller is configured to determine a distance between the integrated representative pattern of the cluster and the feature pattern as an uncertainty value of the feature pattern.”
In an analogous system, SEO teaches “The apparatus of claim 4, wherein the controller is configured to determine a distance between the integrated representative pattern of the cluster and the feature pattern as an uncertainty value of the feature pattern,”
See SEO in [0017] describe “the present invention, the K-means clustering of step (b) comprises: (b1) a step in which an associative clustering module assumes the cluster center value calculated… Using (where Gi is the i-th group among c cluster groups, xk is an arbitrary coordinate point belonging to Gi , and Ji is the result of the sum of distances to an arbitrary center point in the i-th cluster group,”
PNG
media_image5.png
118
520
media_image5.png
Greyscale
See SEO from the above equation and in [0080] for details. Here, SEO mentions this J value is the sum of distance from an arbitrary center point to a point in the i-th cluster group. SEO connects this J value from [0017] to be an uncertainty value since If cluster centers are close, data points between them may belong to either cluster, which represents high uncertainty. This shows an integrated result since the value is a calculated sum.
Further, see SEO in [0091] mention “the updated center value is input into the second process above, and the second through fourth processes above are repeated. The center point where the J value shows no change or reaches the acceptable range (a range where the amount of change is judged to be negligible) is set as the optimal cluster result (center value and coordinates belonging to each cluster).” Here, SEO also describes determining the Euclidean distance (i.e. determine a distance) between a cluster (i.e. feature pattern) and an optimal cluster result (i.e. integrated representative pattern of the cluster) as a J value (i.e. uncertainty value).
Further, see SEO in equation 16 regarding the J value.
PNG
media_image6.png
560
1310
media_image6.png
Greyscale
See SEO in [0085] for more details. Regarding the term integrated representative pattern, the specification in [0077] states "the controller 40 may determine that the first feature pattern 420 is included in the 2-1-2-1 cluster. In this case, the integrated representative pattern of the 2-1-2-1 cluster may be expressed as a sequential combination of the second representative pattern of the first section, the first representative pattern of the second section, the second representative pattern of the third section, and the first representative pattern of the fourth section," which SEO shows this integrated representative pattern also includes a second representative pattern and subsequent representative patterns. Since SEO mentions the process is repeated, this applies to subsequent clusters and their representative patterns.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the references of SHAFFER, SHAHID, Shrestha, ERRICO, WUNSCH, and IKEDA along with the teachings of SEO by using the teachings of SHAFFER, SHAHID, Shrestha, ERRICO, WUNSCH, and IKEDA of using feature patterns to define clusters and generate estimates for a neural network, with SEO’s teaching of using a computer to determine a distance between representative pattern and feature pattern as an uncertainty values for feature pattern.
One of ordinary skill in the art would be motivated to do so because by integrating SEO’s framework into the system of SHAFFER, SHAHID, Shrestha, ERRICO, WUNSCH, and IKEDA, one with ordinary skill in the art would achieve the goal of providing “ a development method that can improve the efficiency and accuracy of an artificial neural network model by applying it to the configuration of training data and the training stage of the model during the development process of an artificial neural network model” (SEO, [0025]), and “By applying associative clustering, the accuracy (coefficient of determination) of the model was significantly improved, and the results of the artificial neural network ensemble model using data classified into three categories showed very high accuracy with a coefficient of determination of 0.88, which increased by up to 0.3” (SEO, [0134]).
Claim 10 is rejected under 35 U.S.C. 103 as being unpatentable over SHAFFER in view of SHAHID, further in view of Shrestha, further in view of ERRICO, further in view of WUNSCH, further in view of IKEDA, further in view of SEO, and further in view of KOMKOV.
Claim 10:
Regarding claim 10, SHAFFER in view of SHAHID, further in view of Shrestha, further in view of ERRICO, further in view of WUNSCH, further in view of IKEDA, and further in view of SEO teaches the limitations of claim 9. However, SHAFFER in view of SHAHID, further in view of Shrestha, further in view of ERRICO, further in view of WUNSCH, further in view of IKEDA, and further in view of SEO, did not teach “The apparatus of claim 9, wherein the controller is configured to determine the feature pattern as an out of distribution (OOD) pattern when a distance between the integrated representative pattern of the cluster and the feature pattern exceeds a threshold distance.”
In an analogous system, KOMKOV teaches “The apparatus of claim 9, wherein the controller is configured to determine the feature pattern as an out of distribution (OOD) pattern when a distance between the integrated representative pattern of the cluster and the feature pattern exceeds a threshold distance,”
See KOMKOV in [0126] describe "In the functional block 480, the, the characteristic value daggr is compared with a threshold. If daggr is bigger than the threshold, then the first input data (e.g. image 401) is an OOD data 495, since it (better said its distribution of an intermediate tensor) is far from the common distribution (of the intermediate tensor) 455_1 to 455_C of the training dataset. If daggr is not bigger than the threshold, then the first input data is ordinary data (not OOD data) 490. If equality applies, it may be defined by a fixed convention that the first data is OOD." Here, KOMKOV describes if the distance value daggr is larger than a threshold (i.e. exceeds a threshold distance), then that classifies the first input data (i.e. feature pattern) as an out of distribution (OOD) pattern. See SEO in [0046, 0088, 0116] for more details.
Further, see KOMKOV in paragraph [0197] mention "Alternatively, or in addition, the determining of the characteristic of the first input data may correspond to determining whether or not the first input data belong to one of the predetermined classes of data. This enables distinction between the out of distribution and in distribution cases of the first input data." Here, KOMKOV mentions comparing if a first input data (i.e. feature pattern) belong to one of the predetermined classes of data (i.e. integrated representative pattern). See [0119] and figure 4 in KOMKOV for more details for OOD representing out of distribution data in item 495.
PNG
media_image7.png
1028
688
media_image7.png
Greyscale
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the references of SHAFFER, SHAHID, Shrestha, ERRICO, WUNSCH, IKEDA, and SEO, along with the teachings of KOMKOV by using the teachings of SHAFFER, SHAHID, Shrestha, ERRICO, WUNSCH, IKEDA, and SEO, of using feature patterns to define clusters and generate estimates for a neural network, with KOMKOV’s teaching of determining a feature pattern as an out of distribution pattern if a distance exceeds a threshold distance between a representative pattern and a feature pattern.
One of ordinary skill in the art would be motivated to do so because by integrating KOMKOV’s framework into the system of SHAFFER, SHAHID, Shrestha, ERRICO, WUNSCH, IKEDA, and SEO, one with ordinary skill in the art would achieve the goal of providing a method that “may improve the main rate of the model, e.g. accuracy. The obtained similarity function may improve testing results on big representative production test sets,” (KOMKOV, [0079]).
Claims 12, 13, and 14 are rejected under 35 U.S.C. 103 as being unpatentable over SHAFFER in view of SHAHID, further in view of Shrestha, further in view of ERRICO, and further in view of KOMKOV.
Claim 12:
Regarding claim 12, SHAFFER in view of SHAHID, further in view of Shrestha, teach the limitations of claim 11.
Further, SHAHID teaches “The apparatus of claim 11, wherein the controller is configured to: determine a first representative pattern that is most similar to the feature pattern among representative patterns of a first layer in the tree-structured clustering model;”
See SHAHID in page 5, first full paragraph describes “to visualise weights (related to input vectors’ elements), we obtain a weight plane figure. A weight plane is configured corresponding to each element of an input vector (there are 16 input vectors, each having an 18 number of elements). It visualises the weights connecting each input to each of the neurons. Larger and smaller weights in the plot are represented by lighter and darker colours, respectively. The similar connection pattern of input elements demonstrates that those elements are highly correlated.” Here, SHAHID describes the connection pattern of input elements as a feature pattern that is part of a clustering model. See SHAHID in page 2, part of Introduction section, first full paragraph for details on tree-structure clustering model.
Further, see SHAHID in page 6, third point, describe “observing the weight planes of inputs Mg and K, a small cluster of yellow neurons in 4th quadrant shows larger weights associated to these neurons. In view of similar pattern of strongest positive connections, these two variables can be placed in a distinct cluster.” Here, SHAHID notes that once there is a similar pattern of strong connections, (i.e. most similar to feature pattern among representative patterns), then the variables are placed into a distinct cluster (i.e. determine a first representative pattern). The term representative pattern is construed by examiner to mean any pattern or trend that categorizes variables within a data, supported by specification [0062] which states a cluster can also be a part of the representative pattern, see specification [0074] for more details on additional metrics of shortest dot product or Euclidean distance, etc.
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to combine the base reference of SHAFFER and incorporate into the teachings of SHAHID because both references teach using feature patterns to define clusters and generate estimates for a neural network, incorporate into determining a first representative pattern that is most similar to the feature pattern among representative patterns of a first layer in the tree-structured clustering model.
One of ordinary skill in the art would be motivated to do so because incorporating the system of SHAFFER into the methods of SHAHID would bring “the synthesis of hierarchical segmentation and GH-EXIN neural-network was deemed to improve the accuracy of clustering” (page 2, first full paragraph, SHAHID).
However, SHAFFER in view of SHAHID, further in view of Shrestha, did not teach “determine a second representative pattern that is most similar to the feature pattern among sub-representative patterns of the first representative pattern; and determine a cluster corresponding to the second representative pattern as a cluster of the feature pattern.”
In an analogous art, ERRICO teaches “determine a second representative pattern that is most similar to the feature pattern among sub-representative patterns of the first representative pattern;”
See ERRICO in col. 6, lines 29-33 describe “A first decision boundary 74 is arbitrarily chosen to separate the clusters according to class. The first cluster 60 represents a sub-region of the o-class feature space, while the second and third clusters 61-63 represent sub-regions in the x-class feature space. The first cluster 60 includes overlapping example signals, that is, examples of x-class patterns.” Here, ERRICO shows that the second cluster (i.e. determine a second representative pattern) that represent sub-regions in the x-class feature space (i.e. most similar to feature pattern among sub-representative patterns) which includes data from the first cluster (i.e. of the first representative pattern).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the references of SHAFFER, SHAHID, and Shrestha, with the teachings of ERRICO by using teachings of SHAFFER, SHAHID, and Shrestha, of a computer system that uses feature patterns to define clusters and generate estimates for a neural network, with ERRICO’s teaching of determining a second representative pattern that is most similar to the feature pattern among sub-representative patterns of the first representative pattern.
One of ordinary skill in the art would be motivated to do so because by integrating ERRICO’s framework into the methods of SHAFFER, SHAHID, and Shrestha, one with ordinary skill in the art would achieve the goal of a method that "enhances the fidelity of a training set and consequently improves the reliability of an associated pattern recognition system. It is also an advantage of the present invention to provide a method and system that improves the accuracy of a recognition system's classifiers without repetitive training," (ERRICO, col. 3, lines 5-9), and "this significantly improves the reliability of pattern classifiers because their training will emphasize distinguishing between classes that are otherwise easily confused," (ERRICO, col. 3, lines 65-67).
However, SHAFFER in view of SHAHID, further in view of Shrestha, and further in view of ERRICO did not teach, “and determine a cluster corresponding to the second representative pattern as a cluster of the feature pattern,”
In an analogous field, KOMKOV teaches “and determine a cluster corresponding to the second representative pattern as a cluster of the feature pattern,”
See KOMKOV in [0077] describe “For example, for the purpose of open-set or closed-set classification, the distribution of the input data tensor may be compared with a similarly obtained distribution for other data. In case of closed-set classification, each class may be associated with a representative distribution of the tensor. The, the comparison between the input data tensor distribution and the representative tensor distribution for a class provides a measure (an indication) of compliance with that class.”
Further, see KOMKOV in [0037] mention “ the second distribution is obtained by averaging of a plurality of distributions determined for respective plurality of input data belonging to a same class.” Here, KOMKOV in [0037] describes determining if input data belongs to a class or (i.e. determine a cluster), and this corresponds to a second distribution which was specified by KOMKOV in [0077] that this distribution could be associated with a representative distribution of the tensor (i.e. second representative pattern as a cluster of the feature pattern). Since KOMKOV mentions a first distribution and second distribution, this process was repeated. See KOMKOV mention in [0030] that the meanings of classes and clusters are equivalent.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the references of SHAFFER, SHAHID, Shrestha, and ERRICO, along with the teachings of KOMKOV by using the teachings of SHAFFER, SHAHID, Shrestha, and ERRICO, of using feature patterns to define clusters and generate estimates for a neural network, with KOMKOV’s teaching of determining a cluster corresponding to the second representative pattern as a cluster of the feature pattern.
One of ordinary skill in the art would be motivated to do so because by integrating KOMKOV’s framework into the system of SHAFFER, SHAHID, Shrestha, and ERRICO, one with ordinary skill in the art would achieve the goal of providing a method that “may improve the main rate of the model, e.g. accuracy. The obtained similarity function may improve testing results on big representative production test sets,” (KOMKOV, [0079]).
Claim 13:
Regarding claim 13, SHAFFER in view of SHAHID, further in view of Shrestha, further in view of ERRICO, and further in view of KOMKOV, teaches the limitations of claim 12.
Further, KOMKOV teaches “the apparatus of claim 12, wherein the controller is configured to determine a representative pattern that is most similar to the feature pattern for each layer,”
See KOMKOV in [0079] mention "as will be described below in detail, the distributions obtained for two pieces of input data (their feature tensors) may be used to determine the distance value and used the distance value to modify the result of the normal operation of the neural network. For example, the NN may output some similarities to some predefined classes. By comparing the input data tensor distribution with representative class tensor distributions, a class with highest degree of compliance can be obtained. This may differ from the class determined by the NN... Similarly, for open-set classification, the input data tensor distribution may be compared with data tensor distributions of other data input previously to obtain the distance value and the distance value may be used to modify similarity between NN-output feature vectors before comparing their similarity." Here, KOMKOV shows comparing the input data tensor distribution (i.e. feature pattern) to see if that is similar to the representative class tensor distributions (i.e. representative patterns).
Further, see KOMKOV in [0077] describe “each class may be associated with a representative distribution of the tensor... the comparison between the input data tensor distribution and the representative tensor distribution for a class provides a measure (an indication) of compliance with that class.” Here, KOMKOV teaches this method applies for each of the classes of the divided sections.
Further, see KOMKOV in [0076] show “ input data is processed by a neural network with at least one intermediate layer. A tensor of a predetermined layer is analyzed for input data to obtain a discrete distribution function of the tensor features.” Here, KOMKOV shows this method applies to each layer of the neural network.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the references of SHAFFER, SHAHID, Shrestha, and ERRICO, along with the teachings of KOMKOV by using the teachings of SHAFFER, SHAHID, Shrestha, and ERRICO of using feature patterns to define clusters and generate estimates for a neural network, with KOMKOV’s teaching of determining a representative pattern that is most similar to the feature patterns for each divided section.
One of ordinary skill in the art would be motivated to do so because by integrating KOMKOV’s framework into the system of SHAFFER, SHAHID, Shrestha, and ERRICO one with ordinary skill in the art would achieve the goal of providing a method that “may improve the main rate of the model, e.g. accuracy. The obtained similarity function may improve testing results on big representative production test sets,” (KOMKOV, [0079]).
Claim 14:
Regarding claim 14, SHAFFER in view of SHAHID, further in view of Shrestha, further in view of ERRICO, further in view of KOMKOV, teaches the limitations of claim 12.
Further, SHAFFER teaches “the apparatus of claim 12, wherein the controller is configured to select a representative pattern having a shortest dot product or Euclidean distance, least mean square (LMS), or cross correlation (CC) with respect to the feature pattern for each layer,”
See SHAFFER in col. 5, lines 58-67, through col. 6, lines 1-10 mention "Further objects and advantages of the present invention are achieved … by a PNN training computer system used to identify chemical analytes. This device uses a series of partially selective sensors to generate a set of sensory data ... A pattern extraction unit connected to the partially selective sensors is used to extract a set of pattern vectors from the set of sensory data which form a set of initial hidden layer neurons of an initial probabilistic neural network. Then an LVQ classifier unit, connected to the pattern extraction unit, is used to reduce the number of neurons in the set of hidden layer neurons using learning vector quantization. Once the LVQ classifier completes processing, a… unit connected to the LVQ classifier unit identifies neurons in the set of hidden layer neurons which have the shortest Euclidean distance to the pattern vectors and eliminates neurons in the set of hidden layer neurons which have not been identified as having the shortest Euclidean distance to any pattern vectors of the set of pattern vectors". Here, SHAFFER teaches a unit that is configured to identify a pattern vector (i.e. representative pattern) that has a shortest Euclidean distance from the neurons in the hidden layer (i.e. feature pattern).
Further, SHAFFER in col. 4, lines 11-18 notes that “The classification of new patterns is performed by propagating the pattern vector through the PNN. The input layer 30 is used to store the new pattern while it is serially passed through the hidden layer neurons 40. At each neuron in the hidden layer 40, the distance (either dot product or Euclidean distance) is computed between the new pattern and the input layer 30 pattern stored in that particular hidden neuron 40.” Here, SHAFFER notes that this process applies to each input layer of the neural network, which stores the new pattern that is transmitted through the hidden layer (i.e. for each layer of the neural network).
Claims 15, 16, 17, and 18 are rejected under 35 U.S.C. 103 as being unpatentable over SHAFFER in view of SHAHID, further in view of Shrestha, further in view of ERRICO, further in view of KOMKOV, and further in view of SEO.
Claim 15:
Regarding claim 15, SHAFFER in view of SHAHID, further in view of Shrestha, further in view of ERRICO, and further in view of KOMKOV, teaches the limitations of claim 12. However, SHAFFER in view of SHAHID, further in view of ERRICO, and further in view of KOMKOV, did not teach “the apparatus of claim 12, wherein the storage is configured to further store a table in which uncertainty values for each of the plurality of clusters are recorded.”
In an analogous system, SEO teaches “The apparatus of claim 12, wherein the storage is configured to further store a table in which uncertainty values for each of the plurality of clusters are recorded.”
See SEO in [0077] describe “meanwhile, the aforementioned series of processes is a program in which algorithms are directly coded using a programming language to be finally executed by a computer, and such a program, containing the aforementioned mathematical formulas, is stored in the main memory of the computer, so that each corresponding module produces results using the input data and the program.” SEO mentions a storage that can store information.
Further, see SEO in [0134] mention “As a specific embodiment of the present invention, the above-mentioned turbidity (Turb) data (Turb t, Turb t-1, Turb t+1) were classified into 2 to 4 groups (classes) using a combinatorial clustering method, and an artificial neural network ensemble model was constructed to predict the turbidity concentration of tomorrow (t+1) using the turbidity concentration of today (t) 28 and the turbidity concentration of yesterday (t-1) as input parameters using an ensemble modeling technique. The results are shown in Table 2 below. By applying associative clustering, the accuracy (coefficient of determination) of the model was significantly improved, and the results of the artificial neural network ensemble model using data classified into three categories showed very high accuracy with a coefficient of determination of 0.88, which increased by up to 0.3. Figure 7 illustrates the verification results of an artificial neural network ensemble model when the number of clusters is 3.” See table 2 (translated) where SEO displays clustering data with various metrics.
PNG
media_image8.png
588
186
media_image8.png
Greyscale
Here, SEO illustrates displaying metrics such as root mean square error (RMSE), which can also be used to measure model uncertainty, representing the standard deviation of residuals (prediction errors). Here, RMSE values per cluster are stored in a table showing (i.e. uncertainty values for each of the plurality of clusters are recorded).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the references of SHAFFER, SHAHID, Shrestha, ERRICO, and KOMKOV, along with the teachings of SEO by using the teachings of SHAFFER, SHAHID, Shrestha, ERRICO, and KOMKOV, of using feature patterns to define clusters and generate estimates for a neural network, with SEO’s teaching of storing uncertainty values for each cluster in a table.
One of ordinary skill in the art would be motivated to do so because by integrating SEO’s framework into the system of SHAFFER, SHAHID, Shrestha, ERRICO, and KOMKOV, one with ordinary skill in the art would achieve the goal of providing “ a development method that can improve the efficiency and accuracy of an artificial neural network model by applying it to the configuration of training data and the training stage of the model during the development process of an artificial neural network model” (SEO, [0025]), and “By applying associative clustering, the accuracy (coefficient of determination) of the model was significantly improved, and the results of the artificial neural network ensemble model using data classified into three categories showed very high accuracy with a coefficient of determination of 0.88, which increased by up to 0.3” (SEO, [0134]).
Claim 16:
Regarding claim 16, SHAFFER in view of SHAHID, further in view of Shrestha, further in view of ERRICO, and further in view of KOMKOV, and further in view of SEO, teaches the limitations of claim 15.
Further, SEO teaches “the apparatus of claim 15, wherein the controller is configured to determine an uncertainty value corresponding to the cluster based on the table,”
See SEO in [0105] mention “the aforementioned series of processes is a program in which algorithms are directly coded using a programming language to be finally executed by a computer, and such a program, containing the aforementioned mathematical formulas, is stored in the main memory of the computer, so that each corresponding module produces results using the input data and the program.” Here, SEO mentions a computer that processes data and program, and relates to a controller. Note the controller is construed by examiner as a computing system that can process or analyze data, extract information from data, or perform calculations or estimates from data.
Further, SEO in [0134] mention “…the results are shown in Table 2 below. By applying associative clustering, the accuracy (coefficient of determination) of the model was significantly improved, and the results of the artificial neural network ensemble model using data classified into three categories showed very high accuracy with a coefficient of determination of 0.88, which increased by up to 0.3. Figure 7 illustrates the verification results of an artificial neural network ensemble model when the number of clusters is 3.” See table 2 (translated) where SEO displays clustering data with various metrics.
PNG
media_image8.png
588
186
media_image8.png
Greyscale
Here, SEO illustrates displaying metrics such as root mean square error (RMSE), which can also be used to measure model uncertainty, representing the standard deviation of residuals (prediction errors). Here, RMSE values per cluster are displayed in a table showing (i.e. uncertainty values corresponding to each of the clusters are recorded).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the references of SHAFFER, SHAHID, Shrestha, ERRICO, and KOMKOV, along with the teachings of SEO by using the teachings of SHAFFER, SHAHID, Shrestha, ERRICO, and KOMKOV, of using feature patterns to define clusters and generate estimates for a neural network, with SEO’s teaching of storing uncertainty values for each cluster in a table.
One of ordinary skill in the art would be motivated to do so because by integrating SEO’s framework into the system of SHAFFER, SHAHID, Shrestha, ERRICO, and KOMKOV, one with ordinary skill in the art would achieve the goal of providing “ a development method that can improve the efficiency and accuracy of an artificial neural network model by applying it to the configuration of training data and the training stage of the model during the development process of an artificial neural network model” (SEO, [0025]), and “By applying associative clustering, the accuracy (coefficient of determination) of the model was significantly improved, and the results of the artificial neural network ensemble model using data classified into three categories showed very high accuracy with a coefficient of determination of 0.88, which increased by up to 0.3” (SEO, [0134]).
Claim 17:
Regarding claim 17, SHAFFER in view of SHAHID, further in view of Shrestha, further in view of ERRICO, and further in view of KOMKOV, teaches the limitations of claim 12.
However, SHAFFER in view of SHAHID, further in view of Shrestha, further in view of ERRICO, and further in view of KOMKOV, did not teach “the apparatus of claim 12, wherein the controller is configured to determine a distance between the second representative pattern of the cluster and the feature pattern as an uncertainty value of the feature pattern.”
In an analogous system, SEO teaches “the apparatus of claim 12, wherein the controller is configured to determine a distance between the second representative pattern of the cluster and the feature pattern as an uncertainty value of the feature pattern,”
See SEO in [0017] describe “the present invention, the K-means clustering of step (b) comprises: (b1) a step in which an associative clustering module assumes the cluster center value calculated… Using (where Gi is the i-th group among c cluster groups, xk is an arbitrary coordinate point belonging to Gi , and Ji is the result of the sum of distances to an arbitrary center point in the i-th cluster group,”
PNG
media_image5.png
118
520
media_image5.png
Greyscale
See SEO from the above equation and in [0080] for details. Here, SEO mentions this J value is the sum of distance from an arbitrary center point to a point in the i-th cluster group. SEO connects this J value from [0017] to be an uncertainty value since If cluster centers are close, data points between them may belong to either cluster, which represents high uncertainty. Since this method can be applied to the i-th group, this can apply to the first, second, third clusters, and subsequent clusters.
Further, see SEO in [0091] mention “the updated center value is input into the second process above, and the second through fourth processes above are repeated. The center point where the J value shows no change or reaches the acceptable range (a range where the amount of change is judged to be negligible) is set as the optimal cluster result (center value and coordinates belonging to each cluster).” Here, SEO also describes determining the Euclidean distance (i.e. determine a distance) between a cluster (i.e. feature pattern) and an optimal cluster result (i.e. representative pattern of the cluster) as a J value (i.e. uncertainty value). Since SEO mentions the process is repeated from [0091], this method applies to subsequent clusters and their respective representative patterns, and includes a second representative pattern of the cluster. See SEO in [0085] for more details. Regarding the second representative pattern, this item is part of the integrated representative pattern, shown from specification in [0077].
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the references of SHAFFER, SHAHID, Shrestha, ERRICO, and KOMKOV along with the teachings of SEO by using the teachings of SHAFFER, SHAHID, Shrestha, ERRICO, and KOMKOV, of using feature patterns to define clusters and generate estimates for a neural network, with SEO’s teaching of using a computer to determine a distance between a second representative pattern and feature pattern as an uncertainty values for feature pattern.
One of ordinary skill in the art would be motivated to do so because by integrating SEO’s framework into the system of SHAFFER, SHAHID, Shrestha, ERRICO, and KOMKOV one with ordinary skill in the art would achieve the goal of providing “ a development method that can improve the efficiency and accuracy of an artificial neural network model by applying it to the configuration of training data and the training stage of the model during the development process of an artificial neural network model” (SEO, [0025]), and “By applying associative clustering, the accuracy (coefficient of determination) of the model was significantly improved, and the results of the artificial neural network ensemble model using data classified into three categories showed very high accuracy with a coefficient of determination of 0.88, which increased by up to 0.3” (SEO, [0134]).
Claim 18:
Regarding claim 18, SHAFFER in view of SHAHID, further in view of Shrestha, further in view of ERRICO, and further in view of KOMKOV, and further in view of SEO, teaches the limitations in claim 17. Further, KOMKOV teaches “the apparatus of claim 17, wherein the controller is configured to determine the feature pattern as an out of distribution (OOD) pattern when a distance between the second representative pattern of the cluster and the feature pattern exceeds a threshold distance,”
See KOMKOV in [0126] describe “In the functional block 480, the, the characteristic value daggr is compared with a threshold. If daggr is bigger than the threshold, then the first input data (e.g. image 401) is an OOD data 495, since it (better said its distribution of an intermediate tensor) is far from the common distribution (of the intermediate tensor) 455_1 to 455_C of the training dataset. If daggr is not bigger than the threshold, then the first input data is ordinary data (not OOD data) 490. If equality applies, it may be defined by a fixed convention that the first data is OOD.” Here, KOMKOV describes if the distance value d aggr is larger than a threshold (i.e. exceeds a threshold distance), then that classifies the first input data (i.e. feature pattern) as an out of distribution (OOD) pattern. See SEO in paragraphs [0046, 0075, 0088, 0116, 0185] for more details.
Further, see KOMKOV in paragraph [0197] mention “ the determining of the characteristic of the first input data may correspond to determining whether or not the first input data belong to one of the predetermined classes of data. This enables distinction between the out of distribution and in distribution cases of the first input data.” Since in the specification [0077], which states "the controller 40 may determine that the first feature pattern 420 is included in the 2-1-2-1 cluster. In this case, the integrated representative pattern of the 2-1-2-1 cluster may be expressed as a sequential combination of the second representative pattern of the first section, …" which SEO shows this integrated representative pattern also includes a second representative pattern and subsequent representative patterns.
In [0197], KOMKOV mentions comparing if a first input data (i.e. feature pattern) belong to one of the predetermined classes of data (i.e. second representative pattern), and KOMKOV in [0030] notes “In this way, the open-set classification may be performed, i.e. by comparing input data with other input data and building possibly new classes/clusters,” where KOMKOV shows building new classes mean predetermined classes include first cluster, second cluster, and subsequent clusters as representative patterns. The term representative pattern is construed by examiner to mean any pattern or trend that categorizes variables within a data, supported by specification [0062] which states a cluster can also be a part of the representative pattern.
See [0119] and figure 4 in KOMKOV for more details for OOD representing out of distribution data in item 495.
PNG
media_image7.png
1028
688
media_image7.png
Greyscale
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the references of SHAFFER, SHAHID, Shrestha, ERRICO, along with the teachings of KOMKOV by using the teachings of SHAFFER, SHAHID, Shrestha, ERRICO, of using feature patterns to define clusters and generate estimates for a neural network, with KOMKOV’s teaching of determining a feature pattern as an out of distribution pattern if a distance exceeds a threshold distance between a representative pattern and a feature pattern.
One of ordinary skill in the art would be motivated to do so because by integrating KOMKOV’s framework into the system of SHAFFER, SHAHID, Shrestha, ERRICO, one with ordinary skill in the art would achieve the goal of providing a method that “may improve the main rate of the model, e.g. accuracy. The obtained similarity function may improve testing results on big representative production test sets,” (KOMKOV, [0079]).
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to WENWEI ZENG whose telephone number is (571)272-7111. The examiner can normally be reached Monday-Friday, 8am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Usmaan Saeed can be reached at (571) 272-4046. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/WenWei Zeng/Examiner, Art Unit 2146
/USMAAN SAEED/Supervisory Patent Examiner, Art Unit 2146
