Final Rejection
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This action is in response to the amendment filed on Nov. 17th, 2025. The amendments are linked to the original application filed on Dec. 17th 2021.
Response to Amendment
The Examiner thanks the applicant for the remarks, edits and arguments.
Regarding Claim Rejections – 35 U.S.C. 101
Applicant Remarks:
The applicant has amended the independent claims to recite the use of a machine learning model. Therefore, the applicant believes that the claims no longer recite an abstract idea and the rejection should be withdrawn.
Examiner Response:
The applicant has amended the claims to recite the use of a machine learning model. According to the MPEP the examiner must examine the claims to see if they recite an abstract idea concept, such as observation, evaluation, judgment, opinion. When evaluating the claims in this application, there are some limitations in the independent claims which recite an abstract concept of evaluation, such as in claim 1, “determining whether the data input is an outlier based on the threshold entropy-based distance metric.”. A human, with the assistance of pen and paper, would be able to evaluate and provide opinions on data to determine if data is considered outlying data. However, in this case a human would not be able to process the data effectively or efficiently when compared to a computer and per the specification and claims a computing device is used instead of a human. Per the MPEP 2106.04(a)(III)(C): “In evaluating whether a claim that requires a computer recites a mental process, examiners should carefully consider the broadest reasonable interpretation of the claim in light of the specification. For instance, examiners should review the specification to determine if the claimed invention is described as a concept that is performed in the human mind and applicant is merely claiming that concept performed 1) on a generic computer, or 2) in a computer environment, or 3) is merely using a computer as a tool to perform the concept. In these situations, the claim is considered to recite a mental process.”. Considering this and the abstract limitations, it is clear that some of the abstract ideas are being performed on a computer system or in a computer environment. Therefore, since this considered an abstract idea and this limitation is designed to execute in a computing environment, under the MPEP section above, this limitation would still be considered be reciting an abstract idea.
After each amendment the examiner must review the claims and assess for patent eligibility. The Alice/Mayo test is used to review the claims to ensure patentability. In doing this, the examiner has found that the amended claims still recite abstract ideas, which are performed in a computing environment. Therefore, the amended claims are still rejected under 35 U.S.C. 101, see 101 rejection below.
Regarding Claim Rejections – 35 U.S.C. 102
Applicant Remarks:
The applicant has amended the independent claims in a way that Huang is no longer able to anticipate the claims. Further the applicant states that Huang fails to teach the use of sub-classes which are based on neural network state of the machine learning model and generation of activation values associated with neuron outputs.
Examiner Response:
After each amendment the examiner must reevaluate the proposed art used in the previous rejection. During this evaluation, the examiner found that the original art, Huang, does fail to explicitly teach the current amendments to the standards of a 102 rejection. It is noted that Huang does fail to teach every element in the independent claims, however Huang is still able to teach some of the amended claims. Further search was conducted and no single art was able to teach the independent claims explicitly. Therefore, the examiner has withdrawn the rejection under 35 U.S.C. 102.
Regarding Claim Rejections – 35 U.S.C. 103
Applicant Remarks:
The applicant argues that independent claims have been amended and Huang is unable to anticipate these limitations. Because of this the art used for some of the dependent claims would be invalid because they fail to cover the material that Huang is unable to cover. Therefore, the proposed art fails to cover the claims as a whole and therefore the 103 rejection should be withdrawn.
Examiner Response:
The applicant argues that the art used in the 103 rejection would not be able to overcome the deficiencies of Huang and therefore the art proposed would fail to teach, in combination with Huang, the dependent claims. The examiner has reviewed the current amendments and has found that Huang was unable to solely teach the independent claims and the art used in the dependent claims was unable to teach the deficiencies of Huang. However, after each amendment, the examiner must perform a complete and thorough search. In doing this the examiner must reexamine the claims for potential double patent issues and other art that could be used to teach the amended claims. During this search, the examiner found art which, in combination with Huang, does teach he amended claims. The examiner believes that the combination of the new art and the previously proposed art, teaches the amended claims. Therefore, the rejection under 35 U.S.C. 103 is upheld, see 103 rejection below.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C 101 because the claimed invention is
directed to an abstract idea without significantly more. The analysis of the claims will
follow the 2019 Revised Patent Subject Matter Eligibility Guidance, 84 Fed. Reg. 50
(“2019 PEG”).
Claim 1
Step 1 – Is the claim to a process, machine, manufacture or composition of matter?
Claim 1, recites “A method for determining outlier inputs for a machine learning system, the method comprising:” therefore it is directed to the statutory category of a process.
Step 2A Prong 1 – Does the claim recite an abstract idea, law of nature, or natural
phenomenon?
The claim recites, inter alia:
“operating on the data input at the machine learning model to generate classification values to classify the data input and to generate activation values associated with neuron outputs from the neural network state;” Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of evaluating and observing data, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper. A human is able to evaluate data and apply judgements to that data in determining outlier input data. The limitation is merely applying an abstract idea on generic computer system. See MPEP 2106.04(a)(2)(III)(c).
“determining, at the outlier identifier, whether an entropy score derived from the classification values and the activation values corresponding to the data input is below a threshold entropy-based distance metric; and” Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of evaluating and observing data, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper. A human is able to evaluate data and make judgements and opinions on the results of the evaluation. The limitation is merely applying an abstract idea on generic computer system. See MPEP 2106.04(a)(2)(III)(c).
“determining whether the data input is an outlier based on the threshold entropy-based distance metric.” Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of evaluating and observing data, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper. A human is able to determine if data is outlying data through observations and evaluations. The limitation is merely applying an abstract idea on generic computer system. See MPEP 2106.04(a)(2)(III)(c).
Step 2A Prong 2 – Does the claim recite additional elements that integrate the judicial exception into a practical application?
The claim recites the additional elements, “receiving a data input at a machine learning model, wherein the machine learning model operates as a trained classifier having multiple classes as an output for data classification with each class of the multiple classes having sub-classes which are based on neural network state of the machine learning model;” is an insignificant extra-solution activity required for any uses of the mental processes (see MPEP § 2106.05(g)) As such, the claim is ineligible.
“receiving the classification values and activation values at an outlier identifier;” is an insignificant extra-solution activity required for any uses of the mental processes (see MPEP § 2106.05(g)) As such, the claim is ineligible.
Step 2B – Does the claim recite additional elements that amount to significantly more than the judicial exception?
Finally, the claim taken as a whole does not contain an inventive concept which provides significantly more than the abstract idea. The additional elements, “receiving a data input at a machine learning model, wherein the machine learning model operates as a trained classifier having multiple classes as an output for data classification with each class of the multiple classes having sub-classes which are based on neural network state of the machine learning model;” is an insignificant extra-solution activity required for any uses of abstract ideas (see MPEP § 2106.05(g)), and is a well-understood, routine, conventional activity (see MPEP § 2106.05(d)(i); “Receiving or transmitting data over a network, e.g., using the Internet to gather data”.
“receiving the classification values and activation values at an outlier identifier;” is an insignificant extra-solution activity required for any uses of abstract ideas (see MPEP § 2106.05(g)), and is a well-understood, routine, conventional activity (see MPEP § 2106.05(d)(i); “Receiving or transmitting data over a network, e.g., using the Internet to gather data”.
Taken alone or in combination, the additional elements of the claim do not provide an inventive concept and thus the claim is subject-matter ineligible.
Claim 2
Step 1 – Is the claim to a process, machine, manufacture or composition of matter?
A process, as above.
Step 2A Prong 1 – Does the claim recite an abstract idea, law of nature, or natural
phenomenon?
The claim recites the abstract ideas of the preceding claims from which it depends.
Step 2A Prong 2 – Does the claim recite additional elements that integrate the judicial exception into a practical application?
The claim recites the additional elements, “generating a reference probability distribution database from a training data set for the trained classifier.” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
Step 2B – Does the claim recite additional elements that amount to significantly more than the judicial exception?
Finally, the claim taken as a whole does not contain an inventive concept which provides significantly more than the abstract idea. The additional elements, “generating a reference probability distribution database from a training data set for the trained classifier.” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
Taken alone or in combination, the additional elements of the claim do not provide an inventive concept and thus the claim is subject-matter ineligible.
Claim 3
Step 1 – Is the claim to a process, machine, manufacture or composition of matter?
A process, as above.
Step 2A Prong 1 – Does the claim recite an abstract idea, law of nature, or natural
phenomenon?
The claim recites the abstract ideas of the preceding claims from which it depends.
Step 2A Prong 2 – Does the claim recite additional elements that integrate the judicial exception into a practical application?
The claim recites the additional elements, “calibrating the reference probability distribution database to reduce a number of activation values utilized for the method based on relevance of each of the activation values in identifying the sub-classes.” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
Step 2B – Does the claim recite additional elements that amount to significantly more than the judicial exception?
Finally, the claim taken as a whole does not contain an inventive concept which provides significantly more than the abstract idea. The additional elements, “calibrating the reference probability distribution database to reduce a number of activation values utilized for the method based on relevance of each of the activation values in identifying the sub-classes.” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
Taken alone or in combination, the additional elements of the claim do not provide an inventive concept and thus the claim is subject-matter ineligible.
Claim 4
Step 1 – Is the claim to a process, machine, manufacture or composition of matter?
A process, as above.
Step 2A Prong 1 – Does the claim recite an abstract idea, law of nature, or natural
phenomenon?
The claim recites, inter alia:
“wherein the determining whether the entropy score is below the threshold further comprises: determining whether the entropy score is below the threshold of any one of the sub-classes.” Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of evaluating and observing data, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper. A human is able to evaluate data and check if it meets a threshold or not. The limitation is merely applying an abstract idea on generic computer system. See MPEP 2106.04(a)(2)(III)(c).
Step 2A Prong 2 – Does the claim recite additional elements that integrate the judicial exception into a practical application?
This claim does not recite any additional limitations which integrate the abstract idea into a practical application.
Step 2B – Does the claim recite additional elements that amount to significantly more than the judicial exception?
Finally, the claim taken as a whole does not contain an inventive concept which provides significantly more than the abstract idea and thus the claim is subject-matter ineligible.
Claim 5
Step 1 – Is the claim to a process, machine, manufacture or composition of matter?
A process, as above.
Step 2A Prong 1 – Does the claim recite an abstract idea, law of nature, or natural
phenomenon?
The claim recites, inter alia:
“sorting each item in a training data set into the sub-classes by using a clustering algorithm.” Under its broadest reasonable interpretation in light of the specification, this limitation encompasses mathematical concept of utilizing a mathematical formula to perform calculations. A human is able to use mathematical algorithms to evaluate data. This claim discloses a math operation and therefore is ineligible.
Step 2A Prong 2 – Does the claim recite additional elements that integrate the judicial exception into a practical application?
This claim does not recite any additional limitations which integrate the abstract idea into a practical application.
Step 2B – Does the claim recite additional elements that amount to significantly more than the judicial exception?
Finally, the claim taken as a whole does not contain an inventive concept which provides significantly more than the abstract idea and thus the claim is subject-matter ineligible.
Claim 6
Step 1 – Is the claim to a process, machine, manufacture or composition of matter?
A process, as above.
Step 2A Prong 1 – Does the claim recite an abstract idea, law of nature, or natural
phenomenon?
The claim recites, inter alia:
“wherein a count is maintained of each item assigned to each sub-class.” Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of evaluating and observing data, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper. A human is able to evaluate data and with pen and paper maintain a count. The limitation is merely applying an abstract idea on generic computer system. See MPEP 2106.04(a)(2)(III)(c).
Step 2A Prong 2 – Does the claim recite additional elements that integrate the judicial exception into a practical application?
This claim does not recite any additional limitations which integrate the abstract idea into a practical application.
Step 2B – Does the claim recite additional elements that amount to significantly more than the judicial exception?
Finally, the claim taken as a whole does not contain an inventive concept which provides significantly more than the abstract idea and thus the claim is subject-matter ineligible.
Claim 7
Step 1 – Is the claim to a process, machine, manufacture or composition of matter?
A process, as above.
Step 2A Prong 1 – Does the claim recite an abstract idea, law of nature, or natural
phenomenon?
The claim recites the abstract ideas of the preceding claims from which it depends.
Step 2A Prong 2 – Does the claim recite additional elements that integrate the judicial exception into a practical application?
The claim recites the additional elements, “wherein a threshold is managed as a hyperparameter for each sub-class to assign items in a training data set to a respective sub-class that exceeds the threshold.” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
Step 2B – Does the claim recite additional elements that amount to significantly more than the judicial exception?
Finally, the claim taken as a whole does not contain an inventive concept which provides significantly more than the abstract idea. The additional elements, “wherein a threshold is managed as a hyperparameter for each sub-class to assign items in a training data set to a respective sub-class that exceeds the threshold.” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
Taken alone or in combination, the additional elements of the claim do not provide an inventive concept and thus the claim is subject-matter ineligible.
Claim 8
Step 1 – Is the claim to a process, machine, manufacture or composition of matter?
Claim 8, recites “An electronic device to execute a method for determining outlier inputs for a machine learning system, the electronic device comprising: a non-transitory computer-readable medium having stored therein an outlier identifier; and a processor coupled to the non-transitory computer-readable medium, the processor to execute on the outlier identifier, for the electronic device to:” therefore it is directed to the statutory category of a machine.
Step 2A Prong 1 – Does the claim recite an abstract idea, law of nature, or natural
phenomenon?
The claim recites, inter alia:
“operate on the data input at the machine learning model to generate classification values to classify the data input and to generate activation values associated with neuron outputs from the neural network state;” Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of evaluating and observing data, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper. A human is able to evaluate data and apply judgements to that data in determining outlier input data. The limitation is merely applying an abstract idea on generic computer system. See MPEP 2106.04(a)(2)(III)(c).
“determine whether an entropy score derived from the classification values and the activation values corresponding to the data input is below a threshold entropy-based distance metric; and” Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of evaluating and observing data, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper. A human is able to evaluate data and make judgements and opinions on the results of the evaluation. The limitation is merely applying an abstract idea on generic computer system. See MPEP 2106.04(a)(2)(III)(c).
“determine whether the data input is an outlier based on the threshold entropy- based distance metric.” Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of evaluating and observing data, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper. A human is able to determine if data is outlying data through observations and evaluations. The limitation is merely applying an abstract idea on generic computer system. See MPEP 2106.04(a)(2)(III)(c).
Step 2A Prong 2 – Does the claim recite additional elements that integrate the judicial exception into a practical application?
The claim recites the additional elements, “receive a data input at a machine learning model, wherein the machine learning model operates as a trained classifier having multiple classes as an output for data classification with each class of the multiple classes having sub- classes which are based on neural network state of the machine learning model;” is an insignificant extra-solution activity required for any uses of the mental processes (see MPEP § 2106.05(g)) As such, the claim is ineligible.
“receive the classification values and activation values to be operated on by the outlier identifier;” is an insignificant extra-solution activity required for any uses of the mental processes (see MPEP § 2106.05(g)) As such, the claim is ineligible.
Step 2B – Does the claim recite additional elements that amount to significantly more than the judicial exception?
Finally, the claim taken as a whole does not contain an inventive concept which provides significantly more than the abstract idea. The additional elements, “receive a data input at a machine learning model, wherein the machine learning model operates as a trained classifier having multiple classes as an output for data classification with each class of the multiple classes having sub- classes which are based on neural network state of the machine learning model;” is an insignificant extra-solution activity required for any uses of abstract ideas (see MPEP § 2106.05(g)), and is a well-understood, routine, conventional activity (see MPEP § 2106.05(d)(i); “Receiving or transmitting data over a network, e.g., using the Internet to gather data”.
“receive the classification values and activation values to be operated on by the outlier identifier;” is an insignificant extra-solution activity required for any uses of abstract ideas (see MPEP § 2106.05(g)), and is a well-understood, routine, conventional activity (see MPEP § 2106.05(d)(i); “Receiving or transmitting data over a network, e.g., using the Internet to gather data”.
Taken alone or in combination, the additional elements of the claim do not provide an inventive concept and thus the claim is subject-matter ineligible.
Claim 9
Step 1 – Is the claim to a process, machine, manufacture or composition of matter?
A machine, as above.
Step 2A Prong 1 – Does the claim recite an abstract idea, law of nature, or natural
phenomenon?
The claim recites the abstract ideas of the preceding claims from which it depends.
Step 2A Prong 2 – Does the claim recite additional elements that integrate the judicial exception into a practical application?
The claim recites the additional elements, “wherein the outlier identifier is further to generate a reference probability distribution database from a training data set for the trained classifier.” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
Step 2B – Does the claim recite additional elements that amount to significantly more than the judicial exception?
Finally, the claim taken as a whole does not contain an inventive concept which provides significantly more than the abstract idea. The additional elements, “wherein the outlier identifier is further to generate a reference probability distribution database from a training data set for the trained classifier.” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
Taken alone or in combination, the additional elements of the claim do not provide an inventive concept and thus the claim is subject-matter ineligible.
Claim 10
Step 1 – Is the claim to a process, machine, manufacture or composition of matter?
A machine, as above.
Step 2A Prong 1 – Does the claim recite an abstract idea, law of nature, or natural
phenomenon?
The claim recites the abstract ideas of the preceding claims from which it depends.
Step 2A Prong 2 – Does the claim recite additional elements that integrate the judicial exception into a practical application?
The claim recites the additional elements, “wherein the outlier identifier is further to calibrate the reference probability distribution database to reduce a number of activation values utilized based on relevance of each of the activation values in identifying the sub-classes.” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
Step 2B – Does the claim recite additional elements that amount to significantly more than the judicial exception?
Finally, the claim taken as a whole does not contain an inventive concept which provides significantly more than the abstract idea. The additional elements, “wherein the outlier identifier is further to calibrate the reference probability distribution database to reduce a number of activation values utilized based on relevance of each of the activation values in identifying the sub-classes.” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
Taken alone or in combination, the additional elements of the claim do not provide an inventive concept and thus the claim is subject-matter ineligible.
Claim 11
Step 1 – Is the claim to a process, machine, manufacture or composition of matter?
A machine, as above.
Step 2A Prong 1 – Does the claim recite an abstract idea, law of nature, or natural
phenomenon?
The claim recites, inter alia:
“wherein the outlier identifier determines whether the entropy score is below the threshold by determining whether the entropy score is below the threshold of any one of the sub-classes.” Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of evaluating and observing data, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper. A human is able to evaluate data and check if it meets a threshold or not. The limitation is merely applying an abstract idea on generic computer system. See MPEP 2106.04(a)(2)(III)(c).
Step 2A Prong 2 – Does the claim recite additional elements that integrate the judicial exception into a practical application?
This claim does not recite any additional limitations which integrate the abstract idea into a practical application.
Step 2B – Does the claim recite additional elements that amount to significantly more than the judicial exception?
Finally, the claim taken as a whole does not contain an inventive concept which provides significantly more than the abstract idea and thus the claim is subject-matter ineligible.
Claim 12
Step 1 – Is the claim to a process, machine, manufacture or composition of matter?
A machine, as above.
Step 2A Prong 1 – Does the claim recite an abstract idea, law of nature, or natural
phenomenon?
The claim recites, inter alia:
“wherein the outlier identifier is further to sort each item in a training data set into the sub-classes by using a clustering algorithm.” Under its broadest reasonable interpretation in light of the specification, this limitation encompasses mathematical concept of utilizing a mathematical formula to perform calculations. A human is able to use mathematical algorithms to evaluate data. This claim discloses a math operation and therefore is ineligible.
Step 2A Prong 2 – Does the claim recite additional elements that integrate the judicial exception into a practical application?
This claim does not recite any additional limitations which integrate the abstract idea into a practical application.
Step 2B – Does the claim recite additional elements that amount to significantly more than the judicial exception?
Finally, the claim taken as a whole does not contain an inventive concept which provides significantly more than the abstract idea and thus the claim is subject-matter ineligible.
Claim 13
Step 1 – Is the claim to a process, machine, manufacture or composition of matter?
A machine, as above.
Step 2A Prong 1 – Does the claim recite an abstract idea, law of nature, or natural
phenomenon?
The claim recites, inter alia:
“wherein a count is maintained of each item assigned to each sub-class.” Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of evaluating and observing data, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper. A human is able to evaluate data and with pen and paper maintain a count. The limitation is merely applying an abstract idea on generic computer system. See MPEP 2106.04(a)(2)(III)(c).
Step 2A Prong 2 – Does the claim recite additional elements that integrate the judicial exception into a practical application?
This claim does not recite any additional limitations which integrate the abstract idea into a practical application.
Step 2B – Does the claim recite additional elements that amount to significantly more than the judicial exception?
Finally, the claim taken as a whole does not contain an inventive concept which provides significantly more than the abstract idea and thus the claim is subject-matter ineligible.
Claim 14
Step 1 – Is the claim to a process, machine, manufacture or composition of matter?
A machine, as above.
Step 2A Prong 1 – Does the claim recite an abstract idea, law of nature, or natural
phenomenon?
The claim recites the abstract ideas of the preceding claims from which it depends.
Step 2A Prong 2 – Does the claim recite additional elements that integrate the judicial exception into a practical application?
The claim recites the additional elements, “wherein a threshold is managed as a hyperparameter for each sub-class to assign items in a training data set to a respective sub-class that exceeds the threshold.” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
Step 2B – Does the claim recite additional elements that amount to significantly more than the judicial exception?
Finally, the claim taken as a whole does not contain an inventive concept which provides significantly more than the abstract idea. The additional elements, “wherein a threshold is managed as a hyperparameter for each sub-class to assign items in a training data set to a respective sub-class that exceeds the threshold.” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
Taken alone or in combination, the additional elements of the claim do not provide an inventive concept and thus the claim is subject-matter ineligible.
Claim 15
Step 1 – Is the claim to a process, machine, manufacture or composition of matter?
Claim 15, recites “A computing device to implement a plurality of virtual machines, the plurality of virtual machines to implement network function virtualization (NFV), where at least one virtual machine from the plurality of virtual machines implements a method for determining outlier inputs for a machine learning system, the computing device comprising: a non-transitory computer-readable medium having stored therein a outlier identifier; and a processor coupled to the non-transitory computer-readable medium, the processor to execute on the at least one virtual machine from the plurality of virtual machines, the at least one virtual machine to execute the outlier identifier, for the processor to:” therefore it is directed to the statutory category of a machine.
Step 2A Prong 1 – Does the claim recite an abstract idea, law of nature, or natural
phenomenon?
The claim recites, inter alia:
“operate on the data input at the machine learning model to generate classification values to classify the data input and to generate activation values associated with neuron outputs from the neural network state;” Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of evaluating and observing data, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper. A human is able to evaluate data and apply judgements to that data in determining outlier input data. The limitation is merely applying an abstract idea on generic computer system. See MPEP 2106.04(a)(2)(III)(c).
“determine whether an entropy score derived from the classification values and the activation values corresponding to the data input is below a threshold entropy-based distance metric; and” Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of evaluating and observing data, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper. A human is able to evaluate data and make judgements and opinions on the results of the evaluation. The limitation is merely applying an abstract idea on generic computer system. See MPEP 2106.04(a)(2)(III)(c).
“determine whether the data input is an outlier based on the threshold entropy- based distance metric.” Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of evaluating and observing data, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper. A human is able to determine if data is outlying data through observations and evaluations. The limitation is merely applying an abstract idea on generic computer system. See MPEP 2106.04(a)(2)(III)(c).
Step 2A Prong 2 – Does the claim recite additional elements that integrate the judicial exception into a practical application?
The claim recites the additional elements, “receive a data input at a machine learning model, wherein the machine learning model operates as a trained classifier having multiple classes as an output for data classification with each class of the multiple classes having sub- classes which are based on neural network state of the machine learning model;” is an insignificant extra-solution activity required for any uses of the mental processes (see MPEP § 2106.05(g)) As such, the claim is ineligible.
“receive the classification values and activation values, to be operated on by the outlier identifier;” is an insignificant extra-solution activity required for any uses of the mental processes (see MPEP § 2106.05(g)) As such, the claim is ineligible.
Step 2B – Does the claim recite additional elements that amount to significantly more than the judicial exception?
Finally, the claim taken as a whole does not contain an inventive concept which provides significantly more than the abstract idea. The additional elements, “receive a data input at a machine learning model, wherein the machine learning model operates as a trained classifier having multiple classes as an output for data classification with each class of the multiple classes having sub- classes which are based on neural network state of the machine learning model;” is an insignificant extra-solution activity required for any uses of abstract ideas (see MPEP § 2106.05(g)), and is a well-understood, routine, conventional activity (see MPEP § 2106.05(d)(i); “Receiving or transmitting data over a network, e.g., using the Internet to gather data”.
“receive the classification values and activation values, to be operated on by the outlier identifier;” is an insignificant extra-solution activity required for any uses of abstract ideas (see MPEP § 2106.05(g)), and is a well-understood, routine, conventional activity (see MPEP § 2106.05(d)(i); “Receiving or transmitting data over a network, e.g., using the Internet to gather data”.
Taken alone or in combination, the additional elements of the claim do not provide an inventive concept and thus the claim is subject-matter ineligible.
Claim 16
Step 1 – Is the claim to a process, machine, manufacture or composition of matter?
A machine, as above.
Step 2A Prong 1 – Does the claim recite an abstract idea, law of nature, or natural
phenomenon?
The claim recites the abstract ideas of the preceding claims from which it depends.
Step 2A Prong 2 – Does the claim recite additional elements that integrate the judicial exception into a practical application?
The claim recites the additional elements, “wherein the outlier identifier is further to generate a reference probability distribution database from a training data set for the trained classifier.” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
Step 2B – Does the claim recite additional elements that amount to significantly more than the judicial exception?
Finally, the claim taken as a whole does not contain an inventive concept which provides significantly more than the abstract idea. The additional elements, “wherein the outlier identifier is further to generate a reference probability distribution database from a training data set for the trained classifier.” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
Taken alone or in combination, the additional elements of the claim do not provide an inventive concept and thus the claim is subject-matter ineligible.
Claim 17
Step 1 – Is the claim to a process, machine, manufacture or composition of matter?
A machine, as above.
Step 2A Prong 1 – Does the claim recite an abstract idea, law of nature, or natural
phenomenon?
The claim recites the abstract ideas of the preceding claims from which it depends.
Step 2A Prong 2 – Does the claim recite additional elements that integrate the judicial exception into a practical application?
The claim recites the additional elements, “wherein the outlier identifier is further to calibrate the reference probability distribution database to reduce a number of activation values utilized based on relevance of each of the activation values in identifying the sub-classes.” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
Step 2B – Does the claim recite additional elements that amount to significantly more than the judicial exception?
Finally, the claim taken as a whole does not contain an inventive concept which provides significantly more than the abstract idea. The additional elements, “wherein the outlier identifier is further to calibrate the reference probability distribution database to reduce a number of activation values utilized based on relevance of each of the activation values in identifying the sub-classes.” amounts to generic computer components used as a tool to perform an existing process. Thus, the additional element amounts to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
Taken alone or in combination, the additional elements of the claim do not provide an inventive concept and thus the claim is subject-matter ineligible.
Claim 18
Step 1 – Is the claim to a process, machine, manufacture or composition of matter?
A machine, as above.
Step 2A Prong 1 – Does the claim recite an abstract idea, law of nature, or natural
phenomenon?
The claim recites, inter alia:
“wherein the outlier identifier determines whether the entropy score is below the threshold by determining whether the entropy score is below the threshold of any one of the sub-classes.” Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of evaluating and observing data, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper. A human is able to evaluate data and check if it meets a threshold or not. The limitation is merely applying an abstract idea on generic computer system. See MPEP 2106.04(a)(2)(III)(c).
Step 2A Prong 2 – Does the claim recite additional elements that integrate the judicial exception into a practical application?
This claim does not recite any additional limitations which integrate the abstract idea into a practical application.
Step 2B – Does the claim recite additional elements that amount to significantly more than the judicial exception?
Finally, the claim taken as a whole does not contain an inventive concept which provides significantly more than the abstract idea and thus the claim is subject-matter ineligible.
Claim 19
Step 1 – Is the claim to a process, machine, manufacture or composition of matter?
A machine, as above.
Step 2A Prong 1 – Does the claim recite an abstract idea, law of nature, or natural
phenomenon?
The claim recites, inter alia:
“wherein the outlier identifier is further to sort each item in a training data set into the sub-classes by using a clustering algorithm.” Under its broadest reasonable interpretation in light of the specification, this limitation encompasses mathematical concept of utilizing a mathematical formula to perform calculations. A human is able to use mathematical algorithms to evaluate data. This claim discloses a math operation and therefore is ineligible.
Step 2A Prong 2 – Does the claim recite additional elements that integrate the judicial exception into a practical application?
This claim does not recite any additional limitations which integrate the abstract idea into a practical application.
Step 2B – Does the claim recite additional elements that amount to significantly more than the judicial exception?
Finally, the claim taken as a whole does not contain an inventive concept which provides significantly more than the abstract idea and thus the claim is subject-matter ineligible.
Claim 20
Step 1 – Is the claim to a process, machine, manufacture or composition of matter?
A machine, as above.
Step 2A Prong 1 – Does the claim recite an abstract idea, law of nature, or natural
phenomenon?
The claim recites, inter alia:
“wherein a count is maintained of each item assigned to each sub-class.” Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of evaluating and observing data, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper. A human is able to evaluate data and with pen and paper maintain a count. The limitation is merely applying an abstract idea on generic computer system. See MPEP 2106.04(a)(2)(III)(c).
Step 2A Prong 2 – Does the claim recite additional elements that integrate the judicial exception into a practical application?
This claim does not recite any additional limitations which integrate the abstract idea into a practical application.
Step 2B – Does the claim recite additional elements that amount to significantly more than the judicial exception?
Finally, the claim taken as a whole does not contain an inventive concept which provides significantly more than the abstract idea and thus the claim is subject-matter ineligible.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 4, 7, 8, 11, 14, 15, and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Huang et al., (Huang et al., “Improper Neural Network Input Detection and Handling”, US 2020/0184254 A1, Filed Dec. 11th 2018, hereinafter “Huang”) in view of Zhao et al., (Zhao et al., “Outlier detection based on approximation accuracy entropy”, Nov. 20th 2018, hereinafter “Zhao”).
Regarding claim 1, Huang discloses, “A method for determining outlier inputs for a machine learning system, the method comprising:” (Detailed Description, pp. 6, [0046]; "FIG. 4B illustrates an example of an outlier detection scheme 460 that can be employed by improper input detection module 432 to determine whether an output data element is an outlier based on a distribution of reference outputs (e.g., distribution 300). "This article discloses a system for detecting outlier input data to a system.)
“receiving the classification values and activation values at an outlier identifier;” (Detailed Description, pp. 8, [0065]; "At operation 602, improper input detection module 432 receives, from hardware circuits, computation outputs of a neural network based on input data provided by an application. The hardware circuits may include activation function engine 430, and the computation outputs may include outputs of the activation function engine 430 for computations of a neural network layer." This system is designed to find outlier data from activation values and the determined end result. This teaches that the data input into the system can evaluate whether and input is an outlier based on how it was altered during the training processes and the outcome of the output.)
Huang fails to explicitly disclose, “receiving a data input at a machine learning model, wherein the machine learning model operates as a trained classifier having multiple classes as an output for data classification with each class of the multiple classes having sub-classes which are based on neural network state of the machine learning model;”, “operating on the data input at the machine learning model to generate classification values to classify the data input and to generate activation values associated with neuron outputs from the neural network state;”, “determining, at the outlier identifier, whether an entropy score derived from the classification values and the activation values corresponding to the data input is below a threshold entropy-based distance metric; and”, and “determining whether the data input is an outlier based on the threshold entropy-based distance metric.”.
However, Zhao discloses, “receiving a data input at a machine learning model, wherein the machine learning model operates as a trained classifier having multiple classes as an output for data classification with each class of the multiple classes having sub-classes which are based on neural network state of the machine learning model;” (Preliminaries, pp. 2485; “Definition 1 (Information Table) An information table is a quadruple IS = (U, A, V, f ), where: 1. U is a non-empty finite set of objects; 2. A is a non-empty finite set of attributes; 3. V is the union of attribute domains, i.e.,
v
=
U
a
∈
A
V
a
, where
V
a
denotes the domain of attribute a; 4. f ∶ U × A → V is an information function which associates a unique value of each attribute with every object belonging to U, such that for any a ∈ A and x ∈ U, f(x, a) ∈
V
a
.” The information table represents the above elements. The IS is used as an input for the proposed approximation accuracy entropy-based outlier detection algorithm, which is called ODAAE.) And (Algorithm 2, pp. 2492; This is the pseudo code for the ODAAE method. As seen in the top of the algorithm, The IS is the input for the algorithm.)
“operating on the data input at the machine learning model to generate classification values to classify the data input and to generate activation values associated with neuron outputs from the neural network state;” (Some Definition related to AAE-Based Outliers, pp. 2499; In order to find outliers in a given information table IS, we need to define two kinds of sequences in IS [23, 51]: the weight-based sequence of attributes and the weight-based sequence of attribute subsets, where the weight of each attribute is defined as follows. Definition 8 (Weight of Attribute) Given an information table IS = (U, A, V, f), for any a ∈ A, the weight of attribute a in IS is defined as:
W
a
=
S
i
g
(
a
)
log
2
U
, where Sig(a) is the AAE-based significance of attribute a (as defined in Definition 6). Definition 9 (Weight-Based Sequence of Attrib-utes) Given an information table IS = (U, A, V, f), where
A
=
a
1
,
…
a
v
, for each 1 ≤ j ≤ v, let W(
a
j
) be the weight of attribute
a
j
in IS. By using the set
W
a
j
:
1
≤
j
≤
v
, we can generate a weight-based sequence
S
=
a
1
'
,
…
a
v
,
of attributes in IS, such that for any 1 ≤ j ≤ v,
a
j
,
∈ A, and for any
1
≤
j
≤
v
,
W
(
a
j
,
)
≤
W
(
a
j
+
1
,
)
. If we gradually delete attributes from the original attribute set A, then we can generate another sequence. Definition 10 (Weight-Based Sequence of Attrib-ute Subsets) Given an information table IS = (U, A, V, f), where
A
=
a
1
,
…
a
v
, let
S
=
a
1
'
,
…
a
v
,
be the weight-based sequence of attributes (as defined above). Let
A
S
=
A
1
,
…
A
v
-
1
be a sequence of attribute subsets in IS, where for each
1
≤
j
<
v
,
A
j
⊆
A
. If
A
1
=
A
-
a
1
,
,
A
v
-
1
=
a
1
,
and
A
j
+
1
=
A
j
-
{
a
j
+
1
,
}
for each 1 ≤ j < v − 1, then AS is called a weight-based sequence of attribute subsets in IS.” This model will input the information table and process the data. This will evaluate the data and assess weights for the attributes of the data given. This is done to find the significance of certain attributes to help define what an outlier would be.)
“determining, at the outlier identifier, whether an entropy score derived from the classification values and the activation values corresponding to the data input is below a threshold entropy-based distance metric; and” (Definition 11, 2490; “In Definition 11, for any object x ∈ U, the significances of x with respect to different attribute subsets of A are used to calculate AAEOF(x). From Eq. (7), we can see that AAEOF(x) is proportional to the significances of x, that is, those objects with higher significances have more likelihood of being outliers. The reason is as follows. Given a set
Q
i
⊆
A
,
1
≤
i
≤
k
of attribute subsets, if for each 1 ≤ i ≤ k, the effect of x on the approximation accuracy entropy
A
A
E
(
A
-
Q
i
|
Q
i
)
(i.e., the significance of x with respect to A −
Q
i
and
Q
i
) is always disproportionately high compared with other objects in U, then we may consider x as behaving abnormally and hence AAEOF(x) will be high.” This system processes the input data as the information table. The data will be evaluated for significance and attributes. Once the entropy accuracy score is calculated in algorithm 1, this will take data and classify potential outliers at the end of algorithm 2.)
“determining whether the data input is an outlier based on the threshold entropy-based distance metric.” (Definition 12, pp. 2491; “In our method, the setting of the threshold - in Definition 12 is related to the value h provided by the users. If we have calculated the AAE-based outlier factor AAEOF(x) for each object x ∈ U, and sorted these objects according to their outlier factors in descending order. Let
x
1
,
x
2
,
…
x
U
be the sequence of objects in U after sorting, where
A
A
E
O
F
x
i
≥
A
A
E
O
F
x
i
+
1
,
1
≤
i
≤
U
. Then we can set
μ
- as follows:
A
A
E
O
F
(
x
h
+
1
)
≤
μ
<
A
A
E
O
F
(
x
h
)
. By virtue of the above setting for -
μ
, our method can find h outliers
(
i
.
e
.
,
x
1
,
x
2
,
…
x
h
)
from U, since the degrees of outlierness of the h objects are higher than those of other objects in U.” This is the process will determine if the input data in the information table contains outlier input data. This process will evaluate the data and will sort the possible outliers and the input items that contain more attributes of being an outlier it will be higher on the sorted list.) And (Algorithm 2, pp. 2493; Lines 28-30 show the process in action. This algorithm uses the many definitions in the article to get an initial set of outlier data. In lines 28-30 the initial set of outliers are processes and if they meet the criteria in line 29, the input data items will be assigned to set O, which contains all of the outliers to be handled by the system.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Huang and Zhao. Huang teaches a system which detects and handles improper input data to a neural network. Zhao teaches a method that uses approximation accuracy entropy to detect outlying data in machine learning applications. One of ordinary skill would have motivation to combine a system which uses machine learning to detect outlying input data in a neural network with a model that uses approximation accuracy entropy to detect outlying input data, “The results on the Lymp data set also demonstrate the effectiveness of ODAAE. On this data set, the performance of ODAAE is equal to that of EODSP, and they perform better than other methods. For instance, if “Top ratio (number of objects)” is set as 5% (7), then ODAAE and EODSP successfully cover all the six outliers in Lymp, but for ABOD, Gaussian, INFLO, KNN, LDOF, LOF, LoOP and SOD, only 1, 3, 5, 4, 4, 4, 5 and 1 outlier(s) are detected, respectively. From another point of view, to detect all outliers in Lymp, ODAAE and EODSP need to check 5% of objects in Lymp, but for the other eight methods, they need to check much more objects in Lymp.” (Zhao, Experiment results, pp. 2495).
Regarding claim 4, Huang discloses, “wherein the determining whether the entropy score is below the threshold further comprises: determining whether the entropy score is below the threshold of any one of the sub-classes.” (Detailed Description, pp. 8, [0067]; "At operation 606, improper input detection module 432 determines that the input data are improper based on the relationship. The determination can be based on, for example, the count of outlier computation outputs exceeding a threshold, as described with respect to FIG. 4C and FIG. 4D. The determination can also be based on comparing the distributions of the computation outputs and reference outputs." The outputs are ranked on many different features. This includes relationships to other classified data and outputs. This example shows that an exceeding a threshold will determine whether an output an outlier or not.)
Regarding claim 7, Huang discloses, “wherein a threshold is managed as a hyperparameter for each sub-class to assign items in a training data set to a respective sub-class that exceeds the threshold.” (Detailed Description, pp. 6, [0049]; In addition, thresholds generator 476 can also receive standard deviation a from reference output statistics parameters 452, as well as standard deviation multiplier 480, to generate thresholds 462 and 464 of FIG. 4B. With range 466 centered at zero, thresholds generator476 can generate thresholds 462 and 464 by multiplying standard deviation a with a multiplier value (e.g., three) set by standard deviation multiplier480 to generate a multiple. Thresholds 462 and 464 can be, respectively, a negative version and a positive version of the multiple." A Threshold generator is used to set and store the different thresholds for the different classifications. This model uses a system that generates different thresholds for detecting the outlying data.)
Regarding claim 8, Huang discloses, “An electronic device to execute a method for determining outlier inputs for a machine learning system, the electronic device comprising: a non-transitory computer-readable medium having stored therein an outlier identifier; and a processor coupled to the non-transitory computer-readable medium, the processor to execute on the outlier identifier, for the electronic device to:” (Detailed Description, pp. 9, [0069]; "FIG. 7 illustrates an example of a computing device 700. Functionality and/or several components of the computing device 700 may be used without limitation with other embodiments disclosed elsewhere in this disclosure, without limitations. A computing device 800 may perform computations to facilitate processing of a task. As an illustrative example, computing device 800 can be part of a server in a multi-tenant compute service system." This method was designed to execute and perform actions on an ecteronic system containing different electronic modules.) And (Detailed Description, pp. 10, [0077]; "The modules described herein may be software modules, hardware modules or a suitable combination thereof. If the modules are software modules, the modules can be embodied on a non-transitory computer readable medium and processed by a processor in any of the computer systems described herein." This system is used by a device which contains the method on a form of memory.) And (Detailed Description, pp. 10, [0077]; "The modules described herein may be software modules, hardware modules or a suitable combination thereof. If the modules are software modules, the modules can be embodied on a non-transitory computer readable medium and processed by a processor in any of the computer systems described herein." This system is used by a device which contains the method on a form of memory.)
“receive the classification values and activation values to be operated on by the outlier identifier;” (Detailed Description, pp. 8, [0065]; "At operation 602, improper input detection module 432 receives, from hardware circuits, computation outputs of a neural network based on input data provided by an application. The hardware circuits may include activation function engine 430, and the computation outputs may include outputs of the activation function engine 430 for computations of a neural network layer." This system is designed to find outlier data from activation values and the determined end result. This teaches that the data input into the system can evaluate whether and input is an outlier based on how it was altered during the training processes and the outcome of the output.)
Huang fails to explicitly disclose, “receive a data input at a machine learning model, wherein the machine learning model operates as a trained classifier having multiple classes as an output for data classification with each class of the multiple classes having sub- classes which are based on neural network state of the machine learning model;”, “operate on the data input at the machine learning model to generate classification values to classify the data input and to generate activation values associated with neuron outputs from the neural network state;”, “determine whether an entropy score derived from the classification values and the activation values corresponding to the data input is below a threshold entropy-based distance metric; and”, and “determine whether the data input is an outlier based on the threshold entropy- based distance metric.”.
However, Zhao discloses, “receive a data input at a machine learning model, wherein the machine learning model operates as a trained classifier having multiple classes as an output for data classification with each class of the multiple classes having sub- classes which are based on neural network state of the machine learning model;” (Preliminaries, pp. 2485; “Definition 1 (Information Table) An information table is a quadruple IS = (U, A, V, f ), where: 1. U is a non-empty finite set of objects; 2. A is a non-empty finite set of attributes; 3. V is the union of attribute domains, i.e.,
v
=
U
a
∈
A
V
a
, where
V
a
denotes the domain of attribute a; 4. f ∶ U × A → V is an information function which associates a unique value of each attribute with every object belonging to U, such that for any a ∈ A and x ∈ U, f(x, a) ∈
V
a
.” The information table represents the above elements. The IS is used as an input for the proposed approximation accuracy entropy-based outlier detection algorithm, which is called ODAAE.) And (Algorithm 2, pp. 2492; This is the pseudo code for the ODAAE method. As seen in the top of the algorithm, The IS is the input for the algorithm.)
“operate on the data input at the machine learning model to generate classification values to classify the data input and to generate activation values associated with neuron outputs from the neural network state;” (Some Definition related to AAE-Based Outliers, pp. 2499; In order to find outliers in a given information table IS, we need to define two kinds of sequences in IS [23, 51]: the weight-based sequence of attributes and the weight-based sequence of attribute subsets, where the weight of each attribute is defined as follows. Definition 8 (Weight of Attribute) Given an information table IS = (U, A, V, f), for any a ∈ A, the weight of attribute a in IS is defined as:
W
a
=
S
i
g
(
a
)
log
2
U
, where Sig(a) is the AAE-based significance of attribute a (as defined in Definition 6). Definition 9 (Weight-Based Sequence of Attrib-utes) Given an information table IS = (U, A, V, f), where
A
=
a
1
,
…
a
v
, for each 1 ≤ j ≤ v, let W(
a
j
) be the weight of attribute
a
j
in IS. By using the set
W
a
j
:
1
≤
j
≤
v
, we can generate a weight-based sequence
S
=
a
1
'
,
…
a
v
,
of attributes in IS, such that for any 1 ≤ j ≤ v,
a
j
,
∈ A, and for any
1
≤
j
≤
v
,
W
(
a
j
,
)
≤
W
(
a
j
+
1
,
)
. If we gradually delete attributes from the original attribute set A, then we can generate another sequence. Definition 10 (Weight-Based Sequence of Attrib-ute Subsets) Given an information table IS = (U, A, V, f), where
A
=
a
1
,
…
a
v
, let
S
=
a
1
'
,
…
a
v
,
be the weight-based sequence of attributes (as defined above). Let
A
S
=
A
1
,
…
A
v
-
1
be a sequence of attribute subsets in IS, where for each
1
≤
j
<
v
,
A
j
⊆
A
. If
A
1
=
A
-
a
1
,
,
A
v
-
1
=
a
1
,
and
A
j
+
1
=
A
j
-
{
a
j
+
1
,
}
for each 1 ≤ j < v − 1, then AS is called a weight-based sequence of attribute subsets in IS.” This model will input the information table and process the data. This will evaluate the data and assess weights for the attributes of the data given. This is done to find the significance of certain attributes to help define what an outlier would be.)
“determine whether an entropy score derived from the classification values and the activation values corresponding to the data input is below a threshold entropy-based distance metric; and” (Definition 11, 2490; “In Definition 11, for any object x ∈ U, the significances of x with respect to different attribute subsets of A are used to calculate AAEOF(x). From Eq. (7), we can see that AAEOF(x) is proportional to the significances of x, that is, those objects with higher significances have more likelihood of being outliers. The reason is as follows. Given a set
Q
i
⊆
A
,
1
≤
i
≤
k
of attribute subsets, if for each 1 ≤ i ≤ k, the effect of x on the approximation accuracy entropy
A
A
E
(
A
-
Q
i
|
Q
i
)
(i.e., the significance of x with respect to A −
Q
i
and
Q
i
) is always disproportionately high compared with other objects in U, then we may consider x as behaving abnormally and hence AAEOF(x) will be high.” This system processes the input data as the information table. The data will be evaluated for significance and attributes. Once the entropy accuracy score is calculated in algorithm 1, this will take data and classify potential outliers at the end of algorithm 2.)
“determine whether the data input is an outlier based on the threshold entropy- based distance metric.” (Definition 12, pp. 2491; “In our method, the setting of the threshold - in Definition 12 is related to the value h provided by the users. If we have calculated the AAE-based outlier factor AAEOF(x) for each object x ∈ U, and sorted these objects according to their outlier factors in descending order. Let
x
1
,
x
2
,
…
x
U
be the sequence of objects in U after sorting, where
A
A
E
O
F
x
i
≥
A
A
E
O
F
x
i
+
1
,
1
≤
i
≤
U
. Then we can set
μ
- as follows:
A
A
E
O
F
(
x
h
+
1
)
≤
μ
<
A
A
E
O
F
(
x
h
)
. By virtue of the above setting for -
μ
, our method can find h outliers
(
i
.
e
.
,
x
1
,
x
2
,
…
x
h
)
from U, since the degrees of outlierness of the h objects are higher than those of other objects in U.” This is the process will determine if the input data in the information table contains outlier input data. This process will evaluate the data and will sort the possible outliers and the input items that contain more attributes of being an outlier it will be higher on the sorted list.) And (Algorithm 2, pp. 2493; Lines 28-30 show the process in action. This algorithm uses the many definitions in the article to get an initial set of outlier data. In lines 28-30 the initial set of outliers are processes and if they meet the criteria in line 29, the input data items will be assigned to set O, which contains all of the outliers to be handled by the system.)
Regarding claim 11, Huang discloses, “wherein the outlier identifier determines whether the entropy score is below the threshold by determining whether the entropy score is below the threshold of any one of the sub-classes.” (Detailed Description, pp. 8, [0067]; "At operation 606, improper input detection module 432 determines that the input data are improper based on the relationship. The determination can be based on, for example, the count of outlier computation outputs exceeding a threshold, as described with respect to FIG. 4C and FIG. 4D. The determination can also be based on comparing the distributions of the computation outputs and reference outputs." The outputs are ranked on many different features. This includes relationships to other classified data and outputs. This example shows that an exceeding a threshold will determine whether an output an outlier or not.)
Regarding claim 14, Huang discloses, “wherein a threshold is managed as a hyperparameter for each sub-class to assign items in a training data set to a respective sub-class that exceeds the threshold.” (Detailed Description, pp. 6, [0049]; In addition, thresholds generator 476 can also receive standard deviation a from reference output statistics parameters 452, as well as standard deviation multiplier 480, to generate thresholds 462 and 464 of FIG. 4B. With range 466 centered at zero, thresholds generator476 can generate thresholds 462 and 464 by multiplying standard deviation a with a multiplier value (e.g., three) set by standard deviation multiplier480 to generate a multiple. Thresholds 462 and 464 can be, respectively, a negative version and a positive version of the multiple." A Threshold generator is used to set and store the different thresholds for the different classifications. This model uses a system that generates different thresholds for detecting the outlying data.)
Regarding claim 15, Huang discloses, “A computing device to implement a plurality of virtual machines, the plurality of virtual machines to implement network function virtualization (NFV), where at least one virtual machine from the plurality of virtual machines implements a method for determining outlier inputs for a machine learning system, the computing device comprising: a non-transitory computer-readable medium having stored therein a outlier identifier; and a processor coupled to the non-transitory computer-readable medium, the processor to execute on the at least one virtual machine from the plurality of virtual machines, the at least one virtual machine to execute the outlier identifier, for the processor to:” (Detailed Description, pp. 9, [0072]; "The access to processing logic 702 can be granted to a client to provide the personal assistant service requested by the client. For example, computing device 700 may host a virtual machine, on which an image recognition software application can be executed. The image recognition software application, upon execution, may access processing logic 702 to predict, for example, an object included in an image. As another example, access to processing logic 702 can also be granted as part of bare-metal instance, in which an image recognition software application executing on a client device (e.g., a remote computer, a smart phone, etc.) can directly access processing logic 702 to perform the recognition of an image." This method can be run on multiple client virtual machines. It can perform over a network as well.) And (Detailed Description, pp. 10, [0077]; "The modules described herein may be software modules, hardware modules or a suitable combination thereof. If the modules are software modules, the modules can be embodied on a non-transitory computer readable medium and processed by a processor in any of the computer systems described herein." This system is used by a device which contains the method on a form of memory.) And (Detailed Description, pp. 9, [0071]; "The processing logic 702 may include one or more integrated circuits, which may include application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), systems-on-chip (Socs), network processing units (NPUs), processors configured to execute instructions or any other circuitry configured to perform logical arithmetic and floating-point operations. Examples of processors that may be included in the processing logic 702 may include processors developed by ARM®, MIPS®, AMO®, Intel®, Qualcomm®, and the like." This system can be executed with different processors listed as examples. These processors are used in conjunction with memory which stores the instructions and methods described in this patent application publication.)
“receive the classification values and activation values, to be operated on by the outlier identifier;” (Detailed Description, pp. 8, [0065]; "At operation 602, improper input detection module 432 receives, from hardware circuits, computation outputs of a neural network based on input data provided by an application. The hardware circuits may include activation function engine 430, and the computation outputs may include outputs of the activation function engine 430 for computations of a neural network layer." This system is designed to find outlier data from activation values and the determined end result. This teaches that the data input into the system can evaluate whether and input is an outlier based on how it was altered during the training processes and the outcome of the output.)
Huang fails to explicitly disclose, “receive a data input at a machine learning model, wherein the machine learning model operates as a trained classifier having multiple classes as an output for data classification with each class of the multiple classes having sub- classes which are based on neural network state of the machine learning model;”, “operate on the data input at the machine learning model to generate classification values to classify the data input and to generate activation values associated with neuron outputs from the neural network state;”, “determine whether an entropy score derived from the classification values and the activation values corresponding to the data input is below a threshold entropy-based distance metric; and”, and “determine whether the data input is an outlier based on the threshold entropy- based distance metric.”.
However, Zhao discloses, “receive a data input at a machine learning model, wherein the machine learning model operates as a trained classifier having multiple classes as an output for data classification with each class of the multiple classes having sub- classes which are based on neural network state of the machine learning model;” (Preliminaries, pp. 2485; “Definition 1 (Information Table) An information table is a quadruple IS = (U, A, V, f ), where: 1. U is a non-empty finite set of objects; 2. A is a non-empty finite set of attributes; 3. V is the union of attribute domains, i.e.,
v
=
U
a
∈
A
V
a
, where
V
a
denotes the domain of attribute a; 4. f ∶ U × A → V is an information function which associates a unique value of each attribute with every object belonging to U, such that for any a ∈ A and x ∈ U, f(x, a) ∈
V
a
.” The information table represents the above elements. The IS is used as an input for the proposed approximation accuracy entropy-based outlier detection algorithm, which is called ODAAE.) And (Algorithm 2, pp. 2492; This is the pseudo code for the ODAAE method. As seen in the top of the algorithm, The IS is the input for the algorithm.)
“operate on the data input at the machine learning model to generate classification values to classify the data input and to generate activation values associated with neuron outputs from the neural network state;” (Some Definition related to AAE-Based Outliers, pp. 2499; In order to find outliers in a given information table IS, we need to define two kinds of sequences in IS [23, 51]: the weight-based sequence of attributes and the weight-based sequence of attribute subsets, where the weight of each attribute is defined as follows. Definition 8 (Weight of Attribute) Given an information table IS = (U, A, V, f), for any a ∈ A, the weight of attribute a in IS is defined as:
W
a
=
S
i
g
(
a
)
log
2
U
, where Sig(a) is the AAE-based significance of attribute a (as defined in Definition 6). Definition 9 (Weight-Based Sequence of Attrib-utes) Given an information table IS = (U, A, V, f), where
A
=
a
1
,
…
a
v
, for each 1 ≤ j ≤ v, let W(
a
j
) be the weight of attribute
a
j
in IS. By using the set
W
a
j
:
1
≤
j
≤
v
, we can generate a weight-based sequence
S
=
a
1
'
,
…
a
v
,
of attributes in IS, such that for any 1 ≤ j ≤ v,
a
j
,
∈ A, and for any
1
≤
j
≤
v
,
W
(
a
j
,
)
≤
W
(
a
j
+
1
,
)
. If we gradually delete attributes from the original attribute set A, then we can generate another sequence. Definition 10 (Weight-Based Sequence of Attrib-ute Subsets) Given an information table IS = (U, A, V, f), where
A
=
a
1
,
…
a
v
, let
S
=
a
1
'
,
…
a
v
,
be the weight-based sequence of attributes (as defined above). Let
A
S
=
A
1
,
…
A
v
-
1
be a sequence of attribute subsets in IS, where for each
1
≤
j
<
v
,
A
j
⊆
A
. If
A
1
=
A
-
a
1
,
,
A
v
-
1
=
a
1
,
and
A
j
+
1
=
A
j
-
{
a
j
+
1
,
}
for each 1 ≤ j < v − 1, then AS is called a weight-based sequence of attribute subsets in IS.” This model will input the information table and process the data. This will evaluate the data and assess weights for the attributes of the data given. This is done to find the significance of certain attributes to help define what an outlier would be.)
“determine whether an entropy score derived from the classification values and the activation values corresponding to the data input is below a threshold entropy-based distance metric; and” (Definition 11, 2490; “In Definition 11, for any object x ∈ U, the significances of x with respect to different attribute subsets of A are used to calculate AAEOF(x). From Eq. (7), we can see that AAEOF(x) is proportional to the significances of x, that is, those objects with higher significances have more likelihood of being outliers. The reason is as follows. Given a set
Q
i
⊆
A
,
1
≤
i
≤
k
of attribute subsets, if for each 1 ≤ i ≤ k, the effect of x on the approximation accuracy entropy
A
A
E
(
A
-
Q
i
|
Q
i
)
(i.e., the significance of x with respect to A −
Q
i
and
Q
i
) is always disproportionately high compared with other objects in U, then we may consider x as behaving abnormally and hence AAEOF(x) will be high.” This system processes the input data as the information table. The data will be evaluated for significance and attributes. Once the entropy accuracy score is calculated in algorithm 1, this will take data and classify potential outliers at the end of algorithm 2.)
“determine whether the data input is an outlier based on the threshold entropy- based distance metric.” (Definition 12, pp. 2491; “In our method, the setting of the threshold - in Definition 12 is related to the value h provided by the users. If we have calculated the AAE-based outlier factor AAEOF(x) for each object x ∈ U, and sorted these objects according to their outlier factors in descending order. Let
x
1
,
x
2
,
…
x
U
be the sequence of objects in U after sorting, where
A
A
E
O
F
x
i
≥
A
A
E
O
F
x
i
+
1
,
1
≤
i
≤
U
. Then we can set
μ
- as follows:
A
A
E
O
F
(
x
h
+
1
)
≤
μ
<
A
A
E
O
F
(
x
h
)
. By virtue of the above setting for -
μ
, our method can find h outliers
(
i
.
e
.
,
x
1
,
x
2
,
…
x
h
)
from U, since the degrees of outlierness of the h objects are higher than those of other objects in U.” This is the process will determine if the input data in the information table contains outlier input data. This process will evaluate the data and will sort the possible outliers and the input items that contain more attributes of being an outlier it will be higher on the sorted list.) And (Algorithm 2, pp. 2493; Lines 28-30 show the process in action. This algorithm uses the many definitions in the article to get an initial set of outlier data. In lines 28-30 the initial set of outliers are processes and if they meet the criteria in line 29, the input data items will be assigned to set O, which contains all of the outliers to be handled by the system.)
Regarding claim 18, Huang discloses, “wherein the outlier identifier determines whether the entropy score is below the threshold by determining whether the entropy score is below the threshold of any one of the sub-classes.” (Detailed Description, pp. 8, [0067]; "At operation 606, improper input detection module 432 determines that the input data are improper based on the relationship. The determination can be based on, for example, the count of outlier computation outputs exceeding a threshold, as described with respect to FIG. 4C and FIG. 4D. The determination can also be based on comparing the distributions of the computation outputs and reference outputs." The outputs are ranked on many different features. This includes relationships to other classified data and outputs. This example shows that an exceeding a threshold will determine whether an output an outlier or not.)
Claims 2, 3, 9, 10, 16, and 17 are rejected under 35 U.S.C. 103 as being unpatentable over Huang and Zhao in view of Daneshpazhouh et al., (Daneshpazhouh et al., “Entropy-based outlier detection using semi-supervised approach with few positive examples”, Jul. 6th, 2014, hereinafter “Daneshpazhouh”).
Regarding claim 2, Huang and Zhao fail to explicitly disclose, “generating a reference probability distribution database from a training data set for the trained classifier.”.
However, Daneshpazhouh discloses, “generating a reference probability distribution database from a training data set for the trained classifier.” (Extracting Reliable Negative Instances/Algorithm: Extract-RN, pp. 79; "The task of this step is to extract the most reliable negative examples set RN, given the training set p that contains only few positive instances, and the unlabeled set U including both positive and negative instances." During the training of the outlier detection model training data is taken in and evaluated. The positive scoring data is untouched and remains in the unlabeled set U. The more reliable negative examples are extracted and placed in set RN to be used for as negative examples.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Huang, Zhao and Daneshpazhouh. Huang teaches a system which detects and handles improper input data to a neural network. Zhao teaches a method that uses approximation accuracy entropy to detect outlying data in machine learning applications. Daneshpazhouh teaches a system used to take in many invalid inputs and detect the top outliers of that input and handle them accordingly. One of ordinary skill would have motivation to combine a system which uses machine learning to detect outlying input data in a neural network with a model that uses approximation accuracy entropy to detect outlying input data and a process to detect top outliers with data sets that contain many invalid data points to further improve outlier detection, "The results on Lymphography data set demonstrates the superiority of the EODSP (Entropy-based Outlier Detection based on Semi-supervised learning from Positive data) method over the eight unsupervised algorithms. In this data set, EODSP successfully covered all the outliers in Top Ratio of 5%. As the second algorithm, LDOF identified 4 outliers out of 6 when the number of records was 7. However, it performed as the EODSP method when the Top Ratio increased up to 10%" (Daneshpazhouh, Results on UCI ML data sets, pp. 83).
Regarding claim 3, Huang and Zhao fail to explicitly disclose, “calibrating the reference probability distribution database to reduce a number of activation values utilized for the method based on relevance of each of the activation values in identifying the sub-classes.”.
However, Daneshpazhouh discloses, “calibrating the reference probability distribution database to reduce a number of activation values utilized for the method based on relevance of each of the activation values in identifying the sub-classes.” (Algorithm: Extract-RN, pp. 79; Lines 6 and 7 teaches that the top negative instances are ranked and stored in the set RN. The set RN contains the top extracted reliable negative instances of the training data.)
Regarding claim 9, Huang and Zhao fail to explicitly disclose, “wherein the outlier identifier is further to generate a reference probability distribution database from a training data set for the trained classifier.”.
However, Daneshpazhouh discloses, “wherein the outlier identifier is further to generate a reference probability distribution database from a training data set for the trained classifier.” (Extracting Reliable Negative Instances/Algorithm: Extract-RN, pp. 79; "The task of this step is to extract the most reliable negative examples set RN, given the training set p that contains only few positive instances, and the unlabeled set U including both positive and negative instances." During the training of the outlier detection model training data is taken in and evaluated. The positive scoring data is untouched and remains in the unlabeled set U. The more reliable negative examples are extracted and placed in set RN to be used for as negative examples.)
Regarding claim 10, Huang and Zhao fail to explicitly disclose, “wherein the outlier identifier is further to calibrate the reference probability distribution database to reduce a number of activation values utilized based on relevance of each of the activation values in identifying the sub-classes.”.
However, Daneshpazhouh discloses, “wherein the outlier identifier is further to calibrate the reference probability distribution database to reduce a number of activation values utilized based on relevance of each of the activation values in identifying the sub-classes.” (Algorithm: Extract-RN, pp. 79; Lines 6 and 7 teaches that the top negative instances are ranked and stored in the set RN. The set RN contains the top extracted reliable negative instances of the training data.)
Regarding claim 16, Huang and Zhao fail to explicitly disclose, “wherein the outlier identifier is further to generate a reference probability distribution database from a training data set for the trained classifier.”.
However, Daneshpazhouh discloses, “wherein the outlier identifier is further to generate a reference probability distribution database from a training data set for the trained classifier.” (Extracting Reliable Negative Instances/Algorithm: Extract-RN, pp. 79; "The task of this step is to extract the most reliable negative examples set RN, given the training set p that contains only few positive instances, and the unlabeled set U including both positive and negative instances." During the training of the outlier detection model training data is taken in and evaluated. The positive scoring data is untouched and remains in the unlabeled set U. The more reliable negative examples are extracted and placed in set RN to be used for as negative examples.)
Regarding claim 17, Huang and Zhao fail to explicitly disclose, “wherein the outlier identifier is further to calibrate the reference probability distribution database to reduce a number of activation values utilized based on relevance of each of the activation values in identifying the sub-classes.”.
However, Daneshpazhouh discloses, “wherein the outlier identifier is further to calibrate the reference probability distribution database to reduce a number of activation values utilized based on relevance of each of the activation values in identifying the sub-classes.” (Algorithm: Extract-RN, pp. 79; Lines 6 and 7 teaches that the top negative instances are ranked and stored in the set RN. The set RN contains the top extracted reliable negative instances of the training data.)
Claims 5, 6, 12, 13, 19, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Huang and Zhao in view of Jiang et al., (Jiang et al., “Clustering-Based Outlier Detection Method”, 2008, hereinafter “Jiang”).
Regarding claim 5, Huang and Zhao fail to explicitly disclose, “sorting each item in a training data set into the sub-classes by using a clustering algorithm.”.
However, Jiang discloses, “sorting each item in a training data set into the sub-classes by using a clustering algorithm.” (One-pass clustering algorithm, pp. 430; “The goal of clustering is that the intra-cluster similarity is maximized while the inter-cluster similarity is minimized. Many efficient clustering algorithms have been proposed by the database research community. Clustering algorithm can be selected according to data, objective of clustering and application. In this paper, we use one-pass clustering algorithm [13] divide dataset into hyper spheres with almost the same radius. The algorithm is described as follows: Step 1: Initialize the set of clusters, S, as the empty set, read a new object p. Step 2: Create a cluster with the object p. Step 3: If no objects are left in the database, go to step 6, otherwise read a new object p, and find the cluster C* in S that is closest to the object p. In other words, find a cluster C* in S, such that for all C in S,
d
p
,
C
*
≤
d
p
,
C
.
Step 4: If
d
p
,
C
*
>
r
, go to step 2. Step 5: Merge object p into cluster C* and modify the CSI of cluster C*, go to step 3. Step 6: Stop” In this article all of the data is evaluated and clustered. This data can be placed into already known clusters or generate a new cluster.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Huang, Zhao and Jiang. Huang teaches a system which detects and handles improper input data to a neural network. Zhao teaches a method that uses approximation accuracy entropy to detect outlying data in machine learning applications. Jiang teaches a system uses a clustering method to cluster training data for outlier data detection. One of ordinary skill would have motivation to combine a system which uses machine learning to detect outlying input data in a neural network with a model that uses approximation accuracy entropy to detect outlying input data and a method which used clustering techniques to help identify outlier input data, "KDDCU P99 Dataset contains around 4,900,000 simulated intrusion records with 41 attributes (34 continuous and 7 categorical). We randomly produced a sub-dataset which consists of 38841 normal records and 1618 attack records. By computing, we obtain EX=0.23. Table 5 gives the partial experiment data with different threshold r. Table 6 shows the comparison results among different methods. The experimental results show that the detection re suits by CBOD outperformed those by Eskin, E." (KDDCUP99 Dataset, Jiang, pp. 432).
Regarding claim 6, Huang and Zhao fail to explicitly disclose, “wherein a count is maintained of each item assigned to each sub-class.”.
However, Jiang discloses, “wherein a count is maintained of each item assigned to each sub-class.” (Outlier Detection Method, pp. 430; “On the basis of the outlier factor of cluster, we present a clustering-based outlier detection method (CBOD), which consists of two stages. Stage 1. Clustering: Cluster on data set D and produce clustering results
C
=
C
1
,
C
2
,
.
.
.
C
k
.” All of the data is evaluated and placed into different subsets of clusters which are denoted as Ck. Since each cluster is a set, the length of the set would be interpreted as the total count of the cluster. As each item is added to the cluster the set length increases by one.)
Regarding claim 12, Huang and Zhao fail to explicitly disclose, “wherein the outlier identifier is further to sort each item in a training data set into the sub-classes by using a clustering algorithm.”.
However, Jiang discloses, “wherein the outlier identifier is further to sort each item in a training data set into the sub-classes by using a clustering algorithm.” (One-pass clustering algorithm, pp. 430; “The goal of clustering is that the intra-cluster similarity is maximized while the inter-cluster similarity is minimized. Many efficient clustering algorithms have been proposed by the database research community. Clustering algorithm can be selected according to data, objective of clustering and application. In this paper, we use one-pass clustering algorithm [13] divide dataset into hyper spheres with almost the same radius. The algorithm is described as follows: Step 1: Initialize the set of clusters, S, as the empty set, read a new object p. Step 2: Create a cluster with the object p. Step 3: If no objects are left in the database, go to step 6, otherwise read a new object p, and find the cluster C* in S that is closest to the object p. In other words, find a cluster C* in S, such that for all C in S,
d
p
,
C
*
≤
d
p
,
C
.
Step 4: If
d
p
,
C
*
>
r
, go to step 2. Step 5: Merge object p into cluster C* and modify the CSI of cluster C*, go to step 3. Step 6: Stop” In this article all of the data is evaluated and clustered. This data can be placed into already known clusters or generate a new cluster.)
Regarding claim 13, Huang and Zhao fail to explicitly disclose, “wherein a count is maintained of each item assigned to each sub-class.”.
However, Jiang discloses, “wherein a count is maintained of each item assigned to each sub-class.” (Outlier Detection Method, pp. 430; “On the basis of the outlier factor of cluster, we present a clustering-based outlier detection method (CBOD), which consists of two stages. Stage 1. Clustering: Cluster on data set D and produce clustering results
C
=
C
1
,
C
2
,
.
.
.
C
k
.” All of the data is evaluated and placed into different subsets of clusters which are denoted as Ck. Since each cluster is a set, the length of the set would be interpreted as the total count of the cluster. As each item is added to the cluster the set length increases by one.)
Regarding claim 19, Huang and Zhao fail to explicitly disclose, “wherein the outlier identifier is further to sort each item in a training data set into the sub-classes by using a clustering algorithm.”.
However, Jiang discloses, “wherein the outlier identifier is further to sort each item in a training data set into the sub-classes by using a clustering algorithm.” (One-pass clustering algorithm, pp. 430; “The goal of clustering is that the intra-cluster similarity is maximized while the inter-cluster similarity is minimized. Many efficient clustering algorithms have been proposed by the database research community. Clustering algorithm can be selected according to data, objective of clustering and application. In this paper, we use one-pass clustering algorithm [13] divide dataset into hyper spheres with almost the same radius. The algorithm is described as follows: Step 1: Initialize the set of clusters, S, as the empty set, read a new object p. Step 2: Create a cluster with the object p. Step 3: If no objects are left in the database, go to step 6, otherwise read a new object p, and find the cluster C* in S that is closest to the object p. In other words, find a cluster C* in S, such that for all C in S,
d
p
,
C
*
≤
d
p
,
C
.
Step 4: If
d
p
,
C
*
>
r
, go to step 2. Step 5: Merge object p into cluster C* and modify the CSI of cluster C*, go to step 3. Step 6: Stop” In this article all of the data is evaluated and clustered. This data can be placed into already known clusters or generate a new cluster.)
Regarding claim 20, Huang and Zhao fail to explicitly disclose, “wherein a count is maintained of each item assigned to each sub-class.”.
However, Jiang discloses, “wherein a count is maintained of each item assigned to each sub-class.” (Outlier Detection Method, pp. 430; “On the basis of the outlier factor of cluster, we present a clustering-based outlier detection method (CBOD), which consists of two stages. Stage 1. Clustering: Cluster on data set D and produce clustering results
C
=
C
1
,
C
2
,
.
.
.
C
k
.” All of the data is evaluated and placed into different subsets of clusters which are denoted as Ck. Since each cluster is a set, the length of the set would be interpreted as the total count of the cluster. As each item is added to the cluster the set length increases by one.)
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PAUL MICHAEL GALVIN-SIEBENALER whose telephone number is (571)272-1257. The examiner can normally be reached Monday - Friday 8AM to 5PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Viker Lamardo can be reached at (571) 270-5871. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/PAUL M GALVIN-SIEBENALER/Examiner, Art Unit 2147
/ERIC NILSSON/Primary Examiner, Art Unit 2151