Notice of Pre-AIA or AIA Status
1. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
2. This action is in response to the original filing on 04/17/2023. Claims 1-20 are pending and have been considered below.
Information Disclosure Statement
3. The information disclosure statement (IDS(s)) submitted on 04/17/2023 is/are in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Interpretation - 35 USC § 112(f)
4. The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
5. The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are: “a set of compute nodes configured to generate, perform, identify, input, compare, obtain, apply, group, calculate” in claims 11-20.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 101
6. 35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to the abstract idea without significantly more.
Step 1, the claims are directed to manufacture and machine.
Claim 1:
Step 2A Prong 1, Claim 1 recites, in part
generate scaled NA2D features based on feature scaling and dimensional reduction operations performed on the set of source vote features (Mathematical concepts, feature scaling and dimensional reduction are mathematical relationships and calculations).
apply the scaled NA2D features to a network address-domain (NAD) classification model to obtain a prediction dataset … indicates a probability that at least one domain maps to at least one IP address (Mathematical concepts, algorithmic calculation producing a probability).
Step 2A Prong 2, this judicial exception is not integrated into a practical application.
The additional elements:
One or more non-transitory computer readable media (NTCRM) comprising instructions for predicting network addresses to domain (NA2D) mappings using machine learning, wherein execution of the instructions by one or more processors is to cause a computing system (mere instructions to apply the exception using a generic computer component).
identify a set of source vote features from NA2D source data (mere data gathering and recited at a high level of generality, and thus are insignificant extra-solution activity).
Step 2B, the claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception, either alone or in combination.
The additional elements:
One or more non-transitory computer readable media (NTCRM) comprising instructions for predicting network addresses to domain (NA2D) mappings using machine learning, wherein execution of the instructions by one or more processors is to cause a computing system (mere instructions to apply the exception using a generic computer component).
identify a set of source vote features from NA2D source data (mere data gathering and output recited at a high level of generality, and thus are insignificant extra-solution activity).
Claim 2 provides further limitations “generate a training dataset; perform classification on the training dataset; and generate the NAD classification model based on the classification” to the abstract idea (Mathematical concepts) as rejected above. However, they do not disclose any additional elements that would amount to a practical application or significantly more than an abstract idea (adding insignificant extra-solution activity to the judicial exception).
Claim 3 provides further limitations “perform feature scaling and dimensional reduction on the training dataset; and perform classification on the feature scaled and dimension reduced training dataset” to the abstract idea (Mathematical concepts) as rejected above. However, they do not disclose any additional elements that would amount to a practical application or significantly more than an abstract idea (adding insignificant extra-solution activity to the judicial exception).
Claim 4 provides further limitations “generate labeled NA2D training data by labeling a training set of source vote features with known correct and incorrect NA2D labels” to the abstract idea (Mental processes) as rejected above. However, they do not disclose any additional elements that would amount to a practical application or significantly more than an abstract idea (adding insignificant extra-solution activity to the judicial exception).
Claim 5 provides further limitations “generate a first version of the NAD classification model using a first set of the labeled NA2D training data; input a second set of the labeled NA2D training data into the first version of the NAD classification model; and compare predictions output by the first version of the NAD classification model based on the second set of the labeled NA2D training data with labeled NA2D relationships” to the abstract idea (Mathematical concepts and Mental processes) as rejected above. However, they do not disclose any additional elements that would amount to a practical application or significantly more than an abstract idea (adding insignificant extra-solution activity to the judicial exception).
Claim 6 provides further limitations “generate a second version of the NAD classification model by refining feature weightings in the NAD classification model when the predictions output by the first version of the NAD classification model is less than a threshold value” to the abstract idea (Mathematical concepts and Mental processes) as rejected above. However, they do not disclose any additional elements that would amount to a practical application or significantly more than an abstract idea (adding insignificant extra-solution activity to the judicial exception).
Claim 7 provides further limitations of “the NA2D source data includes NAD mappings generated by one or more sources, a timestamp of each NAD pair in the NAD mappings, and a profile associated with each NAD pair, the profiles of each NAD pair comprising a unique identifier associated with a user, organization, computing device, or network session event; and the one or more sources include email sniffers, email logins, email opens, offline lookups, historical domain name registration records, and tags or scripts included in information objects or applications”. However, they do not disclose any additional elements that would amount to a practical application or significantly more than an abstract idea (adding insignificant extra-solution activity to the judicial exception).
Claim 8 provides further limitations of “generate a vote matrix to include the set of source vote features”. However, they do not disclose any additional elements that would amount to a practical application or significantly more than an abstract idea (adding insignificant extra-solution activity to the judicial exception).
Claim 9 provides further limitations “group the NA2D source data into network address-domain-source (ADS) keys; and calculate a domain count, profile count, and a confusion value for each ADS key, the domain count is a total number of times a source of the one or more sources maps to an individual domain and to an individual IP address, the profile count is a number of unique profiles associated with each unique NAD mapping for a same source of the one or more sources, and the confusion value is an amount of source confusion or entropy associated with an NAD pair” to the abstract idea (Mathematical concepts and Mental processes) as rejected above. However, they do not disclose any additional elements that would amount to a practical application or significantly more than an abstract idea (adding insignificant extra-solution activity to the judicial exception).
Claim 10 provides further limitations of “wherein the training data includes network addresses associated with known entities or network addresses associated with known networks”. However, they do not disclose any additional elements that would amount to a practical application or significantly more than an abstract idea (adding insignificant extra-solution activity to the judicial exception).
Claim 11:
Step 2A Prong 1, Claim 11 recites, in part
by labeling a set of source vote features with known correct and incorrect NA2D labels (Mental processes, evaluation/judgment about information).
perform classification on the NA2D training dataset (Mathematical concepts, mathematical calculation on data).
generate an NA2D classifier model based on the classification (Mathematical concepts, mathematical operations).
Step 2A Prong 2, this judicial exception is not integrated into a practical application.
The additional elements:
(NA2D) resolution system and a set of compute nodes (mere instructions to apply the exception using a generic computer component).
generate an NA2D training dataset (mere data gathering and recited at a high level of generality, and thus are insignificant extra-solution activity).
Step 2B, the claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception, either alone or in combination.
The additional elements:
(NA2D) resolution system and a set of compute nodes (mere instructions to apply the exception using a generic computer component).
generate an NA2D training dataset (mere data gathering and recited at a high level of generality, and thus are insignificant extra-solution activity).
Claim 12 provides further limitations “perform feature scaling and dimensional reduction on the NA2D training dataset; and perform the classification on the feature scaled and dimension reduced NA2D training dataset” to the abstract idea (Mathematical concepts) as rejected above. However, they do not disclose any additional elements that would amount to a practical application or significantly more than an abstract idea (adding insignificant extra-solution activity to the judicial exception).
Claim 13 provides further limitations “identify organization characteristics for IP addresses; and combine the identified organization characteristics with the source vote features” to the abstract idea (Mental processes) as rejected above. However, they do not disclose any additional elements that would amount to a practical application or significantly more than an abstract idea (adding insignificant extra-solution activity to the judicial exception).
Claim 14 provides further limitations “generate a first version of the NA2D classifier model using a first set of a labeled NA2D training data; input a second set of the labeled NA2D training data into the first version of the NA2D classifier model; and compare predictions output by the first version of the NA2D classifier model based on the second set of the labeled NA2D training data with labeled NA2D relationships” to the abstract idea (Mathematical concepts and Mental processes) as rejected above. However, they do not disclose any additional elements that would amount to a practical application or significantly more than an abstract idea (adding insignificant extra-solution activity to the judicial exception).
Claim 15 provides further limitations “generate a second version of the NA2D classifier model by refining feature weightings in the NA2D classifier model when the predictions output by the first version of the NA2D classifier model is less than a threshold value” to the abstract idea (Mathematical concepts and Mental processes) as rejected above. However, they do not disclose any additional elements that would amount to a practical application or significantly more than an abstract idea (adding insignificant extra-solution activity to the judicial exception).
Claim 16 provides further limitations “obtain the set of source vote features from NA2D source data; generate scaled NA2D features based on feature scaling and dimensional reduction operations performed on the set of source vote features; and apply the scaled NA2D features to the NA2D classifier model to obtain a prediction dataset, wherein the prediction dataset indicates a probability that at least one domain maps to at least one IP address” to the abstract idea (Mathematical concepts) as rejected above. However, they do not disclose any additional elements that would amount to a practical application or significantly more than an abstract idea (adding insignificant extra-solution activity to the judicial exception, mere data gathering).
Claim 17 provides further limitations of “the NA2D source data includes NAD mappings generated by one or more sources, a timestamp of each NAD pair in the NAD mappings, and a profile associated with each NAD pair, the profiles of each NAD pair comprising a unique identifier associated with a user, organization, computing device, or network session event; and the one or more sources include email sniffers, email logins, email opens, offline lookups, historical domain name registration records, and tags or scripts included in information objects or applications”. However, they do not disclose any additional elements that would amount to a practical application or significantly more than an abstract idea (adding insignificant extra-solution activity to the judicial exception).
Claim 18 provides further limitations of “generate a vote matrix to include the set of source vote features”. However, they do not disclose any additional elements that would amount to a practical application or significantly more than an abstract idea (adding insignificant extra-solution activity to the judicial exception).
Claim 19 provides further limitations “group the NA2D source data into network address-domain-source (ADS) keys; and calculate a domain count, profile count, and a confusion value for each ADS key, the domain count is a total number of times a source of the one or more sources maps to an individual domain and to an individual IP address, the profile count is a number of unique profiles associated with each unique NAD mapping for a same source of the one or more sources, and the confusion value is an amount of source confusion or entropy associated with an NAD pair) as rejected above. However, they do not disclose any additional elements that would amount to a practical application or significantly more than an abstract idea (adding insignificant extra-solution activity to the judicial exception).
Claim 20 provides further limitations of “wherein the NA2D resolution system is implemented by a set of technologies selected from a group comprising: a cloud computing service, a content delivery network (CDN), and an edge computing network”. However, they do not disclose any additional elements that would amount to a practical application or significantly more than an abstract idea (mere instructions to apply the exception using a generic computer component).
Claim Rejections – 35 USC § 103
7. The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
8. Claims 1-5, 7, 8, 10, 12, 13, 16-19, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Teller et al. (U.S. Patent Application Pub. No. US 20180349599 A1) in view of Nandy et al. (U.S. Patent Application Pub. No. US 20150341376 A1).
Claim 1: Teller teaches one or more non-transitory computer readable media (NTCRM) comprising instructions (i.e. The computer programs include processor-executable instructions that are stored on at least one non-transitory computer-readable medium; para. [0076]) for predicting network addresses to domain (NA2D) mappings using machine learning, wherein execution of the instructions by one or more processors is to cause a computing system (i.e. The system may include a processor, memory, and a botnet detection application that is stored in the memory and executed by the processor. The botnet detection application may be configured to obtain (i) Netflow data indicating one or more IP addresses accessed by a computer and (ii) passive Domain Name System (DNS) data indicating respective one or more domains associated with each of the one or more IP addresses; para. [0005]) to:
identify a set of source vote features (i.e. fig. 4, the feature generation module 418 of the Botnet detection system 412 is configured to generate features 432 associated with the potential bot computer 402 based on at least the Netflow data 428 and the passive DNS data 430; para. [0049]) from NA2D source data (i.e. the Netflow data 428 and the passive DNS data 430; para. [0049]);
generate scaled NA2D features based on feature scaling operations performed on the set of source vote features (i.e. fig. 4, The feature weighting module 422 of the Botnet detection system 412 is configured to assign weights to the features 432 based on the probability data 434 to provide weighted features 436. More specifically, the feature weighting module 422 assigns weights to particular features based on the probability that the feature information (i.e., the various indications discussed above) is accurate—as measured by the probability data 434; para. [0054-0056]); and
apply the scaled NA2D features to a network address-domain (NAD) classification model to obtain a prediction dataset (i.e. The supervised machine learning algorithm module 424 is configured to determine whether the computer 402 is likely to be part of a Botnet based on the weighted features 436; para. [0057]), weighted features are input to a classifier that produces a decision, wherein the prediction dataset indicates a probability (i.e. The probability data 434 indicates a probability that the computer 402 accessed the one or more domains indicated by the passive DNS data 430. More specifically, the probability data 434 may, in some examples, constitute a probabilistic map of domain access associated with the computer 402; para. [0054]) that at least one domain maps to at least one IP address (i.e. At 514, a Botnet score is calculated for the computer by applying a supervised machine learning algorithm to the weighted features and a determination is made as to whether the calculated Botnet score is greater than or equal to a predetermined threshold. If so, the method 500 progresses to 516 where the computer is determined to likely be part of a Botnet and the method concludes. If the computer's calculated Botnet score is less than the predetermined threshold, the method proceeds to 518 and the computer is determined to not likely be part of a Botnet and the method concludes; para. [0063]), these paragraphs applying weighted features to supervised classifier that produce number scores and probabilistic maps to domain access.
Teller does not explicitly teach dimensional reduction operations.
However, Nandy teaches dimensional reduction operations (i.e. The principal component analysis (PCA) operation is a known method (operation) for detecting a network anomaly. The PCA operation works based on the dimensional reduction property of the PCA method, and was shown to be effective in finding and diagnosing network anomalies in large networks where the dimension of the network is relatively large; para. [0017-0019]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention to modify the invention of Teller to include the feature of Nandy. One would have been motivated to make this modification because dimensional reduction (PCA) is a standard, well-known preprocessing step in machine learning to reduce feature-dimensionality, improve classifier performance, and reduce overfitting.
Claim 2: Teller and Nandy teach the one or more NTCRM of claim 1. Teller further teaches wherein execution of the instructions is to cause the computing system to: generate a training dataset (i.e. the supervised machine learning algorithm module 424 is trained using historical data and known labels; para. [0059, 0060]); perform classification on the training dataset (i.e. the supervised machine learning algorithm module 424 is configured to correlate the weighted features 436 to one of two different labels applicable to the computer 402 under examination: (1) likely to be part of a Botnet or (2) not likely to be part of a Botnet; para. [0057, 0058]); and generate the NAD classification model based on the classification (i.e. the supervised machine learning algorithm module 424 is trained using historical data and known labels; para. [0059, 0060]).
Claim 3: Teller and Nandy teach the one or more NTCRM of claim 2. Teller further teaches wherein execution of the instructions is to cause the computing system to: perform feature scaling on the training dataset (i.e. The feature weighting module 422 of the Botnet detection system 412 is configured to assign weights to the features 432 based on the probability data 434 to provide weighted features 436; para. [0056]); and perform classification on the feature scaled training dataset (i.e. The supervised machine learning algorithm module 424 is configured to determine whether the computer 402 is likely to be part of a Botnet based on the weighted features 436. More specifically, the supervised machine learning algorithm module 424 is configured to correlate the weighted features 436 to one of two different labels applicable to the computer 402 under examination: (1) likely to be part of a Botnet or (2) not likely to be part of a Botnet; para. [0057]).
Teller does not explicitly teach dimensional reduction and dimension reduced.
However, Nandy further teaches dimensional reduction and dimension reduced (i.e. The principal component analysis (PCA) operation is a known method (operation) for detecting a network anomaly. The PCA operation works based on the dimensional reduction property of the PCA method, and was shown to be effective in finding and diagnosing network anomalies in large networks where the dimension of the network is relatively large; para. [0017-0019]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention to modify the invention of Teller to include the feature of Nandy. One would have been motivated to make this modification because dimensional reduction (PCA) is a standard, well-known preprocessing step in machine learning to reduce feature-dimensionality, improve classifier performance, and reduce overfitting.
Claim 4: Teller and Nandy teach the one or more NTCRM of claim 3. Teller further teaches wherein, to generate the training dataset, execution of the instructions is operable to cause the computing system to: generate labeled NA2D training data by labeling (i.e. The feature generation module 418 of the Botnet detection system 412 is configured to generate features 432 associated with the potential bot computer 402 based on at least the Netflow data 428 and the passive DNS data 430; para. [0049]) a training set of source vote features (i.e. The features 432 include one or more indications that influence the ultimate determination as to whether the potential bot computer 402 is likely to be part of a botnet or not. Such features 432 may include, but are not limited to: (i) an indication that the computer 402 has accessed rare IP addresses or domains; (ii) an indication that the computer 402 has accessed domains having less than or equal to a predetermined age; (iii) an indication that the computer 402 has received a number of NX domain responses (i.e., DNS responses indicating that the queried domain does not exist) to DNS queries exceeding a predetermined threshold; para. [0050]) with known correct and incorrect NA2D labels (i.e. The supervised machine learning algorithm module 424 is configured to determine whether the computer 402 is likely to be part of a Botnet based on the weighted features 436. More specifically, the supervised machine learning algorithm module 424 is configured to correlate the weighted features 436 to one of two different labels applicable to the computer 402 under examination: (1) likely to be part of a Botnet or (2) not likely to be part of a Botnet; para. [0057]).
Claim 5: Teller and Nandy teach the one or more NTCRM of claim 4. Teller further teaches wherein, to perform the classification, execution of the instructions is operable to cause the computing system to: generate a first version of the NAD classification model using a first set of the labeled NA2D training data (i.e. the supervised machine learning algorithm module 424 is trained using historical data and known labels 426; para. [0059, 0060]); input a second set of the labeled NA2D training data into the first version of the NAD classification model (i.e. At 514, a Botnet score is calculated for the computer by applying a supervised machine learning algorithm to the weighted features; para. [0057, 0063]); and compare predictions output by the first version of the NAD classification model based on the second set of the labeled NA2D training data with labeled NA2D relationships (i.e. the supervised machine learning algorithm module 424 may be configured to determine whether the computer 402 is likely to be part of a Botnet by comparing the weighted features 436 with corresponding features associated with different computers known to be part of a Botnet (as reflected by the historical data and known labels 426); para. [0057, 0060]).
Claim 7: Teller and Nandy teach the one or more NTCRM of claim 1. Teller further teaches wherein:
the NA2D source data includes NAD mappings generated by one or more sources (i.e. the passive DNS data 430 may include a log of mappings from domains to IPs at certain time intervals as provided, for example, by DNS servers; para. [0048]), a timestamp of each NAD pair in the NAD mappings (i.e. the passive DNS data 430 may include a log of mappings from domains to IPs at certain time intervals as provided, for example, by DNS servers; para. [0048]), and a profile associated with each NAD pair, the profiles of each NAD pair comprising a unique identifier associated with a user, organization, computing device, or network session event (i.e. the feature generation module 418 is further configured to obtain domain registration data 440 from the domain registration data source 408 and generate the features 432 based additionally on the domain registration data 440. As noted above, the domain registration data 440 may include data concerning the registrant and date of registration associated with each domain of one or more domains accessed (or potentially accessed) by the potential bot computer 402; para. [0049]); and
the one or more sources include email sniffers, email logins, email opens, offline lookups, historical domain name registration records, and tags or scripts included in information objects or applications (i.e. the domain registration data source 408 may include a data store or the like that stores registrant and registration date information for one or more domains. In another example, the domain registration data source 408 may include a website, such as WHOIS.com, that includes registrant and registration date information for one or more domains; para. [0044, 0049, 0077]).
Claim 8: Teller and Nandy teach the one or more NTCRM of claim 7. Teller further teaches wherein execution of the instructions is to cause the computing system to: generate a vote to include the set of source vote features (i.e. the feature generation module 418 of the Botnet detection system 412 is configured to generate features 432 associated with the potential bot computer 402 based on at least the Netflow data 428 and the passive DNS data 430; para. [0049, 0050, 0054]).
Teller does not explicitly teach a vote matrix.
However, Nandy further teaches generate a vote matrix to include the set of source vote features (i.e. generating an [m×n] input network traffic matrix 400 including attributes extracted from data associated with the collected network data flow 116; para. [0050, 0207, 0251]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention to modify the invention of Teller to include the feature of Nandy. One would have been motivated to make this modification because dimensional reduction (PCA) is a standard, well-known preprocessing step in machine learning to reduce feature-dimensionality, improve classifier performance, and reduce overfitting.
Claim 10: Teller and Nandy teach the one or more NTCRM of claim 2. Teller further teaches wherein the training data includes network addresses associated with known entities (i.e. the domain registration data source 408 may include a data store or the like that stores registrant and registration date information for one or more domains. In another example, the domain registration data source 408 may include a website, such as WHOIS.com, that includes registrant and registration date information for one or more domains; para. [0044]) or network addresses associated with known networks (i.e. the supervised machine learning algorithm module 424 is trained using historical data and known labels 426 before being applied to the weighted features 436 associated with the potential bot computer 402; para. [0059]).
Claim 12 is similar in scope to Claim 3 and is rejected under a similar rationale.
Claim 13: Teller and Nandy teach the NA2D resolution system of claim 12. Teller further teaches wherein, to generate the NA2D training dataset, the set of compute nodes are configured to: identify organization characteristics for IP addresses (i.e. the domain registration data source 408 may include a website, such as WHOIS.com, that includes registrant and registration date information for one or more domains; para. [0044, 0055]); and combine the identified organization (i.e. the probability module 422 is further configured to obtain domain registration data 440 from the domain registration data source 408 and generate the probability data 434 based additionally on the domain registration data 440. Furthermore, in one example, the probability module 422 is configured to generate the probability data 434 based on count data, which may be extracted from the Netflow data 428 and/or passive DNS data 430. The count data may indicate, for example, the number of times a given domain may be mapped to a given IP for all servers that provided a response at a particular time; para. [0055]) characteristics with the source vote features (i.e. The feature generation module 418 of the Botnet detection system 412 is configured to generate features 432 associated with the potential bot computer 402 based on at least the Netflow data 428 and the passive DNS data 430; para. [0049]).
Claims 16-19 are similar in scope to Claims 1, 7-9 and are rejected under a similar rationale.
Claim 20: Teller and Nandy teach the NA2D resolution system of claim 17. Teller further teaches wherein the NA2D resolution system is implemented by a set of technologies selected from a group comprising: a cloud computing service, a content delivery network (CDN), and an edge computing network (i.e. fig. 1, a cloud computing environment; para. [0031, 0042, 0071]).
9. Claim 6 is rejected under 35 U.S.C. 103 as being unpatentable over Teller in view of Nandy, and further in view of Fu et al. (U.S. Patent Application Pub. No. US 20200410390 A1).
Claim 6: Teller and Nandy teach the one or more NTCRM of claim 5. Teller further teaches wherein, to perform the classification, execution of the instructions is to cause the computing system to: generate a second version of the NAD classification model by refining feature weightings in the NAD classification model when the predictions output by the first version of the NAD classification model (i.e. More specifically, the supervised machine learning algorithm module 424 may be trained by extracting feature sets associated with computers having known labels, i.e., computers either (i) known to be part of a Botnet (one type of label) or (ii) known to not be part of a Botnet (the other type of label). In this way, the supervised machine learning algorithm module 424 may regressively improve its Botnet detection accuracy as more features associated with known labels are analyzed; para. [0056, 0060]).
Teller does not explicitly teach refining feature weightings in the model when the predictions output by the model is less than a threshold value.
However, Fu teaches refining feature weightings in the model when the predictions output by the model is less than a threshold value (i.e. The one or more programs include instructions that: monitor operation of a machine learning model with a target application; generate a first metric that reflects an ability of the machine learning model to make a prediction given input features; generate a second metric that reflects usage of predictions made by the machine learning model; and when the first metric or the second metric falls below a threshold, retrain the machine learning model with a new training dataset; para. [0066]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention to modify the combination of Teller and Nandy to include the feature of Fu. One would have been motivated to make this modification because it provides an improved version of the classification model.
10. Claim 9 is rejected under 35 U.S.C. 103 as being unpatentable over Teller in view of Nandy, and further in view of Bajaria et al. (U.S. Patent Application Pub. No. US 20190199677 A1).
Claim 9: Teller and Nandy teach the one or more NTCRM of claim 8. Teller further teaches wherein, to generate the vote matrix, execution of the instructions is to cause the computing system to: group the NA2D source data into network (i.e. The feature generation module 418 of the Botnet detection system 412 is configured to generate features 432 associated with the potential bot computer 402 based on at least the Netflow data 428 and the passive DNS data 430. In one example, the feature generation module 418 is further configured to obtain domain registration data 440 from the domain registration data source 408 and generate the features 432 based additionally on the domain registration data 440. As noted above, the domain registration data 440 may include data concerning the registrant and date of registration associated with each domain of one or more domains accessed (or potentially accessed) by the potential bot computer 402; para. [0049]); and calculate a domain count (i.e. the probability module 422 is configured to generate the probability data 434 based on count data, which may be extracted from the Netflow data 428 and/or passive DNS data 430. The count data may indicate, for example, the number of times a given domain may be mapped to a given IP for all servers that provided a response at a particular time; para. [0055]), and a confusion value for each ADS key, the domain count is a total number of times a source of the one or more sources maps to an individual domain and to an individual IP address (i.e. the probability module 422 is configured to generate the probability data 434 based on count data, which may be extracted from the Netflow data 428 and/or passive DNS data 430. The count data may indicate, for example, the number of times a given domain may be mapped to a given IP for all servers that provided a response at a particular time; para. [0055]), and the confusion value is an amount of source confusion or entropy associated with an NAD pair (i.e. the feature generation module 418 may be configured to calculate an idiosyncratic score for IPs/domains accessed, or potentially accessed, by the computer 402, and determine whether the calculated idiosyncratic score exceeds a predetermined threshold. According to this example, the idiosyncratic score may be calculated as follows … calculating idiosyncratic scores is similar to Kullback-Leibler (KL) Divergence; para. [0051-0053]).
Teller does not explicitly teach address-domain-source (ADS) keys; the profile count is a number of unique profiles associated with each unique NAD mapping for a same source of the one or more sources.
However, Bajaria teaches address-domain-source (ADS) keys; the profile count is a number of unique profiles associated with each unique NAD mapping for a same source of the one or more sources (i.e. receive a plurality of events from a plurality of sources, wherein each of the plurality of events represents an online activity and comprises an IP address and event information; aggregate subsets of the plurality of events into a plurality of mappings, wherein each of the plurality of mappings associates the IP address, shared by a subset of the plurality of events, with an account, and is associated with a plurality of statistics regarding the subset of events; para. [0007, 0009]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention to modify the combination of Teller and Nandy to include the feature of Bajaria. One would have been motivated to make this modification because it reduces misclassification risk and improve precision.
11. Claim 15 is rejected under 35 U.S.C. 103 as being unpatentable over Teller in view of Fu et al. (U.S. Patent Application Pub. No. US 20200410390 A1).
Claim 15: Teller teaches the NA2D resolution system of claim 14. Teller further teaches wherein, to perform the classification, the set of compute nodes are configured to: generate a second version of the NA2D classifier model by refining feature weightings in the NA2D classifier model when the predictions output by the first version of the NA2D classifier (i.e. More specifically, the supervised machine learning algorithm module 424 may be trained by extracting feature sets associated with computers having known labels, i.e., computers either (i) known to be part of a Botnet (one type of label) or (ii) known to not be part of a Botnet (the other type of label). In this way, the supervised machine learning algorithm module 424 may regressively improve its Botnet detection accuracy as more features associated with known labels are analyzed; para. [0056, 0060]).
Teller does not explicitly teach refining feature weightings in the model when the predictions output by the model is less than a threshold value.
However, Fu teaches refining feature weightings in the model when the predictions output by the model is less than a threshold value (i.e. The one or more programs include instructions that: monitor operation of a machine learning model with a target application; generate a first metric that reflects an ability of the machine learning model to make a prediction given input features; generate a second metric that reflects usage of predictions made by the machine learning model; and when the first metric or the second metric falls below a threshold, retrain the machine learning model with a new training dataset; para. [0066]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention to modify the invention of Teller to include the feature of Fu. One would have been motivated to make this modification because it provides an improved version of the classification model.
Claim Rejections - 35 USC § 102
12. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
13. Claims 11 and 14 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Teller et al. (U.S. Patent Application Pub. No. US 20180349599 A1).
Claim 11: Teller teaches a network address to domain (NA2D) resolution system, comprising:
a set of compute nodes (i.e. FIG. 1 shows a simplified example of a distributed computing system 100. The distributed computing system 100 includes a distributed communications system 110, one or more client devices 120-1, 120-2, . . . , and 120-M (collectively, client devices 120), and one or more servers 130-1, 130-2, . . . , and 130-M (collectively, servers 130); para. [0031]) configured to:
generate an NA2D training dataset (i.e. The feature generation module 418 of the Botnet detection system 412 is configured to generate features 432 associated with the potential bot computer 402 based on at least the Netflow data 428 and the passive DNS data 430; para. [0049]) by labeling a set of source vote features (i.e. The features 432 include one or more indications that influence the ultimate determination as to whether the potential bot computer 402 is likely to be part of a botnet or not. Such features 432 may include, but are not limited to: (i) an indication that the computer 402 has accessed rare IP addresses or domains; (ii) an indication that the computer 402 has accessed domains having less than or equal to a predetermined age; (iii) an indication that the computer 402 has received a number of NX domain responses (i.e., DNS responses indicating that the queried domain does not exist) to DNS queries exceeding a predetermined threshold; para. [0050]) with known correct and incorrect NA2D labels (i.e. The supervised machine learning algorithm module 424 is configured to determine whether the computer 402 is likely to be part of a Botnet based on the weighted features 436. More specifically, the supervised machine learning algorithm module 424 is configured to correlate the weighted features 436 to one of two different labels applicable to the computer 402 under examination: (1) likely to be part of a Botnet or (2) not likely to be part of a Botnet; para. [0057]);
perform classification on the NA2D training dataset (i.e. the supervised machine learning algorithm module 424 is configured to correlate the weighted features 436 to one of two different labels applicable to the computer 402 under examination: (1) likely to be part of a Botnet or (2) not likely to be part of a Botnet; para. [0057, 0058]); and
generate an NA2D classifier model based on the classification (i.e. the supervised machine learning algorithm module 424 is trained using historical data and known labels; para. [0059, 0060]).
Claim 14: Teller teaches the NA2D resolution system of claim 11. Teller further teaches wherein, to perform the classification, the set of compute nodes are configured to: generate a first version of the NA2D classifier model using a first set of a labeled NA2D training data (i.e. the supervised machine learning algorithm module 424 is trained using historical data and known labels 426; para. [0059, 0060]); input a second set of the labeled NA2D training data into the first version of the NA2D classifier model (i.e. At 514, a Botnet score is calculated for the computer by applying a supervised machine learning algorithm to the weighted features; para. [0057, 0063]); and compare predictions output by the first version of the NA2D classifier model based on the second set of the labeled NA2D training data with labeled NA2D relationships (i.e. the supervised machine learning algorithm module 424 may be configured to determine whether the computer 402 is likely to be part of a Botnet by comparing the weighted features 436 with corresponding features associated with different computers known to be part of a Botnet (as reflected by the historical data and known labels 426); para. [0057, 0060]).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant’s disclosure.
Grove et al. (Pub. No. US 20160371601 A1), it may be determined or judged as to whether the quality of the model is below a quality threshold set for retraining criteria. The processing at 308, 310 and 312 are performed in one embodiment responsive to determining that the model should be retrained, for example, the quality of the model is below the quality threshold, and thus retraining is needed.
It is noted that any citation to specific pages, columns, lines, or figures in the prior art references and any interpretation of the references should not be considered to be limiting in any way. A reference is relevant for all it contains and may be relied upon for all that it would have reasonably suggested to one having ordinary skill in the art. In re Heck, 699 F.2d 1331, 1332-33, 216 U.S.P.Q. 1038, 1039 (Fed. Cir. 1983) (quoting In re Lemelson, 397 F.2d 1006, 1009, 158 U.S.P.Q. 275, 277 (C.C.P.A. 1968)).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to TAN TRAN whose telephone number is (303)297-4266. The examiner can normally be reached on Monday - Thursday - 8:00 am - 5:00 pm MT.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Matt Ell can be reached on 571-270-3264. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/TAN H TRAN/Primary Examiner, Art Unit 2141