Using Threat Feeds

The topics in this how-to provide an introduction to using threat feeds in Kentik to reveal compromised hosts:

• Threat Feed Dimensions
• Query for Bots
• Set Up a Threat Report
• Alert on Compromised Hosts

One of the ways that Kentik helps users to visualize and investigate network traffic patterns is the ability to find traffic from infected or compromised hosts. What makes this possible is the fact that we enrich flow records (NetFlow, sFlow, IPFIX, etc.) in Kentik with IP reputation data from our friends at Spamhaus. Based on this information, we expose two dimensions that you can use for group-by or filtering: Bot Net CC and Threat List Host (see Threat Feed Columns).

In this post, we’ll walk through a use case for the Bot Net CC dimension, but first let’s look deeper into the threat feed information from Spamhaus and the corresponding dimensions in Kentik:

• Bot Net CC: This dimension is based on a list of IPs where botnet command and control (CC) servers are known to be running (or were running in the past). By looking for IPs that are communicating with these CC IPs you can identify hosts (or subscribers) in your network that are potentially infected with malware or otherwise participating in botnet activity.
• Threat List Host: This dimension includes a broader set of potentially malicious IPs on the Internet. The IPs are identified as malware distribution points, phishing websites, spam sources, etc. When this column is populated in Kentik’s datastore we identify the specific type of threat associated with the IP. This dimension can help hosting providers identify IPs in their own network that are "doing bad stuff," but it can also enable any network operator to identify client IPs that are talking to remote IPs on the list.

To illustrate the use of Bot Net CC, let’s imagine a fictional enterprise called Pear, Inc. The savvy reader is probably already aware that a popular method of Distributed Denial of Service (DDoS) attacks is to infect hosts with a virus. These hosts then communicate with a Command and Control (C&C) server that tells them where to send attack traffic. Pear’s IT Security department would want to see if they have any hosts on their network that have been compromised and are communicating with a C&C server, thereby participating in a botnet.

We can check this by building a query in the sidebar panes of the Data Explorer (Data Explorer » Explorer View). Using the Group By Dimensions selector in the Query pane, we set the query to look at two dimensions that, when combined (shown at right), will give the list of compromised hosts on our network: Source IP/CIDR and Destination Bot Net CC.

Also in the sidebar, we set the Time pane (see Time Pane) to look back at the last hour, and in the Devices pane (Data Sources Pane) we choose All so that we can look at our entire network.

We’ll also build a filter in the Filtering pane (shown at right; see Filtering Pane) that will ignore our normal network traffic and look only at botnet traffic. We do this by filtering out traffic whose botnet value is blank:

• Click anywhere in the pane to open the Filtering Options dialog.
• In the Ad-Hoc Filter Groups pane, click Add Filter Group.
• Change Include to Exclude.
• Change the dimension to Destination Bot Net CC.
• Click Save Changes.

After our query configuration is complete, we click on the Run Query button at the top of the sidebar. If there were four infected hosts then the query would return a graph that looks similar to the below. Based on these results, the IT Security staff at Pear would know which hosts to examine for botnet malware, so it can be identified and removed.

Pear could take this a step further by running this query as a scheduled report that is emailed to subscribers (specified users). Reports are based on dashboards (see Subscriptions). The following steps show how to save the query to a new dashboard and then set up a weekly report:

• From the drop-down View Options menu at the upper right of the Data Explorer display area (above the graph), choose Add to Dashboard » New Dashboard. The Add Dashboard dialog will open.
• Enter “Bot Traffic” in the Name field, then click Add Dashboard in the lower right corner. The Add Panel dialog will open.
• Leave the dialog settings at defaults (the name of the panel will be “Top Source IP/CIDR, Dest Bot Net CC by Average bits/s”) and click the Add Panel button. You’ll be taken to the new “Bot Traffic” dashboard.
• Click Admin in the main portal navbar, then choose the Subscriptions page from the sidebar.
• Click the Add Subscription button at upper right to open the Add Subscription dialog.
• Fill out the subscription settings as shown in the screenshot at right, then click the Add Subscription button.

If the Pear IT Security department were to follow the steps above, they would receive an email every Monday with a list of infected hosts covering the previous seven days.

Getting a scheduled report is nice, but it would be even more helpful to be notified automatically whenever an infected host is found. By creating an alert policy in Kentik’s alerting system, Pear’s IT department could get a Slack notification in their #itsec channel any time a new host starts talking to a C&C server.

To do this, we would choose Alerting from the main portal navbar, then click Policies in the sidebar. On the resulting Policies page we click the Add Policy button. In the resulting Add Alert Policy dialog, we make settings on the tabs covered in the following topics.

In this tab we give our policy a name and change the Policy Dashboard dropdown to Bot Traffic, the dashboard we created in the previous section. Note that we leave Silent Mode enabled so that the alert does not activate while the alerting system is determining what’s normal (the baseline). It’s also nice to add a description so that everyone in the organization can see at a glance what this policy is for.

In the Data Funneling pane we first confirm that Devices is set to All, then click anywhere in the Dimensions selector to add the same dimensions we used in our original query: Source IP/CIDR and Destination Bot Net CC. We’ll also click in the Secondary Metrics selector and choose Bits/second. Lastly, we'll once again set a filter to exclude traffic whose botnet value is blank.

Moving down to the Building Your Dataset pane, we change the Maximum Number of Keys Analyzed Per Evaluation to 100. We leave everything else at default so the system auto-calculates the Minimum Traffic Threshold.

We choose a criticality level (e.g. Major) from the list at left, then set the switch at right to Enabled, which reveals a number of panes containing controls. In the Threshold Configuration pane, we choose Check If Certain Conditions Exist from the drop-down This Threshold Will menu. Leave the other settings at their defaults.

Moving down to the Conditions pane, we set a threshold of Greater Than 1 kpps. This will trigger the alarm if any of the hosts on Pear’s network are sending more than 1000 packets per second of traffic to a known Botnet C&C server.

The final pane that needs our attention is Notifications. Click anywhere in the Send Alert Status Updates To selector and choose Slack itsec. Note that this notification channel must already have been configured under Alerting » Channels (see Notifications).

On the Historical Baseline tab, leave the default settings. Once all of the required fields have been specified there will be a green check mark on all of the tabs. When we click on the Add Alert Policy button, the policy will begin in Silent Mode, studying the normal patterns of traffic. After the Silent Mode end date (see General Policy Settings), the IT Security department will get a Slack notification every time a new host starts talking to a botnet C&C server.

This is just a taste of the ways that Kentik’s new Threat Feeds information can enable your company to see compromised hosts on their network that they’ve never been able to see before. This insight can be used for both proactive monitoring and data forensics to protect the company’s infrastructure. If you'd like help setting up or using this feature, get in touch with your Kentik support team (support@kentik.com).