Host Configuration |
Note: kprobe (beta) has replaced nProbe as the host agent software used by Kentik to collect traffic data from hosts. While existing nProbe devices will continue to work, it’s no longer possible to create new nProbe devices. |
Hosts that send flow data to Kentik do so via kprobe host agent software. The use of kprobe with Kentik is covered in the following topics:
Note: For help installing and configuring kprobe, please contact support@kentik.com.
top |
About kprobe |
Kentik is designed to enable flow monitoring not only of routers and switches but also of hosts, which can send augmented flow data. In addition to flow records (NetFlow, sFlow, IPFIX), this augmented data includes Network Performance Metrics (retransmits, network latency, and application latency) as well as Layer 7 info such as requests/responses for both DNS and HTTP. This information is unified in the Kentik Data Engine (KDE) with other data such as GEO and BGP, providing the user — and our anomaly detection/alerting system — with a comprehensive view of where traffic is originating and terminating, traffic performance relative to Internet routing paths, and actual HTTP and DNS requests.
The host application/agent that enables the above functionality is kprobe, which is agent software that runs on Linux hosts. kprobe enables customers to send encrypted flow records from a host to Kentik. The kprobe agent listens for incoming and outgoing packets on any network interface and generates flow data from the packets received.
Notes:
- kprobe is included with your Kentik subscription or trial.
- Each instance of kprobe will send HTTPS-encrypted data directly to Kentik on port 443. If it’s not possible to connect to the Internet, the data can be sent through an HTTP proxy instead, see Host Flow Via Proxy.
top |
Host Metrics and Dimensions |
Based on the data sent from kprobe, Kentik is able to make available a comprehensive set of host-related metrics and dimensions, which are covered in the following topics:
top |
kprobe Requirements |
The following resources must be available to support the use of kprobe:
In addition to the above, communication between kprobe and Kentik will require you to enable kprobe to open multiple https sessions to multiple *.kentik.com hosts destined to port 443 (or *.kentik.eu if your organization is registered on Kentik’s EU cluster). Please ensure that any proxies, firewalls, routers, and NAT boxes permit this communication. kprobe will work properly through NAT and proxies.
top |
kprobe Traffic Capacity |
Each monitored interface requires its own individual instance of kprobe, with only one such instance per interface. Each of these instances uses only a single core, which prevents excessive CPU utilization but also determines the volume of traffic that can be handled per interface. The following table provides a very rough guide to how kprobe’s traffic capacity per interface (maximum in-plus-out bits) varies by sampling rate.
Sampling ratio | Max traffic volume |
1:1 or 1:2 | 100 Mbps |
1:40 | 500 Mbps |
1:80 | 1 Gbps |
1:256 | 3 Gbps |
1:1024 | 10 Gbps |
1:4092 | > 10 Gbps |
Notes:
- Actual performance is affected by a number of factors. A procedure for matching sample rate to traffic volume is outlined in Setting Sampling Rate.
- If protocol decoding (e.g. DNS/WWW data) is not needed, you can optimize performance by disabling decoding (see Disabling Protocol Decoding).
top |
Registering kprobe Devices |
Each kprobe instance that will be sending flow records to Kentik must be registered as a device with Kentik. Device registration may be handled in either of the following ways:
Registering a host in Kentik involves specifying the fields that are described in the KB topic Device Admin Dialogs. The following information about specific fields will help ensure correct registration of a kprobe host:
Once a kprobe host is registered as a device it will be represented as a row in the Device List on the Devices page (Admin » Devices; see Device List):
top |
kprobe Download and Install |
To use kprobe, you’ll download and install the executable on each host that you want to monitor:
Note: kprobe must run as root.
top |
kprobe Configuration |
kprobe is configured with command line settings that are covered in the following topics:
top | section |
kprobe Command Line |
The following command line parameters are used for the standard kprobe setup:
The following example shows the structure of a typical command line using the arguments described above (with placeholder values highlighted):
# /usr/local/bin/kprobe --email user@domain.suffix --token user_api_token --interface eth0 --device-id ##### --device-plan ##### --sample ####
Notes:
- The above example would result in protocol decoding (e.g. DNS/WWW data), which could impact kprobe traffic capacity (see kprobe Traffic Capacity). To disable protocol decoding, use the --no-decode flag (see kprobe Optional Features).
- To send encrypted flow from kprobe to Kentik via Kentik’s kproxy (NetFlow proxy agent), use the optional --proxy-url parameter.
top | section |
kprobe Optional Features |
The following command line parameters and flags are used to enable optional features and behaviors:
HTTP Status Server |
The following additional command line options are used to start a simple HTTP status server:
The server started with the above parameters will be accessible via a GET to http://host:port/v1/status. The call will return some basic flow statistics in JSON, as shown in the following example:
{
"flows-in": {
"count": 13889,
"1m.rate": 76.20784200079075,
"5m.rate": 40.91552674366451
},
"flows-out": {
"count": 13889,
"1m.rate": 76.20784200079075,
"5m.rate": 40.91552674366451
}
}
Print-related Configuration |
The following additional command line flags are used to print kprobe-related information:
Debug-only Parameters |
Consult with Kentik support (support@kentik.com) before using these command line parameters for debugging:
Note: The above parameters and flags should not be changed in normal use.
top | section |
Setting Sampling Rate |
General considerations related to setting the flow sampling rate for Kentik devices are covered in Flow Sampling. The following additional examples may help you optimize the sampling rate when using kprobe.
For a host handling 10-20 Gbps:
For a host handling only a few hundred Mb/s:
Note: The maximum FPS available for any given device depends on the Plan (see About Plans) to which that device belongs. If the maximum FPS is exceeded, Kentik will downsample.
top | section |
Disabling Protocol Decoding |
Collection of DNS/WWW data (see Host Traffic Dimensions) is enabled by default. When monitoring host interfaces where collection of DNS/WWW data is not required, kprobe’s traffic capacity can be optimized by disabling protocol decoding. To disable decoding, add the following optional flag to the kprobe command line.
--no-decode
top |
Host Flow Via Proxy |
In situations where it’s not possible for kprobe to communicate with Kentik directly via the Internet, kprobe can be used in conjunction with an HTTP proxy such as kproxy, Kentik’s NetFlow proxy agent. The proxy agent will enable Kentik customers to route flow data from multiple hosts to Kentik via a single point of contact rather than directly from each individual host. To do so:
Configure kprobe for kproxy |
To use kprobe with kproxy, add a --proxy-url parameter when configuring kprobe (see kprobe Command Line) and set the value to the IP address on which you want kproxy to listen, as shown in the following example (placeholder in italics):
--proxy-url http://#.#.#.#:2020
Configure kproxy for kprobe |
The command line arguments used when configuring kproxy for use with kprobe are described in the following list.
The following example shows the structure of a typical command line using the arguments described above (with placeholder values highlighted):
kproxy -api_email=api_email -api_token=api_token -proxy-http=0.0.0.0:2020
Alternatively, if you prefer to keep the API token invisible, you can hide it in an ENV variable:
KENTIK_API_TOKEN=api_token kproxy -api_email=api_email -proxy-http=0.0.0.0:2020
Notes:
- If kproxy fails to launch, add the -verbose flag and try again so that you can provide the output to support@kentik.com in order to facilitate troubleshooting.
- Use -h to return a list of arguments.
In this article: