Kappa Host Agent

Kentik's kappa software agent is covered in the following topics:

• About Kappa
• Kappa Dimensions & Metrics
• Kappa Kubernetes Deployment
• Kappa Host Deployment
• Kappa Cloud Deployment

Note: Kappa is a rapidly-evolving proof of concept software agent. We encourage your feedback. Please direct comments or questions to kubernetes@kentik.com.

General information about Kentik's kappa agent is covered in the following topics:

• Kappa Overview
• Kappa Advantages

Kappa is a host-based telemetry agent providing ultra-efficient observability across production settings including both on-premises data centers and cloud infrastructure. Kappa enhances your organization's ability to:

• Understand traffic flows from all vantage points, including containers and Kubernetes clusters.
• Find congestion and performance hotspots.
• Identify application dependencies.
• Perform network forensics.

Kappa generates kernel flow data using eBPF, which allows Kentik to see the total traffic passing between any source and destination IP, port, and protocol across every conversation taking place within a host, cluster, or data center. Because this information is generated using the Linux kernel, kappa also reports performance characteristics, such as session latency and TCP retransmit statistics.

Kappa also enriches these flow summaries with the application context. Kappa associates conversations with the process name, PID, and the command-line syntax used to launch the IP conversation. If the process is running inside of a container, the container name and ID are also associated. And if the container was scheduled by Kubernetes, kappa enriches the flow record with the name, namespace, workload, and node identifiers of the Kubernetes pod.

Before exporting these records to Kentik, kappa looks for any records associated with other nodes in an environment and joins the source and destination traffic records together with the source and destination host, container, and process metadata, painting the complete picture of application communications within a data center.

Note: Kappa does not “de-duplicate” records. If kappa produces a record for a given flow once at a source node and again at the destination node, the system will preserve and enrich both records. For an accurate volume count or data-rate metric when querying, include a filter whose dimension limits results to a single vantage point (source or destination).

Kappa's design and implementation enable significant advantages over existing approaches to host-based telemetry:

• Easy, flexible deployment: In contrast to application performance monitoring, kappa is flow-based instrumentation that runs with no changes to application code. That makes kappa easy to deploy, and enables you to start monitoring 100% of your on-premises and cloud infrastructure within minutes.
• Versatile single agent: Existing products designed to solve similar problems are typically narrow in scope. But kappa seamlessly covers — in a single agent — the full range of situations where you need to monitor traffic. Deploy kappa as a process or container onto bare-metal hosts, or directly into a Kubernetes cluster to monitor containerized cloud-native workloads.
• Low resource utilization: Kappa uses virtually no resources because it's built on eBPF, which allows sandboxed programs to run within the operating system kernel. Unlike other agent-based solutions, kappa can scale to persistent traffic throughput of 10 Gbps while consuming only a single core.
• Built on the Kentik platform: While open source tools typically rely upon difficult-to-scale backends, kappa sends its data to Kentik Data Engine (KDE), the high-availability cluster that powers the Kentik SaaS. At KDE ingest the kappa flow records are further enriched with data including Threat, Geolocation, and Internet context.
• Advanced analytics included: Kentik's portal and APIs offer a rich set of visualizations that help you understand your network's utilization, performance, vulnerabilities, and costs. Our proactive baselining also yields actionable insights and rapidly detects unusual or unexpected patterns in your traffic.

Kappa generates Metrics & Dimensions whose values are stored in the flow records of the Kentik Data Engine (KDE) and used for queries in the portal and in APIs, either as group-by and filtering parameters (for dimensions) or as the unit by which results are calculated, expressed, and ranked (for metrics).

Kappa supports dimensions (see About Dimensions) that can be used in Kentik APIs or the Kentik portal for group-by and filtering. Listed in the table below, these dimensions are found in the portal under the category "Process-Aware Telemetry Agent" (see Dimensions in the Portal).

Kappa supports the following metrics (see About Metrics), which can be used in Kentik APIs or the Kentik portal (see Metrics in the Portal):

• bits/s: Bits per second.
• packet/s: Packets per second.
• Application Latency: One-way network latency (in milliseconds) derived by examining request/response pairs at the application layer.

Deployment of kappa for Kubernetes is covered in the following topics:

• Kappa Kubernetes Environments
• Clone Kappa to Workstation
• Modify Kappa Variables
• Deploy Kappa to Cluster

You can deploy Kappa in the Kubernetes environments shown in the table below.

Kappa is distributed as a Kubernetes daemon set/config map. Kentik maintains a GitHub repository for kappa at https://github.com/kentik/kappa-cfg. The following command line clones the kappa configuration to your workstation:

$ git clone https://github.com/kentik/kappa-cfg.git

The command line above should result in the following response:

Cloning into 'kappa-cfg'...
remote: Enumerating objects: 12, done.
remote: Counting objects: 100% (12/12), done.
remote: Compressing objects: 100% (10/10), done.
remote: Total 12 (delta 1), reused 12 (delta 1), pack-reused 0
Unpacking objects: 100% (12/12), done.

Once kappa is cloned to your workstation you can open the included kustomize.yml file and modify the following variables to suit your environment:

• email: Set to your Kentik-registered email address (your user level must be Administrator or higher).
• token: Set to your API token from the authentication tab of your Profile in the Kentik portal (from user menu at right of main navbar).
• capture: Specify interface strings/substrings where Kubernetes traffic transits nodes or pods according to your deployed networking model. For example, Calico deployments will want to match on all tun* and cali* interfaces.
• plan: Specify the ID of the Kentik plan within which kappa will create a device. The ID of a given plan is shown to the right of the plan's name in the list of plans on the portal's Licenses page.
  Note: If left unspecified, you must create the device manually in the portal prior to deploying kappa (see Add a Device).
• device: Specify the name that kappa should assign to the device that it will create within the specified plan. Kappa will auto-create one device per deployment.
• bytecode: Set to the path to a file containing the bytecode for your Linux kernel (see Kappa Bytecode).
• newTag: To use Google Kubernetes Engine, set to the value given in Options for GKE.

Kentik uses bytecode generated from kernel header files to retrieve the metadata that is used to enrich the flow records produced by kappa. Bytecode for the following kernel versions may be found at our kappa-cfg GitHub repo:

• Generic Linux systems running Linux Kernel 4.16
• Debian Buster running Linux Kernel 4.19
• Amazon Linux running Linux Kernel 5.4 and 4.14
• Ubuntu 18.04 for AKS nodes running Linux Kernel 5.4
• Container Optimized Linux running Linux Kernel 5.4 and 4.14

To deploy kappa on a system whose kernel is not yet included in the kappa-cfg repo, please email kubernetes@kentik.com and tell us your distribution name and kernel version. We will build a custom bytecode file for you, supply you with a compiled header file, and add it to the kappa-cfg repo.

Kentik currently supports Google Kubernetes Engine using a specialized build that does not include all of the features of the standard Debian/Centos builds. To use this build, set the newTag variable in kustomization.yml to 6fae1a7.

Once the configuration file has been customized for your specific situation, kappa may be deployed as follows:

1. Create a Kentik namespace on your cluster:
   $ kubectl create namespace kentik
2. Switch directories to where you downloaded the kappa-cfg deployment
   $ cd /home/user/git/kappa-cfg
3. Run the apply -k command to apply the configuration (the -k flag uses the Kubernetes kustomization feature):
   $ kubectl apply -k.
   The response should be:
   serviceaccount/kubetags created
   clusterrole.rbac.authorization.k8s.io/kubetags created
   clusterrolebinding.rbac.authorization.k8s.io/kubetags created
   configmap/kappa-bytecode-82cg99776h created
   configmap/kappa-config-h7h9f79t28 created
   configmap/kappa-init-t888g4h9hk created
   secret/kentik-api-secrets-fthc6fd5hm created
   service/kappa-agg created
   deployment.apps/kappa-agg created
   deployment.apps/kubetags created
   daemonset.apps/kappa-agent created
4. Ensure that the kappa pods are running and healthy:
   $ kubectl get pods -n kentik
   The response should be:
   NAME READY STATUS RESTARTS AGE
   kappa-agent-5xzfc 1/1 Running 0 4m17s
   kappa-agent-gl5vv 1/1 Running 0 4m17s
   kappa-agent-sg478 1/1 Running 0 4m17s
   kappa-agg-85c64ff885-9njjl 1/1 Running 0 4m17s
   kubetags-747b9d99fd-m9hcl 1/1 Running 0 4m17s

Coming soon.

Coming soon.