Cloud Overview

A high-level introduction to Kentik Cloud is provided in the following sections:

The Kentik Cloud page is a dashboard providing an overview of traffic involving your cloud resources.
 

Public Clouds and Flow Logs

In Kentik, a "public cloud" is a cloud provider from which we support the export of flow logs about resources in the cloud. We currently support flow logs from AWS, Azure, GCP, and IBM Cloud.

Depending on the provider, a log-generating resource may be a Virtual Private Cloud (VPC), a subnet, a VM, or an interface. A flow log consists of a set of records about the flows that either originated or ended in a given cloud resource. Each individual flow record is made up of a set of fields giving information about a single flow, and is analogous to a flow record generated by a device on a physical network (e.g. NetFlow, sFlow, etc.; see About Flow).

 

Cloud Exports and Devices

Cloud exports and cloud devices for the various cloud providers whose flow logs are supported by Kentik are explained in the following topics:

 
top  |  section

About Exports and Devices

Flow logs are exported to Kentik via a "cloud export" — sometimes referred to as simply a "cloud" — which is an object whose properties are the values that Kentik needs to access logs from a given set of resources in a given cloud provider. Kentik supports multiple "cloud exports" per cloud provider. Each such export is represented as a row in the Cloud Exports list on the Public Clouds Page (Settings » Public Clouds) and has a corresponding Cloud Details Page.

Cloud exports are listed on the Public Clouds page.

The Devices column of the Cloud Exports list will show one or more "cloud devices" that Kentik has derived from each flow export, and these devices will also be listed in the Devices list on the cloud export's Cloud Details page. The name shown in the Devices list for each cloud device can be used to reference the device in Kentik queries using the filtering and group-by dimension "Device Name."

Cloud devices are listed on the details page for a cloud export.

It's important to note that a cloud device in Kentik does not necessarily correspond to an individual resource — subnet, VPC, or interface — hosted by your cloud provider. The way that Kentik derives cloud devices for a given cloud export depends on a number of factors, primarily how log collection is set up in the cloud provider but also the volume of flow records. The topics below explain the default approach used for each cloud provider. If the default approach results in inefficient allocation of flow to cloud devices, Kentik Customer Support will contact your organization to propose alternative allocation strategies that we can use.

 
top  |  section

Exports and Devices in AWS

Each cloud export in AWS represents one S3 bucket, which is specified on the Monitor your AWS Cloud page (see AWS Cloud Setup). The bucket may optionally include multiple "destination log folders," enabling you to configure different resources to send logs to different folders within a single bucket. By default:

  • If the bucket contains multiple folders then Kentik's cloud export from that bucket will have multiple cloud devices, with each device representing the resources that send logs to one folder.
  • If the bucket doesn't contain multiple folders then Kentik's cloud export from that bucket will have a single cloud device representing all resources that send logs to the bucket.
 
top  |  section

Exports and Devices in Azure

Flow logs in Azure are generated by NSGs (network security groups, which represent resources like VMs) and collected in storage accounts that include logs from all NSGs that have been assigned to one uniquely named resource group in one Azure location (e.g. UK South, UK East, etc.). Each cloud export in Kentik (see Azure Cloud Setup) represents one such storage account. By default, each cloud export will have a single cloud device.

 
top  |  section

Exports and Devices in GCP

In GCP, you configure your subnets to publish logs to a Pub/Sub topic, and each Kentik cloud export represents a Subscription to one such topic (see GCP Cloud Setup). By default we identify from the logs themselves which subnets are publishing to the topic corresponding to a given cloud export and we represent each of those subnets as one cloud device.

 
top  |  section

Exports and Devices in IBM

In IBM Cloud, a flow log collector may be configured for a network interface, a VM instance, a subnet, or a VPC. One or more collectors may write log files to a COS bucket. A cloud export in Kentik (see IBM Logging Setup Overview) represents the logs in a single COS bucket, which are forwarded to Kentik by an instance of the Blueflow agent. By default, Kentik identifies from the logs themselves the VPCs in the export and represents each VPC as a cloud device.

 

Cloud Provider Setup

Cloud exports may be added to Kentik in the following situations:

Before you add a cloud you need to prepare the cloud resource for logging, which you'll do in the cloud provider UI (console) rather than in Kentik. Preparing a resource for logging typically involves the following:

  • Enable the creation of flow logs in a VPC.
  • Create a destination to which the logs are published.
  • Set permissions enabling Kentik to access the logs so they can be pulled into Kentik.

The setup workflow varies depending on the cloud provider:

  • To set up flow log ingest from a VPC in AWS, see Kentik for AWS to configure AWS logging manually, or AWS Automated Setup to use Terraform to help automate the process.
  • To set up flow log ingest from a VPC in Google Cloud, see Kentik for GCP.
  • To set up flow log ingest from a Network Security Group in Microsoft Azure, see Kentik for Azure.
  • To set up flow log ingest from a COS service instance in IBM Cloud, see Kentik for IBM Cloud.

Once you've completed the required setup tasks, you'll use the Public Clouds page to manage your organization's cloud exports and to view information about them.

 

Portal Cloud Section

The Cloud section of the Kentik v4 portal provides both high-level and detailed information about your cloud-hosted resources in areas such as traffic volume, performance, costs, connectivity, and utilization. The information is presented in various forms depending on the cloud provider:

  • Kentik Cloud page: Currently visible only in organizations with at least one AWS cloud registered in Kentik, the Kentik Cloud page provides an additional set of AWS-specific visualizations related to your cloud traffic and infrastructure.
  • Cloud dashboards: The AWS Traffic Overview, GCP Traffic Overview, and Azure Traffic Overview pages are Kentik-provided Dashboards for your Kentik-monitored resources in AWS, GCP, and Azure; see Cloud Traffic Overviews.
  • Cloud Performance page: Currently visible only in organizations with at least one AWS cloud registered in Kentik, the Cloud Performance page uses AWS metadata to identify the elements of your AWS resources and give you a clear view of the critical paths to and from the AWS cloud and your organization, and enables you to monitor the performance of those links with synthetic testing (see Synthetics Overview).
  • Connectivity Checker: The Connectivity Checker workflow (Beta) analyzes cloud metadata collected from Kentik-monitored cloud environments and inventories subnets, instances, and VPCs to determine how they communicate (see Connectivity Checker).
    Note: Currently for AWS only.
  • Kentik Kube: Designed for Kentik customers who deploy Kubernetes clusters in the cloud and/or on premises, the Kentik Kube module uses a lightweight eBPF agent to provide detailed network traffic and performance visibility both inside and among Kubernetes clusters.
The cloud dashboards each provide an array of flow-based visualizations for a given cloud provider.
© 2014- Kentik
In this article:
×