Cloud Overview

A high-level introduction to Kentik Cloud is provided in the following sections:

• Public Clouds and Flow Logs
• Cloud Exports and Devices
• Cloud Provider Setup
• Portal Cloud Section

In Kentik, a "public cloud" is a cloud provider from which we support the export of flow logs about resources in the cloud. We currently support flow logs from AWS, Azure, GCP, and OCI.

Depending on the provider, a log-generating resource may be a Virtual Private Cloud (VPC), a subnet, a VM, or an interface. A flow log consists of a set of records about the flows that either originated or ended in a given cloud resource. Each individual flow record is made up of a set of fields giving information about a single flow, and is analogous to a flow record generated by a device on a physical network (e.g. NetFlow, sFlow, etc.; see About Flow).

Cloud exports and cloud devices for the various cloud providers whose flow logs are supported by Kentik are explained in the following topics:

• About Exports and Devices
• Exports and Devices in AWS
• Exports and Devices in Azure
• Exports and Devices in GCP
• Exports and Devices in OCI

Flow logs are exported to Kentik via a "cloud export" — sometimes referred to as simply a "cloud" — which is an object whose properties are the values that Kentik needs to access logs from a given set of resources in a given cloud provider. Kentik supports multiple "cloud exports" per cloud provider. Each such export is represented as a row in the Cloud Exports list on the Public Clouds Page (Settings » Public Clouds) and has a corresponding Cloud Details Page.

The Devices column of the Cloud Exports list will show one or more "cloud devices" that Kentik has derived from each flow export, and these devices will also be listed in the Devices list on the cloud export's Cloud Details page. The name shown in the Devices list for each cloud device can be used to reference the device in Kentik queries using the filtering and group-by dimension "Device Name."

It's important to note that a cloud device in Kentik does not necessarily correspond to an individual resource — subnet, VPC, or interface — hosted by your cloud provider. The way that Kentik derives cloud devices for a given cloud export depends on a number of factors, primarily how log collection is set up in the cloud provider but also the volume of flow records. The topics below explain the default approach used for each cloud provider. If the default approach results in inefficient allocation of flow to cloud devices, Kentik Customer Support will contact your organization to propose alternative allocation strategies that we can use.

Each cloud export in AWS represents one S3 bucket, which is specified on the Monitor your AWS Cloud page (see AWS Cloud Setup). By default, each VPC, subnet, or interface sending logs to the bucket will be represented in Kentik as a "cloud device."

Flow logs in Azure are generated by NSGs (network security groups, which represent resources like VMs) and collected in storage accounts that include logs from all NSGs that have been assigned to one uniquely named resource group in one Azure location (e.g. UK South, UK East, etc.). Each cloud export in Kentik (see Azure Cloud Setup) represents one such storage account. By default, each cloud export will have a single cloud device.

In GCP, you configure your subnets to publish logs to a Pub/Sub topic, and each Kentik cloud export represents a Subscription to one such topic (see GCP Cloud Setup). By default we identify from the logs themselves which subnets are publishing to the topic corresponding to a given cloud export and we represent each of those subnets as one cloud device.

As detailed in the Oracle article VCN Flow Logs, logging may be enabled in OCI for an entire VCN — meaning on all existing and future Virtual Network Interface Cards (VNICs) in all of the VCN's subnets — or selectively on specific subnets, VNICs, or resources (e.g. instances or load balancers). The logs are sent to an OCI Object Storage bucket, which aggregates logs from a specific compartment or across multiple compartments. Kentik accesses the flow logs from this bucket and represents the bucket as a single "cloud export" that is listed in the Cloud Exports List. All resources (VCNs, etc.) that contribute logs to an exported bucket are typically represented in Kentik as a single device that is listed in the Devices list on the export's Cloud Details Page.

Note: In some cases (e.g. high volume of flow records) Kentik may optimize the ingest of flow records by creating multiple cloud devices within a single cloud export.

Cloud exports may be added to Kentik in the following situations:

• At initial Kentik onboarding; see Cloud Setup.
• At any time via the Public Clouds Page (Settings » Public Clouds).

Before you add a cloud you need to prepare the cloud resource for logging, which you'll do in the cloud provider UI (console) rather than in Kentik. Preparing a resource for logging typically involves the following:

• Enable the creation of flow logs in a VPC.
• Create a destination to which the logs are published.
• Set permissions enabling Kentik to access the logs so they can be pulled into Kentik.

The setup workflow varies depending on the cloud provider:

• To set up flow log ingest from a VPC in AWS, see Kentik for AWS to configure AWS logging manually, or AWS Automated Setup to use Terraform to help automate the process.
• To set up flow log ingest from a VPC in Google Cloud, see Kentik for GCP.
• To set up flow log ingest from a Network Security Group in Microsoft Azure, see Kentik for Azure.
• To set up flow log ingest from resources in Oracle Cloud Infrastructure, see Kentik for OCI.

Once you've completed the required setup tasks, you'll use the Public Clouds page to manage your organization's cloud exports and to view information about them.

The Cloud section of the Kentik v4 portal provides both high-level and detailed information about your cloud-hosted resources in areas such as traffic volume, performance, costs, connectivity, and utilization. The information is presented in various forms depending on the cloud provider:

• Kentik Cloud page: Currently visible only in organizations with at least one AWS cloud registered in Kentik, the Kentik Cloud page provides an additional set of AWS-specific visualizations related to your cloud traffic and infrastructure.
• Cloud dashboards: The AWS Traffic Overview, GCP Traffic Overview, and Azure Traffic Overview pages are Kentik-provided Dashboards for your Kentik-monitored resources in AWS, GCP, and Azure; see Cloud Traffic Overviews.
• Cloud Performance page: Currently visible only in organizations with at least one AWS cloud registered in Kentik, the Cloud Performance page uses AWS metadata to identify the elements of your AWS resources and give you a clear view of the critical paths to and from the AWS cloud and your organization, and enables you to monitor the performance of those links with synthetic testing (see Synthetics Overview).
• Connectivity Checker: The Connectivity Checker workflow (Beta) analyzes cloud metadata collected from Kentik-monitored cloud environments and inventories subnets, instances, and VPCs to determine how they communicate (see Connectivity Checker).
  Note: Currently for AWS only.
• Kentik Kube: Designed for Kentik customers who deploy Kubernetes clusters in the cloud and/or on premises, the Kentik Kube module uses a lightweight eBPF agent to provide detailed network traffic and performance visibility both inside and among Kubernetes clusters.