Public Clouds

The setup of cloud exports and cloud devices in the Kentik portal is discussed in the following topics:

• Public Clouds Page
• Cloud Details Page
• Configuration Status Page
• Cloud Export Settings
• Manage Cloud Exports

Notes:
- For a high-level explanation of public clouds, see Cloud Overview.
- For an explanation of cloud exports and cloud devices, see Cloud Exports and Devices.
- For information about adding a cloud export, see Cloud Setup.
- If you would like assistance with any aspect of monitoring your cloud resources with Kentik, please contact Customer Support.

The Public Clouds page of the Kentik portal (Settings » Public Clouds) shows the cloud exports from all of your organization's public cloud resources that are currently configured to export flow logs to Kentik. The Public Clouds page is covered in the following topics:

• Public Clouds Page UI
• Cloud Exports List

The Public Clouds page includes the following UI elements:

• Add Cloud: The cloud providers currently supported by Kentik are each represented by a button to the right of the page title. Click a button to configure a new cloud export (see Cloud Setup) for one of the following providers:
  - Amazon Web Services
  - Google Cloud Platform
  - Microsoft Azure
  - Oracle Cloud Infrastructure
  
• Flows per second: A horizontal bar chart showing the volume of flow logs per second ingested by Kentik from each of your cloud providers (no bar will be shown for a provider with no cloud exports). Each provider's bar is segmented by color to show the portion of FPS that comes from each cloud export from that provider.
  
• Cloud Config Status: The Cloud Config Status pane appears to the right of the Flows per Second chart. The pane is a list showing the latest heath status (e.g. warning, critical, etc.) of the resources hosted with your different cloud providers. For each provider:
  - Click the expand icon to view the status or your resources in the provider's regions.
  - Click the View Details link to go to the Configuration Status Page for that provider.
• Show/Hide Filters (filter icon): A button that toggles the Filters pane between expanded and collapsed.
• Search: A field that you can use to filter the Cloud Exports List. When text is entered, the list will show only rows containing that text in one of the following columns: Provider, Name, Properties, or Devices.
  Note: Filters added with the Filters pane will also appear in this field. You can remove a filter by clicking the X in its tag.
• Cloud Exports: A table listing your organization’s cloud exports (see Cloud Exports List).
• Filters pane: A set of checkboxes to the left of the Cloud Exports list that enables you to filter the list to specific providers. The pane is either expanded or collapsed by the Show/Hide Filters button.

The Cloud Exports list is a table that shows information about your organization's cloud exports (see Cloud Exports and Devices). Click on a column heading to sort the list.

The Cloud Exports list includes the following columns:

• Provider: The cloud service provider (e.g. AWS, Azure, etc.) from which the flow logs for this cloud export are being pulled by Kentik.
• Status: The current status of the cloud export (see Cloud Export Status Icons).
• Name: The name specified for the cloud export when it was added or edited. Click a name to go to the cloud export's Cloud Details Page.
• Properties: A summary of settings made in the dialog with which the Cloud was added or edited. The properties vary by provider:
  - AWS: Properties include Path and Role.
  - GCP: Properties include Project and Subscription.
  - Azure: Properties include Resource Group and Storage Account.
  - IBM: There are no IBM-specific properties.
  - OCI: Properties include Tenancy ID, Compartment ID, User ID, and Default Region.
• Devices: The devices auto-derived by Kentik when the cloud export was created. The relationship between a cloud export and a device varies depending on provider (see Cloud Exports and Devices).
• FPS Sampled: The peak rate of flow records ingested by Kentik from this cloud export's devices over the last 6 hours.
• Edit: A button that opens the configuration dialog for the cloud export (see Cloud Export Settings).
• Refresh: A button that retrieves the latest flow logs for this cloud export.
• Remove: A button that pops up a confirmation dialog that allows you to remove the cloud export from your organization.

The following icons display the status of the cloud export:

• Start (blue circle): Kentik is setting up the cloud export based on information provided in the add or edit dialog.
• Pending (orange clock): Kentik is in the process of setting up the cloud export based on information provided in the add or edit dialog.
• Error (red exclamation): A configuration or connection issue is preventing Kentik from accessing the flow logs for this cloud export.
• OK (green checkmark): Kentik is successfully ingesting flow logs from the source specified with this cloud export.
• Disabled (cloud with strikethrough): Export is disabled in the cloud export's settings.
• Halt (red hand): Kentik is halting the cloud export. Please contact Customer Support to determine possible causes and solutions.

The details page for an individual cloud export, which provides access to the properties and settings of that cloud export, is covered in the following topics:

• Cloud Details Page UI
• Cloud Summary Information
• Cloud Devices List

The Cloud Details page includes the following UI elements:

• Cloud provider (logo): The logo of the provider that hosts this cloud export.
• Cloud name: The name of the cloud export.
• Configure: A button that opens the configuration dialog for this cloud export (see Cloud Export Settings).
• Cloud Summary: Information about the cloud export, including the plan to which it is assigned and other information that varies depending on the provider (see Cloud Summary Information).
• Devices list: A table listing the devices associated with this cloud export (see Cloud Devices List).

As detailed in the topics below, some cloud export summary information is common to all providers, and other information is specific to a given provider.

The following fields are common to the summary information for cloud exports from all providers:

• Export ID: A unique ID assigned to the cloud export by Kentik.
• Billing Plan: The name of the Kentik plan to which the cloud is assigned (see About Plans).
• Plan ID: The ID of the Kentik plan to which the cloud export containing this cloud export is assigned.

The following additional fields are included in the summary information for cloud exports from AWS:

• S3 Bucket Region: The AWS region of the S3 bucket from which Kentik pulls the flow logs for this cloud.
• S3 Bucket Name: The name (not the full ARN) of the S3 bucket.
• IAM Role ARN: The full Amazon resource name of the role that permits services in Kentik’s AWS account to access the needed resources in your AWS account.
• Delete After Read: The current Delete After Read setting for this cloud, which determines whether flow logs will be automatically deleted in AWS after they've been ingested into Kentik.
• Description: The description entered for the cloud when it was created or edited in Kentik.

The following additional fields are included in the summary information for cloud exports from Azure:

• Subscription ID: The Subscription ID of the Azure instance from which Kentik's NSG Flow Exporter application will export flow logs.
• Resource Group: The resource group of the Azure resources for which flow logs are collected for this cloud.
• Location: The location of the Azure resources for which flow logs are collected for this cloud.
• Storage Account: The name of the Azure storage account to which logs from the above-described Azure resources are exported.

The following additional fields are included in the summary information for cloud exports from GCP:

• Project: The name of the GCP project containing the Pub/Sub topic to which flow logs are exported for this cloud export.
• Subscription: The name of the subscription via which Kentik subscribes to your Pub/Sub topic.

The following additional fields are included in the summary information for cloud exports from IBM Cloud:

• Description: The description entered for the cloud when it was created or edited in Kentik.

The Devices list is a table providing information about the individual devices that make up a cloud. Click a column heading to sort by that column. The table includes the following columns:

• Status: The Flow Status of the device.
• Name: The name of the device. The ID of the device is shown in parentheses.
• FPS Sampled: The peak rate of flow records ingested by Kentik from this cloud export's devices over the last 6 hours.

The status of flow for a cloud device is shown as follows:  

• Ok (green checkmark): Kentik is successfully ingesting flow logs from the device.
• Error (orange exclamation): A configuration or connection issue is preventing Kentik from accessing the flow logs for this device.
• Pending (orange clock): Kentik is setting up the device based on information provided in the add or edit dialog for this cloud export.

The Configuration Status pages for cloud providers are covered in the following topics:

• About Cloud Config Status
• Cloud Config Status UI
• Configuration Status List
• Config Status Details

Note: The pages for all supported providers are essentially the same, with differences as noted in the topics below.

The Configuration Status pages help with cloud troubleshooting by revealing issues with the configuration of your organization's cloud exports. The pages — one for each of your cloud providers — display the results of Kentik’s cloud configuration checker, which looks at each AWS account (ARN), Azure account (Resource ID), GCP account (Resource ID), and OCI account (Tenancy ID) for which your organization has a cloud export. The results for each provider are displayed as a set of tables, one for each region (AWS, Azure, and OCI) or project (GCP) in which you have cloud resources.

The configuration checker evaluates whether:

• Kentik is able to access the required API endpoints within the cloud environment.
• Kentik is receiving flow logs.

The Configuration Status page includes the following UI elements:

• Filter: Enter text to filter the page's config status lists to rows in which the Account ID or Export IDs columns contain the entered text.
• Configuration status: A set of tables, one for each region (AWS, Azure, and OCI) or project (GCP) in which the configuration checker is reporting an issue with one or more of your cloud exports (see Configuration Status List).

A cloud status list is a table listing the status of cloud exports in a given region (AWS and Azure) or project (GCP). Click on a row in the table to open the Config Status Details sidebar for the corresponding account, subscription, or project.

The status table for each region includes the following columns:

• Account ID (AWS only): The ID of the AWS account containing the resources.
• Subscription ID (Azure and GCP only): The Subscription ID (Azure) or Project Number (GCP) of the account containing the resources.
• API: A status icon indicating whether Kentik is able to access the API endpoints used to export metadata for this cloud export.
• Flow: A status icon indicating whether Kentik is receiving flow logs for this cloud export.
• Exports: A status icon indicating whether any issues were found on the Kentik side that would prevent correct operation of the cloud export.
• Export IDs: The unique IDs assigned by Kentik to the cloud export(s) in this region or project that are associated with this account ID. Click an ID to go to the Cloud Details Page for the corresponding cloud export.

Note: If a given status icon is not a green checkmark, hover over it to pop up a status description.

The Config Status Details sidebar opens at the right of the Configuration Status list when you click on a row in a table for a given region or project (see Configuration Status List). The sidebar details the status of the operations required for the success of the cloud export corresponding to that row.

The sidebar includes the following elements:

• Owner ID (AWS only): The ID of the AWS account.
• Subscription ID (Azure and GCP only): The ID of the subscription (Azure) or project (GCP) account.
• Tenancy ID (OCI only): The ID of the OCI account.
• Region (AWS, Azure, and OCI only): The cloud provider region of the export.
• GCP Project (GCP only): The name of the GCP project associated with the cloud export.
• Subnets with Sampling (GCP only): The number of subnets with sampling for the GCP cloud export.
• Average Sampling Rate (GCP only): The average sampling rate for the GCP cloud export.
• API Access Status: A pane containing a table listing the endpoints — grouped by API — that Kentik needs for the cloud export. The table has the following columns:
  - Endpoint: An endpoint.
  - Can Access: An icon indicating whether Kentik is currently able to access the endpoint.
• Flow Access Status (AWS only): A pane containing a table indicating status for individual VPCs (see Flow Access Status).
• Cloud Export Status: A pane containing a table listing that lists Kentik cloud exports. The table has the following columns:
  - Status: An icon indicating the status of the export. If the icon is not a green checkmark, hover on it to open a popup with a status description.
  - Export ID: The ID of the export.
  - Name: The name given to the export when it was added or last edited in Kentik. Click the name to go to the Cloud Details Page for this export.

The Flow Access Status table contains the following columns:

• Entity ID: The Entity IDs of the AWS VPCs in the account.
  - If Has Flow is true (green checkmark), the count of the buckets involved will be stated in blue below the ID. Hover over the count to open a popup identifying the bucket(s) in which the logs for the VPC are collected.
  - If Has Flow is false (red), then "Flow logs not configured." will be displayed in orange below the ID.
• Has Flow: An icon indicating whether Kentik is currently receiving logs from a given VPC.
• Open in Data Explorer: Click to open Data Explorer in a new tab, with the Filtering pane (in the Query sidebar) set to filter results to the individual VPC.

Editing a cloud export via the Kentik portal involves specifying information in the fields of the Cloud Export Settings dialog, which is covered in the following topics.

• About Cloud Export Settings
• Cloud Export Settings UI
• Common Cloud Settings
• AWS Provider Settings
• Azure Provider Settings
• GCP Provider Settings
• IBM Cloud Provider Settings
• OCI Provider Settings

Note: The Cloud Export Settings dialog enables you to modify settings for an existing cloud export. The settings for a new cloud export are covered in Cloud Setup.

The Cloud Export Settings dialog enables you to modify the information that Kentik uses for a cloud export. The dialog is accessed from the following portal locations:

• Public Clouds page: Click the edit icon in the row for a given cloud export in the Cloud Exports List.
• Cloud Details page: Click the Configure button at the upper right (see Cloud Details Page UI).

The fields of the Cloud Export Settings dialog vary depending on the provider. As covered in the topics below, some fields are common to all providers (see Common Cloud Settings) while other fields are specific to a given provider.

The Cloud Export Settings dialog for all providers share the same general layout and the following common UI elements:

• Close button: Click the X in the upper right corner to close the dialog. All elements will be restored to their values at the time the dialog was opened.
• Settings fields: The common and provider-specific fields containing configuration settings for the cloud export (see topics below).
• Cancel button: Cancel the add Cloud or edit Cloud operation and exit the dialog. All elements will be restored to their values at the time the dialog was opened.
• Save button: Save changes to Cloud settings and exit the dialog.

The Cloud Export Settings dialogs for all providers contain the following settings fields:

• Name (required): A user-supplied name string for the cloud export.
• Description: A user-supplied description string.
• Billing Plan: A drop-down from which you can choose the Kentik plan to which the cloud export will be assigned (see About Plans).
• Enabled: A switch that turns on/off the ingest of flow logs for this cloud export.
  Note: Disabling a cloud has no effect on the cloud provider side. Publication and collection of flow logs will continue until discontinued on the cloud provider.

Kentik pulls flow logs for a given cloud export from an AWS VPC, subnet, or interface via an Amazon S3 bucket (for more details, see AWS Logging Setup Overview). To establish the connection that enables us to ingest those logs into Kentik we need information that you provide in the following fields:

• S3 Bucket Region: A drop-down list from which you choose the region in which you created the S3 bucket from which Kentik will pull the flow logs (see Create an S3 Bucket).
• Collect logs from alternative S3 bucket(s): A switch that enables the cloud to represent logs from multiple buckets that are not in the same region. If the switch is turned on, the S3 Bucket Name field will be inactivated.
• S3 Bucket Name: The name (not the full ARN) of the S3 bucket.
• IAM Role ARN: The full ARN (Amazon resource name) of the role that you created to establish a “trusted relationship” that permits services in Kentik’s AWS account to access the needed resources in your AWS account.
• Delete After Read: Determines whether or not the logs for this cloud export should be deleted after they've been ingested into Kentik. Turn off if you prefer to manage log deletion on your own.
• Sampling: A set of controls that configure the sampling rate for this cloud export (see Cloud Export Sampling).

The following settings determine the sampling rate for this cloud export:

• Sampling type:
  - Legacy (AWS only): A maximum of 10k flows are randomly sampled per-file.
  - Sampling Rate: Use the rate specified in the Sampling Rate field.
  - Unsampled: No sampling is enabled.
• Sampling rate (present only when Sampling Type is Sampling Rate): Enter the sampling rate in the form of 1:N. Value must be between 2 and 2000.

Kentik pulls flow logs for a given cloud export from an Azure "storage account" that represents all resources within a given Azure subscription that share a location and have been assigned to the same resource group. The storage account is accessed by NSG Flow Exporter (a Kentik-built enterprise application for Azure), which forwards the flow logs to KDE. The following settings and controls enable creation of the storage account and authorize access to it by NSG Flow Exporter:

• Subscription ID: Enter the Subscription ID of the Azure instance from which Kentik's NSG Flow Exporter application will export flow logs (see Authorize Access to Azure).
• Authorize: Click to authorize the Azure portal to create a Service Principal representing NSG Flow Exporter.
  Note: Your Azure role (e.g. Global Administrator) must allow you to grant access by enterprise applications.
• Resource Group to Monitor : Enter the resource group of the Azure resources for which you want to generate flow logs for this cloud (see Specify Azure Resources).
• Location: Enter the location of the Azure resources for which you want to generate flow logs for this cloud.
• Storage Account Name: Enter a name for the Azure storage account to which logs will be exported from the above-specified Azure resources..
  Note: The name must not be already in use by any other storage account, whether in your subscription or that of another Azure user.
• Sampling: A set of controls that configure the sampling rate for this cloud export (see Cloud Export Sampling).
• Configure Manually: Click the button to open a dialog containing a set of manual configuration steps, then go to the Azure portal and follow the instructions (see Choose Configuration Method).
• Configure Using PowerShell: Click the button to open the Logging Configuration Script dialog, which includes a Kentik-generated script (see Generate PowerShell Script).
• Validate configuration: Click the button to begin validation of your flow log export configuration (see Validate Azure Setup).
  Note: Validation may take up to an hour, during which time the cloud's status (e.g. on the Public Clouds Page) will be indicated as "Pending" until Kentik completes registration.

Kentik pulls flow logs for a given cloud export from the Google Cloud Platform (GCP) by subscribing to a Pub/Sub topic that is publishing flow logs from one or more subnets/VPCs (for more details, see GCP Process Overview). To establish the connection that enables us to ingest those logs into Kentik we need information that you provide in the following fields:

• Project: All GCP resources (e.g. compute engine services or cloud storage buckets) are contained within a GCP project. Enter the name of the GCP project that contains the Cloud Pub/Sub topic that you created as a destination for the export of flow logs from your VPC (see Create a New Topic).
• Subscription: Enter the name of the subscription that you created to enable Kentik to subscribe to your Pub/Sub topic (see Create a Pull Subscription).

The settings for IBM cloud exports currently include only Common Cloud Settings.

Kentik pulls flow logs for a given cloud export from Oracle Cloud Infrastructure (OCI) using the OCI object storage service. This service groups all resources within a given OCI tenancy that share a region and have been assigned to the same compartment. The flow logs for these resources are collected through a service connector and stored in an OCI bucket.

In Kentik, an OCI cloud export is configured on the Monitor your OCI Cloud page, which opens from the Add OCI Cloud button on the Public Clouds Page. The settings and controls of the Monitor your OCI Cloud page are organized into the panes covered in the topics below.

Note: The settings in the Name this Cloud Export pane are covered in Common Cloud Settings.

The following settings and controls enable the setup of an OCI cloud export:

• Tenancy ID: A field for the unique identifier (OCID) of the OCI tenancy from which flow logs will be collected.
• Compartment ID: A field for the OCID of the compartment containing the resources from which flow logs will be collected.
• User ID: A field for the OCID of the user whose credentials will be used to access OCI resources.
• OCI Default Region: A dropdown with which to select the region of the OCI resources.
• Verify: A button that you can click to verify authentication for the flow log export configuration. A notification lozenge to the left of the button will display the result:
  - Valid: The entered credentials (OCIDs) have been successfully verified.
  - Not Authenticated: The credentials can't be validated.
• Flow Log Collection: A switch (off by default) that enables/disables the collection of flow logs, and shows the fields and controls covered in Flow Log Collection Controls.

Note: In OCI, the term "OCID" refers to any ID assigned to a resource, whether that resource is a tenancy, a compartment, or a user.

The following fields and controls are shown only when the Flow Log Collection switch is on (collection is enabled):

• Bucket Name: A field for the name of the OCI bucket where flow logs will be stored.
• Bucket Namespace: A field for the namespace of the OCI object storage where the bucket resides.
• Service Connector OCID: A field for the OCID of the service connector that will be used to route flow logs to the bucket.
• Flow Object Name Prefix: A field for a prefix that can be added to the names of flow log objects.
• Verify Flow Logs Bucket Access: A button to initiate a verification process to check if Kentik has the required access to the flow logs bucket. A notification lozenge to the left of the button will display the result.

Clouds are added and edited via the Cloud Export Settings. The add/edit process is covered in the following sections:

• Add a Cloud Export
• Edit a Cloud Export

To add a new Cloud to Kentik:

1. Choose Settings from the main Kentik menu.
2. On the Settings page, click Public Clouds.
3. On the Public Clouds page, click the Add Cloud button corresponding to the provider (AWS, Azure, etc.) containing the cloud resources that you'd like to monitor in Kentik.
4. The resulting page will correspond to the type of cloud you chose to add (e.g. the Monitor your AWS Cloud page if you clicked the Add AWS Cloud button). Follow the steps on the page to fill in the form (see Cloud Setup).

To edit the settings for an existing Cloud:

1. Choose Settings from the main Kentik menu.
2. On the Settings page, click Public Clouds.
3. In the Cloud Exports list on the Public Clouds page, click the Edit icon in the row corresponding to the cloud that you'd like to edit.
4. In the resulting dialog, change any fields that you'd like to modify:
   - For fields that are common to cloud exports from all cloud providers, see Common Cloud Settings.
   - For provider-specific fields, find your provider in the topics listed under Cloud Export Settings.
5. To save changes, click the Save button (lower right).

Note: To remove a cloud export from your organization's collection of cloud exports, click the Remove button (trash icon) at the right of the cloud's row of the Cloud Exports list.