Resolution Overview

Kentik generates two independent dataseries at ingest into the Kentik Data Engine (KDE), allowing for efficient querying over extended periods without losing detail for shorter timespans.

The following topics explain how these two dataseries are used by the system:

• About Dataseries Resolution
• Query Resolution Selection

Kentik builds both the Full and Fast dataseries at the time of data ingest, maintaining them as independent entities:

• Full dataseries: Captures every flow record sent by a customer, adhering to the service agreement limits.
• Fast dataseries: Contains a subset of flow records for quicker responses for querings over 24 hours. To optimize storage and speed:
  - Ports above 32767 (i.e., ephemeral ports) are grouped and represented as port 65535.
  - Flows sharing the same 7-tuple are aggregated.
  - Duplicate sequential flow records within minutes are combined.
  - Data is downsampled to 30 flows per second (fps) or 1800 aggregated flows per minute (fpm) per device.
  Note: For different downsampling options, contact Customer Care.

By default, the choice of dataseries for a query in Kentik is based on the query’s timespan:

• Full Dataseries: Used for timespans of less than 24 hours.
• Fast Dataseries: Used for timespans of 24 hours or more.

Note: The 24-hour criterion is based solely on the duration of the query's timespan, not how far back in time the query is looking. Thus, a query with a duration of less than 24 hours will default to the Full dataseries even if data is from weeks ago (unless manually overridden, see Overriding the Default Dataseries).

The dataseries resolution determines the interval (granularity) used for reporting packets and bytes:

• Fast Dataseries: Query intervals "snap" to the nearest hour. For example, a query from 1:05 PM on 5/12 to 10:31 PM on 5/15, returned results will cover 1:00 PM on 5/12 to 11:00 PM on 5/15.
• Full Dataseries: Query intervals vary depending on query width.

In some circumstances (see Query Resolution Selection), the default choice of dataseries may be manually overridden:

• Fast Dataseries: Can be manually selected for timespans as short as 3 hours for quicker responses.
• Full Dataseries: Can be manually selected for timespans of up to 72 hours for detailed results.

The default dataseries selection may be overridden in both the portal and the API. Manual selection of the dataseries for a given query is covered in the following topics:

• Portal Resolution Selection
• API Resolution Selection

In the Kentik portal, the dataseries selection is based on the timespan set in Data Explorer and Dashboards:

• Dashboards:
  - Automatically selects Full for timespans less than 24 hours (see About Dataseries Resolution).
  - Selects Fast for timespans of 24 hours or more.
• Data Explorer:
  - Three hours or less: Always uses Full dataseries.
  - More than three hours but less than 24 hours: Defaults to Full, but can be manually switched to Fast.
  - 24 to 72 hours: Defaults to Fast, but can be manually switched to Full.
  - More than 72 hours: Always uses Fast dataseries.

The Kentik API uses the same dataseries defaults as the portal, with the option to override using the i_fast_dataset query parameter:

• Without i_fast_dataset:
  - Automatically selects Full for timespans less than 24 hours (see About Dataseries Resolution).
  - Selects Fast for timespans of 24 hours or more.
• With i_fast_dataset:
  - false: Uses Full dataseries for timespans up to 72 hours.
  - true: Uses Fast dataseries regardless of timespan.

Note: Avoid using the Fast dataseries for timespans under 3 hours and the Full dataseries for timespans over 72 hours.