CH logo® Knowledge Base
Contents Search
   

 

Dimensions and Metrics

The features and use of the Data Explorer in the Kentik Detect portal are discussed in the following topics:

Note: In addition to being used for settings in the Kentik Detect portal, dimensions and metrics are also used for the Query API.
- In addition to the standard dimensions and metrics included with Kentik Detect, users can create custom dimensions (see Custom Dimensions).

 

 
 top

About Dimensions

Dimensions in Kentik Detect represent specific data about flow (see Flow Overview). This data is either taken directly from flow records (e.g. NetFlow, sFlow), incorporated from correlated sources (e.g. GeoIP or threat feeds), or derived by Kentik from some combination of the two. Each dimension represents a column in the main tables of the KDE (see Main Table Schema).

Dimensions are primarily used in the following contexts in the Kentik Detect portal and the Query API:

  • Group-by dimensions: Selected in the Query pane of the sidebar (e.g. in Data Explorer).
  • Filters: Selected in the Ad-Hoc Filter Groups section of the Filtering Options dialog accessed from the Filtering pane of the sidebar.

The following table shows more specifically the various places in the Kentik Detect portal where dimensions are specified and used:

Portal section Group-by Filters
Dashboards - Add Panel or Edit Panel dialog » Edit Query in Data Explorer button » sidebar Query pane.
- See Panel Dialogs.
- Sidebar Filtering pane » Filtering Options dialog » Ad-Hoc Filter Groups pane.
- See Ad-Hoc Filter Interface.
Data Explorer - Sidebar Query pane » Group-by Dimensions dialog.
- See Query Dimension Selectors.
- Sidebar Filtering pane » Filtering Options dialog » Ad-Hoc Filter Groups pane
- See Ad-Hoc Filter Interface.
Alerting » Policies Add Policy or Edit Policy dialog » Dataset tab » Data Funneling pane » Dimensions. Add Policy or Edit Policy dialog » Dataset tab » Data Funneling pane » Filters.
Analytics » Peering, Add Dataset N.A. - Add Dataset dialog » Filters pane.
- See Filtering Pane Settings.
Analytics » Peering, Analysis View N.A. - Sidebar » Filters pane.
- See Filtering Pane Settings.
Analytics » Route Traffic N.A. - Sidebar Filtering pane » Filtering Options dialog » Ad-Hoc Filter Groups pane
- See Ad-Hoc Filter Interface.
Analytics » Capacity Sidebar Options pane » Group-by Dimensions dialog (includes only subset of dimensions). N.A.
Admin » User N.A. - Add User or Edit User dialog » User Specific Filters pane.
- See User Admin Dialogs.
Admin » Saved Filters N.A. - Add Saved Filter or Edit Saved Filter dialog » Ad-Hoc Filter Groups pane.
- See Saved Filter Admin Dialogs.

 

 
 top

Dimension Categories

In general, dimensions are available in dimension and filter selectors for queries involving traffic from all types of devices (both routers and hosts), though some dimensions apply only to traffic from host agents such as kprobe (see About kprobe).

Dimensions, which represent or are derived from columns in KDE (see Main Table Schema) fall into the following functional categories:

Category Description Requires host agent (kprobe)
Time Used to specify time ranges. No
IP IP addresses (Ipv4 or Ipv6) of flow, as well as protocol (e.g. TCP or UDP), TCP flags, and ToS. No
Device & Port Information related to devices including interface names and descriptions, port IDs, etc. No
Geo Country codes, region names, and city names for the flow’s source and destination. No
BGP Routing information including source and destination AS, AS path, AS names, community, prefixes, and hops. No
LAN Source and destination local area network IDs and MAC addresses. No
Tags Matches to user-specified tags (see About Flow Tags). No
Query (KDE) Used to specify dataseries (Full or Fast; see Resolution Overview), and other query-related data. No
DNS/WWW Data related to DNS lookup and HTTP, including domain name, referrer, status, etc.). Yes
Ultimate Exit Information about where the traffic leaves the network (device, interface, site, etc.); see Using Ultimate Exit. No
Interface Classification Information about the interface though which a flow enters and leaves a device; see Interface Classification. No
Provider Classification Information (derived during interface classification) about the provider via which traffic from a given externally facing interface reaches the Internet; see Provider Classification. No
Threat Feed Information about source and destination hosts and IPs that have been identified as a security threat by Spamhaus (updated daily). No

Notes:
- For more detailed information about dimensions requiring a host agent, see Host Traffic Dimensions.
- For lists of the specific dimensions in each of the above categories, see Main Table Schema.
- In addition to the dimension categories listed above, KDE main tables also include some columns that store metrics (e.g. NPM), which are covered in General Metrics and Host Traffic Metrics.

 

 
 top

Host Traffic Dimensions

While most dimensions are available (for both group-by and filters) on traffic from all devices, the following dimensions are available only for traffic from a host agent such as kprobe (see About kprobe):

  • DNS Query: A query from a DNS resolver to a DNS name server that translates a user-friendly domain name (e.g. www.domain_name.com) to a numeric IP address, either 32-bit IPv4 (93.184.216.119) or 128-bit IPv6 (2606:2800:220:6d:26bf:1447:1097:aa7).
  • DNS Query Type: The resource record type requested by the DNS query. For a list of record types, see https://en.wikipedia.org/wiki/List_of_DNS_record_types.
  • DNS Response: The response from a DNS server to a DNS query. DNS responses are comprised of resource records (RRs). kprobe collects information from the following RRs:
    - A: IPv4 address for given host.
    - AAAA: IPv6 address for given host.
    - CNAME: A domain name that must be queried to resolve the original DNS query.
    - PTR: Used to look up a domain name based on an IP address.
    - MX: A mail exchange server for a DNS domain name.
    - NS: An authoritative name server for given host.
    - TXT: A non-formatted text string typically used by Sender Policy Framework (SPF) to prevent the sending of emails using a fake identity.
  • DNS Return Code: Status code returned from a DNS query (see https://www.iana.org/assignments/dns-parameters/dns-parameters.xhtml#dns-parameters-6).
  • HTTP Host Header: A mandatory HTTP header field that identifies the domain name of the server.
  • HTTP Referrer: A non-mandatory HTTP header field that identifies the address from which a destination webpage is requested. Logging referrers allows websites to keep track of how users arrive at a page.
  • HTTP Return Code: Status codes defined in the following document: https://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html
  • HTTP User Agent: A non-mandatory HTTP header field that identifies the client that submitted a request. User agent information, such as operating system and browser software, helps websites determine how content will be displayed.
  • HTTP URL: The filename portion of a path to a web resource, with query string (if any).

For additional information on querying using host data, refer to Query Pane Settings. Options include matching substrings in the values of group-by dimensions (see DNS/WWW Extract Function).

 

 
 top

About Metrics

A metric is a combination of a unit (e.g. a bit) with a method of calculation (e.g. average per second) to create a quantifiable measurement (average bits/second). In Kentik Detect, metrics represent measurements made on flows, which are used for counts, rankings (e.g. in a top-X list), and thresholds (e.g. in alerting). The following table shows the main places in the Kentik Detect portal where metrics are specified and used:

Portal location Primary Metric Secondary Metric
Library » Dashboards: Add View Panel or Edit View Panel dialog Query tab » Query pane, either:
- Metrics field
- Customize Metrics » Metrics dialog (see Metrics Dialog).
Query tab » Query pane » Customize Metrics » Metrics dialog.
Data Explorer: Sidebar Query pane. Sidebar » Query pane, either:
- Metrics drop-down (see Query Basic Options).
- Customize Metrics » Metrics dialog.
Sidebar » Query pane » Customize Metrics » Metrics dialog.
Alerting » Policies: Add Policy or Edit Policy dialog Dataset tab » Data Funneling pane » Primary Metric drop-down. Dataset tab » Data Funneling pane » Secondary Metric field.

 

 
 top

General Metrics

The metrics available in a given setting (e.g. drop-down Metric list in Query pane of Data Explorer sidebar) vary depending on the device type. If the traffic being queried comes only from routers then metric options include only the following (for descriptions of the KDE columns from which metrics are derived, see Metrics Columns):

Metric in portal Derived from these main table column(s)
Bits per second in_bytes, out_bytes
Packets per second in_pkts, out_pkts
Count (flows per second) Based on rows per table
Unique IPs, source and destination inet_src_addr, inet_dst_addr
Unique prefixes, source and destination inet_src_route_prefix, inet_dst_route_prefix
Unique ASNs, source and destination src_as, dst_as
Unique next-hop ASN (destination) i_dst_nexthop_as_name
Unique countries, source and destination src_geo, dst_geo
Unique regions, source and destination src_geo_region, dst_geo_region
Unique cities, source and destination src_geo_city, dst_geo_city
Sampling rate, max and average sample_rate

Note: Metrics whose name includes “unique” involve evaluating all instances across all queried devices within each individual time slice covered by the query’s time range (see Table Time-slicing) but not between time slices.

If the traffic being queried includes traffic from any hosts (i.e. Kentik-registered devices representing a kprobe host agent), the following additional metric options are available (for descriptions see Host Traffic Metrics):

Metric in portal Derived from these main table column(s)
Retransmits, per second and % tcp_retransmit (and both_pkts for %)
Out-of-order packets, per second and % ooorder_in_pkts, ooorder_out_pkts (and both_pkts for %)
Fragments, per second and % fragments (and both_pkts for %)
Network latency per client/server/application (ms) client_nw_latency_ms, server_nw_latency_ms, appl_latency_ms

 

 
 top

Host Traffic Metrics

In addition to the metrics described in General Metrics, Kentik Detect uses data from kprobe (see About kprobe) to make available the following metrics on traffic from hosts:

  • Retransmits/second: Packets per second that are re-sent from source to destination; applies only to traffic using a reliable transport protocol such as TCP. Retransmission indicates network delivery issues that can degrade application/service performance.
  • % Retransmits: Resent packets (see above) as a percentage of all packets sent. This measure reveals aggregate efficiency in the full context of expected total capacity.
  • Out of order/second: Packets per second that arrived at the destination out of sequence. A high value indicates potential variability in delivery paths and is particularly detrimental to latency-sensitive traffic such as real-time audio or video.
  • % Out of order: Out of order packets (see above) as a percentage of all packets sent.
  • Fragments/second: Packets per second that have been split into smaller packets for delivery across the network. Fragmentation and reassembly can increase CPU load and also cause retransmits when fragments arrive out-of-order.
  • % Fragments: Fragmented packets (see above) as a percentage of all packets sent.
  • RTT/2 client latency (derived): One-way network latency as measured from the client perspective (in a client-server relationship). High values indicate latency problems in the network or at the server end of the flow.
  • RTT/2 server latency (derived): One-way network latency as measured from the server perspective (in a client-server relationship). High values indicate latency problems in the network or at the client end of the flow.
  • RTT/2 application latency (derived): A measure of one-way network latency that is derived by examining request/response pairs at the application layer (i.e. HTTP GET vs. first response). While this measure can be a reasonable proxy for end-user experience of application response, it only works for application protocols that have clear request/response pairings.
  • First Payload Exchange Latency: Used to measure application response time, particularly when the protocol isn’t understood or can’t be decoded (e.g. HTTPS, SQL, etc.). The time, which excludes TCP setup, starts with the first packet sent (typically the request) and stops with the first packet returned (typically the response).

When one of the host-only metrics above is chosen in the Query pane of the sidebar (e.g. in Data Explorer), a PPS Threshold field will appear to the right of the Metric selector, enabling you to set a level below which matching data will not be returned. The default value is 10. You may need to lower this number if you are looking at a small volume of traffic.

Notes:
- For greater clarity when querying latency, include a filter limiting the query to traffic using the TCP protocol (source, destination, or full).
- Sankey diagrams are not currently accurate for displaying latency or percent metrics. This is scheduled to be addressed in 2018 Q1.
- The KDE columns corresponding to the metrics listed above are shown in NPM Columns.

 

In this article: