Query Settings Overview

The following topics provide an overview of the settings for queries the Kentik portal, including in Data Explorer and dashboard panels:

• Query Sidebar Controls
• Visualization Pane
• Time Pane
• Advanced Query Settings

Several sections of the Kentik portal include a Query sidebar that contains a set of controls presented as a series of panes. The controls in the panes are used to set values for queries whose results (graph and table) are shown in the main display area of the page. The standard set of controls includes the following:

• Visualization: Sets the chart type (see Chart View Types) for the query as well as various result display options that are specific to individual views. See Visualization Pane.
• Data Sources pane: Displays the data sources (e.g. devices) currently selected for the query, and enables access to the dialog for selecting data sources. See Data Source Settings.
• Dimensions pane: Displays the group-by dimensions currently selected for the query, and enables access to the dialog for selecting dimensions. See Dimension Settings.
  Note: If no dimensions are specified then the query will run on the total traffic (less traffic filtered out, if any).
• Matrix With (shown only when the Visualization Type is set to Matrix; see Matrix View): Displays the matrix-with dimensions for a matrix query. This pane is nearly identical to the Dimensions pane. See Matrix Pane UI.
• Metrics pane: Displays the primary metric currently selected for the query, and enables access to the dialog for selecting primary and secondary metrics. See Metric Settings.
• Time pane: Sets the time range for the query. See Time Pane.
• Filtering pane: Displays the filter groups and conditions currently applied to the query, and enables access to the dialog for specifying filters. See Filtering Settings.
• Bracketing pane: Displays the upper bound and the color of each defined range. See Bracketing Settings.
• Advanced pane: A collection of controls used to fine-tune the query. See Advanced Query Settings.

Note: Some of the above controls are documented in this article; more complicated settings are covered in individual chapters.

The following table shows which of the above-listed panes appear in the various portal locations that include sidebars with query settings:

The Visualization pane, which is used to specify how query results will be displayed as a visualization, includes the following controls:

• Visualization Type: A drop-down menu used to set the type of visualization (see Chart View Types) to display in the chart display area.
• Sync Axes: Sets the scale of both left and right axes to the same numeric values (even if the metric is different). This option is present only when Bi-directional Charting is on (see Advanced Pane UI) or Secondary Metric is not None (see About the Metrics Dialog).
  Note: The greater the difference in value between the scale of the two axes, the greater the likelihood that syncing the axes will effectively hide data plotted against the lower-value axis.
• Use Logarithmic Scale: When on, these switches cause one or more axis of the visualization to be plotted against a logarithmic scale; see Logarithmic Scale Options.
• Visualization depth: Determines how many rows, from 1-40, will be plotted in the graph. As this setting is increased, more detail is provided in the visualization, and there's a decrease in the gap between the individual plotted data and the blue line representing Total.
  Note: When View Type is set to Table, the Visualization Depth determines how many rows will be displayed in the table.

Logarithmic scale switches are present for the following view types:

• Stacked Area Chart:
  - Use Logarithmic Scale on Primary Axis
  - Use Logarithmic Scale on Secondary Axis
• Line Chart:
  - Use Logarithmic Scale on Primary Axis
  - Use Logarithmic Scale on Secondary Axis
• Stacked Column chart:
  - Use Logarithmic Scale on Primary Axis

Time-range settings in the sidebar's Time pane define the timespan covered by the query whose results are displayed in the graph and table in the display area. When the Time pane is expanded its initial control set includes the following:

• Compare over previous period: A switch that enables "period over period" comparison of the query results from the currently defined time range with results from the same query looking at a previous time range of the same duration. By default the switch is off. When the switch is on, the Time Range drop-down is labelled "Current" and the Previous controls are displayed.
  Note: This switch results in multiple changes to the content and behavior of the table and chart displayed in Data Explorer. For information about these changes, see Compare Periods.
• Time Range: A three-part control to display and change the time range of the query:
  - Shift back: The arrow at the left end of the control shifts the time range back by the range's current duration, e.g. if the range is one hour starting at 11 AM today it will shift back to one hour starting at 10 AM today.
  - Range drop-down: The currently defined time range is displayed in the central part of the control. Click the control to open the Time Range popup to choose a different lookback duration or define a different custom range.
  - Shift forward: The arrow at the right end of the control shifts the time range forward by the range's current duration. If shifting forward would extend the range beyond the present the control will be inactive.
• Previous (shown only when Compare over previous period is on): A two-part control for setting the time range against which the current time range will be compared:
  - Count: A field in which you can enter the number of units back from the current range.
  - Unit: A drop-down from which you can select a duration unit, either hour, day, week, or month.
• Zone: The time zone in which times are expressed, either UTC or local.
  Note: To change the default for this setting, see user User-specific Defaults.
• Use AWS Timestamps: Choose the timestamps to use for flow records originating in a cloud export from AWS (see Cloud Exports and Devices). By default the switch is off, meaning that flow records are timestamped upon intake by Kentik. When the switch is on, the timestamp for records from AWS flow logs will instead be AWS's timestamp.

Note: When zooming the display area (see Data Explorer Chart), the time range is automatically set to the start and end times of the zoomed region.

When Custom is chosen from the Time Range drop-down, the Time Range popup opens to show the following controls, which are used to define a range:

• Cancel: A button that closes the popup and leaves the time range as it was before the popup was opened.
• Apply: A button that applies the time range from the values in the start and end fields and hides the popup. The applied range will be shown in the Time-range Drop-down
• Lookback list: Preset durations back from the current time, listed along the left side of the popup. Click a list item to set the time range. For example, if the current time is 11:00 and the chosen duration is Last 15 Minutes, the resulting time range will be from 10:45 to 11:00.
• From: Date and time fields that set the start of the time range.
• To: Date and time fields that set the end of the time range.
• Calendars: Side-by-side monthly calendars that show the time range (if it spans more than one day) and enable you to change the range (shown in the time range fields above) by clicking on a date. The start and end days of the time range are indicated in blue, and the intervening days are indicated in light gray. The calendar controls include forward and back buttons to change the displayed months as well as drop-down selectors for month and year.

Kentik supports a number of advanced options that can help you tailor queries to return precisely what you need to see. The controls for these options are located in the Advanced pane of the Query sidebar in Data Explorer and in the configuration dialog for individual dashboard panels. The following topics explain the associated options and their controls:

• Advanced Pane UI
• Compound Queries
• Generated Charts
• DNS/WWW Extract Function
• Time Series Settings

The Advanced pane includes the following controls:

• Dataseries: A drop-down menu to set the resolution of the KDE dataseries on which the query will be run, either Auto, Full, or Fast. See Resolution Overview.
  Note: The available dataseries settings depend on the current time range.
• Show Total Overlay: Enables/disables plotting of the total traffic returned from the query. The effect varies depending on view type:
  - Time series stacked and bar graphs: Total appears in the graph as a blue line above the other plots.
  - Time series line graph: Total appears in the graph as a dashed blue line above the other plots.
  - Comparison bar chart: Total appears as blue line at right.
  - Pie chart: If total overlay is on, the chart includes a ring segment labeled "Other" (traffic not plotted ion the other segments); if off, the Other segment is not included.
  - Sankey diagram: If on, a row for total is included in the table below the diagram.
  - Matrix: If on, a total line will be plotted on the detail graphs that are rendered below the matrix when you click on a matrix cell, column heading, or row heading (depending on the view type selected for those graphs; see Matrix Detail Graph).
  Note: Turning off Total Overlay rescales the axis, which may improve your ability to see smaller-value plots.
• Show Historical Overlay: Enables/disables plotting of the total from the same query run on a time range from a number of days earlier. Historical values are plotted as a dashed gray line. Historical Overlay is not available if the Visualization Type is set to Time Series Line Graph, and is automatically switched off when the type is Pie Chart.
• Days Back: If Historical Overlay is on, this field sets the number of days back that will be plotted in the historical display. For example, if the time range is last 6 hours, the time is 11:00, and the days back is set to 7 (default) then the historical plot will show the total from 5:00 to 11:00 seven days ago.
• Enable Reverse DNS Lookups: Determines whether reverse DNS (rDNS) lookup will be performed to determine associated host names when querying IPs.
  Notes:
  - When this switch is on and the query includes an IP/CIDR dimension, the host name for each IP will appear in parentheses in the IP/CIDR column of the results table.
  - To use an alternate DNS server (instead of the default) for reverse lookup, see Custom DNS.
  - Queries return faster when this option is off.
• Use AS Groups: When this switch is on the results from all ASes of each AS group (see About AS Groups) will be summed for top-X evaluation, graph plotting, and display in the results table.
  Note: The switch is visible only if at least one AS group exists in your organization. When visible, the switch is enabled by default.
• Bi-directional Mode (shown only if view type is stacked or line; see Chart View Types): Enables simultaneous charting of two graphs, one based on the current group-by dimensions and the other based on the opposite of those dimensions (see Compound Queries).
• Generate One Chart Per Series: Changes the visualization in the display area from a single chart plotting multiple top-X series to a set of charts that each show a single top-X series; see Generated Charts.
• Extract From (shown only for specific group-by dimensions): Enables group-by on substrings in certain DNS/WWW dimension values; see DNS/WWW Extract Function.
• Time Series: Settings enabling greater control over the aggregation of returned data into a time series in the query's chart and top-X table (see Time Series Settings).

The Bi-directional Mode switch (Advanced Pane UI) enables compound queries that include, on a single chart, graphs resulting from multiple simultaneous underlying queries. Compound queries fall into two general categories, but in some cases (though not all) the categories can be applied simultaneously to create a chart incorporating four graphs:

• Bidirectional: An “original” graph of traffic is based on the current Group-by Dimension setting, and an “opposite” graph is based on the opposite of those dimensions. For example, if the group-by dimensions are Source Country and Destination AS Number, the opposite graph would show traffic based on Destination Country and Source AS Number.
  Note: Filters are also flipped for the opposite view, meaning that filters on source in the original are on destination in the opposite, and vice versa.
• Secondary Metric: A “primary” graph of traffic is based on the primary Metric setting, and a “secondary” graph is based on the Secondary Display Metric setting (see Metrics Dialog UI).

Note: The two categories of compound queries described above are independent, meaning that you can create either a bidirectional query with no secondary metric or a secondary metric query that's not bidirectional (see Compound Query Variations).

As detailed in the table below, the results (graphs and tables) that are returned from a compound query depend on the interaction of several settings, including the Bi-directional Mode switch in the Query sidebar’s Advanced pane, the Secondary Display Metric in the Metrics dialog, and the View Type setting in the chart display area (see Chart View Types).

The following additional considerations apply when using compound queries:

• In line charts, plots against the left axis are drawn with a solid line, while plots against the right axis are dashed.
• On bidirectional charts, the flipping of dimensions (e.g. from Source ASN to Destination ASN) for opposite graphs applies only to dimensions in the Source and Destination groups (see Dimension Selection Groups). Dimensions in the Full and DNS/WWW categories are treated the same on both original and opposite graphs.
• Dimensions in the Custom category (see Custom Dimensions) will be flipped only if there are two dimensions whose names are identical except in one of the following ways:
  - One includes "src" where the other has "dst."
  - One includes "in" where the other has "out."
  - One includes "to" where the other has "from."

Generated charts are visualizations made up of a set of individual charts, each of which represents one top-X series in the current query. The generated charts feature, which is enabled by the Generate One Chart Per Series switch (see Advanced Pane UI), is particularly useful for defining a dashboard panel (see Add View to Dashboard).

Like standard single-chart visualizations, generated charts are displayed in the Data Explorer display area (see Explorer Chart Display) above the query’s results table. The number of generated charts is set by the query's visualization depth. If bracketing is set (see Bracketing Pane) then each graph will be outlined in the color of the bracketing range corresponding to the value (in the bracketing metric, e.g. average Gbits/s) of the series graphed in that chart.

The following UI elements are shown in the Advanced pane of the Query sidebar when the Generate One Chart Per Series switch is on:

• Chart Titles: The title that will appear at the upper left of each chart.
  - By default, the title of each chart is set with the generator_series_name variable, which is based on values for dimensions that make up the query's key. For example, if the query's group-by dimensions are Source Country and Destination Country then each chart name is a combination of a source country code and a destination country code (e.g. "US ---- GB").
  - If you enter a string into this field, then all of the generated charts will have the same title (the entered string).
• Charts per Row: The number of charts (up to four) that will appear (horizontally) in each row of the display area.
• Chart Height: The vertical space allocated to a row of charts. The settings (Short, Medium, and Tall) are relative; their actual effect depends on the height of the browser window and the position of the Resize bar (see Chart Display UI) that divides the display area from the results table.
• Chart-level Group By Dimensions: A dimension selector that opens a Group By Dimension dialog (see Dimension Selectors) that enables you to choose one or more group-by dimensions. The top-X for the chosen dimensions will be calculated and plotted individually for each chart rather than across the entire query.
• Chart Visualization Depth (shown only when there is a panel dimension): Determines the number of series (top-X combinations of panel group-by dimensions) that will be shown in each chart.

Note: When the space allocated to a given chart is insufficient to show all lines of the chart's key then up and down icons (triangles) will be shown below that chart, enabling you to page through the lines of the key.

The dimensions available in the Group By Dimension selector vary depending on the device type. If any device currently selected in the Data Sources pane (see Data Source Settings) is a host of type kprobe (see Host Configuration) the dimension selector will include a set of DNS/WWW dimensions (see Host Traffic Dimensions).

Some of these DNS/WWW dimensions will, when selected, result in the addition of Extract From settings to the Advanced pane of the Query sidebar. These Extract From settings are available for the following dimensions:

• DNS Query
• HTTP URL
• HTTP Host Header

The Extract settings change how Kentik evaluates the values of the dimensions listed above:

• With no extraction, each unique value in the column will be treated separately.
• With extraction, a regex-defined pattern will be used to look for matching substrings within the values, and all values that match the same substring will be grouped together.

In a query that returns Top-X ranking, for example, if the dimension is DNS Query and the extract function is set to Domain then instead of counting each subdomain within a domain (e.g. x.domain.com, y.domain.com, and z.domain.com) as a unique value, all values sharing the same domain will be counted together.

To apply the extract function, use the Extract From settings to choose (from the drop-down menu) the type of substring that you want to match. The Regex and Selector fields will then be populated with the suggested POSIX-style regex (shown in the following table) and selector for that type of match. Because the fields are editable, you can customize the regex and selector as needed to achieve the desired result.

As explained in About Regex Capture Groups, the selector (e.g. $1) will give the index (1-based) of a “capture group,” which is a substring that is surrounded by parentheses in the regex. For example, in the regex for Path in the table above, $1 would refer to a substring that matches the pattern '.* (the first substring surrounded by parentheses).

Note: For additional information on or assistance with using the extract function, please contact support@kentik.com.

Queries are performed on Kentik-stored network traffic data (flow records) in two stages:

• Initial flow aggregation: Bytes or packets (depending on the query's metric) across the query's time range will be summed for all flows matching the query's filters, divided by the value of the flow aggregation window setting, and grouped by the query's selected dimensions, resulting in a set of sample series that represent an average bit/packet rate across all of the summed flows.
• Time series construction: The top-X sample series from the initial flow aggregation are aggregated into time series buckets of a specified width using the specified aggregation technique.

The settings of the Time Series controls in the Query sidebar's Advanced pane enable you to customize the way aggregation will be performed during the two stages described above:

• Flow Aggregation Window: The time window used for initial flow aggregation (see Flow Aggregation Window). The default setting is Auto.
• Bucket Aggregation: The function used to construct time series buckets from flow aggregation samples:
  - None: No further aggregation of the initial flow aggregation into time series buckets.
  - Average: Calculates the average value of the flow aggregation samples in each bucket.
  - Maximum: Selects the flow aggregation sample with the maximum value in each bucket.
  - Minimum: Selects the flow aggregation sample with the minimum value in each bucket.
• Width (hidden when Bucket Aggregation is set to None): The width (in time) of the "buckets" into which flow aggregation samples are aggregated to make a time series for top-X results:
  - If set to Auto, the bucket width will be one hour.
  - If set to any other value (10/20/30 min, 1/6/12 hours, 1 day, or 1 week) the bucket width must be longer than the flow aggregation window.

A query's flow aggregation window depends on a number of factors including the type of dataseries (see About Dataseries Resolution) and the duration of the query's time range. The table below shows the windows that are available for various time ranges.