Data Explorer
The features and use of the Data Explorer in the Kentik Detect portal are discussed in the following topics:
- About Data Explorer
- Explorer Sidebar Controls
- Query Pane Settings
- Query Dimension Dialogs
- Filter-based Dimensions
- Metrics Dialog
- Explorer Chart Display
- Data Explorer Table
- Explorer View Options
Note: The functionality of Data Explorer is also available via API; see V5 Query API.
About Data Explorer
A general explanation of Data Explorer is provided in the following topics:
Data Explorer Overview
Data Explorer is Kentik Detect’s primary interface for manually exploring the network data (flow records, BGP, SNMP, etc.) stored in the main tables of the Kentik Data Engine (KDE; see KDE Tables). The Data Explorer UI enables you to define settings that are translated into queries that return “views” made up of tables and graphs about the traffic on specified devices during a specified timespan.
Views are defined using sidebar controls (see Explorer Sidebar Controls) to specify query parameters such as time range, devices, and dimensions, and to narrow the returned data by filtering on dozens of different main table fields. Views can be saved and reloaded at a later time (see About Saved Views).
Results are displayed in the display area and are typically made up of both of the following:
- A visualization (see Data Explorer Chart), which may be one of a variety of different types (see Chart View Types).
- A corresponding table (see Data Explorer Table) listing the query results in tabular form.
Data Explorer Access
To open Data Explorer:
- Default view: Click Data Explorer on the main portal navbar.
- Saved view: Choose Saved Views from the Data Explorer menu in the main portal navbar, then click on the view that you want to open (see Loading a View).
Data Explorer Areas
Data Explorer is made up of the following main areas:
- Display area: An area for display of the current view, which in most cases is made up of a graph (see Data Explorer Chart) and an accompanying table (see Data Explorer Table).
- Sidebar: An area at left that contains the controls (see Explorer Sidebar Controls) used to specify the query whose results are returned in the display area.
Explorer Sidebar Controls
The Data Explorer sidebar contains the controls used to define the view whose query results are displayed in the chart and table in the Data Explorer display area. These controls are covered in the following topics:
Explorer Sidebar Overview
The Data Explorer sidebar contains the following UI elements:
- General controls:
- Run Query button: Applies changed settings to the graph and table in the display area on the right side of Data Explorer (see Run Query Button).
- Expand/Collapse control: Toggles the sidebar between expanded (panes) and collapsed (icons only) states (see Expand/Collapse Sidebar). - Sidebar panes: A set of panes that are used to set values for the queries whose results (typically graph and table) are shown in the display area at the right. The panes can be in either Edit mode or Summary mode (see Pane Display Modes).
Note: When the sidebar is collapsed its panes are represented by their respective icons.
Explorer Sidebar Panes
In Data Explorer, the sidebar contains the following panes to control the query whose results are displayed in the display area:
- Query pane: Specifies group-by, metric, and display by options, as well as additional advanced options; see Query Pane Settings.
- Time pane: Specifies the time range covered by the query; see Time Pane Settings.
- Filtering pane: Specifies filters that may be applied to the query; see Filtering Pane Settings.
- Devices pane: Specifies the Kentik-registered devices covered by the query; see Devices Pane Settings.
Notes:
- For fastest results, always choose devices via the Devices pane rather than by filtering.
- For additional general information about panes see About Sidebar Panes.
Query Pane Settings
The Query pane contains a set of controls that define the general outlines of the query whose results are displayed in the graph and table in the Data Explorer display area. These controls are covered in the following topics:
Query Basic Options
The Query pane includes the following basic query settings:
- Group by Dimensions: A selection box used to choose dimensions (preset or filter-based) for the query that will be visualized in the Data Explorer display area. The chosen dimensions are included in the SELECT statement of the underlying query. Preset dimensions, which are based on KDE fields, are listed and described in Dimensions Reference. To use the selection box, see Query Dimension Selectors.
- CIDR: A pair of fields that appear only when the selected dimension includes a CIDR component (e.g. Source IP/CIDR):
- v4 CIDR: Use to specify the number of bits of the routing prefix in IPv4. Default is 32.
- v6 CIDR: Use to specify the number of bits of the routing prefix in IPv6. Default is 128.
Note: If you have only v4 traffic, the v6 CIDR field will be ignored (can be left at 128). - Matrix With (shown only when the View Type is set to Matrix; see Matrix View): This selection box, which is used to choose one or more matrix-with dimensions for a matrix query, functions identically to the Group by Dimensions selection box as described in Query Dimension Selectors.
- Metrics: A drop-down menu from which you can choose the primary metric (see About Metrics) in which the graphs and table of the Data Explorer will express the results of the query.
Note: For each metric in the list Kentik has preselected a set of additional metrics that will be shown in the results table when the query is run. To customize these additional metrics, click Customize Metrics/Edit Metrics. - Metrics selector (shown only if custom metrics have been configured with Metrics dialog): A list of the metrics currently configured for the query. The following actions are possible for this list:
- Change metrics: To change the list, click Customize Metrics/Edit Metrics to open the Metrics Dialog.
- Change order: Drag the handle at the left of each selected metric to change the order in which the metrics are displayed in the results table.
- Set Threshold: Use the threshold icon at the right of each metric to open the Set Minimum Threshold popup for that metric, which enables you to exclude from the results all series whose value for the metric is not at least the minimum value that you specify. The icon turns from gray to orange if a threshold has been set.
Note: To be included in results a series must meet all of the currently set thresholds. - Customize Metrics/Edit Metrics: A link that opens the Metrics Dialog, where you can customize the additional metrics that will be shown in the results table, determine how results are sorted, and create a compound query (see Compound Queries).
Note: If custom metrics have been specified in the dialog then the link will change to Edit Metrics. If the metrics are set back to default, the link reverts to Customize Metrics. - PPS Threshold: This field appears if any hosts are selected in the Device List and Metric is set to Retransmits/s or % Retransmits. It filters the graph and table so that the only rows shown are those whose dimension value is at least the threshold value.
- Dataseries: A drop-down menu to set the resolution of the KDE dataseries on which the query will be run, either Auto, Full, or Fast. See Resolution Overview.
Query Advanced Options
The Advanced Options control toggles display of additional Query pane settings. The following controls are visible only when advanced options are shown:
- Visualization depth: Determines how many rows, from 1-40, will be plotted in the graph. As this setting is increased, more detail is provided in the visualization, and there’s a decrease in the gap between the individual plotted data and the blue line representing Total.
Note: When View Type is set to Table, the Visualization Depth determines how many rows will be displayed in the table. - Show Total Overlay: Enables/disables plotting of the total traffic returned from the query. The effect varies depending on view type:
- Time series stacked and bar graphs: Total appears in the graph as a blue line above the other plots.
- Time series line graph: Total appears in the graph as a dashed blue line above the other plots.
- Comparison bar chart: Total appears as blue line at right.
- Pie chart: If total overlay is on, the chart includes a ring segment labeled “Other” (traffic not plotted ion the other segments); if off, the Other segment is not included.
- Sankey diagram: If on, a row for total is included in the table below the diagram.
- Matrix: If on, a total line will be plotted on the detail graphs that are rendered below the matrix when you click on a matrix cell, column heading, or row heading (depending on the view type selected for those graphs; see Matrix Detail Graph).
Note: Turning off Total Overlay rescales the axis, which may improve your ability to see smaller-value plots. - Show Historical Overlay: Enables/disables plotting of the total from the same query run on a time range from a number of days earlier. Historical values are plotted as a dashed gray line. Historical Overlay is not available if the View Type is set to Time Series Line Graph, and is automatically switched off when the View Type is set to Pie Chart.
- Days Back: If Historical Overlay is on, this field sets the number of days back that will be plotted in the historical display. For example, if the time range is last 6 hours, the time is 11:00, and the days back is set to 7 (default) then the historical plot will show the total from 5:00 to 11:00 seven days ago.
- Enable Reverse DNS Lookups: Determines whether reverse DNS (rDNS) lookup will be performed to determine associated host names when querying IPs.
Notes:
- When this switch is on and the query includes an IP/CIDR dimension, the host name for each IP will appear in parentheses in the IP/CIDR column of the results table.
- To use an alternate DNS server (instead of the default) for reverse lookup, see Custom DNS.
- Queries return faster when this option is off. - Use AS Groups: When this switch is on the results from all ASes of each AS group (see About AS Groups) will be summed for top-X evaluation, graph plotting, and display in the results table.
Note: The switch is visible only if at least one AS group exists in your organization. When visible, the switch is enabled by default. - Bi-directional Mode (shown only if view type is stacked or line; see Chart View Types): Enables simultaneous charting of two graphs, one based on the current group-by dimensions and the other based on the opposite of those dimensions (see Compound Queries).
- Generate One Chart Per Series: Changes the visualization in the display area from a single chart plotting multiple top-X series to a set of charts that each show a single top-X series; see Generated Charts.
- Extract From (shown only for specific group-by dimensions): Enables group-by on substrings in certain DNS/WWW dimension values; see DNS/WWW Extract Function.
Query Dimension Selectors
The Query pane includes the following dimension selectors (pictured at right):
- The Group By Dimensions selector, whose dimensions are chosen in the Group By Dimensions dialog. The dialog has two tabs, each of which supports a distinct kind of group-by dimension:
- Preset: Kentik-defined dimensions as described in Dimensions Reference.
- Filter-based: User-defined dimension composed of a set of filter groups that are each displayed (in graph and in table) as a separate result (plot and row); see Filter-based Dimensions. - The Matrix With selector, whose dimensions are chosen in the Matrix By Dimensions dialog. This selector appears only when the view type is Matrix (see Chart View Types). This selector supports only preset dimensions.
Clicking in a dimension selector opens a dialog (see Query Dimension Dialogs) that allows you to choose multiple simultaneous group-by or matrix-with dimensions (see About Dimensions). Dozens of preset dimensions are currently available for traffic in groups including Source, Destination, Full (combined), and DNS/WWW (see Dimension Selection Groups). In addition, up to ten custom dimensions can be defined (customer-wide; see Custom Dimensions) and applied from a dimension selector.
Notes:
- To query on total traffic, the Group By Dimensions selector should be blank (no group-by dimension selected).
- For fastest results, where possible use native dimensions rather than virtual (see Dimensions Reference).
- A query can use no more than eight group-by dimensions and eight matrix-with dimensions.
Compound Queries
Compound queries enable the inclusion on a single chart of graphs resulting from multiple simultaneous underlying queries. Compound queries fall into two general categories, but in some cases (though not all) the categories can be applied simultaneously to create a chart incorporating four graphs:
- Bidirectional: An “original” graph of traffic is based on the current Group-by Dimension setting, and an “opposite” graph is based on the opposite of those dimensions. For example, if the group-by dimensions are Source Country and Destination AS Number, the opposite graph would show traffic based on Destination Country and Source AS Number.
Note: Filters are also flipped for the opposite view, meaning that filters on source in the original are on destination in the opposite, and vice versa. - Secondary Metric: A “primary” graph of traffic is based on the primary Metric setting, and a “secondary” graph is based on the Secondary Display Metric setting (see Metrics Dialog UI).
Note: The two categories of compound queries described above are independent, meaning that you can create either a bidirectional query with no secondary metric or a secondary metric query that’s not bidirectional (see table below).
As detailed in the table below, the results (graphs and tables) that are returned from a compound query depend on the interaction of several settings, including the Bi-directional Mode switch in the sidebar’s Query pane (see Query Advanced Options), the Secondary Display Metric in the Metrics dialog, and the View Type setting in the chart display area (see Chart View Types).
View Type | Secondary display metric | Bidirectional mode | Chart axes | Chart directions | Table tabs |
Stacked or Line | None | Off | Left only | Positive only | Primary only |
Stacked | None | On | • Left: group-by dimensions • Right: opposite dimensions |
• Positive: group-by dimensions • Negative: opposite dimensions |
• Original: group-by dimensions • Opposite: opposite dimensions |
Stacked | not None | Off or On Note: This setting is ignored unless Secondary Display Metric is None. |
• Left: primary metric • Right: secondary metric |
• Positive: primary metric • Negative: secondary metric |
• Primary metric • Secondary metric |
Line | None | On | Left only | • Positive: group-by dimensions • Negative: opposite dimensions |
• Original: group-by dimensions • Opposite: opposite dimensions |
Line | not None | Off | • Left: primary metric • Right: secondary metric |
Positive only | • Primary metric • Secondary metric |
Line | not None | On | • Left: primary metric • Right: secondary metric |
• Positive: group-by dimensions • Negative: opposite dimensions |
• Original group-by, primary metric • Original group-by, secondary metric • Opposite dimensions, primary metric • Opposite dimensions, secondary metric |
The following additional considerations apply when using compound queries:
- In line charts, plots against the left axis are drawn with a solid line, while plots against the right axis are dashed.
- On bidirectional charts, the flipping of dimensions (e.g. from Source ASN to Destination ASN) for opposite graphs applies only to dimensions in the Source and Destination groups (see Dimension Selection Groups). Dimensions in the Full and DNS/WWW categories are treated the same on both original and opposite graphs.
- Dimensions in the Custom category (see Custom Dimensions) will be flipped only if there are two dimensions whose names are identical except in one of the following ways:
- One includes “src” where the other has “dst.”
- One includes “in” where the other has “out.”
- One includes “to” where the other has “from.”
Generated Charts
Generated charts are visualizations made up of a set of individual charts, each of which represents one top-X series in the current query. The generated charts feature, which is enabled by the Generate One Chart Per Series switch in Query Advanced Options, is particularly useful for defining a dashboard panel (see Add View to Dashboard).
Like standard single-chart visualizations, generated charts are displayed in the Data Explorer display area (see Explorer Chart Display) above the query’s results table. The number of generated charts is set by the query’s visualization depth. As shown in the image below, if bracketing is set (see Bracketing Pane Settings) then each graph will be outlined in the color of the bracketing range corresponding to the value (in the bracketing metric, e.g. average Gbits/s) of the series graphed in that chart.
Generated Charts UI
The following UI elements are shown in the Advanced Options section of the Query pane when the Generate One Chart Per Series switch is on:
- Chart Titles: The title that will appear at the upper left of each chart.
- By default, the title of each chart is set with the generator_series_name variable, which is based on values for dimensions that make up the query’s key. For example, if the query’s group-by dimensions are Source Country and Destination Country then each chart name is a combination of a source country code and a destination country code (e.g. “US ---- GB”).
- If you enter a string into this field, then all of the generated charts will have the same title (the entered string). - Charts per Row: The number of charts (up to four) that will appear (horizontally) in each row of the display area.
- Chart Height: The vertical space allocated to a row of charts. The settings (Short, Medium, and Tall) are relative; their actual effect depends on the height of the browser window and the position of the Resize bar (see Chart Display UI) that divides the display area from the results table.
- Chart-level Group By Dimensions: A dimension selector that opens a Group By Dimension dialog (see Query Dimension Dialogs) that enables you to choose one or more group-by dimensions. The top-X for the chosen dimensions will be calculated and plotted individually for each chart rather than across the entire query.
- Chart Visualization Depth (shown only when there is a panel dimension): Determines the number of series (top-X combinations of panel group-by dimensions) that will be shown in each chart.
Note: When the space allocated to a given chart is insufficient to show all lines of the chart’s key then up and down icons (triangles) will be shown below that chart, enabling you to page through the lines of the key.
DNS/WWW Extract Function
The dimensions available in the Group By Dimension selector vary depending on the device type. If any device currently selected in the Devices pane (see Devices Pane Settings) is a host of type kprobe (see Host Configuration) the dimension selector will include a set of DNS/WWW dimensions (see Host Traffic Dimensions).
Some of these DNS/WWW dimensions will, when selected, result in the addition of Extract From settings to the Advanced Options portion of the Query pane (see Query Advanced Options). The Extract From settings are available for the following dimensions:
- DNS Query
- HTTP URL
- HTTP Host Header
The Extract settings change how Kentik evaluates the values of the dimensions listed above:
- With no extraction, each unique value in the column will be treated separately.
- With extraction, a regex-defined pattern will be used to look for matching substrings within the values, and all values that match the same substring will be grouped together.
In a query that returns Top-X ranking, for example, if the dimension is DNS Query and the extract function is set to Domain then instead of counting each subdomain within a domain (e.g. x.domain.com, y.domain.com, and z.domain.com) as a unique value, all values sharing the same domain will be counted together.
To apply the extract function, use the Extract From settings to choose (from the drop-down menu) the type of substring that you want to match. The Regex and Selector fields will then be populated with the suggested POSIX-style regex (shown in the following table) and selector for that type of match. Because the fields are editable, you can customize the regex and selector as needed to achieve the desired result.
Substring type | Regex: DNS Query, HTTP Host Header |
Regex: HTTP URL |
TLD | ['.]+\.(['.]+)$ | N.A. |
Domain | (['.]+\.['.]+)$ | N.A. |
Subdomain | (['.]+\.['.]+\.['.]+)$ | N.A. |
Host | ('.*)\.['.]+\.['.]+$ | N.A. |
Path | N.A. | ('.*)/.*(\?)+ |
Filename | N.A. | '.*(/.*)(\?)+ |
File Path | N.A. | ('.*)(\?)+ |
Query String | N.A. | '.*[\?]+(.*) |
As explained in About Regex Capture Groups, the selector (e.g. $1) will give the index (1-based) of a “capture group,” which is a substring that is surrounded by parentheses in the regex. For example, in the regex for Path in the table above, $1 would refer to a substring that matches the pattern '.* (the first substring surrounded by parentheses).
Note: For additional information on or assistance with using the extract function, please contact support@kentik.com.
Query Dimension Dialogs
The Matrix By Dimensions dialog and the Preset tab of the Group By Dimensions dialog are used to choose preset dimensions for the Matrix With selector and Group By Dimensions selector, respectively. These query dimension dialogs are covered in the following topics:
Notes:
- The Group By Dimensions dialog also includes a tab that can be used to choose a filter-based dimension that overrides the preset dimension in the Group By Dimensions selector; see Filter-based Dimensions.
- A query can use no more than eight group-by dimensions and eight matrix-with dimensions.
Dimension Dialog UI
The query dimension dialogs share the following common UI elements:
- Close button: Click the X in the upper right corner to close the dialog. All elements will be restored to their values at the time the dialog was opened.
- Filter field: Filters the list of available dimensions to those containing the entered text.
- Selected Dimensions list: A list of selected dimensions (up to eight). Use the handle at the left of each selected dimension to change the order in which the dimensions are applied.
- Available dimensions list: A grid containing of all of the dimensions that are currently supported by Kentik Detect, listed by group (see Dimension Selection Groups).
- Clear Selections button: Clears all dimensions from the Selected Dimensions list.
- Cancel button: Cancel changes to the selected dimensions and exit the dialog. The current query’s dimensions will be restored to what they were when the dialog was opened.
- Save button: Save changes to the selected dimensions and exit the dialog.
Dimension Selection Groups
The dimensions available in the query dimension selectors are based on the columns of the main tables of the KDE (see KDE Tables), which each represent a minute of flow records for a given device. The types of dimensions available for querying are discussed in Dimension Categories, and a description of each individual dimension is provided in Dimensions Reference.
In the Query pane dimension selectors (as well as in the dimension selector for filters; see Ad Hoc Filter Controls), the available dimensions are organized into a grid with the following columns:
- Source: Source traffic.
- Destination: Destination traffic.
- Source or Destination (Filter controls only): Matches on either source or destination traffic (saves adding two filters).
- Non-directional/Other: Traffic attributes that are non-directional.
The rows of the grid organize the available dimensions by functional category:
- Network & Traffic Topology (interface, connectivity, provider, etc.)
- IP & BGP Routing (see Dimension Categories and BGP Overview)
- Cloud (e.g. AWS or GCP; see Cloud Admin)
- Geolocation (Country, region, city, and Custom Geo)
- Application Context & Security (port and protocol, Threat Feed Dimensions)
- Application Decodes (DNS lookup and HTTP; see Host Traffic Dimensions)
- Custom (see Custom Dimensions)
Choosing Query Dimensions
To choose dimensions for the Group By Dimensions selector or Matrix With selector:
- By default (e.g. when you first navigate to the Data Explorer), the Group by Dimensions box is populated with a single dimension (e.g. Destination ASN). Click in the.
- Click in the selection box. A menu appears listing all available dimensions, categorized into Source, Destination, Full (all traffic), and Custom.
- Click on a dimension to add it to the selection box.
- Repeat the previous steps to add additional dimensions.
- To change the order in which the dimensions are applied, drag the dimensions into the desired order.
Notes:
- When more than one group-by dimension is selected the combination of dimensions is evaluated together to determine the rows that are included in the results; see Using Multiple Dimensions.
- Dimensions in the category DNS/WWW (see Dimension Categories) will only appear in the selector if one or more device currently selected in the Devices Pane (see Devices Pane Settings) is a host of type DNS/WWW (see Host Configuration).
- The Matrix With selection box operates the same as the Group By Dimension selection box. For more information, see Matrix View.
Using Multiple Dimensions
The following example illustrates how multiple group-by dimensions combine to determine the results returned from a query. The example shows a common use case for multiple dimensions, which is when an organization that generates traffic wants to see where it’s going and which links and devices it’s using to get there (which enables you to see if the traffic is going to the expected geographic locations in the expected proportions):
- Using Query Basic Options, set Metric to Bits/s and choose Average for Display and Sort by.
- Use Time Pane Settings to set the timespan.
- Use Filtering Pane Settings to filter so that you are seeing only outbound traffic. (Assuming that Interface Classification shows that at least 75 percent of your interfaces are classified, the easiest way to do this is to set a Destination Network Boundary filter to External.)
- In the chart display settings (see Chart Display UI) set the View Type to Sankey Flow Diagram.
- Use Devices Pane Settings to select multiple devices.
- Use the Group-by Dimension selector to set following group-by dimensions:
- Full: Device
- Destination: Next Hop AS Number
- Destination: AS Number
- Destination: Region
- Destination: Country - After clicking the Run Query button, the graph and table will show traffic categorized by the specified dimensions.
In the results:
- The Data Explorer Table will include a column for each dimension in the key (the five group-by dimensions specified above).
- All traffic with the same dimension value for each component of the key will be measured in Mb/second and summed onto a single table row, meaning that each row will represent traffic that has the same device, same dst_next-hop-asn, same dst_as, same dst_region, and same dst_country.
- The rows will be listed in descending order of highest average bits/second.
- The Sankey diagram will show the paths of the traffic represented by the top table rows.
Filter-based Dimensions
Filter-based dimension are covered in the following topics:
About Filter-based Dimensions
Filter-based dimensions allow Data Explorer to combine into a single result (graph and table) a set of time-series that can each have different filters applied. You can specify a nominally unlimited number of series, each with its own specific filters; collectively this set is referred to as a single filter-based dimension (of which you can have only one at a time). As shown in the screenshot below, for example, you might want to look at the traffic across your network to two specific ASes and compare the balance of traffic to each AS from two different source countries.
Note that a query can’t mix filter-based dimensions with regular (“preset”) group-by dimensions. Any dimensions that you already have in the group-by selector will be overridden when you save a filter-based dimension.
Filter-based Tab
Filter-based dimensions are configured on the Filter-based tab of the Group By Dimensions dialog (see Query Dimension Dialogs). The tab includes the following elements:
- Enable Filter-based Dimension: A switch that turns on/off filter-based dimensions. When the switch is on, the dimension configured on the Filter-based tab overrides the group-by dimensions set with the Preset tab.
- Dimension Name: A user-specified name string for the dimension. The name will be displayed as the query title in the display area (see Chart Display UI) and in the Group By Dimensions selector in the Query pane (see Query Dimension Selectors).
- Auto-Add Other Series: Include a series for all traffic that has been filtered out of the series configured in the Series section.
- Series: A set of configuration UI for the series that make up the filter-based dimension (see Series UI).
Series UI
The controls of the Series pane of the Group By Dimensions dialog are similar to those of the Ad-Hoc Filter Groups pane of the Filtering Options Dialog. The interface includes the controls described in the topics below.
Series Controls
The following UI elements are high-level controls for the set of series that make up a filter-based dimension:
- Remove All Series: Removes all filter groups. Located at top right of Series pane.
- Add Series button: Adds a new filter group to the set. Located below the last series.
Series-level Controls
A single container for one or more individual filters, which includes the following controls:
- Name: A user-specified string for the name that will be used to identify the series in results table and in the key of the graph.
- Include/Exclude: A drop-down selector that determines whether results that match the group are included or excluded from the results of queries to which the filter set is applied.
- All/Any: A drop-down selector that, if there are multiple filters in a filter group, determines the conjunctive operator used to join those filters:
- All (default) is used to AND the filters.
- Any is used to OR the filters. - Remove Series: Removes the entire filter group.
- Add Nested Group: Adds a filter group that is nested within the series.
- Add Condition: Adds a filter to the group.
Filter Controls
An individual filter in a group, which includes the following controls:
- Dimension selector: Opens a dialog enabling you to choose a dimension on which to filter (see Dimension Selector Dialog).
- Direction button (shown only when the filter dimension supports directionality): toggles between source and destination.
- Operator: The operator to apply in the filter. Options vary depending on dimension, but may include:
- equals/does not equal;
- greater than/less than;
- contains/does not contain (case insensitive);
- matches regex/does not match regex. - Value: The value to match.
- Remove (X): Removes the filter from the group.
Dimension Selector Dialog
The dimension selector presents a list from which you can choose a dimension on which to filter. The dialog includes the following elements:
- Filter field: Filters the available dimensions to show only those whose name contains the entered text.
- Dimension lists: Lists of the available dimensions in each category (source, destination, etc.; see Dimension Categories).
Note: For descriptions of the available dimensions, see Dimensions Reference.
Setting a Filter-based Dimension
To set a filter-based dimension for a query in Data Explorer:
- In the Query pane of the Data Explorer sidebar, click in the Group By Dimensions selector (see Query Basic Options).
- In the resulting Group By Dimensions dialog, click the Filter-based tab.
- Click the Enable Filter-based Dimension button to activate the controls of the tab.
- Fill in the Dimension Name field, and specify the Auto-Add Other Series button (see Filter-based Tab).
- Configure the series that you want to appear in the graph and table (see Series UI).
- Click Save to apply the Filter-based dimension, which will appear in the Group By Dimensions selector. The filter-based dimension will override any preset dimensions specified for the query.
Metrics Dialog
The Metrics dialog, used to specify multiple metrics for a query, is covered in the following topics:
About the Metrics Dialog
A metric is a combination of a unit (e.g. a bit) with a method of calculation (e.g. average per second) to create a quantifiable measurement (average bits/second). In Data Explorer, the Metrics Dialog determines how metrics are used in the chart and table that return results from a query:
- Primary metric: The metric that is plotted on the positive Y axis in the chart, and by which the table will initially be sorted.
- Secondary metric: If configured, this metric is plotted on the negative Y axis in a compound query (see Compound Queries).
- Additional metrics: Multiple additional metrics may be chosen to appear as columns in the results table (see Data Explorer Table).
The Metrics Dialog is accessed via one of the following:
- The Customize Metrics/Edit Metrics link that appears just below the Metrics Selector in the Query Pane (see Query Basic Options).
- The Custom option on the drop-down Metrics menu (see Query Basic Options).
Metrics Dialog UI
The Metrics Dialog includes the following UI elements that are used to set primary, secondary, and additional metrics:
- Close button: Click the X in the upper right corner to close the dialog. All elements will be restored to their values at the time the dialog was opened.
- Use Preset Selections For: A drop-down menu that lists primary metrics. When you choose a primary metric any existing metrics in the Selected Metrics list are removed and the chosen metric is added to the list along with a set of additional metrics chosen by Kentik to supplement the primary metric.
- Clear Selections button: Clears all metrics from the Selected Metrics list.
- Filter field: Filters the list of available metrics to those whose name contains the entered text.
- Selected Metrics list: A list of the metrics currently selected from the Available Metrics list:
- Change order: Drag the handle at the left of each selected metric to change the order in which the metrics are displayed in the results table.
- Set Threshold: Use the threshold icon at the right of each metric to open the Set Minimum Threshold popup for that metric, which enables you to exclude from the results all series whose value for the metric is not at least the minimum value that you specify. The icon turns from gray to orange if a threshold has been set.
Note: To be included in results a series must meet all of the currently set thresholds. - Available metrics list: A list of all of the metrics that are currently supported by Kentik Detect, grouped into unit-based categories (see General Metrics).
- To select all metrics in a given category (e.g. Bits/s or Unique IPs), check the box at the right of the category name.
- To select all calculations (e.g. Average, 95th percentile, and Max) of a given metric, check the box at the right of the metric name.
- To select an individual metric (e.g. Average Flows/s) check the box to the left of the individual metric in the list.
Notes:
- TCP metrics are available only when the currently selected devices (see Devices Pane Settings) include one or more hosts.
- The categories of Bits/s and Packets/s are broken into subcategories based on where flow is sampled: Ingress, Egress, or both. - Primary Display & Sort Metric: A drop-down menu used to choose the metric that is plotted on the positive Y axis in the chart, and by which the table will initially be sorted.
Note: For TCP (host) metrics related to percents and latencies (e.g. % Retransmits, Server latency, etc.) the menu items include the option to sort using Kentik Intellisort, in which the rows of the result table are sorted based on assumptions about which results would logically be the highest priority for users to know. - Secondary Display Metric (shown only when the Selected Metrics list includes metrics from more than one category): A metric that will be plotted on the negative Y axis in a compound query (see Compound Queries).
- Sort Secondary Metric Separately (shown only if Secondary Display Metric is set to something other than None): If the switch is on, the results table will include a second tab in which the rows are sorted by the secondary metric.
- Cancel button: Cancel changes to the selected metrics and exit the dialog. The current query’s metrics will be restored to what they were when the dialog was opened.
- Save button: Save changes to the selected metrics and exit the dialog.
Explorer Chart Display
The chart display area is used to display a visualization of the results of the current query as a time-series graph and a table. The display area is covered in the following topics:
Note: The visualization types supported in the portal’s various data views are covered in Chart View Types.
Chart Display UI
The chart display area contains a number of UI elements in addition to the visualization (chart) itself, some of which are above the chart and others below.
The following elements are found above the chart:
- Query title: The name for the query whose visualization is displayed in the display area. The name is refreshed each time changes to the query are applied with the Run Query button at the top of the sidebar.
- Save View button: Opens the Add Saved View dialog (see Saved Views).
- Pivot: Opens the Pivot dashboard (in a separate window or tab) and presents the traffic represented in the chart within a set of dashboard panels showing different views of the underlying data, such as source and destination IP, port, ASN, country, and devices.
Note: To see a Pivot dashboard for an individual table row (see Data Explorer Table) rather than for all returned results, click Pivot on the action menu at the right of that row (see Explorer Table Actions). - View Type: A drop-down menu used to set the type of visualization to display in the display area; for a list of the options see Chart View Types.
- Refresh: Updates the currently displayed graph and table. The graph and table will only change if the query is set to a relative timespan (Lookback).
- Options menu:
- Add to Dashboard: Opens a dialog allowing the current Data Explorer settings to be displayed as a panel on the dashboard. See Add View to Dashboard.
- Export: Exports the information represented by the display area’s graphic to a file, either a graphic image or table data. See Export Chart or Table.
- View SQL: A nested menu used to go to the Query Editor, where the SQL Query field will be populated with the query for this panel.
Note: This option is now deprecated. For additional information, contact Kentik support.
- Show API Call: Provides access to the query API dialogs (see Show API Call), which contain cURL and JSON that can be used to return the current content of the display area from the Kentik Query API. The code in the dialogs can be copied and pasted to enable programmatic access to Kentik Detect.
- Share View: Provides access to the Share Explorer View, which contains a URL for the current view in the data display area that can be copied and shared.
The following elements are located below the chart:
- Last Updated indicator: The date-time at which the query represented in the chart was most recently run.
- Live Update/Stop Updating: Toggles the query in and out of Live Update Mode. The control changes depending on the current mode:
- Live Update: When the query is not in Live Update mode, the control is a drop-down menu for selecting the interval (60, 90, or 120 seconds) for live updating.
- Stop Updating: When the query is in Live Update mode, the control is a button that stops live updating.
Note: Live Update mode is only available when the Time pane is set to Lookback (see Time Pane Settings). - Sync Axes (only when Bi-directional Charting is on or Secondary Metric is not None): Sets the scale of both left and right axes to the same numeric values (even if the metric is different).
Note: The greater the difference in value between the scale of the two axes, the greater the likelihood that syncing the axes will effectively hide data plotted against the lower-value axis. - Resize bar: A gray horizontal bar that you can drag up or down to change the vertical allocation of the display area between the chart and the table.
Data Explorer Chart
The Data Explorer chart is a visualization based on traffic whose flow records are stored in the Kentik Data Engine (KDE). The available visualization types are covered in Chart View Types.
Most (but not all) portal visualizations are based on the top records returned from the current query, as measured by the metric selected in the Query pane (see Query Basic Options). Many are based on time-series data, plotted over a time range (see Time Pane Settings) represented on the horizontal axis, with the metric represented on the vertical axis.
Each plot in the visualization corresponds to a row in the table that appears under the chart (see Data Explorer Table). The number of table rows plotted in the chart depends on the visualization depth (see Query Advanced Options). Each plotted row is indicated in the table with a colored disc.
The chart in the display area is dynamic:
- Hover over any line in a line chart or area upper boundary in a time series stacked graph to see a popup containing data for a specific record at a specific point in time.
- Drag and release in the chart to select a portion of the time range to zoom in on. When zooming:
- the Time pane is automatically set to From + To with the start and end times defined based on the zoomed region;
- the graph and table in the display area, along with its associated URL, update so that the zoomed range can be shared; and
- a Zoom out button appears at the upper right of the graph, which can be clicked to zoom out to the previous time range. - Clicking on the colored disc at the left of any row in the table will hide the area or line corresponding to that row from the chart. The disc will turn into a circle. Click the circle to restore display of the line or area in the chart.
Note: If at least one AS group exists in your organization (see About AS Groups), the Use AS Groups switch is on in Query Advanced Options, and a query’s group-by dimensions include destination and/or source ASN, then results from all ASes in each AS group will be summed for top-X evaluation, graph plotting, and display in the results table (see Table AS Grouping).
Live Update Mode
Live Update mode allows you to optionally set an interval — 60, 90, or 120 seconds — at which the Data Explorer will automatically refresh the graph and table. The countdown to refresh starts over each time you apply changes and the new result is returned in the display area. To enter Live Update, choose the desired update interval from the drop-down Live Update menu (see Chart Display UI). To leave, click the Stop Updating button.
Notes:
- Live Update mode is only available when the sidebar’s Time pane was set to Lookback when the Run Query button was most recently clicked (see Time Pane Settings).
- Unless the Time pane is set to Lookback when you click Run Query, Data Explorer will automatically exit Live Update mode.
Data Explorer Table
The query results displayed as a visualization in the chart display area are also presented as a table, which is covered in the following topics:
Explorer Table Overview
The Data Explorer table lists (in descending order) the values of selected metrics for the results returned from the current query. The last row (at bottom) will show the combined total of all records returned from the query. The table rows that are marked with a colored disc at left are those that are plotted in the chart above (the number of plotted rows is determined by the Visualization Depth setting; see Query Advanced Options).
The location in which the table is displayed depends on the current view type (see Chart View Types):
- When the view type is a graph or chart, the table is shown below the chart display area (see Explorer Chart Display).
- When the view type is set to Table, the table alone is displayed without a graph or chart. In this mode, the table can be exported (see Export Chart or Table) or added to a dashboard (see Add View to Dashboard).
- The table is not shown when the view type is Matrix.
Notes:
- The number of rows in the results table that accompanies visualizations is dependent on the Visualization Depth setting and limited to a maximum of 350 unless the view type is Table (may include up to 50,000 rows depending on group-by dimension and metric).
- When displaying results from a compound query (see Compound Queries), multiple tables are used, each on a separate tab corresponding to one axis (left/right) and/or direction (positive/negative).
Table AS Grouping
If at least one AS group exists in your organization (see About AS Groups), the Use AS Groups switch is on in Query Advanced Options, and a query’s group-by dimensions include destination and/or source ASN, then results from all ASes in each AS group will be summed for top-X evaluation, graph plotting, and display in the results table. If a table row represents a group it will include a group icon at the left of the group name; click the icon or name to pop up a list of the ASes in the group.
Explorer Table Columns
The left-most columns of the table always correspond to the dimensions selected in the Query pane (see Query Pane Settings).
The other columns depend on the metrics currently selected in the Query pane with either the drop-down Metrics menu (see Query Basic Options) or the Metrics Dialog. The dialog allows you to customize which columns are shown, but if you don’t customize then in most cases the default columns for a given metric will include the following:
- Average
- 95th Percentile
- Max
- Last Datapoint
Notes:
- The table will include a row (at bottom) for the combined total of all records returned from the query.
- If Historical Overlay is on (see Query Advanced Options) the table will also include a row for historical values.
- The Last Datapoint column gives the value of the datapoint at the end of the time series represented in the chart/table.
Explorer Table Actions
A number of actions can be taken in the table to change the display of information in the table and also the corresponding chart (see Data Explorer Chart):
- Hide/show row: Clicking on the colored disc at the left of a given row will hide the area or line corresponding to that row from the chart, and the disc will turn into a circle. Click the circle to restore display of the line or area in the chart.
- Pivot: Choosing Pivot from the Action menu at the right of each table row opens a Pivot dashboard that is filtered based on the values of the row’s dimension columns. The dashboard presents the traffic represented in the row within a set of dashboard panels showing different views of the underlying data, such as source and destination IP, port, ASN, country, and devices.
Note: To see a Pivot dashboard for the traffic in the entire table rather than for an individual row, click the Pivot link above the chart (see Chart Display UI).
- Add filter to query: Choosing Include or Exclude from the Action menu will create filters in the Filtering pane corresponding to the values in the row’s dimension columns. The filters will be applied when you click the Run Query button. The operator used for the filter depends on the dimensions currently selected in the Query pane:
- for Include the filter will use either the = or LIKE operator;
- for Exclude the filter will use either the <> or NOT LIKE operator. When the changes are applied, the row will be excluded from the chart. - Add filter and change dimension: Choosing Show by from the Action menu at the right of each table row results in two combined actions:
- adds an “include” filter as described in “Add filter to query” above;
- opens a Show By Dimension dialog that is identical to the Group By Dimension selector described in Query Dimension Selectors. The dimensions selected in this dialog will replace the dimensions previously shown in the Query pane, and will be applied when you click the Run Query button.
Explorer View Options
The queries created and displayed in Data Explorer can be saved to a panel in the Dashboard or shared with other Kentik Detect users. The dialogs related to these capabilities are covered in the following topics:
Add View to Dashboard
The Add to Dashboard dialog enables you to choose a new or existing dashboard to which you can save a panel whose view corresponds to the current view in Data Explorer. The procedure for adding a Data Explorer view to a dashboard panel is covered in Add Panel From Explorer.
Export Chart or Table
Choosing Export from the Title pane’s Options menu allows you to export to a file the information represented by the chart and/or table in the display area. The information can be exported either as chart, a table, or both. The available file formats depend on the choice of output.
If Data Explorer is not currently displaying the results of a compound query (see Compound Queries) then the following export options are available:
- Chart + Legend: Export, as a single PDF, both the visualization and the results table.
- Chart Image: Export, as either bitmap (PNG) or vector (SVG), just the visualization.
- Data: Export, as CSV, the data for either the visualization or the results table.
If Data Explorer is currently displaying the results of a compound query, then in addition to the options listed above the Export submenu will include a Series Data option (as shown below) from which you can choose to export either the visualization (as PNG) or the results table (as CSV) associated with each individual axis of the query results.
After you choose the file format, you’ll see a notification explaining that the file is being prepared. When the file is ready another notification will give the link for downloading the file.
Show API Call
The Show API Call item on the Options menu (see Chart Display UI) provides access to a set of dialogs that contain code (cURL or JSON) that can be used to return the current view (content of the display area) from the Kentik Query API. The code in these dialogs can be copied and pasted to enable access to Kentik Detect programmatically rather than via the portal.
Query API code is accessed via the following dialogs, which each display code in a text field from which it can be copied either manually or using the Copy to Clipboard button:
- For Chart (cURL): Opens a dialog containing the cURL for returning an image of the Data Explorer’s current chart from a CLI such as Terminal. Equivalent to the Query Chart Method of the Kentik Query API.
- For Data (cURL): Opens a dialog containing the cURL for returning the Data Explorer’s current table from a CLI such as Terminal. Equivalent to the Query Data Method of the Kentik Query API.
- JSON Input: Opens a dialog containing JSON that can be used in the Query Data Method.
When using the cURL, the following placeholders must be replaced with the appropriate information:
- Replace <YOUR_EMAIL_HERE> with the email address used to register you as a Kentik Detect user.
- Replace <YOUR_API_TOKEN_HERE> with your API token, which you’ll find on your User Profile.
- If the cURL is for a chart, replace <CHOOSE ONE OF:pdf|png> with the desired file type.