Query API

The Kentik V5 Query API enables programmatic querying of network data (flow records, BGP, selected SNMP OIDs) from your company's devices (routers, hosts, etc.) stored in Kentik. The V5 Query API is covered in the following topics:

• About V5 Query API
• V5 Query Methods
• Query API JSON
• Query Chart Method
• Query URL Method
• Query Data Method
• Query SQL Method

Notes:
- For an overview of all Kentik APIs, see APIs Overview.
- For documentation of the V5 Admin APIs, see V5 Admin APIs.
- For information on using the Kentik V5 Query API with cURL, see API Access Via cURL.
- For documentation of the V5 Alerting APIs, see V5 Alerting APIs.
- To test V5 APIs, see V5 API Tester.
- For assistance using any API version please contact support@Kentik.com.

The Kentik V5 Query API includes methods for returning information from the main tables of the Kentik Data Engine (see KDE Tables), either as data in JSON or as image file data representing charts like those displayed in Data Explorer.

Kentik V5 Query APIs may be accessed as follows:

• V5 API Tester: Use the V5 API Tester (ioDocs), which allows you to try the API calls on real data from your Kentik service.
• Command line: Use cURL (https://curl.haxx.se/) in an environment like Terminal. See API Access Via cURL.
• Programmatically: Pass a request body to a V5 API endpoint using any application language that supports the HTTP protocol.

The following table shows the methods that are available in the V5 Query API (click on the topic link for more information about a specific method):

Notes:
- Endpoints shown above are relative to a path that depends on the region for which your organization is registered with Kentik: https://api.kentik.com/api/v5 for US and https://api.kentik.eu/api/v5 for EU.
- The Query SQL method requires detailed knowledge of PostgreSQL and of Kentik's implementation of subqueries (see Querying KDE). To return KDE data as JSON, most API users will achieve more accurate results with less development effort using the Query Data call.

The JSON used for requests and responses by the V5 Query API methods is covered in the following topics:

• Query API Request JSON
• Query JSON as cURL
• Query API Response JSON

The Query Chart, Query Data, and Query URL methods of the V5 Query API both use JSON in the request body (the Query SQL Method uses SQL in the request body instead). The JSON array passed into these calls tells Kentik what data to return and how to return it. This JSON is mostly the same for all three calls.

You can create the JSON array for a Query Chart, Query Data, or Query URL call in either of the following ways:

• Build the JSON from scratch based on the information in the tables below.
• Get the JSON from the Kentik portal:
  1. In the portal's Data Explorer (click Data Explorer on the portal's main navbar), define a query by specifying the fields in the panes of the sidebar, including Query, Time, and Devices as well as, optionally, Filters. Click the Apply Changes button.
  2. When the results of the query are rendered, click the drop-down options menu at the upper right of the graph (see Explorer Title Pane), choose Show API Call, then choose Data or Chart. The Query API example dialog will open.
  3. Click the JSON button at top. The query JSON will appear in the code field.
  4. Click the Copy button at bottom. The JSON will be copied to the clipboard.
  5. Paste the JSON to use it in a Query Data call in cURL (see below) or other setting that supports programmatically generated HTTP calls.

The following example illustrates the objects used by the Query Chart, Query Data, and Query URL methods (with placeholder values highlighted), which are described in the sections below.

{
  "queries": [
    {
      "query": {
        "viz_type": "matrix",
        "show_overlay": false,
        "overlay_day": -7,
        "sync_axes": false,
        "metric": "bytes",
        "matrixBy": [
          "dst_geo_city"
        ],
        "cidr": 32,
        "cidr6": 128,
        "pps_threshold": 500,
        "topx": 8,
        "depth": 100,
        "fastData": "Auto",
        "outsort": "max_bits_per_sec",
        "lookback_seconds": 3600,
        "time_format": "UTC",
        "hostname_lookup": "false",
        "starting_time": null,
        "ending_time": null,
        "device_name": "device_name_1, device_name_2",
        "all_selected": true,
        "filters_obj": {
          "connector": "All",
          "filterGroups": [
            {
              "connector": "All",
              "filters": [
                {
                  "filterField": "protocol",
                  "operator": "=",
                  "filterValue": "17",
                }
              ],
              "not": false,
            }
          ],
        },
        "saved_filters": [
          {
            "filter_id": 363
            "is_not": true
          }
        ],
        "descriptor": "Total",
        "aggregates": [
          {
            "name": "avg_bits_per_sec",
            "column": "f_sum_both_bytes",
            "fn": "average",
            "sample_rate": 1
          },
          {
            "name": "p95th_bits_per_sec",
            "column": "f_sum_both_bytes",
            "fn": "percentile",
            "rank": 95,
            "sample_rate": 1
          },
          {
            "name": "max_bits_per_sec",
            "column": "f_sum_both_bytes",
            "fn": "max",
            "raw": true,
            "sample_rate": 1
          }
        ],
        "query_title": "Top Source City vs Top Dest City by Max Bits/s",
        "dimension": [
          "src_geo_city"
        ]
      },
      "bucket": "Matrix",
      "bucketIndex": 0,
      "isOverlay": false
    }
  ],
  "imageType": "image_file_format"
}

The highest-level object of the V5 Query JSON contains the following two members:

The queries array typically contains one element consisting of a query object and the associated objects shown in the following table. To return data or a chart for multiple data-series (see Compound Queries), include multiple elements, one for each desired axis of the multi-series chart.

The query object contains the objects and arrays that define a query to be run on Kentik:

The filters_obj object is a container for a set of one of more filter groups for a Query Chart or Query Data query. For more information on filters in a query see About Filters.

The filterGroups array defines a group of one or more filters for a Query Chart or Query Data query. For more information on filters in a query see About Filters. Each member of the array contains the following objects:

The filters array is used to define one or more individual filters of a filter group in a Query Chart or Query Data query. For more information on filters in a query see About Filters. Each member of the array corresponds to an individual filter and contains the following objects:

The saved_filters array is a list of IDs referencing saved filters (custom or preset; see Saved Filters) that you wish to apply to the query.

The aggregates array defines an aggregate for a column in a table that is returned from a Query Chart query when the display type (viz_type; see Query Object) is Table. Simply put, an aggregate is the result of simple statistical analyses across a dataset.

If no aggregates are included in the Query Object then Kentik will automatically generate a set of aggregates for the query based on the value of the metric field (query object). Like user-specified aggregates, the name of one of these Kentik-generated aggregates may be specified in the outsort field (query object) to determine the aggregate object by which results will be sorted for depth and topx. The following table shows the names of the Kentik-generated aggregates that are valid in the outsort field for each metric.

The following code sample shows the JSON for a Query Chart call within the data (-d) parameter of a cURL HTTP request (with placeholders highlighted):

curl -X POST \
-H 'X-CH-Auth-Email: user@domain.suffix' \
-H 'X-CH-Auth-API-Token: user_api_token' \
-H 'Content-Type: application/json' \
-d '{ "queries": [ { "query": { "viz_type": "stackedArea", "show_overlay": true, "overlay_day": -7, "sync_axes": false, "metric": "bytes", "matrixBy": [], "cidr": 32, "cidr6": 128, "pps_threshold": 500, "topx": 8, "depth": 25, "fastData": "Auto", "outsort": "max_bits_per_sec", "lookback_seconds": 300, "time_format": "UTC", "starting_time": null, "ending_time": null, "device_name": "device_names", "all_selected": false, "filters_obj": {}, "saved_filters": [], "descriptor": "Total", "query_title": "Top Source Country by Max Bits/s", "time_type": "relative", "device_id": "2951", "num_selected": 0, "sql_string": "", "queryChanged": false, "granularity": "total", "ppsThreshold": "500", "last_run": null, "aggregates": [ { "name": "avg_bits_per_sec", "column": "f_sum_both_bytes", "fn": "average", "sample_rate": 1 }, { "name": "p95th_bits_per_sec", "column": "f_sum_both_bytes", "fn": "percentile", "rank": 95, "sample_rate": 1 }, { "name": "max_bits_per_sec", "column": "f_sum_both_bytes", "fn": "max", "raw": true, "sample_rate": 1 } ], "dimension": [ "Geography_src" ] }, "bucket": "Left +Y Axis", "bucketIndex": 0, "isOverlay": false } ], "imageType": "pdf"}' \
https://api.kentik.com/api/v5/query/topXchart | python -m json.tool

Notes:
- The above example includes placeholders (in italics) that must be replaced with actual values, including your email address (user@domain.suffix), your API token (user_api_token), and the names of the devices (comma-delimited list) on which you wish to run the query (device_names).
- The JSON (value of -d parameter) for a Query Data call would be similar but omit certain fields; see the individual field descriptions in Query API Request JSON.
- For more information on using Kentik APIs with cURL, see API Access Via cURL.

The Query Data and Query SQL methods both return an array of JSON data in the HTTP response body (the Query Chart method returns an image file instead). In both cases this data is drawn from the main tables of the Kentik Data Engine (see KDE Tables), which store flow records and associated network data (BGP, GeoIP, SNMP). The specific objects included in the arrays returned from the two methods will differ, however, even when the query that was passed into each call is based on the same underlying settings (e.g. set a query in Data Explorer, then use Show API Call to grab the query for a Query Data call vs. View SQL for a Query SQL call). That's because the results returned from the Query Data method are processed for improved accuracy (e.g. in queries involving averaging).

Note: An example of response JSON is provided in the topic for each method.

This POST method queries Kentik's KDE and returns image data for a graph similar to what is seen in the data display area of the Data Explorer in the Kentik portal (see Data Explorer Chart). The binary data for the image is returned in the HTTP response body as a JSON object with one dataURI member, the value of which is a base64 encoded string that represents a file.

Supported types for the returned image data are SVG, PDF, PNG, or JPG. The default and recommended image type is SVG, which serves the fastest, scales losslessly, and is the simplest to integrate as it can be rendered by simply injecting it directly into the DOM.

Note: To try this call, go to the V5 API Tester.

HTTP Request

The following table shows the path and HTTP request for this call (with placeholders in italics):

Note: If your organization is registered on Kentik's EU cluster, use api.kentik.eu in place of api.kentik.com in the URL above.

The request body includes a JSON array whose elements contain the information needed by Kentik to perform the query. For information about creating this array and definitions of its component parts, see Query API Request JSON.

HTTP Response

A successful response to this call includes the following elements:

• The response headers.
• The HTTP response code.
• A response body containing a JSON object with one dataURI member.

The following example shows the structure of the returned JSON (placeholder highlighted):

{
  "dataUri": "base64_encoded_string"
}

This POST method returns a URL that will open Data Explorer in the Kentik portal with the sidebar set to correspond to the query parameters defined in the fields of the request JSON, and the query result visualized (graphed) in the display area.

Note: To try this call, go to queryURL under Query methods in the V5 API Tester.

HTTP Request

The following table shows the path and HTTP request for this call (with placeholders in italics):

Note: If your organization is registered on Kentik's EU cluster, use api.kentik.eu in place of api.kentik.com in the URL above.

The request body includes a JSON array whose elements contain the information needed by Kentik to specify the settings for the query that will run when Data Explorer is opened. For information about creating this array and definitions of its component parts, see Query API Request JSON.

Note: The request body for this call is identical in every respect to a request for Query Chart Method, with the sole exception being that this call does not use the imageType param.

HTTP Response

A successful response to this call includes the following elements:

• The response headers.
• The HTTP response code.
• A response body containing a short URL (in quotes).

The following example shows an example of a URL returned from this call:

"https://portal.kentik.com/portal/#Charts/shortUrl/d3f24034b386ebf9e8ad6de8f0f715a1"

This POST method runs a query on data in Kentik's KDE (see KDE Tables) and returns results in a JSON results array.

Notes:
- Most API users will achieve more accurate results with less development effort using this method than with the Query SQL Method, which requires detailed knowledge of PostgreSQL and of Kentik's implementation of subqueries (see Querying KDE).
- To try this call, go to the V5 API Tester.

HTTP Request

The following table shows the path and HTTP request for this call (with placeholders in italics):

Note: If your organization is registered on Kentik's EU cluster, use api.kentik.eu in place of api.kentik.com in the URL above.

The request body includes a JSON array whose elements contain the information needed by Kentik to perform the query. For information about creating this array and definitions of its component parts, see Query API Request JSON.

HTTP Response

A successful response to this call includes the following elements:

• The response headers.
• The HTTP response code.
• A response body containing a results array in JSON. The elements included in the array depend on the query passed into the call.

The following example shows the results array from a simple query on top source country by maximum bits per second, abbreviated to show just two results (normally the actual number of results corresponds to the value of the depth field in the request JSON, with a minimum of 25 and a maximum of 350):

• The first result (US), which includes time-series data for a timespan of five minutes (lookback_seconds set to 300), is an example of what returns for the first number of results, where number corresponds to the value of topx field in the request JSON (see Query API Request JSON).
• The second result (AU) is an example of non-topx results.

{
  "results": [
    {
      "bucket": "Left +Y Axis",
      "data": [
        {
          "Geography_src": "US",
          "avg_bits_per_sec": 130548781.51111111,
          "key": "US",
          "max_bits_per_sec": 153616657.06666666,
          "p95th_bits_per_sec": 150816290.13333333,
          "timeSeries": {
            "both_bits_per_sec": {
              "flow": [
                [
                  1482186900000,
                  128568524.8,
                  60
                ],
                [
                  1482186960000,
                  133069482.66666667,
                  60
                ],
                [
                  1482187020000,
                  115999675.73333333,
                  60
                ],
                [
                  1482187080000,
                  153616657.06666666,
                  60
                ],
                [
                  1482187140000,
                  104022425.6,
                  60
                ],
                [
                  1482187258438,
                  148015923.2,
                  60
                ]
              ]
            }
          }
        },
        {
          "Geography_src": "AU",
          "avg_bits_per_sec": 732114.4888888889,
          "key": "AU",
          "max_bits_per_sec": 1145924.2666666666,
          "p95th_bits_per_sec": 1003315.2
        }
      ]
    }
  ]
}

This POST method runs a SQL query on data in Kentik's KDE. The query's results are returned in a JSON rows array whose elements each correspond to column in the KDE Tables.

Notes:
- This method requires detailed knowledge of PostgreSQL and of Kentik's implementation of subqueries (see Querying KDE). To return KDE data as JSON, most API users will achieve more accurate results with less development effort using the Query Data Method.
- To try this call, go to the V5 API Tester.

HTTP Request

The following table shows the path and HTTP request for this call (with placeholders in italics):

Note: If your organization is registered on Kentik's EU cluster, use api.kentik.eu in place of api.kentik.com in the URL above.

The request body includes a single parameter, which is a SQL query. You can create this query in either of the following ways:

• Write the query from scratch. For guidance on writing a Kentik-compliant query, see Querying KDE.
• Create the query in the Kentik portal:
  1. In the portal's Data Explorer (click Data Explorer on the portal's main navbar), define a query by specifying the fields in the panes of the sidebar, including Query, Time, and Devices as well as, optionally, Filters. Click the Apply Changes button.
  2. When the results of the query are rendered, click the drop-down Options menu at the upper right of the graph (see Explorer Title Pane) and choose View SQL. From the resulting menu options choose the type of view (e.g. "Total") and then the type of SQL (e.g. "Chart SQL"). The Query Editor will open, with the text of the SQL query displayed in the Enter SQL Query field.
  3. Copy the SQL to use it in a Query call in cURL (see below) or other setting that supports programmatically generated HTTP calls.

Note: Queries generated in the Query Editor may contain single quotes. Depending on the programmatic context within which copied queries are used you may need to escape these quotes. In cURL, for example, replace single quotes with the unicode equivalent \u0027 as shown in the following example WHERE clause.

-- original
WHERE i_sub_limit = 200
  AND i_start_time >= '2016-12-01 21:54'
-- escaped
WHERE i_sub_limit = 200
  AND i_start_time >= \u00272016-12-01 21:54\u0027

Request as cURL

The following code sample shows a Query SQL call as a cURL HTTP request (with placeholders highlighted):

curl -X POST \
-H 'X-CH-Auth-Email: user@domain.suffix' \
-H 'X-CH-Auth-API-Token: user_api_token' \
-H 'Content-Type: application/json' \
-d '{
  "query": "SELECT * FROM (SELECT i_start_time, dst_geo, CASE dst_geo WHEN \u0027\u0027 THEN \u0027---\u0027 ELSE dst_geo END AS Geography_dst, sum(both_bytes) AS f_sum_both_bytes, max(i_duration) as i_duration, row_number() OVER (PARTITION BY i_start_time ORDER BY sum(both_bytes) DESC) FROM all_devices WHERE ( ( ( (src_geo = \u0027RU\u0027) ) ) AND ( ( (i_dst_as_name ILIKE \u0027%kentik%\u0027) ) )) AND i_sub_limit = 200 AND i_start_time >= \u00272016-12-01 21:28\u0027 GROUP BY i_start_time, dst_geo ORDER BY f_sum_both_bytes DESC) a WHERE row_number <= 200 ORDER BY i_start_time, f_sum_both_bytes DESC"
  }' \
https://api.kentik.com/api/v5/query/sql | python -m json.tool

Note: For more information on using Kentik APIs with cURL, see API Access Via cURL.

HTTP Response

A successful response to this call includes the following elements:

• The response headers.
• The HTTP response code.
• A response body containing a rows array in JSON. The elements included in the array depend on the query passed into the call.

The following example shows the JSON rows array that would be returned from a query like that shown in the cURL example above:

{
  "rows": [
    {
      "dst_geo": "US",
      "f_sum_both_bytes": 9856,
      "geography_dst": "US",
      "i_duration": 300,
      "i_start_time": "2016-12-01T21:25:00Z",
      "row_number": 1
    },
    {
      "dst_geo": "US",
      "f_sum_both_bytes": 16000,
      "geography_dst": "US",
      "i_duration": 300,
      "i_start_time": "2016-12-01T21:30:00Z",
      "row_number": 1
    },
    {
      "dst_geo": "US",
      "f_sum_both_bytes": 26400,
      "geography_dst": "US",
      "i_duration": 300,
      "i_start_time": "2016-12-01T21:35:00Z",
      "row_number": 1
    }
  ]
}