|Note: These settings are accessed via the Flow Tags page, which is displayed to Admin users only (hidden from Member users).|
Kentik supports the use of user-defined flow tags that can make it more convenient to query the Kentik Data Engine (KDE). The following topics cover the setup of flow tags in the Kentik portal :
Note: For information on querying with tags, see Tag-based Queries.
About Flow Tags
Flow tags are labels that are applied to flow data, based on user-defined criteria, as Kentik ingests the data into the Kentik Data Engine (KDE). Because tags are applied as the data is ingested, a tag must already be created before it can be applied to a given flow record (tags cannot be applied retroactively to rows that already exist in the database). However, because Kentik stores complete flow records rather than aggregated summaries, historical queries are not limited to flow attributes that have been defined in advance with tags. In other words, tags enhance your querying options, but they in no way limit or filter the data that is stored in KDE.
Flow tags are created in the portal (see Adding a Flow Tag) or via one of the following Kentik V5 APIs (see About the V5 APIs):
Tag Application at Ingest
When each flow record is sent to Kentik from a given device, the data is evaluated to determine if any of its attributes match any of the existing tags set up by a given customer.
- If all of the values specified in the ANDed tag fields for a given tag are matched in SRC-related flow fields (i.e. SRC IP, SRC port, ASN path associated with SRC IP, or communities associated with SRC IP) then the tag's name is appended as text to the existing tags (if any) in the src_flow_tags column for that flow.
- If all of the values specified in the ANDed tag fields for a given tag are matched in DST-related flow fields (i.e. DST IP, DST port, ASN path associated with DST IP, or communities associated with DST IP) then the tag's name is appended as text to the existing tags (if any) in the dst_flow_tags column for that flow.
Note: Because the tag fields are ANDed for each of the two comparison operations described above, a tag will be applied only when all tag fields are matched.
The result of the tagging process is that the src_flow_tags and dst_flow_tags columns of each device's main table contain a delimited list of tags that can be searched as part of a KDE query.
Note: For greater detail about how tags are applied to flow in KDE, see Tag-based Queries.
Flow Tags Page
The Flow Tags page is home to the Flow Tag List, which is a table that lists all of your organization’s existing tags. To view the page, choose Settings from the main Kentik navbar, then Flow Tags under Data Enrichment. The Flow Tags page is covered in the following topics:
Flow Tags Page UI
The Flow Tags page has the following main UI elements:
Flow Tag List
The Flow Tag List is a table that lists all of your organization’s currently defined flow tags. The table's columns provide the following information and actions for each tag:
- Tag Name: The name of the flow tag as specified at the time the flow tag was created. Click anywhere on a tag’s row to open the Edit Tag dialog for that flow tag.
- Last Edited: The date on which the tag was last edited. Hover over the date for a tool tip with the full date-time (UTC).
- Edited by: The email address of the person who last modified the tag.
- Created: The date on which the tag was created. Hover over the date to see the full date-time (UTC).
- ID: The system-generated unique ID assigned to the tag when it was created.
- View in Chart: Opens the Total Matching Traffic Dialog for either the Source or the Destination.
- Remove (trash icon): Opens a confirmation dialog that allows you to remove the tag from the Kentik portal.
- Click on the column headings for Tag Name or ID to sort the list (ascending or descending).
- To see additional information about a given flow tag, click anywhere in the row for that flow tag, which opens an Edit Tag dialog where you can review settings (see Editing a Flow Tag).
Total Matching Traffic Dialog
The Total Matching Traffic dialog is opened via one of the View in Chart buttons, either Source or Dest. The dialog displays a chart showing the total traffic (expressed as max bits/second), both historically and for the last 24 hours, that had matches for this tag in either source flow (Source button) or destination flow (Dest button).
The dialog includes the following UI elements:
- Close buttons: To close the dialog, click the X in the upper right corner or the Close button at lower right.
- Applied Filters: A button that opens a popup in which you can see which filters are applied to the traffic shown in the chart.
- View Type: A drop-down menu used to set the type of visualization used for the graph (defaults to Line Chart). For descriptions of the options, see Chart View Types.
- Chart: The visualization of traffic (using the current view type). Hover over any part of the chart for more precise information.
- View in Explorer button: Opens Data Explorer for further exploration of the device's traffic. The query settings will be set to show the same traffic that is shown in the Total Matching Traffic dialog.
Tag Admin Dialogs
Adding or editing a flow tag via the Kentik portal involves specifying information in the fields of the tag admin dialogs, which are covered in the following topics.
About Tag Dialogs
The Kentik portal uses tag admin dialogs to collect and display flow tag information. The required information is entered into the fields of either of the following dialogs:
- Add Tag when adding a new tag to Kentik.
- Edit Tag when editing an existing tag.
Tag Dialogs UI
The Add Tag and Edit Tag dialogs share the same layout and the following common UI elements:
- Close button: Click the X in the upper right corner to close the dialog. All elements will be restored to their values at the time the dialog was opened.
- Tab selectors: Choose the tab to display: General, Device Matching, IP Matching, BGP Matching, or Other (see tab-specific topics below).
- Cancel: A button that cancels the add tag or edit tag operation and exits the dialog. All elements will be restored to their values at the time the dialog was opened.
- Add Tag (Add Tag dialog only): A button that saves settings for the new tag and exits the dialog.
- Save (Edit Tag dialog only): A button that saves changes to tag settings and exits the dialog.
Tag Field Definitions
The fields of the tag admin dialogs are used to specify and display flow tag settings. These fields are described in the following topics:
About Tag Fields
Each tag admin dialog is broken into the tabs covered in the topics below, each of which is made up of several fields. Creating a flow tag requires specifying a name in the Tag Name field and specifying additional tag attribute fields that will be evaluated for a match with fields in the incoming flow (or with values derived from fields in the flow). The fields that can be used to define tags are described in the topics below.
The validation columns in each table below indicate whether or not the following validations will be applied to a given tag field:
- Comma: Comma-delimited list
- Database: Database patterns (e.g. % and _)
- Regex: Regex
Note: For additional information on validation of tag field values, see Tag Field Validation.
General Tag Settings
The following table shows the settings on the General tab of tag admin dialogs:
|Tag Name||The string that will be added to the src_flow_tags and/or dst_flow_tags column in the main KDE table of the device sending flow when a match is found in the flow for the values in any of the following fields.
- A tag name must be from 2 to 20 characters long: alphanumeric, hyphen, or underscore (no spaces).
- A tag name must be unique, but tags whose names contain a common string can be ORed in a query (e.g. "tag1" and "tag2" both contain "tag").
Tag Device Matching
The following table shows the settings on the Device Matching tab of tag admin dialogs:
|Device Name||Results in a match if this value appears within the name or equals the IP address of a device that has been configured to send flow records to Kentik. If there's a match, the tag is applied to both src_flow_tags and dst_flow_tags columns.||Yes||Yes||No|
|Device Type||Type of device to match (router, host, etc.; see Supported Device Types).||Yes||No||Yes|
|Site||Results in a match if this value appears within the name of a site to which the device sending the flow record to Kentik has been assigned (see About Sites).||Yes||No||Yes|
|Interface Name||Results in a match if this value appears within the name or description of a source or destination interface. If there's a match, the tag is applied to the src_flow_tags column if the received flow shows traffic entering on the interface, and a tag is applied to the dst_flow_tags column if the received flow shows traffic leaving on the interface.||Yes||Yes||Yes|
Tag IP Matching
The following table shows the settings on the IP Matching tab of tag admin dialogs:
|IP address (IP/CIDR format)||Expressed in IPv4 or IPv6 CIDR notation (e.g. 22.214.171.124/24; see CIDR Notation), this value will result in a match if it corresponds to a range of IP addresses in the flow, either source (SRC IP) or destination (DST IP). If there's a match, the tag is applied to both src_flow_tags and dst_flow_tags columns.
Note: This field can contain up to 249 IP/CIDR items in a comma-delimited list.
|Port||Results in a match if this value appears within a port number in the flow, either source (SRC Port) or destination (DST Port).||Yes||No||No|
|TCP Flag||An integer number between 0 and 255 representing an 8-bit binary bit pattern. At ingest this pattern is used as a bitmask that is ANDed with the composite (ORed) bit pattern of the TCP flags set in the flow. A match will result if the value in both the flow bit pattern and the bitmask is 1 at any of the eight places.||No||No||No|
|Protocol Number||Results in a match if this value is the same as the protocol of the traffic represented by the flow. The protocol of TCP is 6, and of UDP is 17.||Yes||No||No|
Tag BGP Matching
The following table shows the settings on the BGP Matching tab of tag admin dialogs:
|Last-hop (origin) ASN||Results in a match if this value is the same as the last ASN (16- or 32-bit) in the path in the routing table for either the source (SRC IP) or destination (DST IP).||Yes||No||No|
|Last-hop (origin) AS Name||Results in a match if this value represents the name corresponding to the last ASN in the path in the routing table for either the source (SRC IP) or destination (DST IP).||Yes||No||Yes|
|Next-hop ASN||Results in a match if this value is the same as the ASN (16- or 32-bit) of the next hop router based on AS path.||Yes||No||No|
|Next-hop AS Name||Results in a match if this value represents the name corresponding to the ASN of the next hop router based on AS path.||Yes||No||Yes|
|Next-hop IP||If a CIDR grouping (IPv4 or IPv6) is specified, a match can be on any address within that grouping. If no CIDR grouping is specified a match requires an exact IP.
- CIDRs may be expressed in "short form" (e.g. 1::2/127).
|BGP AS Path||Results in a match if this value is the same as the BGP AS path in the route (see Specifying BGP Tag Fields).||Yes||No||Yes|
|BGP Community||Results in a match if this value is the same as the BGP community of BGP route. May be specified with a form of regex (see Specifying BGP Tag Fields).||Yes||No||Yes|
Specifying BGP Tag Fields
Kentik’s collection of BGP data allows incoming flows to be assigned tags that match communities and AS paths or partial paths. Flow tags are applied (separately for source and destination) at the time that flow is ingested into KDE. They can then be used to narrow query results by applying flow tags using source and destination filter functions in Data Explorer.
Note: A given tag is applied only to flows that arrive after the tag was created. A new tag may take up to 20 minutes to take effect.
Matches on the BGP-related tag fields are made on substrings. For ASN and Next-hop ASN, the string(s) to match are specified in a simple comma-delimited list. For both the BGP AS Path and BGP Community fields, the specified values are also evaluated using a subset of standard regex (see table below):
- BGP AS path tags: Entering "10" in the as-path field will match any path that includes "10", "100", "010", etc. Using regex, a value of "_10_" will match only paths that include ASN 10, including "10 ", " 10", and " 10 ". Also allowed are tags where as-path is specified as, for example, "_10 100_".
- BGP community tags: Flow tags on communities are similar to tags on AS paths except that they also support the use of regex periods. This allows you to specify, for example, "2000:1...." to find any flow with community 2000:1xxxx in it.
The following table shows the regex special characters that are supported when specifying the BGP AS Path and BGP Community:
|_ (underscore)||- Start of string
- End of string
- " " (space)
|.||Any single character, including white space.|
||The characters, or a range of characters separated by a hyphen, contained within square brackets.|
|'||The character or null string at the beginning of an input string.|
|?||Zero or one occurrence of the pattern containing the question mark.|
|$||End of string.|
|*||Zero or more sequences of the preceding character. Also acts as a wildcard for matching any number of characters.|
|+||One or more sequences of the preceding character.|
|()||Used for nesting of expressions.|
Note: For BGP community and AS path tags, any spaces at the beginning or end of the input field, and also before and after each comma will be removed.
Other Tag Settings
The following table shows the settings on the Other tab of tag admin dialogs:
|MAC Address||Results in a match if this value matches source or destination Ethernet (L2) address.||Yes||No||No|
|Country||Results in a match if this value includes a two-letter country code associated with the source or destination IP of the flow.||Yes||No||No|
|VLAN(s)||Results in a match if this value includes a VLAN ID associated with the source or destination IP of the flow.||Yes||No||No|
Note: The validation codes used in the table above are defined in Tag Field Validation.
Tag Field Validation
The following general considerations apply to the validation of values in the tag fields described in Tag Field Definitions:
- Some fields support the entry of multiple values as a comma-delimited list (see tables above).
- Commas are supported only as list delimiters (not in actual values or regex).
- Some fields support the use of database patterns (see tables above).
- In fields where regex is supported (see table below), a period (".") may be used in place of a comma.
- The BGP AS Path and BGP Community tags use PostgreSQL POSIX Extended Regular Expressions. For additional information, see Specifying BGP Tag Fields.
- All other tags that support regex use PostgreSQL Advanced Regular Expressions.
Note: Documentation for PostgreSQL regex and database patterns can be found at PostgreSQL documentation:
- Database patterns are documented under "LIKE."
- Regular expressions are documented under "POSIX Regular Expressions."
Add or Edit Tags
Flow tags are created and edited via the Flow Tags Page of the Kentik portal (from the main navbar, select Settings and then Flow Tags). The add/edit process is covered in the following sections:
- Tags can also be added and edited with either the Tag API or the Batch API.
- The preferred alternative to creating a MYNETWORK tag is to group-by or filter for traffic whose Traffic Profile dimension equals Internal (see Network Classification Dimensions). For more information about adding a MYNETWORK tag, see Add a MYNETWORK Tag.
Adding a Flow Tag
To add a new tag:
- Go to the Flow Tags page (Settings » Flow Tags).
- Open the Add Tag dialog by clicking the Add Tag button.
- On the General tab, name the tag in the Tag Name field.
- Specify the values of the tag fields that will be evaluated for a match with the properties of the incoming flow (see Tag Field Definitions).
- Click the Add Tag button (bottom right) to save the new tag with the currently specified values and close the dialog.
Note: A tag name must be unique, but tags whose names contain a common string can be ORed in a query (e.g. "tag1" and "tag2" both contain "tag").
Editing a Flow Tag
To edit an existing tag:
- Go to the Flow Tags page (Settings » Flow Tags).
- In the Flow Tag List, open an Edit Tag dialog by clicking anywhere in the row of the tag that you'd like to edit.
- Edit the tag fields that you'd like to change (see Tag Field Definitions).
- To save changes, click the Save button at bottom right. The dialog will close.