Flow Tags

Kentik supports the use of user-defined flow tags that can make it more convenient to query the Kentik Data Engine (KDE). The setup of tags in the Kentik portal is discussed in the following topics:

• About Flow Tags
• Flow Tags Page
• Tag Admin Dialogs
• Tag Field Definitions
• Add or Edit Tags
• Add a MYNETWORK Tag

Note: For information on querying with tags, see Tag-based Queries.

Flow Tags are labels that are applied to flow data, based on user-defined criteria, as Kentik ingests the data into the Kentik Data Engine (KDE). Because tags are applied as the data is ingested, a tag must already be created before it can be applied to a given flow record (tags cannot be applied retroactively to rows that already exist in the database). However, because Kentik stores complete flow rather than summaries, historical queries are not limited to flow attributes that have been defined in advance with tags. In other words, tags enhance your querying options, but they in no way limit or filter the data that is stored in KDE.

Flow Tags are created in the portal (see Adding a Flow Tag) or via one of the following Kentik V5 APIs (see About the V5 APIs):

• To add individual tags: Tag API.
• To add tags by batch: Batch API.

When each flow record is sent to Kentik from a given device, the data is evaluated to determine if any of its attributes match any of the existing tags set up by a given customer.

• If all of the values specified in the ANDed tag fields for a given tag are matched in SRC-related flow fields (i.e. SRC IP, SRC port, ASN path associated with SRC IP, or communities associated with SRC IP) then the tag’s name is appended as text to the existing tags (if any) in the src_flow_tags column for that flow.
• If all of the values specified in the ANDed tag fields for a given tag are matched in DST-related flow fields (i.e. DST IP, DST port, ASN path associated with DST IP, or communities associated with DST IP) then the tag’s name is appended as text to the existing tags (if any) in the dst_flow_tags column for that flow.

Note: Because the tag fields are ANDed for each of the two comparison operations described above, a tag will be applied only when all tag fields are matched.

The result of the tagging process is that the src_flow_tags and dst_flow_tags columns of each device’s main table contain a delimited list of tags that can be searched as part of a KDE query.

Note: For greater detail about how tags are applied to flow in KDE, see Tag-based Queries.

The Flow Tags page is home to the Flow Tag List, which is a table that lists all of your organization’s existing tags. To view the page, choose Admin from the main Kentik Detect navbar, then Flow Tags from the sidebar at left. The Flow Tags page is covered in the following topics:

• Flow Tags Page UI
• Flow Tag List

The Flow Tags page has the following main UI elements:

• Filter field: Enter text to filter the Flow tag List. The following columns of the list are searched for a match on the string entered in this field: Tag Name, Edited By, and ID.
• Add Tag button: Opens the Add Tag dialog (see Adding a Flow Tag).
• Flow Tag List: A table listing the flow tags currently set up in your organization (see Flow Tag List).

The Flow Tag List is a table that lists all of your organization’s currently defined flow tags. The table provides the following information and actions for each tag:

• Tag Name: The name of the Flow Tag as specified at the time the Flow Tag was created. Click on a name to go to the Edit Flow Tag page for that Flow Tag.
• Last Edited: The date on which the subscription was last edited. Hover over the date for a tool tip with the full date-time (UTC).
• Edited by: The email address of the person who last modified the tag.
• Created: The date on which the subscription was created. Hover over the date for a tool tip with the full date-time (UTC).
• ID: The system-generated unique ID assigned when the Flow Tag is created.
• View in Chart: Opens the Total Matching Traffic Dialog.
• Delete: Opens a confirming dialog that allows you to remove the tag from Kentik.

Click on a column heading to sort the list (ascending or descending).

Note: To see additional information about a given flow tag, click anywhere in the row for that flow tag, which opens a Edit Tag dialog where you can review settings (see Editing a Flow Tag).

The Total Matching Traffic dialog is opened via one of the View in Chart buttons, either Source or Dest. The dialog displays a chart showing the total traffic (expressed as max bits/second), both historically and for the last 24 hours, that had matches for this tag in either source flow (Source button) or destination flow (Dest button).

The dialog includes the following UI elements:

• Close buttons: To close the dialog, click the X in the upper right corner or the Close button at lower right.
• View Type: A drop-down menu used to set the type of visualization used for the graph (defaults to Line Chart); for descriptions of the options see Chart View Types.
• Chart: The visualization of traffic (using the current view type).
• View in Explorer button: Opens Data Explorer for further exploration of the device’s traffic. The sidebar will be set so that query results will show the same traffic that is shown in the dialogs.

Adding or editing a flow tag via the Kentik portal involves specifying information in the fields of the tag admin dialogs, which are covered in the following topics.

• About Tag Dialogs
• Tag Dialogs UI

Note: Tags can also be added and edited with either the Tag API or the Batch API.

The Kentik portal uses tag admin dialogs to collect and display flow tag information. The required information is entered into the fields of either of the following dialogs:

• Add Tag when adding a new tag to Kentik.
• Edit Tag when editing an existing tag.

The Add Tag and Edit Tag dialogs share the same layout and the following common UI elements:

• Close button: Click the X in the upper right corner to close the dialog. All elements will be restored to their values at the time the dialog was opened.
• Tab selectors: Choose the tab to display (see tab-specific topics below).
• Remove button (Edit Tag dialog only): Remove the tag from your organization’s collection of Kentik-registered tags.
• Cancel button: Cancel the add tag or edit tag operation and exit the dialog. All elements will be restored to their values at the time the dialog was opened.
• Add Tag button (Add Tag dialog only): Save settings for the new tag and exit the dialog.
• Save button (Edit Tag dialog only): Save changes to tag settings and exit the dialog.

The fields of the tag admin dialogs are used to specify and display flow tag settings. These fields are described in the following topics:

• About Tag Fields
• General Tag Settings
• Tag Device Settings
• Tag IP Settings
• Tag BGP Settings
• Other Tag Settings
• Tag Field Validation

Each tag admin dialog is broken into the tabs covered in the topics below, each of which is made up of a number of fields. Creating a flow tag requires specifying a name in the Tag Name field, and specifying additional tag attribute fields that will be evaluated for a match with fields in the incoming flow (or with values derived from fields in the flow). The fields that can be used to define tags are described in the topics below.

The validation columns in each table below indicate whether or not the following validations will be applied to a given tag field:

• Comma: Comma-delimited list
• Database: Database patterns (e.g. % and _)
• Regex: Regex

Note: For additional information on validation of tag field values, see Tag Field Validation.

The following table shows the settings on the General tab of tag admin dialogs:

The following table shows the settings on the Device Matching tab of tag admin dialogs:

The following table shows the settings on the IP Matching tab of tag admin dialogs:

The following table shows the settings on the BGP Matching tab of tag admin dialogs:

The collection of BGP data by Kentik Detect allows incoming flows to be assigned tags that match communities and AS paths or partial paths. Flow Tags are applied (separately for source and destination) at the time that flow is ingested into KDE. They can then be used to narrow query results by applying them using src and dst filter functions in the Data Explorer and Query Builder pages of the Kentik Detect portal.

Note: A given tag is applied only to flows that arrive after the tag was created. A new tag may take up to 20 minutes to take effect.

Matches on the BGP-related tag fields are made on substrings. For ASN and Next Hop ASN, the string(s) to match are specified in a simple comma-delimited list. For both the BGP AS Path and BGP Community fields the specified values are also evaluated using a subset of standard regex (see table below):

• BGP AS path tags: Entering “10” in the as-path field will match any path that includes “10”, “100”, “010”, etc. Using regex, a value of “_10_” will match only paths that include ASN 10, including “10 “, “ 10”, and “ 10 “. Also allowed are tags where as-path is specified as, for example, “_10 100_”.
• BGP community tags: Flow Tags on communities are similar to tags on AS paths except that they also support the use of regex periods. This allows you to specify, for example, “2000:1....” to find any flow with community 2000:1xxxx in it.

The following table shows the regex special characters that are supported when specifying the BGP AS Path and BGP Community:

Note: For BGP community and AS path tags, any spaces at the beginning or end of the input field and also before and after each comma will be removed.

The following table shows the settings on the Other tab of tag admin dialogs:

Note: The Validation codes used in the table above are defined in Tag Field Validation.

The following general considerations apply to the validation of values in the tag fields described in Tag Field Definitions:

• Some fields support the entry of multiple values as a comma-delimited list (see tables above).
• Commas are supported only as list delimiters (not in actual values or regex).
• Some fields support the use of database patterns (see tables above).
• In fields where regex is supported (see table below), a period (“.”) may be used in place of a comma.
• The BGP AS Path and BGP Community tags use PostgreSQL POSIX Extended Regular Expressions. For additional information, see Specifying BGP Tag Fields.
• All other tags that support regex use PostgreSQL Advanced Regular Expressions.

Note: Documentation for PostgreSQL regex and database patterns can be found at PostgreSQL documentation:
- Database patterns are documented under “LIKE.”
- Regular expressions are documented under “POSIX Regular Expressions.”

Flow Tags are created and edited via theFlow Tags page of the Kentik Detect portal (choose Flow Tags from the drop-down Admin menu). The add/edit process is covered in the following sections:

• Adding a Flow Tag
• Editing a Flow Tag

Note: Tags can also be added and edited with either the Tag API or the Batch API.

To add a new tag:

1. Go to the Flow Tags page (choose Admin from the Kentik navbar, then Flow Tags from the sidebar at left).
2. Open the Add Tag page by clicking the Add Tag button.
3. Name the tag in the Tag Name field.
4. Specify the values of the tag fields that will be evaluated for a match with the properties of the incoming flow (see Tag Field Definitions).
5. Click the Save button (bottom right) to save the new tag with the currently specified values, after which you’ll be taken back to the Flow Tags page.

Note: A tag name must be unique, but tags whose names contain a common string can be ORed in a query (e.g. “tag1” and “tag2” both contain “tag”).

To edit an existing tag:

1. Go to the Flow Tags page (choose Admin from the Kentik navbar, then Flow Tags from the sidebar at left).
2. In the Flow Tag List, open a Edit Tag dialog by clicking in the row of the tag that you’d like to edit.
3. Edit the tag fields that you’d like to change (see Tag Field Definitions).
4. To save changes, click the Save button at bottom right.

Kentik recommends that all customers create a tag called MYNETWORK to represent their network. A MYNETWORK tag allows a user to quickly and easily see whether traffic came from an internal or external source and is headed toward an internal or external destination. For information on querying using MYNETWORK tags, see MYNETWORK Tag Queries.

Like any other tag, a MYNETWORK tag is created in an Add Tag dialog in the portal (see Adding a Flow Tag). For a comprehensive MYNETWORK tag, specify one or more of the following tag fields:

• ASN: list all of the internal ASNs (ASNs that your network is responsible for). If multiple ASNs are entered, separate with a comma, for example “ASN1,ASN2”, etc.
• IP addresses in IP/CIDR format: list (using CIDR notation) all of the CIDR blocks on your network.
• Interface Names or Descriptions: list all internal interfaces.