Flow Tags

Kentik supports the use of user-defined flow tags that can make it more convenient to query the Kentik Data Engine (KDE). The following topics cover the setup of flow tags in the Kentik portal :

• About Flow Tags
• Flow Tags Page
• Tag Admin Dialogs
• Tag Field Definitions
• Add or Edit Tags

Note: For information on querying with tags, see Tag-based Queries.

Flow tags are labels that are applied to flow data, based on user-defined criteria, as Kentik ingests the data into the Kentik Data Engine (KDE). Because tags are applied as the data is ingested, a tag must already be created before it can be applied to a given flow record (tags cannot be applied retroactively to rows that already exist in the database). However, because Kentik stores complete flow records rather than aggregated summaries, historical queries are not limited to flow attributes that have been defined in advance with tags. In other words, tags enhance your querying options, but they in no way limit or filter the data that is stored in KDE.

Flow tags are created in the portal (see Adding a Flow Tag) or via one of the following Kentik V5 APIs (see About the V5 APIs):

• To add individual tags: Tag API.
• To add tags by batch: Batch API.

When each flow record is sent to Kentik from a given device, the data is evaluated to determine if any of its attributes match any of the existing tags set up by a given customer.

• If all of the values specified in the ANDed tag fields for a given tag are matched in SRC-related flow fields (i.e. SRC IP, SRC port, ASN path associated with SRC IP, or communities associated with SRC IP) then the tag's name is appended as text to the existing tags (if any) in the src_flow_tags column for that flow.
• If all of the values specified in the ANDed tag fields for a given tag are matched in DST-related flow fields (i.e. DST IP, DST port, ASN path associated with DST IP, or communities associated with DST IP) then the tag's name is appended as text to the existing tags (if any) in the dst_flow_tags column for that flow.

Note: Because the tag fields are ANDed for each of the two comparison operations described above, a tag will be applied only when all tag fields are matched.

The result of the tagging process is that the src_flow_tags and dst_flow_tags columns of each device's main table contain a delimited list of tags that can be searched as part of a KDE query.

Note: For greater detail about how tags are applied to flow in KDE, see Tag-based Queries.

The Flow Tags page is home to the Flow Tag List, which is a table that lists all of your organization’s existing tags. To view the page, choose Settings from the main Kentik navbar, then Flow Tags under Data Enrichment. The Flow Tags page is covered in the following topics:

• Flow Tags Page UI
• Flow Tag List

The Flow Tags page has the following main UI elements:

• Filter field: Enter text to filter the Tag Name column of the Flow Tag List for a match on the string entered in this field.
• Add Tag button: Opens the Add Tag dialog (see Adding a Flow Tag).
• Flow Tag List: A table listing the flow tags currently set up in your organization (see Flow Tag List).

The Flow Tag List is a table that lists all of your organization’s currently defined flow tags. The table's columns provide the following information and actions for each tag:

• Tag Name: The name of the flow tag as specified at the time the flow tag was created. Click anywhere on a tag’s row to open the Edit Tag dialog for that flow tag.
• Last Edited: The date on which the tag was last edited. Hover over the date for a tool tip with the full date-time (UTC).
• Edited by: The email address of the person who last modified the tag.
• Created: The date on which the tag was created. Hover over the date to see the full date-time (UTC).
• ID: The system-generated unique ID assigned to the tag when it was created.
• View in Chart: Opens the Total Matching Traffic Dialog for either the Source or the Destination.
• Remove (trash icon): Opens a confirmation dialog that allows you to remove the tag from the Kentik portal.

Note:
- Click on the column headings for Tag Name or ID to sort the list (ascending or descending).
- To see additional information about a given flow tag, click anywhere in the row for that flow tag, which opens an Edit Tag dialog where you can review settings (see Editing a Flow Tag).

The Total Matching Traffic dialog is opened via one of the View in Chart buttons, either Source or Dest. The dialog displays a chart showing the total traffic (expressed as max bits/second), both historically and for the last 24 hours, that had matches for this tag in either source flow (Source button) or destination flow (Dest button).

The dialog includes the following UI elements:

• Close buttons: To close the dialog, click the X in the upper right corner or the Close button at lower right.
• Applied Filters: A button that opens a popup in which you can see which filters are applied to the traffic shown in the chart.
• View Type: A drop-down menu used to set the type of visualization used for the graph (defaults to Line Chart). For descriptions of the options, see Chart View Types.
• Chart: The visualization of traffic (using the current view type). Hover over any part of the chart for more precise information.
• View in Explorer button: Opens Data Explorer for further exploration of the device's traffic. The query settings will be set to show the same traffic that is shown in the Total Matching Traffic dialog.

Adding or editing a flow tag via the Kentik portal involves specifying information in the fields of the tag admin dialogs, which are covered in the following topics.

• About Tag Dialogs
• Tag Dialogs UI

Note: Tags can also be added and edited with either the Tag API or the Batch API.

The Kentik portal uses tag admin dialogs to collect and display flow tag information. The required information is entered into the fields of either of the following dialogs:

• Add Tag when adding a new tag to Kentik.
• Edit Tag when editing an existing tag.

The Add Tag and Edit Tag dialogs share the same layout and the following common UI elements:

• Close button: Click the X in the upper right corner to close the dialog. All elements will be restored to their values at the time the dialog was opened.
• Tab selectors: Choose the tab to display: General, Device Matching, IP Matching, BGP Matching, or Other (see tab-specific topics below).
• Cancel: A button that cancels the add tag or edit tag operation and exits the dialog. All elements will be restored to their values at the time the dialog was opened.
• Add Tag (Add Tag dialog only): A button that saves settings for the new tag and exits the dialog.
• Save (Edit Tag dialog only): A button that saves changes to tag settings and exits the dialog.

The fields of the tag admin dialogs are used to specify and display flow tag settings. These fields are described in the following topics:

• About Tag Fields
• General Tag Settings
• Tag Device Matching
• Tag IP Matching
• Tag BGP Matching
• Other Tag Settings
• Tag Field Validation

Each tag admin dialog is broken into the tabs covered in the topics below, each of which is made up of several fields. Creating a flow tag requires specifying a name in the Tag Name field and specifying additional tag attribute fields that will be evaluated for a match with fields in the incoming flow (or with values derived from fields in the flow). The fields that can be used to define tags are described in the topics below.

The validation columns in each table below indicate whether or not the following validations will be applied to a given tag field:

• Comma: Comma-delimited list
• Database: Database patterns (e.g. % and _)
• Regex: Regex

Note: For additional information on validation of tag field values, see Tag Field Validation.

The following table shows the settings on the General tab of tag admin dialogs:

The following table shows the settings on the Device Matching tab of tag admin dialogs:

The following table shows the settings on the IP Matching tab of tag admin dialogs:

The following table shows the settings on the BGP Matching tab of tag admin dialogs:

Kentik’s collection of BGP data allows incoming flows to be assigned tags that match communities and AS paths or partial paths. Flow tags are applied (separately for source and destination) at the time that flow is ingested into KDE. They can then be used to narrow query results by applying flow tags using source and destination filter functions in Data Explorer.

Note: A given tag is applied only to flows that arrive after the tag was created. A new tag may take up to 20 minutes to take effect.

Matches on the BGP-related tag fields are made on substrings. For ASN and Next-hop ASN, the string(s) to match are specified in a simple comma-delimited list. For both the BGP AS Path and BGP Community fields, the specified values are also evaluated using a subset of standard regex (see table below):

• BGP AS path tags: Entering "10" in the as-path field will match any path that includes "10", "100", "010", etc. Using regex, a value of "_10_" will match only paths that include ASN 10, including "10 ", " 10", and " 10 ". Also allowed are tags where as-path is specified as, for example, "_10 100_".
• BGP community tags: Flow tags on communities are similar to tags on AS paths except that they also support the use of regex periods. This allows you to specify, for example, "2000:1...." to find any flow with community 2000:1xxxx in it.

The following table shows the regex special characters that are supported when specifying the BGP AS Path and BGP Community:

Note: For BGP community and AS path tags, any spaces at the beginning or end of the input field, and also before and after each comma will be removed.

The following table shows the settings on the Other tab of tag admin dialogs:

Note: The validation codes used in the table above are defined in Tag Field Validation.

The following general considerations apply to the validation of values in the tag fields described in Tag Field Definitions:

• Some fields support the entry of multiple values as a comma-delimited list (see tables above).
• Commas are supported only as list delimiters (not in actual values or regex).
• Some fields support the use of database patterns (see tables above).
• In fields where regex is supported (see table below), a period (".") may be used in place of a comma.
• The BGP AS Path and BGP Community tags use PostgreSQL POSIX Extended Regular Expressions. For additional information, see Specifying BGP Tag Fields.
• All other tags that support regex use PostgreSQL Advanced Regular Expressions.

Note: Documentation for PostgreSQL regex and database patterns can be found at PostgreSQL documentation:
- Database patterns are documented under "LIKE."
- Regular expressions are documented under "POSIX Regular Expressions."

Flow tags are created and edited via the Flow Tags Page of the Kentik portal (from the main navbar, select Settings and then Flow Tags). The add/edit process is covered in the following sections:

• Adding a Flow Tag
• Editing a Flow Tag

Note:
- Tags can also be added and edited with either the Tag API or the Batch API.
- The preferred alternative to creating a MYNETWORK tag is to group-by or filter for traffic whose Traffic Profile dimension equals Internal (see Network Classification Dimensions). For more information about adding a MYNETWORK tag, see Add a MYNETWORK Tag.

To add a new tag:

1. Go to the Flow Tags page (Settings » Flow Tags).
2. Open the Add Tag dialog by clicking the Add Tag button.
3. On the General tab, name the tag in the Tag Name field.
4. Specify the values of the tag fields that will be evaluated for a match with the properties of the incoming flow (see Tag Field Definitions).
5. Click the Add Tag button (bottom right) to save the new tag with the currently specified values and close the dialog.

  
Note: A tag name must be unique, but tags whose names contain a common string can be ORed in a query (e.g. "tag1" and "tag2" both contain "tag").

To edit an existing tag:

1. Go to the Flow Tags page (Settings » Flow Tags).
2. In the Flow Tag List, open an Edit Tag dialog by clicking anywhere in the row of the tag that you'd like to edit.
3. Edit the tag fields that you'd like to change (see Tag Field Definitions).
4. To save changes, click the Save button at bottom right. The dialog will close.