Synthetics Test Settings

The Test Control Center in Synthetics is used both to view test results, which is covered in Test Control Center, and to configure synthetic tests, which is covered in the following topics:

• About Test Settings
• Add Test Page
• Test Settings Page
• Common Test Settings
• Test-specific Settings
• Advanced Options Pane

Notes:
- For general information about the Test Control Center, see Test Control Center.
- For information about the types of tests that you manage in the Test Control Center, see Synthetic Test Types.
- For information about how test status (Healthy, Warning, Critical) is derived from test results, see Synthetic Test Status.

In Kentik Synthetics, a test is a collection of subtests that generally involve pings and traceroutes from a set of agents (global or private) toward a target. The type of a test is typically determined by the nature of its target, with each type falling into one of the categories described in Synthetic Test Types.

Test settings are accessed in the following contexts:

• Creating a test: To configure settings for a new test, access the Add Test Page by clicking the Add Test button on the Test Control Center page, then click on the type of test you want to add.
• Editing a test: To configure settings for an existing test, access the test’s Test Settings Page via the Edit icon toward the right of each row in the Tests List on the Test Control Center page or via the Edit Test button on the Test Details page (see SubNav Test Detail Controls).

The configuration settings for a given test are the same whether you are creating or editing the test, but the page for editing existing tests includes additional Test Management Controls.

Note: If your organization's test credit balance (see About Test Credits) is below zero, a notification will appear to the right of the title on the main Test Control Center Page as well as on any Add Test Page or Test Settings Page. To add more credits, click the link in the notification (see Adding Test Credits).

The Add Test page enables you to create a new test by choosing from a list of test types that are categorized as described in Synthetic Test Types. Clicking on one of the listed test types opens a page that is effectively a Test Settings page for the new test, but without the Test Management Controls. As described in the Test Settings Page topics below, the test configuration settings on the settings page for a given test vary depending on the type of the test.

The Test Settings page, which is accessed via an Edit icon found either in the Tests List on the main Test Control Center page or on the Test Details page (see SubNav Test Detail Controls), is covered in the following topics:

• Test Management Controls
• Test Settings Overview
• Select Agents Dialog

Note: The test types in each of the categories covered in the topics below are described in Synthetic Test Types.

In addition to test configuration settings covered in the topics below the Test Settings page includes the following test management controls:

• Pause/Resume Test: A button that does one of the following:
  - Pause: If the test is active it stops all testing related to this test.
  - Resume: If the test is paused it resumes all testing related to this test.
• Remove: Pops up a confirming dialog that enables you to permanently remove this test from your organization's collection of tests.

Note: The above controls are present only when editing an existing test rather than adding a new one.

The table below shows the primary settings required for tests in each of the categories described in Synthetic Test Types. The main variation in settings between tests are related to specifying the target.

Notes:
- In addition to the above parameters, most test types support notifications (see Configure notifications in Common Test Settings).
- The test types in each of the categories listed above are listed on the Add Test Page.

The Select Agent dialog, which opens from the Add Agents or Edit Agents button on a Test Settings page, contains a list from which to choose one or more agents to be used for a test. The dialog has two forms:

• Multi-agent version: Checkboxes to the left of every agent enable you to select multiple agents simultaneously. This version is used on the settings pages for all Mesh tests (to choose agents in the matrix) and Network tests (to choose agents to test from).
• Single-agent version: Radio buttons to the left of every agent enable you to select one agent at a time. This version is used when choosing an agent to test toward, e.g. on the settings page for an Agent-to-Agent test.

The dialog includes the following UI elements

• Filter: Enter text to filter the list to those agents that contain the text in their name or location.
• Close: Click the X in the upper right corner to exit the dialog and keep the set of selected agents the same as it was when the dialog was opened.
• Check all (checkbox; multi-agent dialog only): Check or uncheck the checkboxes for all currently shown agents (agents hidden by filtering will not be selected). Individual checkboxes can still be checked/unchecked after using this checkbox.
• Agents List: A list of agents to select for inclusion in the test. If the test allows selection of both Private and Global agents the list will be broken into two sections. The list includes the following columns:
  - Agent: The name of the agent.
  - Location: The name of the site in which an agent is deployed, or the cloud region of a cloud agent.
• Cancel: Click Cancel to exit the dialog and keep the set of selected agents the same as it was when the dialog was opened.
• Save: Click Save to exit the dialog and change the set of selected agents to the agents currently selected in the dialog.

The following test settings are common to the settings pages for all test types:

• Name: The name by which you want the test to appear in the Kentik portal.
• Choose agents: The control for selecting the agents in the mesh (e.g. agents in sites for a Site Test). The control includes:
  - Agent list: The list of agents currently selected for this test (initial value is "No agents selected.").
  - Edit Agents: Click this button to select agents in the Select Agents Dialog.
• Configure Notifications (not available for Transaction tests): This control enables you to optionally assign to this test Notifications that will go to notification channels in Kentik’s notification system. The control is made up of two parts:
  - Notification Channels: A field that shows the channels to which notifications are currently assigned for this test. To remove a channel from this test, click the X in the channel’s tag. To add a channel, click in the field to drop down a list of channels in your organization from which you can choose.
  - Add New: A button that opens the Add Notification dialog (see Notification Channel Settings), where you can create a new notification channel in your organization so you can add it to the test.
• Advanced Options: Expands/collapses the Advanced Options Pane.
• Expected monthly credit usage: A bar graph that shows the impact of the test, as currently configured, on your organization's overall consumption of credits available for the current month (see About Test Credits). Also shown is the number of credits the test will use in the current month and the remaining number of credits that will be available in the current month if this test is active.
• Save and Start Testing: A button that becomes active when the current settings are sufficient for Kentik to create a test. When the above settings are specified as desired, click the button to start the test.

The following topics cover settings that are specific to tests of a given type:

• Network Meshes & Grids
• Network & Routing Tests
• Web Tests
• Autonomous Tests
• DNS Tests

Test types in the Network Meshes & Grids category use the following fields to specify the target to test toward:

• Network Mesh: Select two or more agents to mesh.
• Network Grid: Enter a comma-separated list of IP Address(es) to target, or add IPs from.txt or.csv file.
• DNS Server Grid: Enter DNS server IP(s) to target.

Test types in the Network & Routing category use the following fields to specify the target to test toward:

• IP Address test: Enter a comma-separated list of IP Address(es) to target.
• Hostname test: Enter a hostname to target.
• SaaS Application test: Enter a SaaS application to target.
• Agent-to-Agent test: Choose a target agent. The UI for this selector is the same as the Choose Agents control described in Common Test Settings.
• BGP Monitor:
  - Prefixes to Monitor: Enter a comma-separated list of up to 10 prefixes to monitor, or add prefixes from.txt or.csv file.
  - Include more specific prefixes: Available when all prefixes listed in the Prefixes to Monitor field are /15 or narrower (e.g. /16, /21, etc.), this switch tells Kentik to automatically discover and monitor more specific prefixes within the listed prefixes.
  - Allowed ASN(s): Enter a comma-separated list of ASNs to monitor for route announcements and withdrawals.
  - Check RPKI: If off, all ASNs that aren't explicitly listed in the Allowed ASNs field will be flagged as invalid. If on (default), ASNs that aren't listed but are RPKI-valid will also be treated as valid.

The following settings are specific to test types in the Web Tests category:

• HTTP(S) or API:
  - Select a Method: Select the type of HTTP request - GET, POST, PUT or PATCH
  - Enter a URL or FQDN: The URL or Fully Qualified Domain Name that will be the target of the selected HTTP request.
  - Enable ping and traceroute towards this host: If this switch is off then on the Subtest Details pages for this test the Metrics tab will not show timeline charts for latency, packet loss, or jitter, and there will be no results for the Path View tab (see Details tabs in Subtest Details UI).
  - Configure Request (optional): Specify headers, parameters, and body for the HTTP request.
• Page Load (uses app agents, which run an instance of Headless Chromium):
  - Enter a URL or IP address: The URL/IP that will be the target of the selected HTTP request.
  - Enable ping and traceroute towards this host: If this switch is off then on the Subtest Details pages for this test the Metrics tab will not show timeline charts for latency, packet loss, or jitter, and there will be no results for the Path View tab (see Details tabs in Subtest Details UI).
  - Configure Request and CSS Selectors Validation (optional): Specify headers, parameters, and CSS selectors for the HTTP request (see About Configure Request Settings).
• Transaction (uses app agents, which run an instance of Headless Chromium):
  - Configure Puppeteer script: By default, this textbox contains an example Puppeteer script that you can use to experiment with transaction tests. To create a test, you'll overwrite the example script with a script that executes the steps that you'd like to test (see About Transaction Test Scripts).

The optional Configure Request settings on a Page Load test enable additional control over how the test is structured and what it looks for:

• Test structure: Use the following tabs to control HTTP configuration for the test:
  - Headers: Specify key and value for headers in the page's HTTP request.
  - Params: Specify parameters to send in the body of the page's HTTP request.
• CSS Selectors: Specify one or more CSS Selectors. The test will look for those CSS selectors in the HTML of the web page that returns from the request.

If a CSS selector is specified in a given test:

• A CSS Validation column will appear in the Test Details Table on the details page for that test. The column will indicate PASS only if all specified selectors are found; otherwise it will indicate FAIL.
  Note: This result is not used in computing test health (which drives alerts and notifications).
• The charts on the Metrics tab of the Subtest Details Page for this test will include a CSS Selectors chart (see Page Load tests in Additional Metrics Charts).

The key to using Transaction tests is to create a script that performs the actions that you'd like to test. You'll typically create the script in the Recorder tab of Chrome Developer Tools and paste it into the textbox in the Configure Puppeteer script step of the Transaction test's Test Settings page. You'll then be able to insert into the script, at whichever stages of the transaction you wish, a command to take a screenshot, which will be displayed in the Screenshots pane of the Results tab on any of the test's subtest details pages.

To create a Transaction test from a script:

1. Open Chrome Developer tools and got to the Recorder tab.
2. Record the sequence of actions that you'd like to test.
   Note: A test may include no more than five separate pages.
3. Export the recording to a Puppeteer script as described in https://developer.chrome.com/docs/devtools/recorder/#export-flows.
4. In the Kentik portal, choose Test Control Center from the navbar menu.
5. On the Test Control Center landing page, click the Add Test button at upper right.
6. On the Add Test page, find the Web Tests category and click Transaction.
7. On the resulting Test Settings page, name the test and choose the agents to test from (see Common Test Settings).
8. In the Configure Puppeteer script textbox, select the entire contents and paste in the Puppeteer script that you created above.
   Note: The script must include declarations of the consts puppeteer, browser, and page as shown at the start of the example script.
   
9. At each point in the series of actions where you'd like a screenshot taken of the page state, insert the following line at the corresponding point in the script:
   await page.screenshot({ path: "screenshot##.jpg" });
   Note: In the above line, replace ## with the number of the screenshot (01 for the first shot, 02 for the second, etc.). A test may include no more than 10 screenshots.
10. If desired, configure notifications for the test (see Common Test Settings).
11. If desired, open the Advanced Options Pane and change default settings for General Options and/or Health Options.
12. Click the Save and Start Testing button to start the test.
    Note: If the button is not active and you've specified the name and agents for the test, see Transaction Script Validation.

The script for a transaction test must meet the following requirements:

• The declarations for the puppeteer, browser, and page consts must match those in the default example script:
  const puppeteer = require('puppeteer');
  const browser = await puppeteer.launch();
  const page = await browser.newPage();
• There must be no more than five page.goto calls.
• There must be no more than ten page.screenshot calls.

Once a Transaction test is configured and running, here's how to see the results:

1. In the Tests List on the Test Control Center landing page, click on the link to the test.
2. On the resulting Test Details Page, click the Details button in the row for a subtest in the Test Details Table at the bottom of the Results tab.
3. On the resulting Subtest Details Page, the Results tab will display a time series chart showing the time it took to complete the transaction during each time slice of the time range. The tab also displays the screenshots for the time slice corresponding to the currently selected segment of the timeline.

The settings pages for tests in the Autonomous category include a control to identify a specific instance of a dimension whose IPs will be tested toward. The dimension corresponds to the type of test — ASN, CDN, Region, Country, or City — and the control’s label varies depending on the type of the test (e.g. for an ASN test the control is labeled “Select an ASN to target”).

The target control includes three tabs that can be used to choose the target instance. Using an ASN test as an example, these tabs are as follows:

• By Inbound Traffic: A table listing the top-X ASNs sending traffic to your network (based on the flow data your organization sends to Kentik), including volume and status information. Click the Select button to choose one ASN.
• By Outbound Traffic: Same as the By Inbound Traffic tab, but listing the top-X ASNs to which your network sends traffic.
• Manual: Two controls:
  - Target: A drop-down from which you choose the target (e.g. ASN) from a filterable list of instances of the dimension corresponding to this test.
  - Direction: A drop-down with which you specify whether the target is tested as the destination of outbound traffic (default) or the source of inbound traffic.

The following settings are specific to test types in the DNS Tests category:

• DNS Server Monitor:
  - Enter DNS server IPs: Enter a comma-separated list of the IP addresses of a DNS server to query.
  - Enter a host name to look up: A field into which you enter the hostname that you want this test to resolve via the specified DNS server(s).

The controls used to set advanced options for tests are covered in the following sections:

• About Advanced Options
• General Options
• Health Options
• Ping Options
• Traceroute Options

The settings in the Advanced Options pane enable you to customize settings that are otherwise set by default, giving you finer control over the details of how a test is conducted. The controls are grouped into the cards covered in the topics below.

Note: Options vary by test type.

Depending on the test type, the following settings may appear in the General Options card on the Advanced Options pane:

• Test Frequency/DNS Lookup Frequency: The interval at which the test should be run. Available intervals currently include:
  - 1 (default), 2, 5, 10, 15, 30, and 60 minutes for tests of all types.
  Note: The default interval for Page Load tests is 5 minutes).
  - 1 second and 15 seconds for tests of all types except “Web Tests” and “DNS Tests” (see Synthetic Test Types).
  - 90 minutes for Transaction tests only.
  Note: All probes for each subtest in the test (see Ping Options and Traceroute Options) will be sent at the start of the test interval.
• IP Version: Determines the type of IP addresses that will be targeted by the test: IPv4 only, IPv6 only, or both (default). This is particularly useful for tests of type Hostname or ASN, where there could be a mix of IPv4 and IPv6 addresses.
• Find the top: Once you've chosen (in the Select Target step; see Autonomous Tests) an entity (ASN, CDN, Country, Region, or City) to test toward, this setting determines whether the tested IPs within that entity are the top source IPs (default) or destination IPs.
• Create tests using target as: This setting determines whether Kentik will test toward IPs within the target entity that are sending traffic to your AS ("Source of inbound traffic") or receiving traffic from your AS ("Destination of outbound traffic").
• Max Providers to Track: The maximum number of providers to track autonomously (see Provider Classification).
• Max IP Targets: The maximum number of IPs toward which Kentik should create subtests for this test. The actual number of subtests will depend on the number of IPs that meet the qualifications determined by other settings; if that number is greater than this setting then the number of tests (and the consumption of test credits) will be capped.
• Flow Eval Frequency: The interval at which your organization's flow data will be evaluated by Kentik to identify new targets (IP addresses).
  Note: This setting does not affect test frequency.
• Use Agent Private IPs: Use for mesh testing between private agents on your network. Kentik will test using the local IP that can be assigned to each private agent via the Private IP field in the Configure Agent Dialog, resulting in a more direct path than testing using the public IP addresses of the private agents.
• Labels: This field shows the labels that are already assigned to this test and also enables you to assign to the test any labels that already exist in your organization, which you can choose from a list that drops down when you click in the field.
  Note: Labels for tests can also be managed on the main Test Control Center page (see Label Controls).
• Test Bidirectionally: Test in both directions when testing between two agents.
• HTTP Timeout: The duration (in ms) for which Kentik will wait for a response to a test HTTP request. If no response is received before the timeout, no results will be shown for the HTTP portion of the test, and health status will be determined by ping, if enabled (see Web Tests).
  Note: The default timeout is 5 seconds (5000 ms) for HTTP(S) or API tests and 20 seconds (20000 ms) for Page Load tests.
• DNS Record Type: A selector for choosing the type of record that the test will request from the server (see https://en.wikipedia.org/wiki/List_of_DNS_record_types).
• Target Port: The port on the DNS server being tested (default is 53).
• Ignore TLS Errors: Ignore errors related to Transport Layer Security (encryption of data in transit).

Notes:
- The above settings that involve selecting “sources” or “destinations” of traffic are used only to determine which IPs to test toward. The subtests in a test are always run from an agent to an IP. For a private agent in your AS, a subtest runs from your AS to the target IP. For a global agent, a subtest runs from the global agent to the target IP.
- To test your own AS, select it on the manual tab of the Select Target step, then select the global agent from which to test.

The table below shows which general options are included in the advanced options for each test type.

The following settings related to health status will appear in the Health Options card for tests of all types:

• Subtest Threshold for Status: The minimum number of unhealthy (warning or critical) subtests that will result in overall test status being classified as unhealthy (default is 1).
• Thresholds: A metric value based on which the status of a subtest in this test will be reported as Warning and/or Critical. The following values are supported (available thresholds vary by test type):
  - Ping Latency: Latency in standard deviation or in milliseconds (choose the metric from the Use drop-down).
  - Ping Packet Loss: Packet loss in percent (greater than 0% for warning and equal to or greater than 50% for critical).
  - Ping Jitter: Jitter in standard deviation or in milliseconds.
  - HTTP Latency: HTTP latency in standard deviation or in milliseconds.
  - Response Time: DNS response time in standard deviation or in milliseconds.
  - Transaction time: Elapsed time of transaction.
  - Reachability: The number of "vantage points" (public BGP routing data collectors) that are able to reach the tested prefixes as a percent of all such vantage points. If reachability drops below the specified thresholds for Warning and/or Critical an alert will be triggered in the Synthetics Incident Log, and a notification will also be triggered if one is set for the test.
  Note: For standard deviation, default values are 1.5 for warning and 3 for critical.
• Valid HTTP Codes: If this switch is on, click in the field to open a drop-down from which you can choose the HTTP status codes that will be considered normal (rather than warning or critical).
  Note: When not specified, all codes above 400 will result in a health status of critical.
• Valid DNS Codes: If this switch is on, click in the field to open a drop-down from which you can choose the HTTP status codes that will be considered normal (rather than warning or critical).
  Note: When not specified, all codes above 400 will result in a health status of critical.
• Alert on Unhealthy Status: Settings defining the conditions that will trigger an unhealthy status alert:
  - the number of incidents of unhealthy overall test status (default is 3);
  - the duration within which the number of incidents must occur (default is 5);
  - the duration between incidents after which the count of incidents will reset (default is 2).

The table below shows which health options are included in the advanced options for each test type.

The following ping-related settings will appear in the Ping Options card for all tests except those in the category DNS Tests:

• Probes per Ping: The number of individual packets that will be sent per ping (default is 5).
• Protocol: The protocol in which packets will be sent, either ICMP (default) or TCP.
• Overall Ping Timeout: The duration (in ms) for which Kentik will wait for the full ping test to complete (default is 3000). A “per-probe” timeout is derived by dividing this number by the Probes per Ping setting above.
• Inter-probe delay: The delay (in ms) that will be added between consecutive ping probes. Default is 0 ms (no delay).

The table below shows which ping options are included in the advanced options for each test type.

The following traceroute-related settings will appear in the Traceroute Options card for all tests except those in the category DNS Tests:

• Probes per Hop: The number of individual packets that will be sent for each router in the path (default is 3).
• Protocol: The protocol in which packets will be sent, either UDP (default) or TCP.
• Target Port: The port to which the packets will be sent (default is 33434).
• Overall Trace Timeout: The duration (in ms) for which Kentik will wait for the full ping traceroute to complete (default is 22500). A “per-probe” timeout is derived by dividing this number by the Probes per Hop setting above.
  Note: To minimize incomplete traces in the diagram on the Path View tab (see Test Path View) Kentik recommends a value of 7500 ms for each probe per hop.
• Max TTL (Hop Limit): The maximum number of hops to trace (default is 30).
  Note: If the route being traced involves more hops than the hop limit then traces in the diagram on the Path View tab will be truncated.
• Inter-probe delay: The delay (in ms) that will be added between consecutive traceroute probes. Default is 0 ms (no delay).

The table below shows which traceroute options are included in the advanced options for each test type.