Skip to main content

Datadog

Datadog is a comprehensive monitoring and observability platform for cloud-scale applications. It provides infrastructure monitoring, application performance monitoring (APM), log management, security monitoring, and incident management capabilities.

Authentication Types​

Datadog supports 2 authentication methods:

  • API Keys - Uses API Key and Application Key for programmatic access. Simpler to set up and ideal for server-side integrations.

    • Pros: Easy setup, good for automation, no user interaction required
    • Cons: Keys have full account access, no granular permissions

  • OAuth - User-delegated access with granular scope-based permissions. More secure for user-facing applications.

    • Pros: Granular permissions, secure token management, per-user access control
    • Cons: More complex setup, requires OAuth client creation

General Settings​

Before using the connector, you need to configure:

  • Datadog Site - Your Datadog site URL determines which regional API to use. Enter one of:
    • app.datadoghq.com (US1 - default)
    • app.datadoghq.eu (EU)
    • us3.datadoghq.com (US3)
    • us5.datadoghq.com (US5)
    • ap1.datadoghq.com (AP1 - Asia Pacific)
    • ap2.datadoghq.com (AP2 - Asia Pacific)
    • app.ddog-gov.com (Government)
tip

To find your Datadog site, check the URL you use to access Datadog in your browser. If you access Datadog at https://app.datadoghq.com, your site is app.datadoghq.com.

Setting up API Keys​

API Keys provide full programmatic access to your Datadog account. You'll need both an API Key and an Application Key.

Creating an API Key​

  1. Log in to your Datadog account

  2. Navigate to Organization Settings (bottom left gear icon) → API Keys

  3. Click New Key

  4. Enter a name for your key (e.g., "Webrix MCP Integration")

  5. Click Create Key

  6. Copy the API Key immediately (you won't be able to see it again)

Creating an Application Key​

  1. In Organization Settings, go to Application Keys

  2. Click New Key

  3. Enter a name for the application key (e.g., "Webrix MCP App Key")

  4. Select the appropriate scopes for the key (or leave as full access)

  5. Click Create Key

  6. Copy the Application Key immediately

Configuring in Webrix​

  1. In the Webrix connector settings, select API Key as the authentication method

  2. Enter your Datadog Site (e.g., app.datadoghq.com)

  3. Enter your API Key in the API Key field

  4. Enter your Application Key in the Application Key field

  5. Click Save Changes

warning

Keep your API and Application Keys secure. They provide full access to your Datadog account. Never commit them to source control or share them publicly.

Setting up OAuth​

OAuth provides more secure, user-delegated access with granular permissions. Each user authenticates with their own Datadog credentials.

Creating an OAuth Client​

  1. Log in to your Datadog account with admin privileges

  2. Navigate to Organization Settings → OAuth Apps

  3. Click New OAuth App

  4. Fill in the application details:

    • Name: "Webrix MCP Integration" (or your preferred name)
    • Description: "MCP connector for AI assistant access"
    • Redirect URIs: Copy from the Webrix platform (you'll see this when configuring)
    • Scopes: Select the scopes your integration needs:
      • monitors_read - Read monitors
      • dashboards_read - Read dashboards
      • metrics_read - Query metrics
      • logs_read_data - Read log data
      • incidents_read - Read incidents
      • slos_read - Read SLOs
      • hosts_read - Read hosts
      • events_read - Read events
      • downtimes_read - Read downtimes
      • tags_read - Read tags
      • Add write scopes if you need to create/modify resources

  5. Click Create

  6. Copy the Client ID (visible immediately)

  7. Copy the Client Secret (shown only once - store it securely)

Configuring in Webrix​

  1. In the Webrix connector settings, select OAuth as the authentication method

  2. Enter your Datadog Site (e.g., app.datadoghq.com)

  3. Enter the Client ID from your OAuth app

  4. Enter the Client Secret from your OAuth app

  5. Click Save Changes

  6. Click Connect to initiate the OAuth flow

  7. Log in with your Datadog credentials and authorize the application

tip

OAuth uses PKCE (Proof Key for Code Exchange) for enhanced security. No additional configuration is needed - it's enabled automatically.

Available Tools​

The Datadog connector provides 30 tools organized into categories:

Monitors (5 tools)​

  • List, search, and retrieve monitor configurations
  • Check monitor downtime schedules
  • Validate monitor configurations

Dashboards (3 tools)​

  • List, search, and retrieve dashboard configurations
  • Inspect dashboard widgets and layouts

Metrics (4 tools)​

  • Query time series metrics data
  • List active metrics and search by name
  • Retrieve metric metadata

Logs (3 tools)​

  • Search logs with advanced queries
  • List log pipelines and indexes
  • Understand log processing configuration

Incidents (3 tools)​

  • List and search incidents
  • Retrieve incident details and timelines

Service Level Objectives (3 tools)​

  • List and retrieve SLO configurations
  • Get SLO history and compliance data

Infrastructure (3 tools)​

  • List and search hosts
  • Retrieve host details and metadata

Events (2 tools)​

  • List and search events from the event stream
  • Filter by time range, source, and tags

Downtimes (2 tools)​

  • List scheduled maintenance windows
  • Retrieve downtime configurations

Tags (2 tools)​

  • List host tags across infrastructure
  • Retrieve tags for specific hosts

Common Use Cases​

Investigating Alerts​

"Show me all monitors that are currently alerting" "Get details for monitor ID 12345" "What downtimes are currently active?"

Analyzing Metrics​

"Query CPU metrics for host web-server-01 over the last hour" "List all active metrics with 'docker' in the name" "Get metadata for metric system.cpu.idle"

Reviewing Logs​

"Search logs for service:api status:error in the last 2 hours" "Show me all log pipelines configured in my account" "List all log indexes and their retention periods"

Incident Management​

"List all active incidents" "Get details for incident INC-123" "Search for incidents with severity SEV-1"

Infrastructure Monitoring​

"List all hosts tagged with env:production" "Get details for host db-primary-01" "Show me all tags assigned to host web-01"

Troubleshooting​

"Authentication Failed" Error​

Cause: Invalid API Key, Application Key, or OAuth credentials.

Solution:

For API Keys:

  • Verify your API Key and Application Key are correct and not expired
  • Check that you copied the full keys without extra spaces
  • Ensure the keys have the necessary permissions

For OAuth:

  • Verify your Client ID and Client Secret are correct
  • Check that the redirect URI matches exactly
  • Ensure your OAuth app has the required scopes

"Invalid Site" Error​

Cause: The Datadog site is not configured correctly.

Solution:

  • Verify you entered the correct site URL (e.g., app.datadoghq.com, not https://app.datadoghq.com)
  • Check that your site matches where you access Datadog in your browser
  • Common sites: app.datadoghq.com (US), app.datadoghq.eu (EU)

"Permission Denied" or "Forbidden" Errors​

Cause: The API/Application Keys or OAuth scopes lack required permissions.

Solution:

For API Keys:

  • Ensure your Application Key has the necessary permissions
  • Create a new Application Key with appropriate scopes if needed

For OAuth:

  • Review the scopes granted to your OAuth app
  • Add missing scopes in the OAuth app configuration
  • Re-authenticate to grant new permissions

"Rate Limit Exceeded" Error​

Cause: Too many API requests in a short time period.

Solution:

  • Datadog has rate limits per endpoint (typically 300 requests per hour)
  • Wait a few minutes before retrying
  • Consider caching frequently accessed data
  • Use pagination and filtering to reduce request count

No Data Returned​

Cause: Query parameters might be too restrictive or data doesn't exist.

Solution:

  • Verify the time range includes data (check "from" and "to" timestamps)
  • Remove filters to see if data exists
  • Check that the resource (monitor, dashboard, host) exists in your account
  • Verify tags and names are spelled correctly

Metric Queries Return Empty Results​

Cause: Metric might not be reporting or query syntax is incorrect.

Solution:

  • Use "List Active Metrics" to verify the metric exists and is reporting
  • Check metric query syntax (e.g., avg:system.cpu.idle{host:myhost})
  • Verify the time range includes when the metric was reporting
  • Ensure host tags in the query match actual host tags

Log Searches Return No Results​

Cause: Query syntax error or logs aren't being ingested.

Solution:

  • Verify log query syntax (e.g., service:api status:error)
  • Check that logs are being ingested in the Datadog Logs UI
  • Verify the time range includes when logs were sent
  • Check that the log index exists and has data
  • Ensure you have logs_read_data permission

Best Practices​

Security​

  • Use OAuth for user-facing integrations to leverage granular permissions
  • Use API Keys for server-side automation where OAuth isn't practical
  • Rotate API and Application Keys regularly
  • Never commit credentials to source control
  • Use minimum required scopes for OAuth applications

Performance​

  • Use specific filters to reduce data returned
  • Paginate through large result sets rather than requesting all at once
  • Cache frequently accessed data (dashboards, monitor configs) locally
  • Use time ranges appropriate for your use case (avoid querying months of data)

Querying​

  • Start with broad queries and narrow down with filters
  • Use tags consistently across your infrastructure for easier filtering
  • Test metric queries in the Datadog UI before using them in tools
  • Reference Datadog's query language documentation for advanced syntax

Organization​

  • Use consistent naming conventions for monitors, dashboards, and SLOs
  • Apply meaningful tags to all resources for easier discovery
  • Document custom metrics and their meanings
  • Keep monitor and dashboard configurations in version control