Implementing Security Monitoring with Datadog

When to Use

• Deploying Cloud SIEM to detect real-time threats across cloud infrastructure (AWS, Azure, GCP)
• Creating custom detection rules for attacker techniques, credential abuse, or anomalous behavior
• Enabling Workload Protection (CSM Threats) to monitor file, process, and network activity on hosts and containers
• Meeting compliance requirements (PCI-DSS, SOC 2, HIPAA) that mandate centralized log monitoring and alerting
• Building security dashboards to provide SOC visibility into threat signals, investigation context, and response metrics

Do not use for endpoint-only monitoring without cloud infrastructure; use a dedicated EDR solution for purely on-premises endpoint detection.

Prerequisites

• Datadog account with Security Monitoring (Cloud SIEM) and/or Cloud Security Management enabled
• Datadog API Key and Application Key from Organization Settings > API Keys
• Datadog Agent v7+ installed on hosts/containers that generate security-relevant logs
• Log sources configured for ingestion: AWS CloudTrail, VPC Flow Logs, GuardDuty, Azure Activity Logs, GCP Audit Logs, or on-host logs (auth.log, syslog, Windows Security Events)
• Python 3.9+ with datadog-api-client library for programmatic rule management
• Network access from monitored hosts to Datadog intake endpoints (port 443)

Workflow

Step 1: Deploy and Configure the Datadog Agent for Security

Install the Datadog Agent and enable security-related features in datadog.yaml:

# /etc/datadog-agent/datadog.yaml

api_key: <YOUR_DATADOG_API_KEY>
site: datadoghq.com   # or datadoghq.eu, us3.datadoghq.com, etc.

# Enable log collection for Cloud SIEM
logs_enabled: true

# Enable security features
runtime_security_config:
  enabled: true          # Workload Protection (CSM Threats)
  activity_dump:
    enabled: true        # Record process activity for investigation

compliance_config:
  enabled: true          # CIS benchmark checks (CSM Misconfigurations)
  host_benchmarks:
    enabled: true

Configure log sources for security-relevant files on Linux:

# /etc/datadog-agent/conf.d/auth.d/conf.yaml
logs:
  - type: file
    path: /var/log/auth.log
    source: auth
    service: linux-auth
    tags:
      - env:production
      - security:authentication

  - type: file
    path: /var/log/syslog
    source: syslog
    service: linux-syslog

For Windows Security Event Logs:

# /etc/datadog-agent/conf.d/win32_event_log.d/conf.yaml
logs:
  - type: windows_event
    channel_path: Security
    source: windows.events
    service: windows-security
    filters:
      - id: [4624, 4625, 4648, 4672, 4688, 4720, 4726, 4740, 4767]

Enable the system-probe for Workload Protection (CSM Threats):

# /etc/datadog-agent/system-probe.yaml
runtime_security_config:
  enabled: true
  fim_enabled: true        # File Integrity Monitoring
  network_enabled: true    # Network activity monitoring

Restart the Agent after configuration changes:

sudo systemctl restart datadog-agent
sudo datadog-agent status | grep -A5 "Security Agent"

Step 2: Configure Cloud Log Sources for SIEM

Set up AWS CloudTrail, VPC Flow Logs, and GuardDuty ingestion for Cloud SIEM:

Datadog App > Security > Cloud SIEM > Configuration > Content Packs

AWS Content Pack:
  1. Enable the AWS integration in Datadog (Integrations > Amazon Web Services)
  2. Configure CloudTrail log forwarding via the Datadog Forwarder Lambda
  3. Enable VPC Flow Logs forwarding to Datadog
  4. Enable GuardDuty findings forwarding

Required IAM permissions for the Datadog role:
  - cloudtrail:LookupEvents
  - logs:FilterLogEvents
  - guardduty:ListDetectors, guardduty:GetFindings
  - s3:GetObject (for CloudTrail S3 bucket)

Azure Content Pack:
  1. Configure Azure Activity Logs via Event Hub to Datadog
  2. Forward Azure AD Sign-in Logs and Audit Logs
  3. Enable Microsoft Defender for Cloud alerts forwarding

GCP Content Pack:
  1. Configure GCP Audit Logs export via Pub/Sub to Datadog
  2. Forward Cloud Audit Logs (Admin Activity, Data Access)

Verify log ingestion is working:

Datadog App > Logs > Search
  Filter: source:(cloudtrail OR aws.guardduty OR azure.activitylogs)
  Verify: Logs appearing with correct source tags and parsed attributes

Step 3: Enable and Customize Detection Rules

…