parsedmarc/docs/source/google_secops.md

Google SecOps (Chronicle) Output

parsedmarc can output DMARC reports to Google SecOps (Chronicle) in UDM (Unified Data Model) format.

Configuration

To enable Google SecOps output, add a [google_secops] section to your configuration file:

[general]
save_aggregate = True
save_forensic = True

[google_secops]
# Optional: Include forensic report message payload (default: False)
# For privacy, message bodies are excluded by default
include_ruf_payload = False

# Optional: Maximum bytes of forensic message payload to include (default: 4096)
ruf_payload_max_bytes = 4096

# Optional: Static observer name for telemetry identification
static_observer_name = my-parsedmarc-instance

# Optional: Static observer vendor (default: parsedmarc)
static_observer_vendor = parsedmarc

# Optional: Static environment label (e.g., prod, dev)
static_environment = prod

Output Format

The Google SecOps output produces newline-delimited JSON (NDJSON) in Chronicle UDM format, which can be ingested into Google SecOps for hunting and dashboarding.

Event Types

1. DMARC_AGGREGATE: One event per aggregate report row, preserving count and period information
2. DMARC_FORENSIC: One event per forensic report
3. SMTP_TLS_REPORT: One event per SMTP TLS failure detail
4. DMARC_PARSE_ERROR: Generated when parsing fails (does not crash)

UDM Schema

Each event includes:

• metadata: Event timestamp, type, product name, and vendor
• principal: Source IP address, location (country), and hostname (reverse DNS)
• target: Domain name (from DMARC policy)
• security_result: Severity level, description, and detection fields for dashboarding
  • detection_fields: Key DMARC dimensions for filtering and grouping (e.g., dmarc.disposition, dmarc.pass, dmarc.header_from, dmarc.report_org)
  • All dashboard-relevant fields use dmarc.* or smtp_tls.* prefixes for easy identification
• additional.fields (optional): Low-value context fields (e.g., detailed auth results) not typically used for dashboarding

Design Rationale: DMARC dimensions are placed in security_result[].detection_fields rather than additional.fields because Chronicle dashboards, stats searches, and aggregations work best with UDM label arrays. The additional.fields is a protobuf Struct intended for opaque context and is not reliably queryable for dashboard operations.

Severity Heuristics

• HIGH: DMARC disposition = reject
• MEDIUM: DMARC disposition = quarantine with partial SPF/DKIM failures
• LOW: DMARC disposition = none or pass

Example Output

Aggregate Report Event

{
  "event_type": "DMARC_AGGREGATE",
  "metadata": {
    "event_timestamp": "2018-06-19T00:00:00+00:00",
    "event_type": "GENERIC_EVENT",
    "product_name": "parsedmarc",
    "vendor_name": "parsedmarc"
  },
  "principal": {
    "ip": ["199.230.200.36"],
    "location": {"country_or_region": "US"}
  },
  "target": {
    "domain": {"name": "example.com"}
  },
  "security_result": [{
    "severity": "LOW",
    "description": "DMARC fail; SPF=pass; DKIM=pass; SPF not aligned; DKIM not aligned; disposition=none",
    "detection_fields": [
      {"key": "dmarc.disposition", "value": "none"},
      {"key": "dmarc.policy", "value": "none"},
      {"key": "dmarc.pass", "value": false},
      {"key": "dmarc.spf_aligned", "value": false},
      {"key": "dmarc.dkim_aligned", "value": false},
      {"key": "dmarc.header_from", "value": "example.com"},
      {"key": "dmarc.envelope_from", "value": "example.com"},
      {"key": "dmarc.report_org", "value": "example.net"},
      {"key": "dmarc.report_id", "value": "b043f0e264cf4ea995e93765242f6dfb"},
      {"key": "dmarc.report_begin", "value": "2018-06-19 00:00:00"},
      {"key": "dmarc.report_end", "value": "2018-06-19 23:59:59"},
      {"key": "dmarc.row_count", "value": 1},
      {"key": "dmarc.spf_result", "value": "pass"},
      {"key": "dmarc.dkim_result", "value": "pass"}
    ]
  }],
  "additional": {
    "fields": [
      {"key": "spf_0_domain", "value": "example.com"},
      {"key": "spf_0_result", "value": "pass"},
      {"key": "dkim_0_domain", "value": "example.com"},
      {"key": "dkim_0_result", "value": "pass"}
    ]
  }
}

Forensic Report Event

{
  "event_type": "DMARC_FORENSIC",
  "metadata": {
    "event_timestamp": "2019-04-30T02:09:00+00:00",
    "event_type": "GENERIC_EVENT",
    "product_name": "parsedmarc",
    "vendor_name": "parsedmarc"
  },
  "principal": {
    "ip": ["10.10.10.10"]
  },
  "target": {
    "domain": {"name": "example.com"}
  },
  "security_result": [{
    "severity": "MEDIUM",
    "description": "DMARC forensic report: authentication failure (dmarc)",
    "detection_fields": [
      {"key": "dmarc.auth_failure", "value": "dmarc"},
      {"key": "dmarc.reported_domain", "value": "example.com"}
    ]
  }],
  "additional": {
    "fields": [
      {"key": "feedback_type", "value": "auth-failure"},
      {"key": "message_id", "value": "<01010101010101010101010101010101@ABAB01MS0016.someserver.loc>"},
      {"key": "authentication_results", "value": "dmarc=fail (p=none; dis=none) header.from=example.com"},
      {"key": "delivery_result", "value": "delivered"}
    ]
  }
}

SMTP TLS Report Event

{
  "event_type": "SMTP_TLS_REPORT",
  "metadata": {
    "event_timestamp": "2016-04-01T00:00:00+00:00",
    "event_type": "GENERIC_EVENT",
    "product_name": "parsedmarc",
    "vendor_name": "parsedmarc"
  },
  "target": {
    "domain": {
      "name": "company-y.example"
    }
  },
  "security_result": [{
    "severity": "LOW",
    "description": "SMTP TLS failure: certificate-expired",
    "detection_fields": [
      {"key": "smtp_tls.policy_domain", "value": "company-y.example"},
      {"key": "smtp_tls.result_type", "value": "certificate-expired"},
      {"key": "smtp_tls.failed_session_count", "value": 100},
      {"key": "smtp_tls.report_org", "value": "Company-X"},
      {"key": "smtp_tls.report_begin", "value": "2016-04-01T00:00:00Z"},
      {"key": "smtp_tls.report_end", "value": "2016-04-01T23:59:59Z"}
    ]
  }],
  "principal": {
    "ip": ["2001:db8:abcd:0012::1"]
  }
}

Parse Error Event

{
  "event_type": "DMARC_PARSE_ERROR",
  "metadata": {
    "event_timestamp": "2026-01-09T16:22:10.933751+00:00",
    "event_type": "GENERIC_EVENT",
    "product_name": "parsedmarc",
    "vendor_name": "parsedmarc"
  },
  "security_result": [{
    "severity": "ERROR",
    "description": "Failed to parse DMARC report: Invalid XML structure"
  }]
}

Google SecOps Searches

Here are some example YARA-L rules you can use in Google SecOps to hunt for DMARC issues:

Find all DMARC aggregate report failures

rule dmarc_aggregate_failures {
  meta:
    author = "parsedmarc"
    description = "Detect DMARC authentication failures in aggregate reports"
    
  events:
    $e.metadata.product_name = "parsedmarc"
    $e.event_type = "DMARC_AGGREGATE"
    $e.security_result.detection_fields.key = "dmarc.pass"
    $e.security_result.detection_fields.value = false
    
  condition:
    $e
}

Find high severity DMARC events (rejected mail)

rule high_severity_dmarc_events {
  meta:
    author = "parsedmarc"
    description = "Detect high severity DMARC aggregate events (rejected mail)"
    
  events:
    $e.metadata.product_name = "parsedmarc"
    $e.event_type = "DMARC_AGGREGATE"
    $e.security_result.severity = "HIGH"
    
  condition:
    $e
}

Find repeated DMARC failures from same source IP

rule repeated_dmarc_failures {
  meta:
    author = "parsedmarc"
    description = "Detect repeated DMARC failures from the same source IP"
    
  events:
    $e.metadata.product_name = "parsedmarc"
    $e.event_type = "DMARC_AGGREGATE"
    $e.security_result.detection_fields.key = "dmarc.pass"
    $e.security_result.detection_fields.value = false
    $e.principal.ip = $source_ip
    
  match:
    $source_ip over 1h
    
  condition:
    #e > 5
}

Find DMARC forensic reports with authentication failures

rule dmarc_forensic_failures {
  meta:
    author = "parsedmarc"
    description = "Detect DMARC forensic reports with authentication failures"
    
  events:
    $e.metadata.product_name = "parsedmarc"
    $e.event_type = "DMARC_FORENSIC"
    $e.security_result.detection_fields.key = "dmarc.auth_failure"
    
  condition:
    $e
}

Find SMTP TLS failures

rule smtp_tls_failures {
  meta:
    author = "parsedmarc"
    description = "Detect SMTP TLS failures"
    
  events:
    $e.metadata.product_name = "parsedmarc"
    $e.event_type = "SMTP_TLS_REPORT"
    
  condition:
    $e
}

Privacy Considerations

By default, forensic report message bodies are excluded from the output to protect privacy. If you need to include message samples for investigation:

1. Set include_ruf_payload = True in your configuration
2. Adjust ruf_payload_max_bytes to limit the amount of data included (default: 4096 bytes)
3. Message samples will be truncated if they exceed the configured maximum

Note: Be aware of data privacy regulations (GDPR, CCPA, etc.) when including message payloads in security telemetry.

Usage

The Google SecOps output works with all parsedmarc input methods, including file processing and mailbox monitoring.

Processing Files

To output DMARC reports from files to Google SecOps, redirect stdout or use the output in your ingestion pipeline:

# Output to stdout
parsedmarc -c config.ini samples/aggregate/*.xml > dmarc_events.ndjson

# Stream to file
parsedmarc -c config.ini samples/aggregate/*.xml >> /var/log/dmarc/events.ndjson

# Use with a log shipper (e.g., Fluentd, Logstash)
parsedmarc -c config.ini samples/aggregate/*.xml | your-log-shipper

Monitoring Mailboxes

The Google SecOps output automatically works when monitoring mailboxes via IMAP, Microsoft Graph, or Gmail API. Configure your mailbox connection and enable watching:

[general]
save_aggregate = True
save_forensic = True

[mailbox]
watch = True
delete = False
batch_size = 10

[imap]
host = imap.example.com
user = dmarc@example.com
password = yourpassword

[google_secops]
include_ruf_payload = False
static_observer_name = mailbox-monitor
static_environment = prod

When watching a mailbox, parsedmarc will continuously output UDM events to stdout as new reports arrive. Pipe this to your log shipper for real-time ingestion:

parsedmarc -c config.ini | fluentd

The output is in newline-delimited JSON format, with one UDM event per line, ready for ingestion into Google SecOps.