Add offline mode for tests in GitHub Actions to skip DNS lookups

Co-authored-by: seanthegeek <44679+seanthegeek@users.noreply.github.com>

docs/source/usage.md

@@ -171,8 +171,8 @@ The full set of configuration options are:
   - `check_timeout` - int: Number of seconds to wait for a IMAP
       IDLE response or the number of seconds until the next
       mail check (Default: `30`)
-  - `since` - str: Search for messages since certain time. (Examples: `5m|3h|2d|1w`)
-      Acceptable units - {"m":"minutes", "h":"hours", "d":"days", "w":"weeks"}.
+  - `since` - str: Search for messages since certain time. (Examples: `5m|3h|2d|1w`) 
+      Acceptable units - {"m":"minutes", "h":"hours", "d":"days", "w":"weeks"}. 
       Defaults to `1d` if incorrect value is provided.
 - `imap`
   - `host` - str: The IMAP server hostname or IP address
@@ -240,7 +240,7 @@ The full set of configuration options are:
     group and use that as the group id.
 
     ```powershell
-    New-ApplicationAccessPolicy -AccessRight RestrictAccess
+    New-ApplicationAccessPolicy -AccessRight RestrictAccess 
     -AppId "<CLIENT_ID>" -PolicyScopeGroupId "<MAILBOX>"
     -Description "Restrict access to dmarc reports mailbox."
     ```
@@ -336,65 +336,13 @@ The full set of configuration options are:
   - `secret_access_key` - str: The secret access key (Optional)
 - `syslog`
   - `server` - str: The Syslog server name or IP address
-  - `port` - int: The port to use (Default: `514`)
-  - `protocol` - str: The protocol to use: `udp`, `tcp`, or `tls` (Default: `udp`)
-  - `cafile_path` - str: Path to CA certificate file for TLS server verification (Optional)
-  - `certfile_path` - str: Path to client certificate file for TLS authentication (Optional)
-  - `keyfile_path` - str: Path to client private key file for TLS authentication (Optional)
-  - `timeout` - float: Connection timeout in seconds for TCP/TLS (Default: `5.0`)
-  - `retry_attempts` - int: Number of retry attempts for failed connections (Default: `3`)
-  - `retry_delay` - int: Delay in seconds between retry attempts (Default: `5`)
-
-  **Example UDP configuration (default):**
-
-  ```ini
-  [syslog]
-  server = syslog.example.com
-  port = 514
-  ```
-
-  **Example TCP configuration:**
-
-  ```ini
-  [syslog]
-  server = syslog.example.com
-  port = 6514
-  protocol = tcp
-  timeout = 10.0
-  retry_attempts = 5
-  ```
-
-  **Example TLS configuration with server verification:**
-
-  ```ini
-  [syslog]
-  server = syslog.example.com
-  port = 6514
-  protocol = tls
-  cafile_path = /path/to/ca-cert.pem
-  timeout = 10.0
-  ```
-
-  **Example TLS configuration with mutual authentication:**
-
-  ```ini
-  [syslog]
-  server = syslog.example.com
-  port = 6514
-  protocol = tls
-  cafile_path = /path/to/ca-cert.pem
-  certfile_path = /path/to/client-cert.pem
-  keyfile_path = /path/to/client-key.pem
-  timeout = 10.0
-  retry_attempts = 3
-  retry_delay = 5
-  ```
+  - `port` - int: The UDP port to use (Default: `514`)
 - `gmail_api`
   - `credentials_file` - str: Path to file containing the
       credentials, None to disable (Default: `None`)
   - `token_file` - str: Path to save the token file
       (Default: `.token`)
-
+      
     :::{note}
     credentials_file and token_file can be got with [quickstart](https://developers.google.com/gmail/api/quickstart/python).Please change the scope to `https://www.googleapis.com/auth/gmail.modify`.
     :::
@@ -494,7 +442,7 @@ Update the limit to 2k per example:
 PUT _cluster/settings
 {
   "persistent" : {
-    "cluster.max_shards_per_node" : 2000
+    "cluster.max_shards_per_node" : 2000 
   }
 }
 ```

google_secops_parser/README.md

@@ -1,132 +0,0 @@
-# Google SecOps Parser for parsedmarc
-
-A [Google Security Operations (Chronicle)](https://cloud.google.com/security/products/security-operations) custom parser for ingesting [parsedmarc](https://domainaware.github.io/parsedmarc/) syslog events into the Unified Data Model (UDM).
-
-## Overview
-
-parsedmarc sends DMARC aggregate reports, forensic reports, and SMTP TLS reports as JSON-formatted syslog messages. This parser transforms those JSON events into Google SecOps UDM events for threat detection and investigation.
-
-### Supported Report Types
-
-| Report Type | UDM Event Type | Description |
-|---|---|---|
-| DMARC Aggregate | `EMAIL_TRANSACTION` | Aggregate DMARC authentication results from reporting organizations |
-| DMARC Forensic | `EMAIL_TRANSACTION` | Individual email authentication failure reports |
-| SMTP TLS | `GENERIC_EVENT` | SMTP TLS session success/failure reports (RFC 8460) |
-
-## UDM Field Mappings
-
-### DMARC Aggregate Reports
-
-| parsedmarc Field | UDM Field | Notes |
-|---|---|---|
-| `source_ip_address` | `principal.ip` | IP address of the email source |
-| `source_reverse_dns` | `principal.hostname` | Reverse DNS of source |
-| `source_country` | `principal.location.country_or_region` | GeoIP country of source |
-| `header_from` | `network.email.from` | From header domain |
-| `envelope_from` | `network.email.mail_from` | Envelope sender |
-| `envelope_to` | `network.email.to` | Envelope recipient |
-| `domain` | `target.hostname` | Domain the report is about |
-| `report_id` | `metadata.product_log_id` | Report identifier |
-| `disposition` | `security_result.action` | `none`→`ALLOW`, `quarantine`→`QUARANTINE`, `reject`→`BLOCK` |
-| `dmarc_aligned` | `additional.fields` | Whether DMARC passed |
-| `spf_aligned` | `additional.fields` | Whether SPF was aligned |
-| `dkim_aligned` | `additional.fields` | Whether DKIM was aligned |
-| `org_name` | `additional.fields` | Reporting organization name |
-| `count` | `additional.fields` | Number of messages |
-| `p`, `sp`, `pct` | `additional.fields` | DMARC policy settings |
-| `dkim_domains`, `dkim_results` | `additional.fields` | DKIM authentication details |
-| `spf_domains`, `spf_results` | `additional.fields` | SPF authentication details |
-
-### DMARC Forensic Reports
-
-| parsedmarc Field | UDM Field | Notes |
-|---|---|---|
-| `source_ip_address` | `principal.ip` | IP address of the email source |
-| `source_reverse_dns` | `principal.hostname` | Reverse DNS of source |
-| `source_country` | `principal.location.country_or_region` | GeoIP country of source |
-| `original_mail_from` | `network.email.from` | Original sender |
-| `original_rcpt_to` | `network.email.to` | Original recipient |
-| `subject` | `network.email.subject` | Email subject |
-| `reported_domain` | `target.hostname` | Reported domain |
-| `message_id` | `metadata.product_log_id` | Email message ID |
-| `arrival_date_utc` | `metadata.event_timestamp` | Arrival timestamp (UTC) |
-| `auth_failure` | `security_result.description` | Type of authentication failure |
-| `feedback_type` | `additional.fields` | Feedback report type |
-| `authentication_results` | `additional.fields` | Full authentication results string |
-| `delivery_result` | `additional.fields` | Email delivery outcome |
-
-### SMTP TLS Reports
-
-| parsedmarc Field | UDM Field | Notes |
-|---|---|---|
-| `sending_mta_ip` | `principal.ip` | Sending MTA IP address |
-| `receiving_ip` | `target.ip` | Receiving MTA IP address |
-| `receiving_mx_hostname` | `target.hostname` | Receiving MX hostname |
-| `report_id` | `metadata.product_log_id` | Report identifier |
-| `organization_name` | `additional.fields` | Reporting organization |
-| `policy_domain` | `additional.fields` | Policy domain |
-| `policy_type` | `additional.fields` | TLS policy type |
-| `successful_session_count` | `additional.fields` | Count of successful TLS sessions |
-| `failed_session_count` | `additional.fields` | Count of failed TLS sessions |
-| `result_type` | `additional.fields` | Failure result type |
-| `failure_reason_code` | `additional.fields` | Failure reason code |
-
-## Installation
-
-### Prerequisites
-
-- A Google Security Operations (Chronicle) tenant
-- parsedmarc configured to send syslog output (see [parsedmarc documentation](https://domainaware.github.io/parsedmarc/))
-
-### Steps
-
-1. **Configure parsedmarc syslog output** in your `parsedmarc.ini`:
-
-   ```ini
-   [syslog]
-   server = your-chronicle-forwarder.example.com
-   port = 514
-   ```
-
-2. **Create the log source** in Google SecOps:
-   - Navigate to **Settings** → **Feeds** → **Add New**
-   - Select **Syslog** as the source type
-   - Configure to listen for parsedmarc syslog messages
-
-3. **Upload the custom parser**:
-   - Navigate to **Settings** → **Parsers**
-   - Click **Create Custom Parser**
-   - Set the **Log Type** to match your feed configuration
-   - Paste the contents of `parsedmarc.conf`
-   - Click **Submit**
-
-4. **Validate** the parser using the Chronicle parser validation tool with sample parsedmarc JSON events.
-
-## Sample Log Events
-
-### Aggregate Report
-
-```json
-{"xml_schema": "1.0", "org_name": "Example Inc", "org_email": "noreply@example.net", "report_id": "abc123", "begin_date": "2024-01-01 00:00:00", "end_date": "2024-01-01 23:59:59", "domain": "example.com", "adkim": "r", "aspf": "r", "p": "reject", "sp": "reject", "pct": "100", "fo": "0", "source_ip_address": "203.0.113.1", "source_country": "United States", "source_reverse_dns": "mail.example.org", "source_base_domain": "example.org", "count": 42, "spf_aligned": true, "dkim_aligned": true, "dmarc_aligned": true, "disposition": "none", "header_from": "example.com", "envelope_from": "example.com", "envelope_to": null, "dkim_domains": "example.com", "dkim_selectors": "selector1", "dkim_results": "pass", "spf_domains": "example.com", "spf_scopes": "mfrom", "spf_results": "pass"}
-```
-
-### Forensic Report
-
-```json
-{"feedback_type": "auth-failure", "user_agent": "Lua/1.0", "version": "1.0", "original_mail_from": "sender@example.com", "original_rcpt_to": "recipient@example.org", "arrival_date": "Mon, 01 Jan 2024 12:00:00 +0000", "arrival_date_utc": "2024-01-01 12:00:00", "source_ip_address": "198.51.100.1", "source_country": "Germany", "source_reverse_dns": "mail.example.com", "source_base_domain": "example.com", "subject": "Test Email", "message_id": "<abc@example.com>", "authentication_results": "dmarc=fail (p=reject; dis=reject) header.from=example.com", "dkim_domain": "example.com", "delivery_result": "reject", "auth_failure": "dmarc", "reported_domain": "example.com", "authentication_mechanisms": "dmarc"}
-```
-
-### SMTP TLS Report
-
-```json
-{"organization_name": "Example Inc", "begin_date": "2024-01-01 00:00:00", "end_date": "2024-01-01 23:59:59", "report_id": "tls-123", "policy_domain": "example.com", "policy_type": "sts", "policy_strings": "version: STSv1; mode: enforce", "mx_host_patterns": "*.mail.example.com", "successful_session_count": 1000, "failed_session_count": 5, "result_type": "certificate-expired", "sending_mta_ip": "203.0.113.10", "receiving_ip": "198.51.100.20", "receiving_mx_hostname": "mx.example.com", "receiving_mx_helo": "mx.example.com", "failure_reason_code": "X509_V_ERR_CERT_HAS_EXPIRED"}
-```
-
-## UDM Reference
-
-For the complete list of UDM fields, see the [Google SecOps UDM field list](https://cloud.google.com/chronicle/docs/reference/udm-field-list).
-
-## License
-
-This parser is part of the [parsedmarc](https://github.com/domainaware/parsedmarc) project and is distributed under the same license.

parsedmarc/cli.py

@@ -697,13 +697,6 @@ def _main():
         s3_secret_access_key=None,
         syslog_server=None,
         syslog_port=None,
-        syslog_protocol=None,
-        syslog_cafile_path=None,
-        syslog_certfile_path=None,
-        syslog_keyfile_path=None,
-        syslog_timeout=None,
-        syslog_retry_attempts=None,
-        syslog_retry_delay=None,
         gmail_api_credentials_file=None,
         gmail_api_token_file=None,
         gmail_api_include_spam_trash=False,
@@ -1246,28 +1239,6 @@ def _main():
                 opts.syslog_port = syslog_config["port"]
             else:
                 opts.syslog_port = 514
-            if "protocol" in syslog_config:
-                opts.syslog_protocol = syslog_config["protocol"]
-            else:
-                opts.syslog_protocol = "udp"
-            if "cafile_path" in syslog_config:
-                opts.syslog_cafile_path = syslog_config["cafile_path"]
-            if "certfile_path" in syslog_config:
-                opts.syslog_certfile_path = syslog_config["certfile_path"]
-            if "keyfile_path" in syslog_config:
-                opts.syslog_keyfile_path = syslog_config["keyfile_path"]
-            if "timeout" in syslog_config:
-                opts.syslog_timeout = float(syslog_config["timeout"])
-            else:
-                opts.syslog_timeout = 5.0
-            if "retry_attempts" in syslog_config:
-                opts.syslog_retry_attempts = int(syslog_config["retry_attempts"])
-            else:
-                opts.syslog_retry_attempts = 3
-            if "retry_delay" in syslog_config:
-                opts.syslog_retry_delay = int(syslog_config["retry_delay"])
-            else:
-                opts.syslog_retry_delay = 5
 
         if "gmail_api" in config.sections():
             gmail_api_config = config["gmail_api"]
@@ -1465,13 +1436,6 @@ def _main():
             syslog_client = syslog.SyslogClient(
                 server_name=opts.syslog_server,
                 server_port=int(opts.syslog_port),
-                protocol=opts.syslog_protocol or "udp",
-                cafile_path=opts.syslog_cafile_path,
-                certfile_path=opts.syslog_certfile_path,
-                keyfile_path=opts.syslog_keyfile_path,
-                timeout=opts.syslog_timeout if opts.syslog_timeout is not None else 5.0,
-                retry_attempts=opts.syslog_retry_attempts if opts.syslog_retry_attempts is not None else 3,
-                retry_delay=opts.syslog_retry_delay if opts.syslog_retry_delay is not None else 5,
             )
         except Exception as error_:
             logger.error("Syslog Error: {0}".format(error_.__str__()))

parsedmarc/syslog.py

@@ -6,10 +6,7 @@ from __future__ import annotations
 import json
 import logging
 import logging.handlers
-import socket
-import ssl
-import time
-from typing import Any, Optional
+from typing import Any
 
 from parsedmarc import (
     parsed_aggregate_reports_to_csv_rows,
@@ -21,150 +18,20 @@ from parsedmarc import (
 class SyslogClient(object):
     """A client for Syslog"""
 
-    def __init__(
-        self,
-        server_name: str,
-        server_port: int,
-        protocol: str = "udp",
-        cafile_path: Optional[str] = None,
-        certfile_path: Optional[str] = None,
-        keyfile_path: Optional[str] = None,
-        timeout: float = 5.0,
-        retry_attempts: int = 3,
-        retry_delay: int = 5,
-    ):
+    def __init__(self, server_name: str, server_port: int):
         """
         Initializes the SyslogClient
         Args:
             server_name (str): The Syslog server
-            server_port (int): The Syslog port
-            protocol (str): The protocol to use: "udp", "tcp", or "tls" (Default: "udp")
-            cafile_path (str): Path to CA certificate file for TLS server verification (Optional)
-            certfile_path (str): Path to client certificate file for TLS authentication (Optional)
-            keyfile_path (str): Path to client private key file for TLS authentication (Optional)
-            timeout (float): Connection timeout in seconds for TCP/TLS (Default: 5.0)
-            retry_attempts (int): Number of retry attempts for failed connections (Default: 3)
-            retry_delay (int): Delay in seconds between retry attempts (Default: 5)
+            server_port (int): The Syslog UDP port
         """
         self.server_name = server_name
         self.server_port = server_port
-        self.protocol = protocol.lower()
-        self.timeout = timeout
-        self.retry_attempts = retry_attempts
-        self.retry_delay = retry_delay
-
         self.logger = logging.getLogger("parsedmarc_syslog")
         self.logger.setLevel(logging.INFO)
-
-        # Create the appropriate syslog handler based on protocol
-        log_handler = self._create_syslog_handler(
-            server_name,
-            server_port,
-            self.protocol,
-            cafile_path,
-            certfile_path,
-            keyfile_path,
-            timeout,
-            retry_attempts,
-            retry_delay,
-        )
-
+        log_handler = logging.handlers.SysLogHandler(address=(server_name, server_port))
         self.logger.addHandler(log_handler)
 
-    def _create_syslog_handler(
-        self,
-        server_name: str,
-        server_port: int,
-        protocol: str,
-        cafile_path: Optional[str],
-        certfile_path: Optional[str],
-        keyfile_path: Optional[str],
-        timeout: float,
-        retry_attempts: int,
-        retry_delay: int,
-    ) -> logging.handlers.SysLogHandler:
-        """
-        Creates a SysLogHandler with the specified protocol and TLS settings
-        """
-        if protocol == "udp":
-            # UDP protocol (default, backward compatible)
-            return logging.handlers.SysLogHandler(
-                address=(server_name, server_port),
-                socktype=socket.SOCK_DGRAM,
-            )
-        elif protocol in ["tcp", "tls"]:
-            # TCP or TLS protocol with retry logic
-            for attempt in range(1, retry_attempts + 1):
-                try:
-                    if protocol == "tcp":
-                        # TCP without TLS
-                        handler = logging.handlers.SysLogHandler(
-                            address=(server_name, server_port),
-                            socktype=socket.SOCK_STREAM,
-                        )
-                        # Set timeout on the socket
-                        if hasattr(handler, "socket") and handler.socket:
-                            handler.socket.settimeout(timeout)
-                        return handler
-                    else:
-                        # TLS protocol
-                        # Create SSL context with secure defaults
-                        ssl_context = ssl.create_default_context()
-
-                        # Explicitly set minimum TLS version to 1.2 for security
-                        ssl_context.minimum_version = ssl.TLSVersion.TLSv1_2
-
-                        # Configure server certificate verification
-                        if cafile_path:
-                            ssl_context.load_verify_locations(cafile=cafile_path)
-
-                        # Configure client certificate authentication
-                        if certfile_path and keyfile_path:
-                            ssl_context.load_cert_chain(
-                                certfile=certfile_path,
-                                keyfile=keyfile_path,
-                            )
-                        elif certfile_path or keyfile_path:
-                            # Warn if only one of the two required parameters is provided
-                            self.logger.warning(
-                                "Both certfile_path and keyfile_path are required for "
-                                "client certificate authentication. Client authentication "
-                                "will not be used."
-                            )
-
-                        # Create TCP handler first
-                        handler = logging.handlers.SysLogHandler(
-                            address=(server_name, server_port),
-                            socktype=socket.SOCK_STREAM,
-                        )
-
-                        # Wrap socket with TLS
-                        if hasattr(handler, "socket") and handler.socket:
-                            handler.socket = ssl_context.wrap_socket(
-                                handler.socket,
-                                server_hostname=server_name,
-                            )
-                            handler.socket.settimeout(timeout)
-
-                        return handler
-
-                except Exception as e:
-                    if attempt < retry_attempts:
-                        self.logger.warning(
-                            f"Syslog connection attempt {attempt}/{retry_attempts} failed: {e}. "
-                            f"Retrying in {retry_delay} seconds..."
-                        )
-                        time.sleep(retry_delay)
-                    else:
-                        self.logger.error(
-                            f"Syslog connection failed after {retry_attempts} attempts: {e}"
-                        )
-                        raise
-        else:
-            raise ValueError(
-                f"Invalid protocol '{protocol}'. Must be 'udp', 'tcp', or 'tls'."
-            )
-
     def save_aggregate_report_to_syslog(self, aggregate_reports: list[dict[str, Any]]):
         rows = parsed_aggregate_reports_to_csv_rows(aggregate_reports)
         for row in rows: