amorfo77/parsedmarc
1
0
Fork 0
mirror of https://github.com/domainaware/parsedmarc.git synced 2026-07-31 04:35:58 +00:00
Code Issues Projects Releases Wiki Activity

Store CBN parser booleans as typed bool_value

Browse Source 
The alignment flags and normalized_timespan were stored as
string_value "true"/"false" in additional.fields, which diverged from
the [gsecops] API output (a protobuf Struct, where they are real
booleans) — a UDM search written for one delivery path would not match
the other for those four fields.

Google's parser extension examples document the boolean idiom this
uses: build a string, convert to boolean, rename into the boolean
field. It is the boolean analogue of the number_value chain already
used for count/source_asn (proven in the content-hub Azure Cosmos DB
parser), and the UDM search docs confirm additional fields are
matchable via value.bool_value. The string conversion in step 1b
remains, because CBN conditionals compare against the preserved JSON
type and the == "true"/"false" guards need strings.

With this, both delivery paths emit identical value types for every
additional.fields key. The prior justification ("content-hub never
uses bool_value") described the absence of precedent, not a
prohibition; the documented boolean example is the stronger authority.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Sean Whalen
2026-07-11 15:32:12 -04:00
co-authored by Claude Fable 5
parent fba083f351
commit c8e8cb54c0
2 changed files with 64 additions and 17 deletions
Download Patch File Download Diff File Expand all files Collapse all files
google_secops_parser/README.md
+15 -6
View File
@@ -71,11 +71,16 @@ DMARC types.
2. **JSON types** — Chronicle's `json{}` filter **preserves the original JSON
type**, so parsedmarc's booleans and numbers are handled differently:
- **Booleans** (`dmarc_aligned` / `spf_aligned` / `dkim_aligned` /
`normalized_timespan`) are converted to strings so `[dmarc_aligned] ==
"false"` works, and stored as `string_value` (Google's content-hub parsers
never use `bool_value`). Note that `testing` is **not** a boolean —
parsedmarc emits the RFC 9990 `t=` flag as the string `"y"`/`"n"` — so it
is passed through as a string and guarded with `!= ""`.
`normalized_timespan`) are converted to strings because CBN conditionals
compare against the preserved type, so `[dmarc_aligned] == "false"` needs
a string. In `additional.fields` they are stored as typed **`bool_value`**
— built as a string, `convert`-ed back to `boolean`, then renamed — the
boolean pattern from Google's
[parser extension examples](https://docs.cloud.google.com/chronicle/docs/event-processing/parser-extension-examples);
UDM search matches them with `value.bool_value`. Note that `testing` is
**not** a boolean — parsedmarc emits the RFC 9990 `t=` flag as the string
`"y"`/`"n"` — so it is passed through as a string and guarded with
`!= ""`.
- **Numbers** (`count` / `*_session_count` / `source_asn`) are stored as
`number_value` — built as a string, `convert`-ed to `uinteger`, then
renamed — so SecOps can range-query and sort them (parsedmarc's "store
@@ -83,6 +88,9 @@ DMARC types.
so a tenant where `%{}` rejects non-string fields degrades to a missing
`additional.fields` entry instead of `_failed_parsing_`.
Both value types match what the `[gsecops]` API output emits, so UDM
searches and dashboards port between the two delivery paths unchanged.
Every `if`-tested field is initialized to `""` *before* `json` and guarded
with `!= ""`: CBN raises `_failed_parsing_` on a conditional referencing an
absent field, and treats an initialized-but-empty field as present. A
@@ -205,7 +213,8 @@ Suggested order:
depends on.
2. The two aggregates — confirm the `dmarc_aligned=false` one yields
`security_result.category = AUTH_VIOLATION`, that `count` and `source_asn`
land as `number_value`, and that the boolean→string handling holds.
land as `number_value`, and that the alignment booleans land as
`bool_value` (queryable as `additional.fields["dmarc_aligned"].value.bool_value`).
3. The SMTP TLS rows — confirm the success row produces a `GENERIC_EVENT` with
`target.hostname` and no `security_result`, and the failure rows produce
`security_result.action = FAIL`.
google_secops_parser/parsedmarc.conf
+49 -11
View File
@@ -58,13 +58,17 @@ filter {
# 2. JSON TYPES ARE PRESERVED. The CBN json{} filter keeps the original JSON
# type (a JSON boolean stays a boolean, a number stays a number).
# Booleans (*_aligned / normalized_timespan) are converted to strings
# (step 1b) so `== "true"` / `== "false"` tests work, and stored as
# string_value (content-hub never uses bool_value). "testing" is NOT a
# boolean: parsedmarc emits the RFC 9990 t= flag as the string "y"/"n".
# Numbers (count / *_session_count / source_asn) are stored as
# number_value -- built as a string, convert-ed to uinteger, then
# renamed to number_value -- so SecOps can range-query and sort them.
# Matches the content-hub parsers.
# (step 1b) because CBN conditionals compare against the preserved type,
# so `== "true"` / `== "false"` tests need strings. In additional.fields
# they are stored as typed bool_value -- built as a string, convert-ed
# back to boolean, renamed to bool_value -- following the boolean
# example in Google's parser extension docs; UDM search matches them
# with value.bool_value. "testing" is NOT a boolean: parsedmarc emits
# the RFC 9990 t= flag as the string "y"/"n". Numbers (count /
# *_session_count / source_asn) are stored the same way as number_value
# so SecOps can range-query and sort them. Value types match the
# [gsecops] API output, so searches port between the two paths.
# https://docs.cloud.google.com/chronicle/docs/event-processing/parser-extension-examples
# 2b. CONDITIONALS. Every field tested in an `if` is initialized to "" in
# step 1a, and guards use `if [field] != ""` (not bare `if [field]`,
# which is true for an initialized-but-empty field). Matches content-hub.
@@ -446,10 +450,23 @@ filter {
merge => { "event.idm.read_only_udm.additional.fields" => "f_disc" }
}
}
# Booleans are stored as typed bool_value via the documented pattern for
# boolean UDM fields (build string -> convert to boolean -> rename), the
# boolean analogue of the number_value chains below. UDM search matches
# them with additional.fields["..."].value.bool_value, and the type
# matches what parsedmarc's [gsecops] API output emits.
# https://docs.cloud.google.com/chronicle/docs/event-processing/parser-extension-examples
if [normalized_timespan] == "true" or [normalized_timespan] == "false" {
mutate {
replace => { "f_norm.key" => "normalized_timespan" "f_norm.value.string_value" => "%{normalized_timespan}" }
merge => { "event.idm.read_only_udm.additional.fields" => "f_norm" }
}
mutate {
convert => { "f_norm.value.string_value" => "boolean" }
on_error => "norm_not_bool"
}
if ![norm_not_bool] {
mutate { rename => { "f_norm.value.string_value" => "f_norm.value.bool_value" } }
mutate { merge => { "event.idm.read_only_udm.additional.fields" => "f_norm" } }
}
}
@@ -457,19 +474,40 @@ filter {
if [dmarc_aligned] == "true" or [dmarc_aligned] == "false" {
mutate {
replace => { "f_dmarc.key" => "dmarc_aligned" "f_dmarc.value.string_value" => "%{dmarc_aligned}" }
merge => { "event.idm.read_only_udm.additional.fields" => "f_dmarc" }
}
mutate {
convert => { "f_dmarc.value.string_value" => "boolean" }
on_error => "dmarc_not_bool"
}
if ![dmarc_not_bool] {
mutate { rename => { "f_dmarc.value.string_value" => "f_dmarc.value.bool_value" } }
mutate { merge => { "event.idm.read_only_udm.additional.fields" => "f_dmarc" } }
}
}
if [spf_aligned] == "true" or [spf_aligned] == "false" {
mutate {
replace => { "f_spfa.key" => "spf_aligned" "f_spfa.value.string_value" => "%{spf_aligned}" }
merge => { "event.idm.read_only_udm.additional.fields" => "f_spfa" }
}
mutate {
convert => { "f_spfa.value.string_value" => "boolean" }
on_error => "spfa_not_bool"
}
if ![spfa_not_bool] {
mutate { rename => { "f_spfa.value.string_value" => "f_spfa.value.bool_value" } }
mutate { merge => { "event.idm.read_only_udm.additional.fields" => "f_spfa" } }
}
}
if [dkim_aligned] == "true" or [dkim_aligned] == "false" {
mutate {
replace => { "f_dkima.key" => "dkim_aligned" "f_dkima.value.string_value" => "%{dkim_aligned}" }
merge => { "event.idm.read_only_udm.additional.fields" => "f_dkima" }
}
mutate {
convert => { "f_dkima.value.string_value" => "boolean" }
on_error => "dkima_not_bool"
}
if ![dkima_not_bool] {
mutate { rename => { "f_dkima.value.string_value" => "f_dkima.value.bool_value" } }
mutate { merge => { "event.idm.read_only_udm.additional.fields" => "f_dkima" } }
}
}
if [disposition] != "" {