feat(rapid7-insightidr): fetch and parse the evidence behind each alert (v1.8.0)
An alert names what fired. The evidence is what happened, and until now it was
never fetched — which is why an incident could show a rule name and nothing
else. On a Fortigate admin login the alert carried an empty
rule_keys_of_interest; the evidence carried the whole event.
It cannot be reached by mapping alone. The evidence's `data` is a
JSON-ENCODED STRING holding {source_data, custom_data}, and JSONata does not
deserialise, so no mapper expression can ever see inside it. The parsing has to
happen in the command, and that is the reason for this change more than the
extra call is.
Once parsed, two views of the same event:
custom_data the vendor's normalised fields, prefixed per source
(Forti_user, Forti_srcip) — 6 keys on the sample
source_data the raw line in key="value" form — 25 keys
Both are merged into evidence_fields with the vendor prefix stripped, so a
mapper names `user` once instead of once per vendor, and source_data is overlaid
last because it carries more and is not affected by the quoting artefact below.
Values are cleaned, and that is not cosmetic: custom_data returns
'10.70.20.66"' with a trailing quote left by the vendor parser. Kept as-is it
yields an address matching no regex — an observable nobody can pivot on, and no
error to say why.
The line itself lands on raw_data and the source's own sentence on message,
both of which the layout builder offers. The Fortigate login now maps to
src_endpoint.ip, dst_endpoint.ip, actor.user.name, device.hostname/name/uid,
activity_name "login" and status_detail "success", and indexes four entities
where it indexed none.
Optional (include_evidence, default true) and capped (max_evidence_per_alert,
default 5), one call per alert under the run's existing time budget. An alert
whose evidence cannot be read keeps an evidence_error and still arrives;
evidence_calls and evidence_errors are reported in the run metadata.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -20,6 +20,8 @@ field_mappings:
|
||||
# alert rule's default severity.
|
||||
severity: "priority"
|
||||
source: "source"
|
||||
# The log line's own sentence beats a generic title when there is one.
|
||||
description: "first_evidence_message"
|
||||
# Rapid7's own triage state. It cannot reach the incident's status and verdict columns
|
||||
# — MapFields accepts only title, severity, source, description and custom:* — so it
|
||||
# rides as custom fields, where an analyst at least sees that the source already closed
|
||||
@@ -140,6 +142,30 @@ ocsf:
|
||||
- { source_path: "alerts[0].rule_keys_of_interest[key='destination_ip'].values[0]", ocsf_field: "dst_endpoint.ip" }
|
||||
- { source_path: "alerts[0].rule_keys_of_interest[key='geoip_country_code'].values[0]", ocsf_field: "src_endpoint.location.country" }
|
||||
- { source_path: "alerts[0].rule_keys_of_interest[key='result'].values[0]", ocsf_field: "status_detail" }
|
||||
# ── The evidence: the log line the alert was raised on ────────────
|
||||
# An alert names what fired; the evidence is what actually happened. It lives
|
||||
# in a JSON-encoded string the command parses and normalises into
|
||||
# evidence_fields — see get_investigations_full.py. Vendor prefixes are
|
||||
# stripped there (Forti_srcip -> srcip), so one entry serves every log source
|
||||
# that names its fields the usual way.
|
||||
- { source_path: "alerts[0].evidence_fields", ocsf_field: "evidence" }
|
||||
- { source_path: "alerts[0].evidence_fields.srcip", ocsf_field: "src_endpoint.ip" }
|
||||
- { source_path: "alerts[0].evidence_fields.dstip", ocsf_field: "dst_endpoint.ip" }
|
||||
- { source_path: "alerts[0].evidence_fields.srcport", ocsf_field: "src_endpoint.port" }
|
||||
- { source_path: "alerts[0].evidence_fields.dstport", ocsf_field: "dst_endpoint.port" }
|
||||
- { source_path: "alerts[0].evidence_fields.user", ocsf_field: "actor.user.name" }
|
||||
- { source_path: "alerts[0].evidence_fields.devname", ocsf_field: "device.name" }
|
||||
- { source_path: "alerts[0].evidence_fields.devname", ocsf_field: "device.hostname" }
|
||||
- { source_path: "alerts[0].evidence_fields.devid", ocsf_field: "device.uid" }
|
||||
- { source_path: "alerts[0].evidence_fields.hostname", ocsf_field: "device.hostname" }
|
||||
- { source_path: "alerts[0].evidence_fields.status", ocsf_field: "status_detail" }
|
||||
# `action` is what the source system did ("login"), which says more than the
|
||||
# investigation's own `source` ("ALERT"). Mapped after it so it wins.
|
||||
- { source_path: "alerts[0].evidence_fields.action", ocsf_field: "activity_name" }
|
||||
# The line itself, on the OCSF field meant for it — and one the layout builder
|
||||
# offers, unlike `evidence` above, which only ever shows in the Context tab.
|
||||
- { source_path: "alerts[0].evidence_raw_line", ocsf_field: "raw_data" }
|
||||
- { source_path: "alerts[0].evidence_fields.msg ? alerts[0].evidence_fields.msg : alerts[0].evidence_fields.logdesc", ocsf_field: "message" }
|
||||
# ── Every alert, not just the first ───────────────────────────────
|
||||
# An investigation aggregates alerts, and the fields above only ever read alerts[0].
|
||||
# This entry lands the whole set as OCSF-shaped objects, so the host and the process of
|
||||
@@ -161,6 +187,7 @@ ocsf:
|
||||
"process": rule_keys_of_interest[key='process.name'].values[0],
|
||||
"cmd_line": rule_keys_of_interest[key='process.cmd_line'].values[0],
|
||||
"technique": rule.mitre_tcodes[$substring($, 0, 2) != 'TA'][0],
|
||||
"detection": rule_keys_of_interest{key: values[0]}
|
||||
"detection": rule_keys_of_interest{key: values[0]},
|
||||
"evidence": evidence_fields
|
||||
}
|
||||
ocsf_field: "evidences"
|
||||
|
||||
Reference in New Issue
Block a user