f3nris
1888648930
Rapid7 exposes a process tree per alert; the layout renderer has a block that
draws one. They do not speak the same shape.
The block reads ctx.process_tree as N FLAT chains, root first, each
{alert_id, rule, host, user, nodes[]} — it merges shared ancestors itself.
Rapid7 answers with a NESTED tree: one root entity and children[]. So the
command walks each root-to-leaf path into its own chain, maps the node fields
(cmdline -> cmd_line, hashes.md5 -> md5, start_time -> started_at, pid coerced
from the string Rapid7 sends), and rolls every alert's chains up onto the
investigation, since the block reads the key at the top level and no mapper
expression reaches past alerts[0].
Verified through the block's own reader: a tree of explorer -> cmd ->
{bcdedit, vssadmin} flattens to two chains and comes back as one tree with the
ancestors shared and both leaves marked.
That mark is worth stating plainly. The block marks the last node of a chain as
the target, and Rapid7 flags no node as the one the alert fired on — so here it
means "end of this branch" and nothing stronger. Naming a target would assert
something the sensor never said.
Gated, because this is a third call per alert on top of the evidence one:
include_process_tree=auto (the default) asks only for the event types an
endpoint sensor produces a tree for, true asks for every alert, false never.
An authentication or firewall alert never has one, and on a feed where those
dominate the default saves a call per alert. 404 is treated as "no tree", which
is what it means, and only other statuses are recorded as errors. Bounded at 12
chains and 24 levels so an unexpected shape cannot expand without limit.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
201 lines
16 KiB
YAML
201 lines
16 KiB
YAML
name: "Rapid7 InsightIDR Investigations (full) → OCSF"
|
|
description: |
|
|
Maps one Rapid7 investigation fetched with its alerts inlined
|
|
(get_investigations_full, results_path = data) to OCSF finding fields.
|
|
|
|
The plain investigation feed cannot carry a detection: idr/v2/investigations answers
|
|
with a title, a priority, a status, an assignee and timestamps, and nothing else. An
|
|
incident built from it indexes zero entities. This mapper reads the same investigation
|
|
plus the alerts the command inlined under `alerts`, where the detection content
|
|
actually lives.
|
|
|
|
Investigation values are mapped first and the first alert's equivalents after, so the
|
|
alert wins wherever the container says nothing. `alerts` is empty on an investigation
|
|
whose alerts could not be read (the command sets alerts_error) — every alert path then
|
|
resolves to nothing and is skipped, and the incident degrades to what the investigation
|
|
alone says rather than failing.
|
|
field_mappings:
|
|
title: "title"
|
|
# toSeverity maps CRITICAL→5, HIGH→4, MEDIUM→3, LOW→2; anything else falls back to the
|
|
# alert rule's default severity.
|
|
severity: "priority"
|
|
source: "source"
|
|
# The log line's own sentence beats a generic title when there is one.
|
|
description: "first_evidence_message"
|
|
# Rapid7's own triage state. It cannot reach the incident's status and verdict columns
|
|
# — MapFields accepts only title, severity, source, description and custom:* — so it
|
|
# rides as custom fields, where an analyst at least sees that the source already closed
|
|
# this and what it concluded.
|
|
"custom:rapid7_status": "status"
|
|
"custom:rapid7_disposition": "disposition"
|
|
# Rapid7's AI suggestion, exposed as a fact and never as a verdict.
|
|
# first_alert_* are lifted to the top level by the command on purpose: this
|
|
# mapper section resolves dotted paths only and cannot index an array, so
|
|
# `alerts[0].x` here would read as nothing — silently, leaving the custom field
|
|
# empty with no error. The ocsf section below is full JSONata and indexes freely.
|
|
"custom:rapid7_ai_disposition": "first_alert_ai_disposition"
|
|
"custom:rapid7_alert_count": "alert_count"
|
|
"custom:rapid7_log_id": "first_alert_log_id"
|
|
"custom:rapid7_logset_id": "first_alert_logset_id"
|
|
"custom:rapid7_rule_rrn": "first_alert_rule_rrn"
|
|
"custom:rapid7_event_type": "first_alert_event_type"
|
|
ocsf:
|
|
# ── Finding: the investigation ────────────────────────────────────
|
|
- { source_path: "rrn", ocsf_field: "finding_info.uid" }
|
|
- { source_path: "title", ocsf_field: "finding_info.title" }
|
|
- { source_path: "created_time", ocsf_field: "finding_info.created_time" }
|
|
- { source_path: "last_accessed", ocsf_field: "finding_info.modified_time" }
|
|
- { source_path: "first_alert_time", ocsf_field: "finding_info.first_seen_time" }
|
|
- { source_path: "latest_alert_time", ocsf_field: "finding_info.last_seen_time" }
|
|
- { source_path: "tags", ocsf_field: "metadata.labels" }
|
|
- { source_path: "alert_count", ocsf_field: "count" }
|
|
# ── Incident state ────────────────────────────────────────────────
|
|
- { source_path: "status", ocsf_field: "status" }
|
|
- { source_path: "priority", ocsf_field: "priority" }
|
|
- { source_path: "disposition", ocsf_field: "disposition" }
|
|
- { source_path: "source", ocsf_field: "activity_name" }
|
|
- { source_path: "organization_id", ocsf_field: "cloud.org.uid" }
|
|
- { source_path: "assignee.name", ocsf_field: "assignee.name" }
|
|
- { source_path: "assignee.email", ocsf_field: "assignee.email_addr" }
|
|
# ── Detection time: the MTTD anchor ───────────────────────────────
|
|
# Weakest first, strongest last. The investigation is created after the alert that
|
|
# opened it; the alert's own alerted_at is when the event happened in the source system.
|
|
- { source_path: "created_time", ocsf_field: "time" }
|
|
- { source_path: "first_alert_time", ocsf_field: "time" }
|
|
- { source_path: "alerts[0].alerted_at", ocsf_field: "time" }
|
|
# ── The detection rule, from the first alert ──────────────────────
|
|
# `rule.name` is documented but absent from live ABA alerts, so `type` — which repeats
|
|
# the rule's display name — is the honest fallback.
|
|
- { source_path: "alerts[0].type", ocsf_field: "finding_info.analytic.name" }
|
|
- { source_path: "alerts[0].rule.rrn", ocsf_field: "finding_info.analytic.uid" }
|
|
- { source_path: "alerts[0].rule.version_rrn", ocsf_field: "finding_info.analytic.version" }
|
|
- { source_path: "alerts[0].rrn", ocsf_field: "finding_info.uid_alt" }
|
|
# ── Product identity ──────────────────────────────────────────────
|
|
- { source_path: "'Rapid7 InsightIDR'", ocsf_field: "metadata.product.name" }
|
|
- { source_path: "'Rapid7'", ocsf_field: "metadata.product.vendor_name" }
|
|
- { source_path: "alerts[0].external_source", ocsf_field: "metadata.log_source" }
|
|
- { source_path: "alerts[0].triggering_event_type", ocsf_field: "metadata.event_code" }
|
|
- { source_path: "alerts[0].log_details[0].log_entry_id", ocsf_field: "metadata.uid" }
|
|
# ── MITRE ATT&CK ──────────────────────────────────────────────────
|
|
# mitre_tcodes mixes both kinds in one array: "TA0040" is a tactic, "T1490" a technique.
|
|
# Splitting on the "TA" prefix keeps a tactic out of the technique field, where it would
|
|
# name a technique that does not exist.
|
|
- { source_path: "alerts[0].rule.mitre_tcodes[$substring($, 0, 2) = 'TA'][0]", ocsf_field: "attacks.tactic.uid" }
|
|
- { source_path: "alerts[0].rule.mitre_tcodes[$substring($, 0, 2) != 'TA'][0]", ocsf_field: "attacks.technique.uid" }
|
|
# ── The affected host ─────────────────────────────────────────────
|
|
- { source_path: "alerts[0].rule_keys_of_interest[key='hostname'].values[0]", ocsf_field: "device.hostname" }
|
|
- { source_path: "alerts[0].rule_keys_of_interest[key='hostname'].values[0]", ocsf_field: "src_endpoint.hostname" }
|
|
# ── The process the alert is about ────────────────────────────────
|
|
- { source_path: "alerts[0].rule_keys_of_interest[key='process.name'].values[0]", ocsf_field: "process.name" }
|
|
- { source_path: "alerts[0].rule_keys_of_interest[key='process.name'].values[0]", ocsf_field: "process.file.name" }
|
|
- { source_path: "alerts[0].rule_keys_of_interest[key='process.cmd_line'].values[0]", ocsf_field: "process.cmd_line" }
|
|
- { source_path: "alerts[0].rule_keys_of_interest[key='process.exe_path'].values[0]", ocsf_field: "process.file.path" }
|
|
- { source_path: "alerts[0].rule_keys_of_interest[key='process.exe_file.hashes.md5'].values[0]", ocsf_field: "process.file.hashes.md5" }
|
|
- { source_path: "alerts[0].rule_keys_of_interest[key='process.exe_file.hashes.sha256'].values[0]", ocsf_field: "process.file.hashes.sha256" }
|
|
# ── Its parent, which says how it was launched ────────────────────
|
|
- { source_path: "alerts[0].rule_keys_of_interest[key='parent_process.name'].values[0]", ocsf_field: "actor.process.parent_process.name" }
|
|
- { source_path: "alerts[0].rule_keys_of_interest[key='parent_process.cmd_line'].values[0]", ocsf_field: "actor.process.parent_process.cmd_line" }
|
|
- { source_path: "alerts[0].rule_keys_of_interest[key='parent_process.exe_path'].values[0]", ocsf_field: "actor.process.parent_process.file.path" }
|
|
- { source_path: "alerts[0].rule_keys_of_interest[key='parent_process.exe_file.hashes.md5'].values[0]", ocsf_field: "actor.process.parent_process.file.hashes.md5" }
|
|
# ── The acting account ────────────────────────────────────────────
|
|
# This tenant's field catalogue has no canonical user namespace, so an account only
|
|
# appears when a rule names it among its keys of interest. Both spellings are mapped
|
|
# because rules are not consistent about which they emit.
|
|
- { source_path: "alerts[0].rule_keys_of_interest[key='user'].values[0]", ocsf_field: "actor.user.name" }
|
|
- { source_path: "alerts[0].rule_keys_of_interest[key='username'].values[0]", ocsf_field: "actor.user.name" }
|
|
# ── Lateral movement: source and destination ──────────────────────
|
|
# An asset_auth rule describes a connection, so it names BOTH ends and neither
|
|
# is "the host": source_asset_address / destination_asset_address, and
|
|
# source_account / destination_account. Mapping them to src_endpoint and
|
|
# dst_endpoint is not only the correct OCSF shape — it is also what makes them
|
|
# appear in the layout builder, whose field picker is a closed <select> over
|
|
# the OCSF catalogue. A path that is not in that catalogue cannot be placed on
|
|
# a layout at all, which is why `detection` below, useful as it is, only ever
|
|
# shows up in the Context tab.
|
|
- { source_path: "alerts[0].rule_keys_of_interest[key='source_asset_address'].values[0]", ocsf_field: "src_endpoint.hostname" }
|
|
- { source_path: "alerts[0].rule_keys_of_interest[key='destination_asset_address'].values[0]", ocsf_field: "dst_endpoint.hostname" }
|
|
- { source_path: "alerts[0].rule_keys_of_interest[key='destination_asset'].values[0]", ocsf_field: "dst_endpoint.hostname" }
|
|
# actor.user is who acted, user is who it was done TO. Keeping them apart is
|
|
# what lets an analyst read a lateral move in one line.
|
|
- { source_path: "alerts[0].rule_keys_of_interest[key='source_account'].values[0]", ocsf_field: "actor.user.name" }
|
|
- { source_path: "alerts[0].rule_keys_of_interest[key='destination_account'].values[0]", ocsf_field: "user.name" }
|
|
# ── Every key the rule found interesting ──────────────────────────
|
|
# rule_keys_of_interest is an OPEN vocabulary: each detection rule names its own
|
|
# keys. A process_start_event rule emits hostname / process.cmd_line /
|
|
# parent_process.*; an ingress_auth rule emits account / source_ip /
|
|
# geoip_country_code / result and none of the above. The explicit entries below
|
|
# can therefore only ever cover the rules someone has already seen — on any
|
|
# other rule they all resolve to nothing and the incident is empty again, which
|
|
# is the exact failure this integration started from.
|
|
#
|
|
# So the whole list is also flattened, verbatim, into one object. Keys nobody
|
|
# anticipated arrive intact and are visible to the analyst and to the AI
|
|
# analysis, which reads the mapped context but never `raw`. The explicit
|
|
# mappings stay because they are what types an entity correctly — a value under
|
|
# `detection.` is only ever matched by the IOC regex, so an account there would
|
|
# never be indexed as a user.
|
|
- { source_path: "alerts[0].rule_keys_of_interest{key: values[0]}", ocsf_field: "detection" }
|
|
- { source_path: "alerts[0].rule_matching_keys{key: values[0]}", ocsf_field: "detection_match" }
|
|
# ── Keys seen on non-process rules ────────────────────────────────
|
|
- { source_path: "alerts[0].rule_keys_of_interest[key='account'].values[0]", ocsf_field: "actor.user.account.name" }
|
|
- { source_path: "alerts[0].rule_keys_of_interest[key='source_ip'].values[0]", ocsf_field: "src_endpoint.ip" }
|
|
- { source_path: "alerts[0].rule_keys_of_interest[key='destination_ip'].values[0]", ocsf_field: "dst_endpoint.ip" }
|
|
- { source_path: "alerts[0].rule_keys_of_interest[key='geoip_country_code'].values[0]", ocsf_field: "src_endpoint.location.country" }
|
|
- { source_path: "alerts[0].rule_keys_of_interest[key='result'].values[0]", ocsf_field: "status_detail" }
|
|
# ── The evidence: the log line the alert was raised on ────────────
|
|
# An alert names what fired; the evidence is what actually happened. It lives
|
|
# in a JSON-encoded string the command parses and normalises into
|
|
# evidence_fields — see get_investigations_full.py. Vendor prefixes are
|
|
# stripped there (Forti_srcip -> srcip), so one entry serves every log source
|
|
# that names its fields the usual way.
|
|
- { source_path: "alerts[0].evidence_fields", ocsf_field: "evidence" }
|
|
- { source_path: "alerts[0].evidence_fields.srcip", ocsf_field: "src_endpoint.ip" }
|
|
- { source_path: "alerts[0].evidence_fields.dstip", ocsf_field: "dst_endpoint.ip" }
|
|
- { source_path: "alerts[0].evidence_fields.srcport", ocsf_field: "src_endpoint.port" }
|
|
- { source_path: "alerts[0].evidence_fields.dstport", ocsf_field: "dst_endpoint.port" }
|
|
- { source_path: "alerts[0].evidence_fields.user", ocsf_field: "actor.user.name" }
|
|
- { source_path: "alerts[0].evidence_fields.devname", ocsf_field: "device.name" }
|
|
- { source_path: "alerts[0].evidence_fields.devname", ocsf_field: "device.hostname" }
|
|
- { source_path: "alerts[0].evidence_fields.devid", ocsf_field: "device.uid" }
|
|
- { source_path: "alerts[0].evidence_fields.hostname", ocsf_field: "device.hostname" }
|
|
- { source_path: "alerts[0].evidence_fields.status", ocsf_field: "status_detail" }
|
|
# `action` is what the source system did ("login"), which says more than the
|
|
# investigation's own `source` ("ALERT"). Mapped after it so it wins.
|
|
- { source_path: "alerts[0].evidence_fields.action", ocsf_field: "activity_name" }
|
|
# The line itself, on the OCSF field meant for it — and one the layout builder
|
|
# offers, unlike `evidence` above, which only ever shows in the Context tab.
|
|
- { source_path: "alerts[0].evidence_raw_line", ocsf_field: "raw_data" }
|
|
- { source_path: "alerts[0].evidence_fields.msg ? alerts[0].evidence_fields.msg : alerts[0].evidence_fields.logdesc", ocsf_field: "message" }
|
|
# ── Process tree ──────────────────────────────────────────────────
|
|
# The layout block reads ctx.process_tree: N flat chains, root first, each
|
|
# {alert_id, rule, host, user, nodes[]}. Rapid7 nests its tree through
|
|
# `children`, so the command flattens each root-to-leaf path into one chain
|
|
# and rolls every alert's chains up to the investigation — see
|
|
# get_investigations_full.py. Passed through verbatim; nothing to reshape here.
|
|
- { source_path: "process_tree", ocsf_field: "process_tree" }
|
|
# ── Every alert, not just the first ───────────────────────────────
|
|
# An investigation aggregates alerts, and the fields above only ever read alerts[0].
|
|
# This entry lands the whole set as OCSF-shaped objects, so the host and the process of
|
|
# the SECOND and THIRD alert are indexed as entities too — the extractor walks arrays
|
|
# and classifies on the path's suffix, so `.hostname` under here counts exactly as
|
|
# `device.hostname` does above.
|
|
- source_path: >
|
|
alerts.{
|
|
"uid": rrn,
|
|
"title": title,
|
|
"time": alerted_at,
|
|
"status": status,
|
|
"disposition": disposition,
|
|
"hostname": rule_keys_of_interest[key='hostname'].values[0],
|
|
"src_hostname": rule_keys_of_interest[key='source_asset_address'].values[0],
|
|
"dst_hostname": rule_keys_of_interest[key='destination_asset_address'].values[0],
|
|
"actor_user": rule_keys_of_interest[key='source_account'].values[0],
|
|
"target_user": rule_keys_of_interest[key='destination_account'].values[0],
|
|
"process": rule_keys_of_interest[key='process.name'].values[0],
|
|
"cmd_line": rule_keys_of_interest[key='process.cmd_line'].values[0],
|
|
"technique": rule.mitre_tcodes[$substring($, 0, 2) != 'TA'][0],
|
|
"detection": rule_keys_of_interest{key: values[0]},
|
|
"evidence": evidence_fields
|
|
}
|
|
ocsf_field: "evidences"
|