feat(rapid7-insightidr): the investigation, with its alerts inlined (v1.4.0)
The previous command returned the alerts as its result rows, which quietly changed the unit: an investigation holding three alerts would have become three incidents. The investigation is what an analyst opens and what the SOC tracks — it stays the incident, and the alerts are its content. get_investigations_full walks the same three hops and inverts the shape: investigations under `data`, each carrying its alerts under `alerts` plus an alert_count. results_path=data and dedup_key=rrn, so it is a drop-in replacement for list_investigations in an existing ingestion rule — only the command name and the mapper change. A few first-alert values are also lifted to the top level, and that is not cosmetic. MapFields resolves dotted paths only and cannot index an array, so `alerts[0].ai_suggested_disposition` in a field_mappings entry reads as nothing — silently, leaving the custom field empty with no error anywhere. Three custom fields were empty for exactly that reason until an end-to-end run showed it. The ocsf section has no such limit: it is evaluated by full JSONata and indexes freely, which is why the host and process mappings worked while the custom fields did not. The mapper maps the investigation first and the first alert after, so the alert wins wherever the container says nothing. It also lands every alert — not just alerts[0] — as OCSF-shaped objects under `evidences`, because the extractor walks arrays and classifies on the path suffix: an investigation aggregating two alerts on two machines now indexes both hosts, where reading alerts[0] alone would have indexed one and silently dropped the other. Verified end to end against the investigation that started this, with a second alert added on another host: field mapping, OCSF mapping and entity extraction. Both hosts are indexed. With the incident type's extraction policy set to selected/host,ip,user, the entity set is exactly those two — the alert RRN's 32-hex suffix, which the IOC regex cannot tell from an MD5, is filtered out there. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,124 @@
|
||||
name: "Rapid7 InsightIDR Investigations (full) → OCSF"
|
||||
description: |
|
||||
Maps one Rapid7 investigation fetched with its alerts inlined
|
||||
(get_investigations_full, results_path = data) to OCSF finding fields.
|
||||
|
||||
The plain investigation feed cannot carry a detection: idr/v2/investigations answers
|
||||
with a title, a priority, a status, an assignee and timestamps, and nothing else. An
|
||||
incident built from it indexes zero entities. This mapper reads the same investigation
|
||||
plus the alerts the command inlined under `alerts`, where the detection content
|
||||
actually lives.
|
||||
|
||||
Investigation values are mapped first and the first alert's equivalents after, so the
|
||||
alert wins wherever the container says nothing. `alerts` is empty on an investigation
|
||||
whose alerts could not be read (the command sets alerts_error) — every alert path then
|
||||
resolves to nothing and is skipped, and the incident degrades to what the investigation
|
||||
alone says rather than failing.
|
||||
field_mappings:
|
||||
title: "title"
|
||||
# toSeverity maps CRITICAL→5, HIGH→4, MEDIUM→3, LOW→2; anything else falls back to the
|
||||
# alert rule's default severity.
|
||||
severity: "priority"
|
||||
source: "source"
|
||||
# Rapid7's own triage state. It cannot reach the incident's status and verdict columns
|
||||
# — MapFields accepts only title, severity, source, description and custom:* — so it
|
||||
# rides as custom fields, where an analyst at least sees that the source already closed
|
||||
# this and what it concluded.
|
||||
"custom:rapid7_status": "status"
|
||||
"custom:rapid7_disposition": "disposition"
|
||||
# Rapid7's AI suggestion, exposed as a fact and never as a verdict.
|
||||
# first_alert_* are lifted to the top level by the command on purpose: this
|
||||
# mapper section resolves dotted paths only and cannot index an array, so
|
||||
# `alerts[0].x` here would read as nothing — silently, leaving the custom field
|
||||
# empty with no error. The ocsf section below is full JSONata and indexes freely.
|
||||
"custom:rapid7_ai_disposition": "first_alert_ai_disposition"
|
||||
"custom:rapid7_alert_count": "alert_count"
|
||||
"custom:rapid7_log_id": "first_alert_log_id"
|
||||
"custom:rapid7_logset_id": "first_alert_logset_id"
|
||||
"custom:rapid7_rule_rrn": "first_alert_rule_rrn"
|
||||
"custom:rapid7_event_type": "first_alert_event_type"
|
||||
ocsf:
|
||||
# ── Finding: the investigation ────────────────────────────────────
|
||||
- { source_path: "rrn", ocsf_field: "finding_info.uid" }
|
||||
- { source_path: "title", ocsf_field: "finding_info.title" }
|
||||
- { source_path: "created_time", ocsf_field: "finding_info.created_time" }
|
||||
- { source_path: "last_accessed", ocsf_field: "finding_info.modified_time" }
|
||||
- { source_path: "first_alert_time", ocsf_field: "finding_info.first_seen_time" }
|
||||
- { source_path: "latest_alert_time", ocsf_field: "finding_info.last_seen_time" }
|
||||
- { source_path: "tags", ocsf_field: "metadata.labels" }
|
||||
- { source_path: "alert_count", ocsf_field: "count" }
|
||||
# ── Incident state ────────────────────────────────────────────────
|
||||
- { source_path: "status", ocsf_field: "status" }
|
||||
- { source_path: "priority", ocsf_field: "priority" }
|
||||
- { source_path: "disposition", ocsf_field: "disposition" }
|
||||
- { source_path: "source", ocsf_field: "activity_name" }
|
||||
- { source_path: "organization_id", ocsf_field: "cloud.org.uid" }
|
||||
- { source_path: "assignee.name", ocsf_field: "assignee.name" }
|
||||
- { source_path: "assignee.email", ocsf_field: "assignee.email_addr" }
|
||||
# ── Detection time: the MTTD anchor ───────────────────────────────
|
||||
# Weakest first, strongest last. The investigation is created after the alert that
|
||||
# opened it; the alert's own alerted_at is when the event happened in the source system.
|
||||
- { source_path: "created_time", ocsf_field: "time" }
|
||||
- { source_path: "first_alert_time", ocsf_field: "time" }
|
||||
- { source_path: "alerts[0].alerted_at", ocsf_field: "time" }
|
||||
# ── The detection rule, from the first alert ──────────────────────
|
||||
# `rule.name` is documented but absent from live ABA alerts, so `type` — which repeats
|
||||
# the rule's display name — is the honest fallback.
|
||||
- { source_path: "alerts[0].type", ocsf_field: "finding_info.analytic.name" }
|
||||
- { source_path: "alerts[0].rule.rrn", ocsf_field: "finding_info.analytic.uid" }
|
||||
- { source_path: "alerts[0].rule.version_rrn", ocsf_field: "finding_info.analytic.version" }
|
||||
- { source_path: "alerts[0].rrn", ocsf_field: "finding_info.uid_alt" }
|
||||
# ── Product identity ──────────────────────────────────────────────
|
||||
- { source_path: "'Rapid7 InsightIDR'", ocsf_field: "metadata.product.name" }
|
||||
- { source_path: "'Rapid7'", ocsf_field: "metadata.product.vendor_name" }
|
||||
- { source_path: "alerts[0].external_source", ocsf_field: "metadata.log_source" }
|
||||
- { source_path: "alerts[0].triggering_event_type", ocsf_field: "metadata.event_code" }
|
||||
- { source_path: "alerts[0].log_details[0].log_entry_id", ocsf_field: "metadata.uid" }
|
||||
# ── MITRE ATT&CK ──────────────────────────────────────────────────
|
||||
# mitre_tcodes mixes both kinds in one array: "TA0040" is a tactic, "T1490" a technique.
|
||||
# Splitting on the "TA" prefix keeps a tactic out of the technique field, where it would
|
||||
# name a technique that does not exist.
|
||||
- { source_path: "alerts[0].rule.mitre_tcodes[$substring($, 0, 2) = 'TA'][0]", ocsf_field: "attacks.tactic.uid" }
|
||||
- { source_path: "alerts[0].rule.mitre_tcodes[$substring($, 0, 2) != 'TA'][0]", ocsf_field: "attacks.technique.uid" }
|
||||
# ── The affected host ─────────────────────────────────────────────
|
||||
- { source_path: "alerts[0].rule_keys_of_interest[key='hostname'].values[0]", ocsf_field: "device.hostname" }
|
||||
- { source_path: "alerts[0].rule_keys_of_interest[key='hostname'].values[0]", ocsf_field: "src_endpoint.hostname" }
|
||||
# ── The process the alert is about ────────────────────────────────
|
||||
- { source_path: "alerts[0].rule_keys_of_interest[key='process.name'].values[0]", ocsf_field: "process.name" }
|
||||
- { source_path: "alerts[0].rule_keys_of_interest[key='process.name'].values[0]", ocsf_field: "process.file.name" }
|
||||
- { source_path: "alerts[0].rule_keys_of_interest[key='process.cmd_line'].values[0]", ocsf_field: "process.cmd_line" }
|
||||
- { source_path: "alerts[0].rule_keys_of_interest[key='process.exe_path'].values[0]", ocsf_field: "process.file.path" }
|
||||
- { source_path: "alerts[0].rule_keys_of_interest[key='process.exe_file.hashes.md5'].values[0]", ocsf_field: "process.file.hashes.md5" }
|
||||
- { source_path: "alerts[0].rule_keys_of_interest[key='process.exe_file.hashes.sha256'].values[0]", ocsf_field: "process.file.hashes.sha256" }
|
||||
# ── Its parent, which says how it was launched ────────────────────
|
||||
- { source_path: "alerts[0].rule_keys_of_interest[key='parent_process.name'].values[0]", ocsf_field: "actor.process.parent_process.name" }
|
||||
- { source_path: "alerts[0].rule_keys_of_interest[key='parent_process.cmd_line'].values[0]", ocsf_field: "actor.process.parent_process.cmd_line" }
|
||||
- { source_path: "alerts[0].rule_keys_of_interest[key='parent_process.exe_path'].values[0]", ocsf_field: "actor.process.parent_process.file.path" }
|
||||
- { source_path: "alerts[0].rule_keys_of_interest[key='parent_process.exe_file.hashes.md5'].values[0]", ocsf_field: "actor.process.parent_process.file.hashes.md5" }
|
||||
# ── The acting account ────────────────────────────────────────────
|
||||
# This tenant's field catalogue has no canonical user namespace, so an account only
|
||||
# appears when a rule names it among its keys of interest. Both spellings are mapped
|
||||
# because rules are not consistent about which they emit.
|
||||
- { source_path: "alerts[0].rule_keys_of_interest[key='user'].values[0]", ocsf_field: "actor.user.name" }
|
||||
- { source_path: "alerts[0].rule_keys_of_interest[key='username'].values[0]", ocsf_field: "actor.user.name" }
|
||||
# ── Every alert, not just the first ───────────────────────────────
|
||||
# An investigation aggregates alerts, and the fields above only ever read alerts[0].
|
||||
# This entry lands the whole set as OCSF-shaped objects, so the host and the process of
|
||||
# the SECOND and THIRD alert are indexed as entities too — the extractor walks arrays
|
||||
# and classifies on the path's suffix, so `.hostname` under here counts exactly as
|
||||
# `device.hostname` does above.
|
||||
- source_path: >
|
||||
alerts.{
|
||||
"uid": rrn,
|
||||
"title": title,
|
||||
"time": alerted_at,
|
||||
"status": status,
|
||||
"disposition": disposition,
|
||||
"device": { "hostname": rule_keys_of_interest[key='hostname'].values[0] },
|
||||
"process": {
|
||||
"name": rule_keys_of_interest[key='process.name'].values[0],
|
||||
"cmd_line": rule_keys_of_interest[key='process.cmd_line'].values[0]
|
||||
},
|
||||
"attacks": { "technique": { "uid": rule.mitre_tcodes[$substring($, 0, 2) != 'TA'][0] } }
|
||||
}
|
||||
ocsf_field: "evidences"
|
||||
Reference in New Issue
Block a user