From b6903d4590e1d00901c27bb1d04233ccb7d33690 Mon Sep 17 00:00:00 2001 From: Guillaume BOURGEOIS Date: Mon, 7 Sep 2026 23:07:53 +0200 Subject: [PATCH] feat(rapid7-insightidr): map alerts to OCSF, with real observables (v1.2.0) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit An incident built from a Rapid7 investigation indexed zero entities. Nothing to pivot on, nothing to correlate, and an AI analysis with nothing to read. The reason turned out not to be the mapper but the feed: an investigation is an envelope, and neither it nor its /alerts list carries a host, an account, a process or a MITRE code. The Alert Triage API does, in rule_keys_of_interest. On the alert behind the investigation that started this, that field holds: hostname NBSGIT01-26 process.name bcdedit.exe process.cmd_line bcdedit /set {default} recoveryenabled No process.exe_path C:\WINDOWS\System32\bcdedit.exe parent_process.name cmd.exe parent_process.cmd_line "C:\WINDOWS\system32\cmd.exe" parent_process.exe_file.hashes.md5 e463e36104d22a2479046a7d4f620816 with rule.mitre_tcodes ["TA0040", "T1490"] — Impact / Inhibit System Recovery, which is what disabling Windows recovery actually is. None of it was reachable before. It is a list of {key, values} pairs rather than an object, so every entry reads through a JSONata predicate — rule_keys_of_interest[key='hostname'].values[0]. A key a rule does not emit resolves to nothing and is skipped, so one file covers every detection rule. Two details worth keeping: - mitre_tcodes mixes tactics and techniques in one array ("TA0040" beside "T1490"). They are split on the "TA" prefix so a tactic never lands in the technique field, where it would name a technique that does not exist. - rule.name is documented but absent from live ABA alerts, so the analytic name falls back to `type`, which repeats the rule's display name. Rapid7's own status, disposition and AI-suggested disposition ride as custom fields. They cannot reach the incident's status and verdict columns — MapFields accepts only title, severity, source, description and custom:* — but an analyst can at least see that the source already closed the alert and what it decided. The AI suggestion is exposed as a fact, never as a verdict. Validated end to end against a live alert: field mapping, OCSF mapping and entity extraction, which now yields the host and the parent process hash where it previously yielded nothing. Co-Authored-By: Claude Opus 5 (1M context) --- integrations/rapid7-insightidr/manifest.yaml | 4 +- .../mappers/search_alerts.yaml | 115 ++++++++++++++++++ 2 files changed, 117 insertions(+), 2 deletions(-) create mode 100644 integrations/rapid7-insightidr/mappers/search_alerts.yaml diff --git a/integrations/rapid7-insightidr/manifest.yaml b/integrations/rapid7-insightidr/manifest.yaml index 653c831..001a9c9 100644 --- a/integrations/rapid7-insightidr/manifest.yaml +++ b/integrations/rapid7-insightidr/manifest.yaml @@ -1,8 +1,8 @@ id: rapid7_insightidr name: Rapid7 InsightIDR -version: 1.1.1 +version: 1.2.0 description: "Rapid7 InsightIDR (REST API) — investigation ingestion and full lifecycle (list/get/search/create/update/assign/set-status/bulk-close), investigation alerts and Rapid7 product alerts, custom threat indicators (add/replace), log management and LEQL log/log-set queries with downloads, and user directory search." -changelog: "1.1.1 — list_alert_fields: default path_depth to 1 and expose the search filter. The V2 fields endpoint refuses an unbounded walk of the tree (\"root path with path depth greater than 1 is not supported for V2 without a search filter\"), so the bare call returned HTTP 400. | 1.1.0 — Add the Alert Triage API (/idr/at): search alerts, get an alert, its evidence, its actors and its process trees, plus the tenant's alert field catalogue. These carry the detection content the investigation endpoints do not — MITRE T-codes, matched rule keys, assets and accounts, command lines. MDR-gated. | 1.0.0 — Initial release: investigation ingestion (list_investigations) with an exhaustive OCSF mapper, 18 commands across investigations, alerts, custom threats, logs/LEQL search and users. API v1/v2 selectable per instance and per command." +changelog: "1.2.0 — Add the search_alerts OCSF mapper: rule_keys_of_interest carries the host, the process, its command line, its parent and the file hashes, so an alert-based feed indexes entities where the investigation feed indexed none. Validated against a live ABA alert. | 1.1.1 — list_alert_fields: default path_depth to 1 and expose the search filter. The V2 fields endpoint refuses an unbounded walk of the tree (\"root path with path depth greater than 1 is not supported for V2 without a search filter\"), so the bare call returned HTTP 400. | 1.1.0 — Add the Alert Triage API (/idr/at): search alerts, get an alert, its evidence, its actors and its process trees, plus the tenant's alert field catalogue. These carry the detection content the investigation endpoints do not — MITRE T-codes, matched rule keys, assets and accounts, command lines. MDR-gated. | 1.0.0 — Initial release: investigation ingestion (list_investigations) with an exhaustive OCSF mapper, 18 commands across investigations, alerts, custom threats, logs/LEQL search and users. API v1/v2 selectable per instance and per command." category: siem # Per-instance configuration. The Insight platform region selects the API host diff --git a/integrations/rapid7-insightidr/mappers/search_alerts.yaml b/integrations/rapid7-insightidr/mappers/search_alerts.yaml new file mode 100644 index 0000000..c7a1ae4 --- /dev/null +++ b/integrations/rapid7-insightidr/mappers/search_alerts.yaml @@ -0,0 +1,115 @@ +name: "Rapid7 InsightIDR Alerts → OCSF" +description: | + Maps one Rapid7 alert from the Alert Triage API (search_alerts, results_path = alerts) + to OCSF finding fields. + + This mapper exists because the investigation feed cannot carry a detection. An + investigation, and the /investigations/{rrn}/alerts list behind it, hold a title, a + priority, a status and timestamps — no host, no account, no process, no MITRE code. An + incident built from one indexes zero entities, so it cannot be pivoted on, cannot be + correlated, and gives an analyst nothing to read. + + The alert carries all of it in rule_keys_of_interest: a list of {key, values} pairs + whose keys are already dotted paths (process.cmd_line, parent_process.name, hostname). + Reading them needs a JSONata predicate per key rather than a plain path, which is why + every entry below looks heavier than in other mappers. A key a given rule does not emit + resolves to nothing and is skipped, so the same file covers every detection rule. +field_mappings: + title: "title" + # toSeverity maps CRITICAL→5, HIGH→4, MEDIUM→3, LOW→2, and anything else to the alert + # rule's default. (The investigation mapper's comment on this is wrong — checked + # against internal/incidenttypes/engine.go.) + severity: "priority" + source: "external_source" + # Rapid7's own triage state. It cannot reach the incident's status or verdict columns — + # MapFields only accepts title, severity, source, description and custom:* — so it is + # kept as custom fields, where an analyst at least sees that the source already closed + # this alert and what it concluded. + "custom:rapid7_status": "status" + "custom:rapid7_disposition": "disposition" + # Rapid7's AI suggestion, exposed as a fact and never as a verdict. + "custom:rapid7_ai_disposition": "ai_suggested_disposition" + "custom:rapid7_responsibility": "responsibility" + # The log identifiers a LEQL query needs to pull the raw event back out of Log Search. + # Kept as custom fields rather than forced onto an OCSF field that means something else. + "custom:rapid7_log_id": "log_details[0].log_id" + "custom:rapid7_logset_id": "log_details[0].logset_id" +ocsf: + # ── Finding ─────────────────────────────────────────────────────── + - { source_path: "rrn", ocsf_field: "finding_info.uid" } + - { source_path: "external_id", ocsf_field: "finding_info.uid_alt" } + - { source_path: "title", ocsf_field: "finding_info.title" } + - { source_path: "created_at", ocsf_field: "finding_info.created_time" } + - { source_path: "updated_at", ocsf_field: "finding_info.modified_time" } + - { source_path: "alerted_at", ocsf_field: "finding_info.first_seen_time" } + - { source_path: "tags", ocsf_field: "metadata.labels" } + # The detection rule. `rule.name` is documented but absent from live ABA alerts, so + # `type` — which repeats the rule's display name — is the honest fallback. + - { source_path: "type", ocsf_field: "finding_info.analytic.name" } + - { source_path: "rule.rrn", ocsf_field: "finding_info.analytic.uid" } + - { source_path: "rule.version_rrn", ocsf_field: "finding_info.analytic.version" } + # ── Detection time: the MTTD anchor ─────────────────────────────── + # alerted_at is when the underlying event happened in the source system; created_at is + # when Rapid7 finished building the alert. Weakest first, strongest last. + - { source_path: "created_at", ocsf_field: "time" } + - { source_path: "alerted_at", ocsf_field: "time" } + # ── State, as the source sees it ────────────────────────────────── + - { source_path: "status", ocsf_field: "status" } + - { source_path: "disposition", ocsf_field: "disposition" } + - { source_path: "priority", ocsf_field: "priority" } + - { source_path: "responsibility", ocsf_field: "comment" } + # ── Product identity ────────────────────────────────────────────── + - { source_path: "'Rapid7 InsightIDR'", ocsf_field: "metadata.product.name" } + - { source_path: "'Rapid7'", ocsf_field: "metadata.product.vendor_name" } + - { source_path: "external_source", ocsf_field: "metadata.log_source" } + - { source_path: "triggering_event_type", ocsf_field: "activity_name" } + - { source_path: "triggering_event_sub_type", ocsf_field: "status_detail" } + # The parent investigation, on the OCSF field meant for joining findings that belong + # together. Every alert Rapid7 folded into one investigation carries the same value, + # which is what lets the correlation engine rebuild that grouping as a case. + - { source_path: "investigation_rrn", ocsf_field: "metadata.correlation_uid" } + - { source_path: "organization.id", ocsf_field: "cloud.org.uid" } + - { source_path: "organization.name", ocsf_field: "cloud.org.name" } + # ── MITRE ATT&CK ────────────────────────────────────────────────── + # mitre_tcodes mixes both kinds in one array: "TA0040" is a tactic, "T1490" a + # technique. Splitting on the "TA" prefix keeps a tactic out of the technique field, + # where it would name a technique that does not exist. + - { source_path: "rule.mitre_tcodes[$substring($, 0, 2) = 'TA'][0]", ocsf_field: "attacks.tactic.uid" } + - { source_path: "rule.mitre_tcodes[$substring($, 0, 2) != 'TA'][0]", ocsf_field: "attacks.technique.uid" } + # ── The affected host ───────────────────────────────────────────── + # device.hostname and src_endpoint.hostname both index the host as an entity; the + # duplication is what other vendor mappers do and what the pivot graph reads. + - { source_path: "rule_keys_of_interest[key='hostname'].values[0]", ocsf_field: "device.hostname" } + - { source_path: "rule_keys_of_interest[key='hostname'].values[0]", ocsf_field: "src_endpoint.hostname" } + # ── The process the alert is about ──────────────────────────────── + - { source_path: "rule_keys_of_interest[key='process.name'].values[0]", ocsf_field: "process.name" } + - { source_path: "rule_keys_of_interest[key='process.name'].values[0]", ocsf_field: "process.file.name" } + - { source_path: "rule_keys_of_interest[key='process.cmd_line'].values[0]", ocsf_field: "process.cmd_line" } + - { source_path: "rule_keys_of_interest[key='process.exe_path'].values[0]", ocsf_field: "process.file.path" } + - { source_path: "rule_keys_of_interest[key='process.exe_file.hashes.md5'].values[0]", ocsf_field: "process.file.hashes.md5" } + - { source_path: "rule_keys_of_interest[key='process.exe_file.hashes.sha256'].values[0]", ocsf_field: "process.file.hashes.sha256" } + - { source_path: "rule_keys_of_interest[key='process.pid'].values[0]", ocsf_field: "process.pid" } + # ── Its parent, which is what says how it was launched ──────────── + - { source_path: "rule_keys_of_interest[key='parent_process.name'].values[0]", ocsf_field: "actor.process.parent_process.name" } + - { source_path: "rule_keys_of_interest[key='parent_process.cmd_line'].values[0]", ocsf_field: "actor.process.parent_process.cmd_line" } + - { source_path: "rule_keys_of_interest[key='parent_process.exe_path'].values[0]", ocsf_field: "actor.process.parent_process.file.path" } + - { source_path: "rule_keys_of_interest[key='parent_process.exe_file.hashes.md5'].values[0]", ocsf_field: "actor.process.parent_process.file.hashes.md5" } + - { source_path: "rule_keys_of_interest[key='parent_process.exe_file.hashes.sha256'].values[0]", ocsf_field: "actor.process.parent_process.file.hashes.sha256" } + # ── The acting account ──────────────────────────────────────────── + # There is no canonical user namespace in this tenant's field catalogue, so the account + # only appears when a rule names it among its keys of interest. Both spellings are + # mapped because rules are not consistent about which one they emit. + - { source_path: "rule_keys_of_interest[key='user'].values[0]", ocsf_field: "actor.user.name" } + - { source_path: "rule_keys_of_interest[key='username'].values[0]", ocsf_field: "actor.user.name" } + - { source_path: "rule_keys_of_interest[key='account'].values[0]", ocsf_field: "actor.user.account.name" } + # ── Network, when the rule is about a flow ──────────────────────── + - { source_path: "rule_keys_of_interest[key='source_address'].values[0]", ocsf_field: "src_endpoint.ip" } + - { source_path: "rule_keys_of_interest[key='destination_address'].values[0]", ocsf_field: "dst_endpoint.ip" } + - { source_path: "rule_keys_of_interest[key='destination_port'].values[0]", ocsf_field: "dst_endpoint.port" } + # ── Who owns it at the source ───────────────────────────────────── + - { source_path: "assignee.first_name & ' ' & assignee.last_name", ocsf_field: "assignee.name" } + - { source_path: "assignee.email", ocsf_field: "assignee.email_addr" } + # ── The log entry behind the alert ──────────────────────────────── + # Not investigable on its own, but it is the handle a LEQL query needs to pull the raw + # event out of Log Search — the only bridge from an alert back to the line that made it. + - { source_path: "log_details[0].log_entry_id", ocsf_field: "metadata.uid" }