diff --git a/integrations/rapid7-insightidr/manifest.yaml b/integrations/rapid7-insightidr/manifest.yaml index 59a35f7..ad2cc07 100644 --- a/integrations/rapid7-insightidr/manifest.yaml +++ b/integrations/rapid7-insightidr/manifest.yaml @@ -1,8 +1,8 @@ id: rapid7_insightidr name: Rapid7 InsightIDR -version: 1.4.1 +version: 1.5.0 description: "Rapid7 InsightIDR (REST API) — investigation ingestion and full lifecycle (list/get/search/create/update/assign/set-status/bulk-close), investigation alerts and Rapid7 product alerts, custom threat indicators (add/replace), log management and LEQL log/log-set queries with downloads, and user directory search." -changelog: "1.4.1 — Read the alert RRN from `rrn` before `id` and drop anything not RRN-shaped: the investigation alert list also returns bare UUIDs, and one of those rejects the entire /alerts/ops/rrns batch (HTTP 400), losing every alert in the run. | 1.4.0 — Add get_investigations_full: the investigation with its alerts inlined, so it stays the unit that becomes an incident. Drop-in for list_investigations (results_path data, dedup rrn). | 1.3.0 — Add search_investigation_alerts: ingest only the alerts that belong to an investigation, with their full detection content, in 1+N+1 calls. Filtering search_alerts on investigation_rrn does not work — the term is accepted and silently matches nothing. | 1.2.1 — search_alerts: report a malformed terms_json with the input named and echoed, instead of json's bare \"Expecting value: line 1 column 1\". | 1.2.0 — Add the search_alerts OCSF mapper: rule_keys_of_interest carries the host, the process, its command line, its parent and the file hashes, so an alert-based feed indexes entities where the investigation feed indexed none. Validated against a live ABA alert. | 1.1.1 — list_alert_fields: default path_depth to 1 and expose the search filter. The V2 fields endpoint refuses an unbounded walk of the tree (\"root path with path depth greater than 1 is not supported for V2 without a search filter\"), so the bare call returned HTTP 400. | 1.1.0 — Add the Alert Triage API (/idr/at): search alerts, get an alert, its evidence, its actors and its process trees, plus the tenant's alert field catalogue. These carry the detection content the investigation endpoints do not — MITRE T-codes, matched rule keys, assets and accounts, command lines. MDR-gated. | 1.0.0 — Initial release: investigation ingestion (list_investigations) with an exhaustive OCSF mapper, 18 commands across investigations, alerts, custom threats, logs/LEQL search and users. API v1/v2 selectable per instance and per command." +changelog: "1.5.0 — Flatten the whole of rule_keys_of_interest into the context. It is an open vocabulary — a process rule and an authentication rule share no keys — so the hardcoded list only covered rules already seen and left every other alert empty. | 1.4.1 — Read the alert RRN from `rrn` before `id` and drop anything not RRN-shaped: the investigation alert list also returns bare UUIDs, and one of those rejects the entire /alerts/ops/rrns batch (HTTP 400), losing every alert in the run. | 1.4.0 — Add get_investigations_full: the investigation with its alerts inlined, so it stays the unit that becomes an incident. Drop-in for list_investigations (results_path data, dedup rrn). | 1.3.0 — Add search_investigation_alerts: ingest only the alerts that belong to an investigation, with their full detection content, in 1+N+1 calls. Filtering search_alerts on investigation_rrn does not work — the term is accepted and silently matches nothing. | 1.2.1 — search_alerts: report a malformed terms_json with the input named and echoed, instead of json's bare \"Expecting value: line 1 column 1\". | 1.2.0 — Add the search_alerts OCSF mapper: rule_keys_of_interest carries the host, the process, its command line, its parent and the file hashes, so an alert-based feed indexes entities where the investigation feed indexed none. Validated against a live ABA alert. | 1.1.1 — list_alert_fields: default path_depth to 1 and expose the search filter. The V2 fields endpoint refuses an unbounded walk of the tree (\"root path with path depth greater than 1 is not supported for V2 without a search filter\"), so the bare call returned HTTP 400. | 1.1.0 — Add the Alert Triage API (/idr/at): search alerts, get an alert, its evidence, its actors and its process trees, plus the tenant's alert field catalogue. These carry the detection content the investigation endpoints do not — MITRE T-codes, matched rule keys, assets and accounts, command lines. MDR-gated. | 1.0.0 — Initial release: investigation ingestion (list_investigations) with an exhaustive OCSF mapper, 18 commands across investigations, alerts, custom threats, logs/LEQL search and users. API v1/v2 selectable per instance and per command." category: siem # Per-instance configuration. The Insight platform region selects the API host diff --git a/integrations/rapid7-insightidr/mappers/get_investigations_full.yaml b/integrations/rapid7-insightidr/mappers/get_investigations_full.yaml index 6aad5c7..5411825 100644 --- a/integrations/rapid7-insightidr/mappers/get_investigations_full.yaml +++ b/integrations/rapid7-insightidr/mappers/get_investigations_full.yaml @@ -101,6 +101,29 @@ ocsf: # because rules are not consistent about which they emit. - { source_path: "alerts[0].rule_keys_of_interest[key='user'].values[0]", ocsf_field: "actor.user.name" } - { source_path: "alerts[0].rule_keys_of_interest[key='username'].values[0]", ocsf_field: "actor.user.name" } + # ── Every key the rule found interesting ────────────────────────── + # rule_keys_of_interest is an OPEN vocabulary: each detection rule names its own + # keys. A process_start_event rule emits hostname / process.cmd_line / + # parent_process.*; an ingress_auth rule emits account / source_ip / + # geoip_country_code / result and none of the above. The explicit entries below + # can therefore only ever cover the rules someone has already seen — on any + # other rule they all resolve to nothing and the incident is empty again, which + # is the exact failure this integration started from. + # + # So the whole list is also flattened, verbatim, into one object. Keys nobody + # anticipated arrive intact and are visible to the analyst and to the AI + # analysis, which reads the mapped context but never `raw`. The explicit + # mappings stay because they are what types an entity correctly — a value under + # `detection.` is only ever matched by the IOC regex, so an account there would + # never be indexed as a user. + - { source_path: "alerts[0].rule_keys_of_interest{key: values[0]}", ocsf_field: "detection" } + - { source_path: "alerts[0].rule_matching_keys{key: values[0]}", ocsf_field: "detection_match" } + # ── Keys seen on non-process rules ──────────────────────────────── + - { source_path: "alerts[0].rule_keys_of_interest[key='account'].values[0]", ocsf_field: "actor.user.account.name" } + - { source_path: "alerts[0].rule_keys_of_interest[key='source_ip'].values[0]", ocsf_field: "src_endpoint.ip" } + - { source_path: "alerts[0].rule_keys_of_interest[key='destination_ip'].values[0]", ocsf_field: "dst_endpoint.ip" } + - { source_path: "alerts[0].rule_keys_of_interest[key='geoip_country_code'].values[0]", ocsf_field: "src_endpoint.location.country" } + - { source_path: "alerts[0].rule_keys_of_interest[key='result'].values[0]", ocsf_field: "status_detail" } # ── Every alert, not just the first ─────────────────────────────── # An investigation aggregates alerts, and the fields above only ever read alerts[0]. # This entry lands the whole set as OCSF-shaped objects, so the host and the process of @@ -119,6 +142,7 @@ ocsf: "name": rule_keys_of_interest[key='process.name'].values[0], "cmd_line": rule_keys_of_interest[key='process.cmd_line'].values[0] }, - "attacks": { "technique": { "uid": rule.mitre_tcodes[$substring($, 0, 2) != 'TA'][0] } } + "attacks": { "technique": { "uid": rule.mitre_tcodes[$substring($, 0, 2) != 'TA'][0] } }, + "detection": rule_keys_of_interest{key: values[0]} } ocsf_field: "evidences" diff --git a/integrations/rapid7-insightidr/mappers/search_alerts.yaml b/integrations/rapid7-insightidr/mappers/search_alerts.yaml index 7699eff..036ce89 100644 --- a/integrations/rapid7-insightidr/mappers/search_alerts.yaml +++ b/integrations/rapid7-insightidr/mappers/search_alerts.yaml @@ -124,3 +124,26 @@ ocsf: - { source_path: "investigation.priority", ocsf_field: "finding_info.related_events.priority" } - { source_path: "investigation.disposition", ocsf_field: "finding_info.related_events.disposition" } - { source_path: "investigation.rrn", ocsf_field: "finding_info.related_events.uid" } + # ── Every key the rule found interesting ────────────────────────── + # rule_keys_of_interest is an OPEN vocabulary: each detection rule names its own + # keys. A process_start_event rule emits hostname / process.cmd_line / + # parent_process.*; an ingress_auth rule emits account / source_ip / + # geoip_country_code / result and none of the above. The explicit entries below + # can therefore only ever cover the rules someone has already seen — on any + # other rule they all resolve to nothing and the incident is empty again, which + # is the exact failure this integration started from. + # + # So the whole list is also flattened, verbatim, into one object. Keys nobody + # anticipated arrive intact and are visible to the analyst and to the AI + # analysis, which reads the mapped context but never `raw`. The explicit + # mappings stay because they are what types an entity correctly — a value under + # `detection.` is only ever matched by the IOC regex, so an account there would + # never be indexed as a user. + - { source_path: "rule_keys_of_interest{key: values[0]}", ocsf_field: "detection" } + - { source_path: "rule_matching_keys{key: values[0]}", ocsf_field: "detection_match" } + # ── Keys seen on non-process rules ──────────────────────────────── + - { source_path: "rule_keys_of_interest[key='account'].values[0]", ocsf_field: "actor.user.account.name" } + - { source_path: "rule_keys_of_interest[key='source_ip'].values[0]", ocsf_field: "src_endpoint.ip" } + - { source_path: "rule_keys_of_interest[key='destination_ip'].values[0]", ocsf_field: "dst_endpoint.ip" } + - { source_path: "rule_keys_of_interest[key='geoip_country_code'].values[0]", ocsf_field: "src_endpoint.location.country" } + - { source_path: "rule_keys_of_interest[key='result'].values[0]", ocsf_field: "status_detail" }