fix(rapid7-insightidr): map both ends of a connection (v1.7.0)

An "RDP connection to DC" investigation arrived with device {}, process {} and
two indexed entities, both of them false — the 32-hex tails of its own alert
RRNs, which the IOC regex cannot tell from an MD5. Its four real values were
sitting in the payload, unmapped:

  source_account             vbalbakakis
  destination_account        vassilisadmin
  source_asset_address       sgazswvdip-01.toto.priv
  destination_asset_address  scazsvpdc01.toto.priv

The mappings assumed an alert has one host and one user, which holds for a
process_start_event and not for an auth rule: a connection names both ends and
neither is "the host". So source and destination now land on src_endpoint and
dst_endpoint, the acting account on actor.user and the account acted upon on
user — the distinction that lets a lateral move be read in one line.

That choice is also what makes the fields placeable. The layout builder's field
picker is a closed <select> over the OCSF catalogue in web/src/lib/ocsf.ts;
a path absent from it cannot be put on a layout at all. All four targets are in
that catalogue. `detection`, the open-vocabulary catch-all, is not and cannot
be — it keeps its value in the Context tab, which renders the whole tree, but a
layout cannot reference it, so anything meant for a layout has to be mapped to a
real OCSF field first.

evidences[] is flattened at the same time. It emitted "device": {}, "process":
{} and "attacks": {"technique": {}} whenever those keys were absent — three
empty objects per alert, filling the context and hiding what was populated.

Verified against the RDP investigation: four entities where there were none,
two hosts and two accounts, and no false hash once the incident type's
extraction policy is set to selected/host,ip,user.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

integrations/rapid7-insightidr/manifest.yaml

@@ -1,8 +1,8 @@
 id: rapid7_insightidr
 name: Rapid7 InsightIDR
-version: 1.6.0
+version: 1.7.0
 description: "Rapid7 InsightIDR (REST API) — investigation ingestion and full lifecycle (list/get/search/create/update/assign/set-status/bulk-close), investigation alerts and Rapid7 product alerts, custom threat indicators (add/replace), log management and LEQL log/log-set queries with downloads, and user directory search."
-changelog: "1.6.0 — Surface region_failures: the Alert Triage API answers HTTP 200 with a partial result when it cannot reach a region, and that list was being dropped. Also report the regions seen in the RRNs against the configured one. | 1.5.0 — Flatten the whole of rule_keys_of_interest into the context. It is an open vocabulary — a process rule and an authentication rule share no keys — so the hardcoded list only covered rules already seen and left every other alert empty. | 1.4.1 — Read the alert RRN from `rrn` before `id` and drop anything not RRN-shaped: the investigation alert list also returns bare UUIDs, and one of those rejects the entire /alerts/ops/rrns batch (HTTP 400), losing every alert in the run. | 1.4.0 — Add get_investigations_full: the investigation with its alerts inlined, so it stays the unit that becomes an incident. Drop-in for list_investigations (results_path data, dedup rrn). | 1.3.0 — Add search_investigation_alerts: ingest only the alerts that belong to an investigation, with their full detection content, in 1+N+1 calls. Filtering search_alerts on investigation_rrn does not work — the term is accepted and silently matches nothing. | 1.2.1 — search_alerts: report a malformed terms_json with the input named and echoed, instead of json's bare \"Expecting value: line 1 column 1\". | 1.2.0 — Add the search_alerts OCSF mapper: rule_keys_of_interest carries the host, the process, its command line, its parent and the file hashes, so an alert-based feed indexes entities where the investigation feed indexed none. Validated against a live ABA alert. | 1.1.1 — list_alert_fields: default path_depth to 1 and expose the search filter. The V2 fields endpoint refuses an unbounded walk of the tree (\"root path with path depth greater than 1 is not supported for V2 without a search filter\"), so the bare call returned HTTP 400. | 1.1.0 — Add the Alert Triage API (/idr/at): search alerts, get an alert, its evidence, its actors and its process trees, plus the tenant's alert field catalogue. These carry the detection content the investigation endpoints do not — MITRE T-codes, matched rule keys, assets and accounts, command lines. MDR-gated. | 1.0.0 — Initial release: investigation ingestion (list_investigations) with an exhaustive OCSF mapper, 18 commands across investigations, alerts, custom threats, logs/LEQL search and users. API v1/v2 selectable per instance and per command."
+changelog: "1.7.0 — Map source/destination assets and accounts to src_endpoint, dst_endpoint, actor.user and user. A connection rule names both ends and neither is \"the host\", so an RDP-to-DC alert produced no entity at all and nothing a layout could show. | 1.6.0 — Surface region_failures: the Alert Triage API answers HTTP 200 with a partial result when it cannot reach a region, and that list was being dropped. Also report the regions seen in the RRNs against the configured one. | 1.5.0 — Flatten the whole of rule_keys_of_interest into the context. It is an open vocabulary — a process rule and an authentication rule share no keys — so the hardcoded list only covered rules already seen and left every other alert empty. | 1.4.1 — Read the alert RRN from `rrn` before `id` and drop anything not RRN-shaped: the investigation alert list also returns bare UUIDs, and one of those rejects the entire /alerts/ops/rrns batch (HTTP 400), losing every alert in the run. | 1.4.0 — Add get_investigations_full: the investigation with its alerts inlined, so it stays the unit that becomes an incident. Drop-in for list_investigations (results_path data, dedup rrn). | 1.3.0 — Add search_investigation_alerts: ingest only the alerts that belong to an investigation, with their full detection content, in 1+N+1 calls. Filtering search_alerts on investigation_rrn does not work — the term is accepted and silently matches nothing. | 1.2.1 — search_alerts: report a malformed terms_json with the input named and echoed, instead of json's bare \"Expecting value: line 1 column 1\". | 1.2.0 — Add the search_alerts OCSF mapper: rule_keys_of_interest carries the host, the process, its command line, its parent and the file hashes, so an alert-based feed indexes entities where the investigation feed indexed none. Validated against a live ABA alert. | 1.1.1 — list_alert_fields: default path_depth to 1 and expose the search filter. The V2 fields endpoint refuses an unbounded walk of the tree (\"root path with path depth greater than 1 is not supported for V2 without a search filter\"), so the bare call returned HTTP 400. | 1.1.0 — Add the Alert Triage API (/idr/at): search alerts, get an alert, its evidence, its actors and its process trees, plus the tenant's alert field catalogue. These carry the detection content the investigation endpoints do not — MITRE T-codes, matched rule keys, assets and accounts, command lines. MDR-gated. | 1.0.0 — Initial release: investigation ingestion (list_investigations) with an exhaustive OCSF mapper, 18 commands across investigations, alerts, custom threats, logs/LEQL search and users. API v1/v2 selectable per instance and per command."
 category: siem
 
 # Per-instance configuration. The Insight platform region selects the API host

integrations/rapid7-insightidr/mappers/get_investigations_full.yaml

@@ -101,6 +101,22 @@ ocsf:
   # because rules are not consistent about which they emit.
   - { source_path: "alerts[0].rule_keys_of_interest[key='user'].values[0]",     ocsf_field: "actor.user.name" }
   - { source_path: "alerts[0].rule_keys_of_interest[key='username'].values[0]", ocsf_field: "actor.user.name" }
+  # ── Lateral movement: source and destination ──────────────────────
+  # An asset_auth rule describes a connection, so it names BOTH ends and neither
+  # is "the host": source_asset_address / destination_asset_address, and
+  # source_account / destination_account. Mapping them to src_endpoint and
+  # dst_endpoint is not only the correct OCSF shape — it is also what makes them
+  # appear in the layout builder, whose field picker is a closed <select> over
+  # the OCSF catalogue. A path that is not in that catalogue cannot be placed on
+  # a layout at all, which is why `detection` below, useful as it is, only ever
+  # shows up in the Context tab.
+  - { source_path: "alerts[0].rule_keys_of_interest[key='source_asset_address'].values[0]",      ocsf_field: "src_endpoint.hostname" }
+  - { source_path: "alerts[0].rule_keys_of_interest[key='destination_asset_address'].values[0]", ocsf_field: "dst_endpoint.hostname" }
+  - { source_path: "alerts[0].rule_keys_of_interest[key='destination_asset'].values[0]",         ocsf_field: "dst_endpoint.hostname" }
+  # actor.user is who acted, user is who it was done TO. Keeping them apart is
+  # what lets an analyst read a lateral move in one line.
+  - { source_path: "alerts[0].rule_keys_of_interest[key='source_account'].values[0]",      ocsf_field: "actor.user.name" }
+  - { source_path: "alerts[0].rule_keys_of_interest[key='destination_account'].values[0]", ocsf_field: "user.name" }
   # ── Every key the rule found interesting ──────────────────────────
   # rule_keys_of_interest is an OPEN vocabulary: each detection rule names its own
   # keys. A process_start_event rule emits hostname / process.cmd_line /
@@ -137,12 +153,14 @@ ocsf:
         "time": alerted_at,
         "status": status,
         "disposition": disposition,
-        "device": { "hostname": rule_keys_of_interest[key='hostname'].values[0] },
+        "hostname": rule_keys_of_interest[key='hostname'].values[0],
-        "process": {
+        "src_hostname": rule_keys_of_interest[key='source_asset_address'].values[0],
-          "name": rule_keys_of_interest[key='process.name'].values[0],
+        "dst_hostname": rule_keys_of_interest[key='destination_asset_address'].values[0],
-          "cmd_line": rule_keys_of_interest[key='process.cmd_line'].values[0]
+        "actor_user": rule_keys_of_interest[key='source_account'].values[0],
-        },
+        "target_user": rule_keys_of_interest[key='destination_account'].values[0],
-        "attacks": { "technique": { "uid": rule.mitre_tcodes[$substring($, 0, 2) != 'TA'][0] } },
+        "process": rule_keys_of_interest[key='process.name'].values[0],
+        "cmd_line": rule_keys_of_interest[key='process.cmd_line'].values[0],
+        "technique": rule.mitre_tcodes[$substring($, 0, 2) != 'TA'][0],
         "detection": rule_keys_of_interest{key: values[0]}
       }
     ocsf_field: "evidences"

integrations/rapid7-insightidr/mappers/search_alerts.yaml

@@ -124,6 +124,22 @@ ocsf:
   - { source_path: "investigation.priority",    ocsf_field: "finding_info.related_events.priority" }
   - { source_path: "investigation.disposition", ocsf_field: "finding_info.related_events.disposition" }
   - { source_path: "investigation.rrn",         ocsf_field: "finding_info.related_events.uid" }
+  # ── Lateral movement: source and destination ──────────────────────
+  # An asset_auth rule describes a connection, so it names BOTH ends and neither
+  # is "the host": source_asset_address / destination_asset_address, and
+  # source_account / destination_account. Mapping them to src_endpoint and
+  # dst_endpoint is not only the correct OCSF shape — it is also what makes them
+  # appear in the layout builder, whose field picker is a closed <select> over
+  # the OCSF catalogue. A path that is not in that catalogue cannot be placed on
+  # a layout at all, which is why `detection` below, useful as it is, only ever
+  # shows up in the Context tab.
+  - { source_path: "rule_keys_of_interest[key='source_asset_address'].values[0]",      ocsf_field: "src_endpoint.hostname" }
+  - { source_path: "rule_keys_of_interest[key='destination_asset_address'].values[0]", ocsf_field: "dst_endpoint.hostname" }
+  - { source_path: "rule_keys_of_interest[key='destination_asset'].values[0]",         ocsf_field: "dst_endpoint.hostname" }
+  # actor.user is who acted, user is who it was done TO. Keeping them apart is
+  # what lets an analyst read a lateral move in one line.
+  - { source_path: "rule_keys_of_interest[key='source_account'].values[0]",      ocsf_field: "actor.user.name" }
+  - { source_path: "rule_keys_of_interest[key='destination_account'].values[0]", ocsf_field: "user.name" }
   # ── Every key the rule found interesting ──────────────────────────
   # rule_keys_of_interest is an OPEN vocabulary: each detection rule names its own
   # keys. A process_start_event rule emits hostname / process.cmd_line /