feat(rapid7-insightidr): ingest only the alerts that belong to an investigation (v1.3.0)
Ingesting from search_alerts brings back every alert the tenant produces, including the operational noise — two of the first three sampled were an agent reporting it lacked Full Disk Access. What is worth an incident is what Rapid7 or the analyst folded into an investigation. Filtering search_alerts on investigation_rrn does not work. The term is accepted — no validation error, HTTP 200 — and matches nothing: zero alerts over fifteen days, on a tenant where an alert carrying an investigation_rrn provably exists in that window. The field is not searchable, and the API says so by returning an empty page rather than by refusing the term. That failure mode is the reason not to build an ingestion feed on it: a filter that silently stops matching drains the feed to zero with no error anywhere, and nothing would ever say why. search_investigation_alerts is guaranteed by construction instead: 1. the investigations in the window (incremental on their created_time) 2. their alert RRNs, one call per investigation — there is no bulk form 3. the alerts themselves via /alerts/ops/rrns, batched 100 at a time So 1+N+1 calls, and an alert with no investigation is never even named. Its failure modes are loud: an unreadable investigation is skipped and counted, and HTTP errors surface as themselves. Step 2 is the one that grows with the window, so it runs under a time budget (60s, against the server's 90s ceiling) and reports truncated=true rather than being killed with the whole run's alerts lost. A partial fetch that says it is partial beats a silent zero — the same reasoning that rules out the filter. The output shape matches search_alerts exactly, so the mapper committed in 1.2.0 serves both. It gains five entries for the parent investigation, which resolve to nothing on the search_alerts feed and are skipped there. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -1,8 +1,8 @@
|
||||
id: rapid7_insightidr
|
||||
name: Rapid7 InsightIDR
|
||||
version: 1.2.1
|
||||
version: 1.3.0
|
||||
description: "Rapid7 InsightIDR (REST API) — investigation ingestion and full lifecycle (list/get/search/create/update/assign/set-status/bulk-close), investigation alerts and Rapid7 product alerts, custom threat indicators (add/replace), log management and LEQL log/log-set queries with downloads, and user directory search."
|
||||
changelog: "1.2.1 — search_alerts: report a malformed terms_json with the input named and echoed, instead of json's bare \"Expecting value: line 1 column 1\". | 1.2.0 — Add the search_alerts OCSF mapper: rule_keys_of_interest carries the host, the process, its command line, its parent and the file hashes, so an alert-based feed indexes entities where the investigation feed indexed none. Validated against a live ABA alert. | 1.1.1 — list_alert_fields: default path_depth to 1 and expose the search filter. The V2 fields endpoint refuses an unbounded walk of the tree (\"root path with path depth greater than 1 is not supported for V2 without a search filter\"), so the bare call returned HTTP 400. | 1.1.0 — Add the Alert Triage API (/idr/at): search alerts, get an alert, its evidence, its actors and its process trees, plus the tenant's alert field catalogue. These carry the detection content the investigation endpoints do not — MITRE T-codes, matched rule keys, assets and accounts, command lines. MDR-gated. | 1.0.0 — Initial release: investigation ingestion (list_investigations) with an exhaustive OCSF mapper, 18 commands across investigations, alerts, custom threats, logs/LEQL search and users. API v1/v2 selectable per instance and per command."
|
||||
changelog: "1.3.0 — Add search_investigation_alerts: ingest only the alerts that belong to an investigation, with their full detection content, in 1+N+1 calls. Filtering search_alerts on investigation_rrn does not work — the term is accepted and silently matches nothing. | 1.2.1 — search_alerts: report a malformed terms_json with the input named and echoed, instead of json's bare \"Expecting value: line 1 column 1\". | 1.2.0 — Add the search_alerts OCSF mapper: rule_keys_of_interest carries the host, the process, its command line, its parent and the file hashes, so an alert-based feed indexes entities where the investigation feed indexed none. Validated against a live ABA alert. | 1.1.1 — list_alert_fields: default path_depth to 1 and expose the search filter. The V2 fields endpoint refuses an unbounded walk of the tree (\"root path with path depth greater than 1 is not supported for V2 without a search filter\"), so the bare call returned HTTP 400. | 1.1.0 — Add the Alert Triage API (/idr/at): search alerts, get an alert, its evidence, its actors and its process trees, plus the tenant's alert field catalogue. These carry the detection content the investigation endpoints do not — MITRE T-codes, matched rule keys, assets and accounts, command lines. MDR-gated. | 1.0.0 — Initial release: investigation ingestion (list_investigations) with an exhaustive OCSF mapper, 18 commands across investigations, alerts, custom threats, logs/LEQL search and users. API v1/v2 selectable per instance and per command."
|
||||
category: siem
|
||||
|
||||
# Per-instance configuration. The Insight platform region selects the API host
|
||||
@@ -414,6 +414,31 @@ commands:
|
||||
required: []
|
||||
outputs_schema: { properties: {} }
|
||||
|
||||
- id: search_investigation_alerts
|
||||
name: rapid7-insight-idr-search-investigation-alerts
|
||||
description: "Ingestion: the alerts that belong to an investigation, with their full detection content. Walks the investigations in a time window, collects their alert RRNs and retrieves the alerts in batches from the Alert Triage API — so an alert Rapid7 never folded into an investigation is never returned. Results path = alerts, dedup on rrn, incremental on start_time (the investigation's creation time). Use the search_alerts mapper: the alert objects are identical, with the parent investigation carried alongside under `investigation`."
|
||||
risk: read
|
||||
inputs_schema:
|
||||
properties:
|
||||
start_time: { type: string, description: "ISO-8601 lower bound on investigation created_time. Incremental fetch watermark." }
|
||||
end_time: { type: string, description: "ISO-8601 upper bound on investigation created_time" }
|
||||
default_days: { type: string, description: "Window size in days used when start_time is absent (default 1)" }
|
||||
statuses: { type: string, description: "Comma-separated investigation statuses to include (open, investigating, closed). Leaving out closed skips what the source has already triaged." }
|
||||
priorities: { type: string, description: "Comma-separated investigation priorities to include (Unspecified, Low, Medium, High, Critical)" }
|
||||
field_ids: { type: string, description: "Comma-separated additional alert field ids to include (see rapid7-insight-idr-list-alert-fields)" }
|
||||
include_investigation: { type: string, description: "Carry the parent investigation on each alert under `investigation` (true/false, default true)" }
|
||||
max_alerts: { type: string, description: "Ceiling on alert RRNs collected in one run (default 500)" }
|
||||
time_budget_seconds: { type: string, description: "Stop walking investigations after this many seconds and report truncated=true, rather than exceed the server's 90-second command budget and lose the whole run (default 60)" }
|
||||
index: { type: string, description: "0-based page index of investigations" }
|
||||
page_size: { type: string, description: "Investigation page size" }
|
||||
limit: { type: number, description: "Maximum number of investigations to walk (default 50)" }
|
||||
required: []
|
||||
outputs_schema: { properties: {} }
|
||||
ingest:
|
||||
results_path: alerts
|
||||
dedup_key: rrn
|
||||
incremental_field: start_time
|
||||
|
||||
# ── Connectivity test ─────────────────────────────────────────────────────
|
||||
- id: test_connection
|
||||
name: rapid7-insight-idr-test-connection
|
||||
|
||||
Reference in New Issue
Block a user