From 80de5399a66c195ef5d07411b07387c59c28c914 Mon Sep 17 00:00:00 2001 From: Guillaume BOURGEOIS Date: Mon, 12 Oct 2026 08:14:10 +0000 Subject: [PATCH] rapid7-insightidr 1.14.9: a user_authentications answer that fits Riposte gives a tool result 8000 characters; two full summaries ran to 11903 and were cut at 3903, the asset list half gone. The answer now keeps only the fields an authentication question is answered from (destination and source asset and address, domain, result, logon type, service) and lists up to 40 new destination assets in full: about 2.5 KB for 1000 events. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_015RhL2gy6Sx6Y3FEsyBE3mZ --- integrations/rapid7-insightidr/manifest.yaml | 2 +- .../scripts/user_authentications.py | 37 ++++++++++++++++++- 2 files changed, 36 insertions(+), 3 deletions(-) diff --git a/integrations/rapid7-insightidr/manifest.yaml b/integrations/rapid7-insightidr/manifest.yaml index 6206db4..27c49be 100644 --- a/integrations/rapid7-insightidr/manifest.yaml +++ b/integrations/rapid7-insightidr/manifest.yaml @@ -1,6 +1,6 @@ id: rapid7_insightidr name: Rapid7 InsightIDR -version: 1.14.8 +version: 1.14.9 description: "Rapid7 InsightIDR (REST API) — investigation ingestion and full lifecycle (list/get/search/create/update/assign/set-status/bulk-close), investigation alerts and Rapid7 product alerts, investigation comments (add/list), custom threat indicators (add/replace), log management and LEQL log/log-set queries with downloads, and user directory search." changelog: "1.13.2 — query_log and query_log_set: trim the quotes and spaces a pasted log or log set ID drags along, and explain the 404. A trailing quote was URL-encoded into the path and the API answered a bare \"HTTP 404 Not Found\" that named nothing. Also send both time bounds on every run: a scheduled run passes only the watermark, and the first run nothing — the window now ends now and, without a start, opens 24 hours back. | 1.13.1 — test_connection: accept an InsightIDR Essential key. Essential answers the Investigations API with a 403 that names the package, so a valid key failed its test; the test now falls back to Log Search and says which commands that package leaves out — the LEQL feeds (query_log, query_log_set) are its way to ingest detections. | 1.13.0 — Add add_comment and list_comments: the investigation comment thread is where an analyst’s reasoning lives, and a playbook had no way to write into it or read it back. The Comments API is v1-only and addresses its target by RRN — the short investigation ID the other v1 endpoints accept is rejected with a 400 that names nothing, so both commands refuse it up front. | 1.12.0 — Make the LEQL queries ingestible: query_log and query_log_set gain an ingest block, a stable event_id for dedup, a bounded page walk and a detection_name stamp, and they now read start_time as ISO-8601, UNIX seconds or milliseconds alike. A tenant whose basic detection rules only notify (email, Slack) creates no investigation and no alert an API can read, so replaying the rule's own query on a schedule is the only way its detections reach a SOAR. | 1.11.1 — Strip the whitespace out of the comma-separated filter params (statuses, priorities, sources, tags) before the URL. \"HIGH, CRITICAL\" as typed sent \" CRITICAL\" to the API, which rejects the whole call — so a priority filter written naturally fetched nothing. | 1.11.0 —Build one evidence row per alert evidence into custom:rapid7_evidence, for the table field: sortable and filterable where the Markdown summary only reads. | 1.10.0 — Render every detection key as Markdown into custom:rapid7_summary, so a layout can show what the OCSF catalogue has no field for. Map both ends of an account action, read source_json top-level scalars, drop placeholder values, and force evidences to stay an array. | 1.9.2 — Read the evidence data in both of its shapes. A parsed source states its event flat, with no source_data/custom_data wrapper, and the parser returned {} for all of them. | 1.9.1 — get_alert_process_trees: explain the 404, which the API returns with an empty body for three different situations. | 1.9.0 — Fetch each alert process tree and flatten it into ctx.process_tree, the shape the layout Process Tree block reads. Gated on the event type by default, since an authentication alert never has one. | 1.8.0 — Fetch each alert evidence at ingestion and parse it: the log line behind the alert lives in a JSON-encoded string no mapper can reach, and it is where the source and destination addresses, the account and the device actually are. | 1.7.0 — Map source/destination assets and accounts to src_endpoint, dst_endpoint, actor.user and user. A connection rule names both ends and neither is \"the host\", so an RDP-to-DC alert produced no entity at all and nothing a layout could show. | 1.6.0 — Surface region_failures: the Alert Triage API answers HTTP 200 with a partial result when it cannot reach a region, and that list was being dropped. Also report the regions seen in the RRNs against the configured one. | 1.5.0 — Flatten the whole of rule_keys_of_interest into the context. It is an open vocabulary — a process rule and an authentication rule share no keys — so the hardcoded list only covered rules already seen and left every other alert empty. | 1.4.1 — Read the alert RRN from `rrn` before `id` and drop anything not RRN-shaped: the investigation alert list also returns bare UUIDs, and one of those rejects the entire /alerts/ops/rrns batch (HTTP 400), losing every alert in the run. | 1.4.0 — Add get_investigations_full: the investigation with its alerts inlined, so it stays the unit that becomes an incident. Drop-in for list_investigations (results_path data, dedup rrn). | 1.3.0 — Add search_investigation_alerts: ingest only the alerts that belong to an investigation, with their full detection content, in 1+N+1 calls. Filtering search_alerts on investigation_rrn does not work — the term is accepted and silently matches nothing. | 1.2.1 — search_alerts: report a malformed terms_json with the input named and echoed, instead of json's bare \"Expecting value: line 1 column 1\". | 1.2.0 — Add the search_alerts OCSF mapper: rule_keys_of_interest carries the host, the process, its command line, its parent and the file hashes, so an alert-based feed indexes entities where the investigation feed indexed none. Validated against a live ABA alert. | 1.1.1 — list_alert_fields: default path_depth to 1 and expose the search filter. The V2 fields endpoint refuses an unbounded walk of the tree (\"root path with path depth greater than 1 is not supported for V2 without a search filter\"), so the bare call returned HTTP 400. | 1.1.0 — Add the Alert Triage API (/idr/at): search alerts, get an alert, its evidence, its actors and its process trees, plus the tenant's alert field catalogue. These carry the detection content the investigation endpoints do not — MITRE T-codes, matched rule keys, assets and accounts, command lines. MDR-gated. | 1.0.0 — Initial release: investigation ingestion (list_investigations) with an exhaustive OCSF mapper, 18 commands across investigations, alerts, custom threats, logs/LEQL search and users. API v1/v2 selectable per instance and per command." category: siem diff --git a/integrations/rapid7-insightidr/scripts/user_authentications.py b/integrations/rapid7-insightidr/scripts/user_authentications.py index ca84f92..5ebcf3f 100644 --- a/integrations/rapid7-insightidr/scripts/user_authentications.py +++ b/integrations/rapid7-insightidr/scripts/user_authentications.py @@ -268,6 +268,39 @@ def _norm_account(v): return v +# The fields an authentication question is answered from, matched by name so a +# source that prefixes or nests them still lands here. Everything else — event +# ids, timestamps, SIDs, the vendor payload — is left out: Riposte gives one +# tool result 8000 characters, and the first version of this answer, two full +# summaries, was cut at 3903 of 11903 with the asset list half gone. +_KEY_FIELDS = ("destination_asset", "source_asset", "destination_domain", "result", "logon_type", "service") + + +def _compact(events, top_assets): + """Counts for the key fields only; destination assets listed in full.""" + counts = {} + first_ts, last_ts = None, None + for e in events: + ts = e.get("timestamp") + if isinstance(ts, (int, float)): + first_ts = ts if first_ts is None or ts < first_ts else first_ts + last_ts = ts if last_ts is None or ts > last_ts else last_ts + for k, v in _fields(e).items(): + kl = k.lower() + if any(f in kl for f in _KEY_FIELDS) and "sid" not in kl: + counts.setdefault(k, {}) + key = str(v)[:120] + counts[k][key] = counts[k].get(key, 0) + 1 + out = {"count": len(events), + "first_event": _iso(first_ts) if first_ts is not None else None, + "last_event": _iso(last_ts) if last_ts is not None else None} + for k, vals in sorted(counts.items()): + ranked = sorted(vals.items(), key=lambda x: -x[1]) + n = top_assets if "destination_asset" in k.lower() and "address" not in k.lower() else 5 + out[k] = {"distinct": len(vals), "top": [[v, c] for v, c in ranked[:n]]} + return out + + def _is_users(fields, user): """The event is this user's: an account or user field holds the account. @@ -331,8 +364,8 @@ def run(): "user_matches": len(events), "truncated": len(matched) >= limit, "window": "%s to %s" % (_iso(start_time), _iso(end_time)), - "all": _summarize(events, limit), - "new_authentications": _summarize(new, limit), + "all": _compact(events, 10), + "new_authentications": _compact(new, 40), "note": ("all and new_authentications hold only this user's events (an account field equals " "the user): keyword_matches counts every line the text appeared in, other accounts " "included. new_authentications holds the events Rapid7 marked as a first "