diff --git a/integrations/rapid7-insightidr/agents/operator.yaml b/integrations/rapid7-insightidr/agents/operator.yaml index 40bf73c..3fb3ed0 100644 --- a/integrations/rapid7-insightidr/agents/operator.yaml +++ b/integrations/rapid7-insightidr/agents/operator.yaml @@ -40,7 +40,9 @@ instructions: | assets", "Domain Credentials": call user_authentications(user=, anchor_time=) ONCE. It does the search, the window and the split itself. Report its - "window" word for word, the totals from "all", and from + "window" word for word, user_matches (the user's own events — + keyword_matches includes other accounts' lines, never report those as + the user's), the totals from "all", and from "new_authentications" the destination assets by name with their counts: those are the new assets. Do not redo this search with query_log_set. Other alerts with evidence_api false, or a question this does not diff --git a/integrations/rapid7-insightidr/manifest.yaml b/integrations/rapid7-insightidr/manifest.yaml index 7096440..6206db4 100644 --- a/integrations/rapid7-insightidr/manifest.yaml +++ b/integrations/rapid7-insightidr/manifest.yaml @@ -1,6 +1,6 @@ id: rapid7_insightidr name: Rapid7 InsightIDR -version: 1.14.7 +version: 1.14.8 description: "Rapid7 InsightIDR (REST API) — investigation ingestion and full lifecycle (list/get/search/create/update/assign/set-status/bulk-close), investigation alerts and Rapid7 product alerts, investigation comments (add/list), custom threat indicators (add/replace), log management and LEQL log/log-set queries with downloads, and user directory search." changelog: "1.13.2 — query_log and query_log_set: trim the quotes and spaces a pasted log or log set ID drags along, and explain the 404. A trailing quote was URL-encoded into the path and the API answered a bare \"HTTP 404 Not Found\" that named nothing. Also send both time bounds on every run: a scheduled run passes only the watermark, and the first run nothing — the window now ends now and, without a start, opens 24 hours back. | 1.13.1 — test_connection: accept an InsightIDR Essential key. Essential answers the Investigations API with a 403 that names the package, so a valid key failed its test; the test now falls back to Log Search and says which commands that package leaves out — the LEQL feeds (query_log, query_log_set) are its way to ingest detections. | 1.13.0 — Add add_comment and list_comments: the investigation comment thread is where an analyst’s reasoning lives, and a playbook had no way to write into it or read it back. The Comments API is v1-only and addresses its target by RRN — the short investigation ID the other v1 endpoints accept is rejected with a 400 that names nothing, so both commands refuse it up front. | 1.12.0 — Make the LEQL queries ingestible: query_log and query_log_set gain an ingest block, a stable event_id for dedup, a bounded page walk and a detection_name stamp, and they now read start_time as ISO-8601, UNIX seconds or milliseconds alike. A tenant whose basic detection rules only notify (email, Slack) creates no investigation and no alert an API can read, so replaying the rule's own query on a schedule is the only way its detections reach a SOAR. | 1.11.1 — Strip the whitespace out of the comma-separated filter params (statuses, priorities, sources, tags) before the URL. \"HIGH, CRITICAL\" as typed sent \" CRITICAL\" to the API, which rejects the whole call — so a priority filter written naturally fetched nothing. | 1.11.0 —Build one evidence row per alert evidence into custom:rapid7_evidence, for the table field: sortable and filterable where the Markdown summary only reads. | 1.10.0 — Render every detection key as Markdown into custom:rapid7_summary, so a layout can show what the OCSF catalogue has no field for. Map both ends of an account action, read source_json top-level scalars, drop placeholder values, and force evidences to stay an array. | 1.9.2 — Read the evidence data in both of its shapes. A parsed source states its event flat, with no source_data/custom_data wrapper, and the parser returned {} for all of them. | 1.9.1 — get_alert_process_trees: explain the 404, which the API returns with an empty body for three different situations. | 1.9.0 — Fetch each alert process tree and flatten it into ctx.process_tree, the shape the layout Process Tree block reads. Gated on the event type by default, since an authentication alert never has one. | 1.8.0 — Fetch each alert evidence at ingestion and parse it: the log line behind the alert lives in a JSON-encoded string no mapper can reach, and it is where the source and destination addresses, the account and the device actually are. | 1.7.0 — Map source/destination assets and accounts to src_endpoint, dst_endpoint, actor.user and user. A connection rule names both ends and neither is \"the host\", so an RDP-to-DC alert produced no entity at all and nothing a layout could show. | 1.6.0 — Surface region_failures: the Alert Triage API answers HTTP 200 with a partial result when it cannot reach a region, and that list was being dropped. Also report the regions seen in the RRNs against the configured one. | 1.5.0 — Flatten the whole of rule_keys_of_interest into the context. It is an open vocabulary — a process rule and an authentication rule share no keys — so the hardcoded list only covered rules already seen and left every other alert empty. | 1.4.1 — Read the alert RRN from `rrn` before `id` and drop anything not RRN-shaped: the investigation alert list also returns bare UUIDs, and one of those rejects the entire /alerts/ops/rrns batch (HTTP 400), losing every alert in the run. | 1.4.0 — Add get_investigations_full: the investigation with its alerts inlined, so it stays the unit that becomes an incident. Drop-in for list_investigations (results_path data, dedup rrn). | 1.3.0 — Add search_investigation_alerts: ingest only the alerts that belong to an investigation, with their full detection content, in 1+N+1 calls. Filtering search_alerts on investigation_rrn does not work — the term is accepted and silently matches nothing. | 1.2.1 — search_alerts: report a malformed terms_json with the input named and echoed, instead of json's bare \"Expecting value: line 1 column 1\". | 1.2.0 — Add the search_alerts OCSF mapper: rule_keys_of_interest carries the host, the process, its command line, its parent and the file hashes, so an alert-based feed indexes entities where the investigation feed indexed none. Validated against a live ABA alert. | 1.1.1 — list_alert_fields: default path_depth to 1 and expose the search filter. The V2 fields endpoint refuses an unbounded walk of the tree (\"root path with path depth greater than 1 is not supported for V2 without a search filter\"), so the bare call returned HTTP 400. | 1.1.0 — Add the Alert Triage API (/idr/at): search alerts, get an alert, its evidence, its actors and its process trees, plus the tenant's alert field catalogue. These carry the detection content the investigation endpoints do not — MITRE T-codes, matched rule keys, assets and accounts, command lines. MDR-gated. | 1.0.0 — Initial release: investigation ingestion (list_investigations) with an exhaustive OCSF mapper, 18 commands across investigations, alerts, custom threats, logs/LEQL search and users. API v1/v2 selectable per instance and per command." category: siem @@ -327,7 +327,7 @@ commands: - id: user_authentications name: rapid7-insight-idr-user-authentications - description: "For a behaviour (UBA) authentication alert — lateral movement, \"attempted to remotely access N new assets\": which assets a user authenticated to around the alert, and which were new. One call: searches the Asset Authentication log set for the user over the 24 hours before the alert's time (to 30 minutes after), and returns per-field counts for all of it and, separately, for the events Rapid7 marked new_authentication — whose destination assets are the new assets. No query syntax, no log set id, no date arithmetic." + description: "For a behaviour (UBA) authentication alert — lateral movement, \"attempted to remotely access N new assets\": which assets a user authenticated to around the alert, and which were new. Only events whose account field is the user count (the keyword search also matches other accounts' lines; keyword_matches says how many). One call: searches the Asset Authentication log set for the user over the 24 hours before the alert's time (to 30 minutes after), and returns per-field counts for all of it and, separately, for the events Rapid7 marked new_authentication — whose destination assets are the new assets. No query syntax, no log set id, no date arithmetic." risk: read inputs_schema: properties: @@ -336,7 +336,7 @@ commands: hours_before: { type: number, description: "Window start, hours before anchor_time (default 24)" } minutes_after: { type: number, description: "Window end, minutes after anchor_time (default 30)" } log_set_name: { type: string, description: "Log set to search (default Asset Authentication)" } - limit: { type: number, description: "Maximum events read (default 2000)" } + limit: { type: number, description: "Maximum events read (default and minimum 10000)" } required: [user, anchor_time] outputs_schema: { properties: {} } diff --git a/integrations/rapid7-insightidr/scripts/user_authentications.py b/integrations/rapid7-insightidr/scripts/user_authentications.py index 2a670c3..ca84f92 100644 --- a/integrations/rapid7-insightidr/scripts/user_authentications.py +++ b/integrations/rapid7-insightidr/scripts/user_authentications.py @@ -255,6 +255,42 @@ def _resolve_name(name): NEW_MARKERS = ("new_authentication",) +_TRUE = ("true", "1", "yes") + + +def _norm_account(v): + """jdoe, DOMAIN\\jdoe, jdoe@domain.tld -> jdoe.""" + v = str(v or "").strip().lower() + if "\\" in v: + v = v.rsplit("\\", 1)[1] + if "@" in v: + v = v.split("@", 1)[0] + return v + + +def _is_users(fields, user): + """The event is this user's: an account or user field holds the account. + + The keyword search matches the text anywhere in the line, so it also brings + other people's logons — on a real run the only "new" event it found was + another account's. Only the account fields decide.""" + want = _norm_account(user) + for k, v in fields.items(): + kl = k.lower() + if ("account" in kl or "user" in kl) and "sid" not in kl and _norm_account(v) == want: + return True + return False + + +def _is_new(fields): + """Rapid7 marks a first-time authentication either with a boolean field + (new_authentication: true) or with that word as a value.""" + for k, v in fields.items(): + if "new_authentication" in k.lower() and str(v).strip().lower() in _TRUE: + return True + if str(v).strip().lower() in NEW_MARKERS: + return True + return False def run(): @@ -278,24 +314,29 @@ def run(): name = str(inp.get("log_set_name") or "Asset Authentication").strip() log_set_id = _resolve_name(name) try: - limit = int(inp.get("limit") or 2000) + limit = max(int(inp.get("limit") or 10000), 10000) except (TypeError, ValueError): - limit = 2000 + limit = 10000 query = 'where("%s")' % user first = request("GET", "log_search/query/logsets/" + urllib.parse.quote(log_set_id, safe=""), params={"query": query, "from": start_time, "to": end_time}) - events = _identify(_collect(first, limit), "", None) - new = [e for e in events - if any(str(v).lower() in NEW_MARKERS for v in _fields(e).values())] + matched = _identify(_collect(first, limit), "", None) + events = [e for e in matched if _is_users(_fields(e), user)] + new = [e for e in events if _is_new(_fields(e))] print(json.dumps({ "user": user, "log_set": name, "query": query, + "keyword_matches": len(matched), + "user_matches": len(events), + "truncated": len(matched) >= limit, "window": "%s to %s" % (_iso(start_time), _iso(end_time)), "all": _summarize(events, limit), "new_authentications": _summarize(new, limit), - "note": ("new_authentications holds only the events Rapid7 marked new_authentication: their " - "destination assets are the new assets a behaviour alert counts."), + "note": ("all and new_authentications hold only this user's events (an account field equals " + "the user): keyword_matches counts every line the text appeared in, other accounts " + "included. new_authentications holds the events Rapid7 marked as a first " + "authentication: their destination assets are the new assets a behaviour alert counts."), }))