The keyword search matches the account name anywhere in a line, so it brought
other accounts' logons: on a real run the only "new" event reported was
another user's failed password. Only events whose account/user field is the
user (DOMAIN\\user and user@domain normalized) are summarized now, with
keyword_matches and user_matches both reported. A first authentication is
recognized as Rapid7 flags it, a boolean new_authentication field, as well as
the word. The event limit cannot be set below 10000 (the agent passed 1000).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015RhL2gy6Sx6Y3FEsyBE3mZ
A new read command answers what a behaviour authentication alert asks —
which assets the user authenticated to around the alert, and which were new —
in one call: it resolves the Asset Authentication log set, builds the
keyword query and the 24 h window itself, and summarizes all events and,
apart, those marked new_authentication. Across real runs gpt-oss mangled log
set ids, computed the window a day off, put a filter inside the LEQL (HTTP
400) and read ten lines out of a thousand; this leaves it nothing to get
wrong. Static calls it first for such alerts; list_investigation_alerts
points to it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015RhL2gy6Sx6Y3FEsyBE3mZ
query_log_set takes only="field=value[, …]", applied to the events' fields
after the search (case ignored), and says how many matched before it. Static
runs a second summary of the same query restricted to what the alert is about
— for a "new assets" alert, the new authentications — and names their
destination assets: the first real run summarized 978 events but left the 64
new ones mixed in.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015RhL2gy6Sx6Y3FEsyBE3mZ
summarize=true returns, over every matching event, each field's most frequent
values with counts, the first and last event time and five samples, instead of
the raw events. On a real run Static got 1000 raw events, could read about
ten, and reported the hosts and accounts of those ten; 300 events now fit in
about 1.5 KB. Static passes summarize always and quotes the returned window.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015RhL2gy6Sx6Y3FEsyBE3mZ
get_investigations_full dropped every alert without an RRN, so a behaviour
(UBA) investigation arrived with alerts: [] and alert_count: 0 and its
reading worked from a title alone. Those alerts are now kept under
alerts_without_rrn as the investigation endpoint returns them, counted in
alert_count, rendered in summary_markdown with where their evidence lives,
and lift first_alert_type / first_alert_source / first_alert_id when they are
all the investigation has.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015RhL2gy6Sx6Y3FEsyBE3mZ
query_log_set takes anchor_time with hours_before / minutes_after and computes
the window itself, and returns it as text: on a real run the agent computed
"24 hours before 2026-10-11T00:04Z" as ending on the 10th and reported a
day-old window. Static passes the alert's first seen time instead of dates,
and lists domain controllers apart from the new assets.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015RhL2gy6Sx6Y3FEsyBE3mZ
query_log_set accepts log_set_name and resolves the id itself (exact name, or
the one set whose name contains it; otherwise refused with the existing
names): on a real run the model mangled a 36-character id it copied. Static
queries by name, retries a 0-event keyword search case-insensitively, takes
the 24 hours before a behaviour alert (they aggregate hours of activity), and
always looks for the user's other investigations.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015RhL2gy6Sx6Y3FEsyBE3mZ
list_investigation_alerts now marks each alert with alert_rrn, evidence_api
and next: a behaviour alert with no RRN says, in the output the agent reads,
that the Alert Triage commands answer 404 for it and that its evidence is in
the authentication logs. Static still called them when only its instructions
said so. Its sheet gets 14 calls spent in a stated order, authentication log
sets for lateral-movement alerts (never file access), and a window built from
first and last seen.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015RhL2gy6Sx6Y3FEsyBE3mZ
agents/operator.yaml: read-only commands, analyst_l2 ceiling, and the procedure
learned on a real UBA alert (start from the investigation's alerts, plain-UUID
alerts have no Alert Triage evidence, LEQL keyword search in parentheses,
search_investigation takes start_time/end_time). README documents agents/.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015RhL2gy6Sx6Y3FEsyBE3mZ
add_comment (POST /idr/v1/comments) and list_comments (GET /idr/v1/comments)
close the gap the Demisto integration never covered: the investigation comment
thread is where an analyst's reasoning is recorded, and a playbook could neither
write into it nor read it back.
The Comments API is v1-only and has no multi-customer parameter, so neither
command takes api_version. It addresses its target by RRN; the short
investigation ID the other v1 endpoints accept is refused with a 400 that names
no parameter, so both commands reject a non-RRN up front and say why.
list_comments walks the pages (size capped at 100 by the API) up to `limit`, or
all of them with all_results, with a 50-page backstop.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A tenant whose basic detection rules only notify — email, Slack — creates no
investigation and no alert any API can read: /investigations stays empty and the
Alert Triage API is MDR-gated. The rule's LEQL query is the only artefact left,
so replay it on a schedule and turn every matching line into an alert.
query_log and query_log_set gain:
- an ingest block (results_path events, dedup_key event_id, watermark start_time);
- event_id = log_id + the line's sequence number, the line's own address in Log
Search. It has to be stable across runs because the scheduler deliberately
re-reads the last minute of the previous window on every tick, and anything
derived from the fetch time would make that overlap a second incident. A hash
of (log_id, timestamp, message) covers a source that does not number entries;
- start_time read as ISO-8601, UNIX seconds or milliseconds alike. The API only
takes milliseconds, but which one the scheduler sends is an operator setting on
the rule — the feed must not fetch the wrong window because of a dropdown;
- a bounded page walk (limit, default 1000): unattended every few minutes, a
query matching more than intended would otherwise hold the run open long
enough for the next tick to find it in flight and skip itself;
- detection_name, stamped on every event. The log line does not know which rule
it is being replayed for, and without it an incident is titled by its own first
characters where the source console shows an alert name.
Plus mappers/query_log{,_set}.yaml: the parsed event lives in `message` as a
JSON-encoded string, so account, addresses, asset and process come out through
$parseJson. A raw unparsed line is not JSON — those entries resolve to nothing
and are skipped, and custom:rapid7_log_line still carries the line verbatim.
Severity is left unmapped: a log line has no priority of its own, so the alert
rule's default applies.
Known limits, by construction: this endpoint rejects calculations, so a rule with
a threshold (N occurrences in M minutes) cannot be reproduced — only pattern
rules are 1:1 — and an inactivity rule never can, since it fires on the absence
of events. The query also becomes a copy: editing the rule in InsightIDR leaves
the feed reading the old one.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UjtRtCzw6FEajDDHJ3mTQf
"HIGH, CRITICAL" as typed sent " CRITICAL" to the API, which rejects
the whole call — a priority filter written naturally fetched nothing.
Normalize statuses/priorities/sources/tags in list_investigations,
get_investigations_full and search_investigation_alerts before the URL.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L1fWBoEKvAw3k78XDkpRQo
The Markdown summary reads as a page. It cannot be sorted or filtered, and an
investigation aggregating several alerts, each with several evidences, is
exactly where those are wanted: "Multiple User Accounts Were Disabled" is a
question about which accounts, and the answer is a column.
So one row per evidence, across every alert, into custom:rapid7_evidence. The
table field reads an array of objects as it stands, so nothing is reshaped on
the mapper side. It sits beside the summary rather than replacing it — one is
read, the other is queried.
Two decisions worth stating.
Each row carries its alert's name first. An investigation aggregates several,
and a row that does not say where it came from cannot be acted on.
Column order is load-bearing, because it is the only ordering signal the field
has: it takes the union of keys in order of first appearance, so whatever the
first row names first is what an analyst sees without scrolling. Known keys
lead — action, result, the source and target pairs, the host, the addresses —
and the rest follow alphabetically.
Width is capped at 30 columns, decided by how many rows carry each key rather
than by which appeared first: an investigation mixing rule types has a sparse
union, and a column present on one row out of eighty would otherwise push out
one present on all of them. Exercised on a 187-column case — the five keys
common to every row all survived, and the count is reported as
evidence_columns_dropped rather than the loss being silent.
Verified against the account-disabled investigation: two evidences become two
rows of twenty columns, `alert` first, the two disabled accounts in
target_account — checked through the table field's own reader, not just the
generator.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
An "Multiple User Accounts Were Disabled" investigation came back with one
entity — the actor — while the layout stayed nearly empty. Four separate
reasons, and the last one is why the first three keep recurring.
evidences was an OBJECT, not an array. JSONata returns a bare object when the
path yields one item, so the key had one shape on an investigation with a
single alert and another with two. Bracketed to force an array.
Only the actor was mapped. An ad_admin rule names who acted AND who it was
done to, and on this alert the disabled accounts are the point of it. Target
account, target user and both domains now map to the OCSF Target User group,
the actor to Actor — the distinction that lets the sentence be read in one
line.
The domain controller was nowhere. A Windows event puts it in
source_json.computerName and nowhere else, and source_json was skipped whole
because the Azure AD one runs to kilobytes. Its top-level SCALARS are now
read — bounded, and where computerName, osVersion and callerIpAddress live —
while its nested part stays skipped.
source_asset was the string "unknown". Mapped as-is it becomes an asset named
"unknown" in the entity index, shared by every incident that had no asset: a
pivot that groups things by the absence of a fact. Placeholders are dropped,
and a key whose value cleans away to nothing is removed rather than kept
empty, since "" is not nil and would still be written to its OCSF field.
Five entities now, from one: both accounts, both display names, the DC.
And the reason this kept happening. The layout builder's field picker is a
closed <select> over the OCSF catalogue in web/src/lib/ocsf.ts, so a path absent
from it cannot be placed on a layout at all — which is what `detection` and
`evidence` are, and what every unanticipated key will be. Mapping each one to a
real OCSF field is right where a field exists, but rule_keys_of_interest is an
open vocabulary: four spellings turned up in as many days.
So the whole of it is also rendered as a Markdown table into
custom:rapid7_summary — keys of interest first, since the rule itself says
those are what matter, then the evidence fields. Declare a custom field of type
markdown with that name, put it on the layout, and a new rule type is readable
the day it appears instead of after another mapping round.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
evidence_fields came back {} on a live run. The evidence was fetched, the run
was clean — one call, 1.3s, no error — and the parser found nothing in it.
`data` does not have one shape. The parser was written against an unparsed
Fortigate source and looked only for {source_data, custom_data}. A PARSED
source has no such wrapper: the event is the object.
{user, account, source_ip, geoip_country_code, geoip_organization,
result, service, user_agent, source_json{…}, r7_context{…},
detection_context{…}}
Every one of those is what the incident was missing, sitting one json.loads
away, and the parser walked past all of it and returned an empty dict. No error
— which is how it survived a run that reported success.
Now both are read as a union rather than one or the other: scalars at the top
level are always taken, so a third shape still yields its plain fields instead
of nothing, and the wrapped keys are overlaid when present. source_json is
skipped — several kilobytes of vendor payload that stay in the raw line —
while r7_context and detection_context are flattened one level, being Rapid7's
own annotations and the same across parsed sources. They carry the account age
in minutes, which is exactly what says whether a sign-in is from a new account.
The Azure AD sign-in now maps to src_endpoint.ip, .isp_org and
.location.country, actor.user.name/.uid/.account.name, http_request.user_agent
and metadata.product.feature.name, and indexes the address and the account.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Running the command on an alert reports `{"error": "HTTP 404", "detail": ""}`.
The API returns 404 with an empty body, so the operator is told a number and
nothing else — and cannot tell a normal answer from a fault.
It is usually normal, and the URL is not at fault: it is built exactly as
get_alert_evidence builds its own, which works. Rapid7 uses the same 404 for
three unrelated situations and does not distinguish them:
- the alert is not an endpoint detection, so no tree exists. This is the
common case on a feed of ingress_auth, asset_auth or firewall events;
- the tree exists in principle but was never generated. The endpoint returns
the CACHED tree unless force_refresh is set, so an ungenerated tree answers
404 while its data is there. Retrying with force_refresh=true is the first
thing to try on an endpoint detection;
- the key lacks the privilege.
All three are now named in the error, and the force_refresh input says what it
is really for rather than describing it only as a staleness control.
The ingestion path already handled this correctly — it treats 404 as "no tree"
and counts only other statuses as errors — so nothing changes there.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Rapid7 exposes a process tree per alert; the layout renderer has a block that
draws one. They do not speak the same shape.
The block reads ctx.process_tree as N FLAT chains, root first, each
{alert_id, rule, host, user, nodes[]} — it merges shared ancestors itself.
Rapid7 answers with a NESTED tree: one root entity and children[]. So the
command walks each root-to-leaf path into its own chain, maps the node fields
(cmdline -> cmd_line, hashes.md5 -> md5, start_time -> started_at, pid coerced
from the string Rapid7 sends), and rolls every alert's chains up onto the
investigation, since the block reads the key at the top level and no mapper
expression reaches past alerts[0].
Verified through the block's own reader: a tree of explorer -> cmd ->
{bcdedit, vssadmin} flattens to two chains and comes back as one tree with the
ancestors shared and both leaves marked.
That mark is worth stating plainly. The block marks the last node of a chain as
the target, and Rapid7 flags no node as the one the alert fired on — so here it
means "end of this branch" and nothing stronger. Naming a target would assert
something the sensor never said.
Gated, because this is a third call per alert on top of the evidence one:
include_process_tree=auto (the default) asks only for the event types an
endpoint sensor produces a tree for, true asks for every alert, false never.
An authentication or firewall alert never has one, and on a feed where those
dominate the default saves a call per alert. 404 is treated as "no tree", which
is what it means, and only other statuses are recorded as errors. Bounded at 12
chains and 24 levels so an unexpected shape cannot expand without limit.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
An alert names what fired. The evidence is what happened, and until now it was
never fetched — which is why an incident could show a rule name and nothing
else. On a Fortigate admin login the alert carried an empty
rule_keys_of_interest; the evidence carried the whole event.
It cannot be reached by mapping alone. The evidence's `data` is a
JSON-ENCODED STRING holding {source_data, custom_data}, and JSONata does not
deserialise, so no mapper expression can ever see inside it. The parsing has to
happen in the command, and that is the reason for this change more than the
extra call is.
Once parsed, two views of the same event:
custom_data the vendor's normalised fields, prefixed per source
(Forti_user, Forti_srcip) — 6 keys on the sample
source_data the raw line in key="value" form — 25 keys
Both are merged into evidence_fields with the vendor prefix stripped, so a
mapper names `user` once instead of once per vendor, and source_data is overlaid
last because it carries more and is not affected by the quoting artefact below.
Values are cleaned, and that is not cosmetic: custom_data returns
'10.70.20.66"' with a trailing quote left by the vendor parser. Kept as-is it
yields an address matching no regex — an observable nobody can pivot on, and no
error to say why.
The line itself lands on raw_data and the source's own sentence on message,
both of which the layout builder offers. The Fortigate login now maps to
src_endpoint.ip, dst_endpoint.ip, actor.user.name, device.hostname/name/uid,
activity_name "login" and status_detail "success", and indexes four entities
where it indexed none.
Optional (include_evidence, default true) and capped (max_evidence_per_alert,
default 5), one call per alert under the run's existing time budget. An alert
whose evidence cannot be read keeps an evidence_error and still arrives;
evidence_calls and evidence_errors are reported in the run metadata.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
An "RDP connection to DC" investigation arrived with device {}, process {} and
two indexed entities, both of them false — the 32-hex tails of its own alert
RRNs, which the IOC regex cannot tell from an MD5. Its four real values were
sitting in the payload, unmapped:
source_account vbalbakakis
destination_account vassilisadmin
source_asset_address sgazswvdip-01.toto.priv
destination_asset_address scazsvpdc01.toto.priv
The mappings assumed an alert has one host and one user, which holds for a
process_start_event and not for an auth rule: a connection names both ends and
neither is "the host". So source and destination now land on src_endpoint and
dst_endpoint, the acting account on actor.user and the account acted upon on
user — the distinction that lets a lateral move be read in one line.
That choice is also what makes the fields placeable. The layout builder's field
picker is a closed <select> over the OCSF catalogue in web/src/lib/ocsf.ts;
a path absent from it cannot be put on a layout at all. All four targets are in
that catalogue. `detection`, the open-vocabulary catch-all, is not and cannot
be — it keeps its value in the Context tab, which renders the whole tree, but a
layout cannot reference it, so anything meant for a layout has to be mapped to a
real OCSF field first.
evidences[] is flattened at the same time. It emitted "device": {}, "process":
{} and "attacks": {"technique": {}} whenever those keys were absent — three
empty objects per alert, filling the context and hiding what was populated.
Verified against the RDP investigation: four entities where there were none,
two hosts and two accounts, and no false hash once the incident type's
extraction policy is set to selected/host,ip,user.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
An RRN states its region — rrn:alerts:eu:<org>:alert:… — while the base URL is
built from the instance's configured `region`. Following that thread found a
gap that matters more than the naming.
The Alert Triage API is multi-region behind one gateway, and eleven of its
endpoints can answer HTTP 200 with a region_failures list. The spec is explicit:
the presence of items in that field IS the partial failure. Three of the
commands here call such endpoints and all three read only the payload, so a
region the gateway could not reach produced fewer alerts and nothing else — no
error, no count, no log line. For an ingestion feed that is indistinguishable
from a quiet period, which is the one failure mode this integration has now hit
three separate times.
So region_failures is carried into the run metadata by every command that can
receive it, alongside the regions actually seen in the collected RRNs, the
configured region, and any mismatch between them.
The mismatch report is deliberately a report and not a correction. On a
single-region tenant the two always agree. On a multi-customer key, which spans
organizations, they need not — and rewriting the host per resource would be a
guess about how Rapid7 routes these calls, made without a tenant to verify it
against. Naming the discrepancy is what can be done honestly today; if it ever
fires, the metadata says so instead of the alerts silently going missing.
list_alert_fields already passed region_failures through untouched, since it
returns the API response as it stands.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Both mappers read that field through a hardcoded list of keys — hostname,
process.cmd_line, parent_process.*, user — taken from the one alert this work
was built against, a process_start_event detection. Every entry is a JSONata
predicate on an exact key, so a rule emitting anything else matches none of
them.
A real alert from a second tenant showed what that costs. It is a custom
ingress_auth rule, and its keys are:
account locuz_hma@totogroup.net
source_ip 183.82.103.101
geoip_country_code IN
result SUCCESS
Not one appears in the hardcoded list. The alert was fetched, inlined and
mapped, and produced an incident as empty as the ones this integration set out
to fix — with the account and the source IP sitting in the payload, unread.
Two changes:
- The whole list is flattened verbatim into `detection`, and rule_matching_keys
into `detection_match`. Keys nobody anticipated now arrive intact, and they
reach the AI analysis, which reads the mapped context and never `raw`.
- account, source_ip, destination_ip, geoip_country_code and result gain
explicit OCSF entries. The explicit mappings are not redundant with the
flattening: a value under `detection.` is only ever seen by the IOC regex,
so an account there would be indexed as an email, or not at all, rather than
as a user. Typing an entity correctly still requires naming its field.
evidences[] carries the flattened keys too, so the same holds for the second and
third alert of an investigation, not only the first.
Verified against both rule shapes end to end. The process rule yields the two
hosts it did before; the authentication rule now yields the source IP and the
account, where it previously yielded nothing.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The run died with HTTP 400 and eight "The provided RRN: [...] is invalid",
every one a plain UUID. Two mistakes compounded.
The alert list under an investigation is not consistent about where the RRN
sits. On the ABA alert this work was built against, `id` held the RRN. On
others it holds a bare UUID. The code read `id` first and `rrn` only as a
fallback, so it picked the UUID whenever both were present.
Worse, the failure was total rather than partial. /alerts/ops/rrns validates the
whole list and rejects the entire request when one entry is malformed, so a
single bad identifier costs every alert in that batch — and with one batch, the
whole run. An investigation feed that stops dead because one alert is shaped
differently is not an acceptable failure mode.
So: read `rrn` first, and drop anything that is not RRN-shaped before the batch
is sent, counting it as alert_ids_without_rrn in the run metadata. The
investigation still arrives, its retrievable alerts still arrive, and what could
not be fetched is a number an operator can see rather than an absence nobody
would notice.
Both commands that walk this list are fixed; the selector is shared and was
checked against the two real shapes plus the empty and null cases.
This turns a total failure into a partial one. It does NOT establish that those
eight alerts are reachable at all: if they carry no RRN anywhere, they are not
in the Alert Triage system and no field_ids or batch call will produce them.
That needs the raw shape of one, which the next run of list_investigation_alerts
on the failing investigation will show.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The previous command returned the alerts as its result rows, which quietly
changed the unit: an investigation holding three alerts would have become three
incidents. The investigation is what an analyst opens and what the SOC tracks —
it stays the incident, and the alerts are its content.
get_investigations_full walks the same three hops and inverts the shape:
investigations under `data`, each carrying its alerts under `alerts` plus an
alert_count. results_path=data and dedup_key=rrn, so it is a drop-in replacement
for list_investigations in an existing ingestion rule — only the command name
and the mapper change.
A few first-alert values are also lifted to the top level, and that is not
cosmetic. MapFields resolves dotted paths only and cannot index an array, so
`alerts[0].ai_suggested_disposition` in a field_mappings entry reads as nothing
— silently, leaving the custom field empty with no error anywhere. Three custom
fields were empty for exactly that reason until an end-to-end run showed it. The
ocsf section has no such limit: it is evaluated by full JSONata and indexes
freely, which is why the host and process mappings worked while the custom
fields did not.
The mapper maps the investigation first and the first alert after, so the alert
wins wherever the container says nothing. It also lands every alert — not just
alerts[0] — as OCSF-shaped objects under `evidences`, because the extractor
walks arrays and classifies on the path suffix: an investigation aggregating
two alerts on two machines now indexes both hosts, where reading alerts[0]
alone would have indexed one and silently dropped the other.
Verified end to end against the investigation that started this, with a second
alert added on another host: field mapping, OCSF mapping and entity extraction.
Both hosts are indexed. With the incident type's extraction policy set to
selected/host,ip,user, the entity set is exactly those two — the alert RRN's
32-hex suffix, which the IOC regex cannot tell from an MD5, is filtered out
there.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Ingesting from search_alerts brings back every alert the tenant produces,
including the operational noise — two of the first three sampled were an agent
reporting it lacked Full Disk Access. What is worth an incident is what Rapid7
or the analyst folded into an investigation.
Filtering search_alerts on investigation_rrn does not work. The term is accepted
— no validation error, HTTP 200 — and matches nothing: zero alerts over fifteen
days, on a tenant where an alert carrying an investigation_rrn provably exists
in that window. The field is not searchable, and the API says so by returning an
empty page rather than by refusing the term. That failure mode is the reason not
to build an ingestion feed on it: a filter that silently stops matching drains
the feed to zero with no error anywhere, and nothing would ever say why.
search_investigation_alerts is guaranteed by construction instead:
1. the investigations in the window (incremental on their created_time)
2. their alert RRNs, one call per investigation — there is no bulk form
3. the alerts themselves via /alerts/ops/rrns, batched 100 at a time
So 1+N+1 calls, and an alert with no investigation is never even named. Its
failure modes are loud: an unreadable investigation is skipped and counted, and
HTTP errors surface as themselves.
Step 2 is the one that grows with the window, so it runs under a time budget
(60s, against the server's 90s ceiling) and reports truncated=true rather than
being killed with the whole run's alerts lost. A partial fetch that says it is
partial beats a silent zero — the same reasoning that rules out the filter.
The output shape matches search_alerts exactly, so the mapper committed in 1.2.0
serves both. It gains five entries for the parent investigation, which resolve
to nothing on the search_alerts feed and are skipped there.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
json.loads answers a bad value with "Expecting value: line 1 column 1 (char 0)",
which names neither the input at fault nor what it actually received. The
command has three string inputs; nothing in that message says which one broke,
so the operator is left guessing.
The mistake it hides is the obvious one. Documentation shows the input name and
the value together, so `terms_json: [{...}]` gets pasted whole into the value
field, and the parser then chokes on the field name it was handed. Echoing the
first 80 characters back makes that visible immediately.
Also carry a worked example in the manifest — the term that keeps only alerts
attached to an investigation, which is the filter this input mainly exists for —
and say explicitly to paste the array alone.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
An incident built from a Rapid7 investigation indexed zero entities. Nothing to
pivot on, nothing to correlate, and an AI analysis with nothing to read. The
reason turned out not to be the mapper but the feed: an investigation is an
envelope, and neither it nor its /alerts list carries a host, an account, a
process or a MITRE code.
The Alert Triage API does, in rule_keys_of_interest. On the alert behind the
investigation that started this, that field holds:
hostname NBSGIT01-26
process.name bcdedit.exe
process.cmd_line bcdedit /set {default} recoveryenabled No
process.exe_path C:\WINDOWS\System32\bcdedit.exe
parent_process.name cmd.exe
parent_process.cmd_line "C:\WINDOWS\system32\cmd.exe"
parent_process.exe_file.hashes.md5 e463e36104d22a2479046a7d4f620816
with rule.mitre_tcodes ["TA0040", "T1490"] — Impact / Inhibit System Recovery,
which is what disabling Windows recovery actually is. None of it was reachable
before.
It is a list of {key, values} pairs rather than an object, so every entry reads
through a JSONata predicate — rule_keys_of_interest[key='hostname'].values[0].
A key a rule does not emit resolves to nothing and is skipped, so one file
covers every detection rule.
Two details worth keeping:
- mitre_tcodes mixes tactics and techniques in one array ("TA0040" beside
"T1490"). They are split on the "TA" prefix so a tactic never lands in the
technique field, where it would name a technique that does not exist.
- rule.name is documented but absent from live ABA alerts, so the analytic
name falls back to `type`, which repeats the rule's display name.
Rapid7's own status, disposition and AI-suggested disposition ride as custom
fields. They cannot reach the incident's status and verdict columns — MapFields
accepts only title, severity, source, description and custom:* — but an analyst
can at least see that the source already closed the alert and what it decided.
The AI suggestion is exposed as a fact, never as a verdict.
Validated end to end against a live alert: field mapping, OCSF mapping and
entity extraction, which now yields the host and the parent process hash where
it previously yielded nothing.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The bare call answered HTTP 400:
root path with path depth greater than 1 is not supported for V2
without a search filter
Sending neither path nor path_depth asks the V2 fields endpoint to walk the
whole hierarchy from the root, which it refuses. That is exactly the call an
operator makes first — open the command, run it, see what the tenant exposes —
so the one entry point into this API failed on its most obvious use.
Default path_depth to 1 when neither path nor search is given: the top-level
nodes come back, and `path` drills down from there. Also expose the `search`
query parameter, which was missing from the command entirely and which lifts
the depth restriction — it is the way to look through the whole tree at once
for a field whose branch you do not know yet (search=hostname).
Found by running the command against a live tenant. The 400 also settles the
open question from v1.1.0: the API answered a validation error rather than
403, so the Alert Triage API is reachable with this key.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
An ingested investigation carried almost nothing: title, priority, status,
assignee and timestamps. Not a host, not an account, not a command line — so
no entity was ever indexed, and neither the pivot graph nor the AI analysis
had anything to read. Fetching the investigation's alerts did not help: that
endpoint answers with the rule name (already the investigation's title), the
alert source and an alert RRN, and nothing else.
The detection content lives in a different API. The Alert Triage API is served
under /idr/at — its own base, next to /idr/v1 and /idr/v2 — and it is where
the evidence, the actors, the MITRE T-codes and the process trees actually
are. It was missing from this package entirely.
Six read commands:
search_alerts POST /alerts/ops/search
get_alert GET /alerts/{rrn}
get_alert_evidence GET /alerts/{rrn}/evidences
get_alert_actors GET /alerts/{rrn}/actors
get_alert_process_trees POST /alerts/{rrn}/process_trees/latest
list_alert_fields GET /alerts/2/fields
search_alerts is the one that changes the shape of the feed. It returns full
alert objects — rule with its mitre_tcodes, rule_keys_of_interest, the
AI-suggested disposition, the parent investigation RRN, the status transition
timings and the log entries behind the alert — and its start_time is required
by the API, which makes it a natural incremental watermark. So the alerts can
be read directly, paginated, with no per-investigation fan-out.
Two things worth knowing before using them:
- field_ids is not optional in practice. The API returns no additional
fields when it is empty, and those additional fields are where the
hostnames, accounts and command lines are. Their ids are per-tenant dotted
paths (process.cmd_line, …), which is what list_alert_fields is for.
/alerts/fields is deprecated, hence /alerts/2/fields.
- the whole API is gated on Managed Detection and Response with access to
the Alerts experience. A key that works on every investigation endpoint
can still be refused here, so 401 and 403 are reported with that named
explanation rather than as a bare status code — a licence boundary should
not read as a bug.
Written against the published OpenAPI spec (SIEM Alerts API 39.10.0); not yet
exercised against a live tenant.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
REST API integration for Rapid7 InsightIDR. Investigation ingestion
(list_investigations) with an exhaustive OCSF mapper and a bundled default
incident type, plus 18 commands across investigations (list/get/search/
create/update/assign/set-status/bulk-close), investigation alerts and Rapid7
product alerts, custom threat indicators (add/replace), log management and
LEQL log/log-set queries with downloads, and user directory search.
API v1/v2 selectable per instance (is_v2) and per command (api_version);
multi-customer query parameter supported on v2 calls.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
f3nris
Guillaume BOURGEOIS