2d6e09774265569eddcc99a645370d28fa67fe3e
5 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 f3nris
|
1888648930 |
feat(rapid7-insightidr): feed the layout Process Tree block (v1.9.0)
Rapid7 exposes a process tree per alert; the layout renderer has a block that
draws one. They do not speak the same shape.
The block reads ctx.process_tree as N FLAT chains, root first, each
{alert_id, rule, host, user, nodes[]} — it merges shared ancestors itself.
Rapid7 answers with a NESTED tree: one root entity and children[]. So the
command walks each root-to-leaf path into its own chain, maps the node fields
(cmdline -> cmd_line, hashes.md5 -> md5, start_time -> started_at, pid coerced
from the string Rapid7 sends), and rolls every alert's chains up onto the
investigation, since the block reads the key at the top level and no mapper
expression reaches past alerts[0].
Verified through the block's own reader: a tree of explorer -> cmd ->
{bcdedit, vssadmin} flattens to two chains and comes back as one tree with the
ancestors shared and both leaves marked.
That mark is worth stating plainly. The block marks the last node of a chain as
the target, and Rapid7 flags no node as the one the alert fired on — so here it
means "end of this branch" and nothing stronger. Naming a target would assert
something the sensor never said.
Gated, because this is a third call per alert on top of the evidence one:
include_process_tree=auto (the default) asks only for the event types an
endpoint sensor produces a tree for, true asks for every alert, false never.
An authentication or firewall alert never has one, and on a feed where those
dominate the default saves a call per alert. 404 is treated as "no tree", which
is what it means, and only other statuses are recorded as errors. Bounded at 12
chains and 24 levels so an unexpected shape cannot expand without limit.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
f3nris
|
995a39ea66 |
feat(rapid7-insightidr): fetch and parse the evidence behind each alert (v1.8.0)
An alert names what fired. The evidence is what happened, and until now it was
never fetched — which is why an incident could show a rule name and nothing
else. On a Fortigate admin login the alert carried an empty
rule_keys_of_interest; the evidence carried the whole event.
It cannot be reached by mapping alone. The evidence's `data` is a
JSON-ENCODED STRING holding {source_data, custom_data}, and JSONata does not
deserialise, so no mapper expression can ever see inside it. The parsing has to
happen in the command, and that is the reason for this change more than the
extra call is.
Once parsed, two views of the same event:
custom_data the vendor's normalised fields, prefixed per source
(Forti_user, Forti_srcip) — 6 keys on the sample
source_data the raw line in key="value" form — 25 keys
Both are merged into evidence_fields with the vendor prefix stripped, so a
mapper names `user` once instead of once per vendor, and source_data is overlaid
last because it carries more and is not affected by the quoting artefact below.
Values are cleaned, and that is not cosmetic: custom_data returns
'10.70.20.66"' with a trailing quote left by the vendor parser. Kept as-is it
yields an address matching no regex — an observable nobody can pivot on, and no
error to say why.
The line itself lands on raw_data and the source's own sentence on message,
both of which the layout builder offers. The Fortigate login now maps to
src_endpoint.ip, dst_endpoint.ip, actor.user.name, device.hostname/name/uid,
activity_name "login" and status_detail "success", and indexes four entities
where it indexed none.
Optional (include_evidence, default true) and capped (max_evidence_per_alert,
default 5), one call per alert under the run's existing time budget. An alert
whose evidence cannot be read keeps an evidence_error and still arrives;
evidence_calls and evidence_errors are reported in the run metadata.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
f3nris
|
c7569fd4d0 |
fix(rapid7-insightidr): map both ends of a connection (v1.7.0)
An "RDP connection to DC" investigation arrived with device {}, process {} and
two indexed entities, both of them false — the 32-hex tails of its own alert
RRNs, which the IOC regex cannot tell from an MD5. Its four real values were
sitting in the payload, unmapped:
source_account vbalbakakis
destination_account vassilisadmin
source_asset_address sgazswvdip-01.toto.priv
destination_asset_address scazsvpdc01.toto.priv
The mappings assumed an alert has one host and one user, which holds for a
process_start_event and not for an auth rule: a connection names both ends and
neither is "the host". So source and destination now land on src_endpoint and
dst_endpoint, the acting account on actor.user and the account acted upon on
user — the distinction that lets a lateral move be read in one line.
That choice is also what makes the fields placeable. The layout builder's field
picker is a closed <select> over the OCSF catalogue in web/src/lib/ocsf.ts;
a path absent from it cannot be put on a layout at all. All four targets are in
that catalogue. `detection`, the open-vocabulary catch-all, is not and cannot
be — it keeps its value in the Context tab, which renders the whole tree, but a
layout cannot reference it, so anything meant for a layout has to be mapped to a
real OCSF field first.
evidences[] is flattened at the same time. It emitted "device": {}, "process":
{} and "attacks": {"technique": {}} whenever those keys were absent — three
empty objects per alert, filling the context and hiding what was populated.
Verified against the RDP investigation: four entities where there were none,
two hosts and two accounts, and no false hash once the incident type's
extraction policy is set to selected/host,ip,user.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
f3nris
|
306450a4b6 |
fix(rapid7-insightidr): rule_keys_of_interest is an open vocabulary (v1.5.0)
Both mappers read that field through a hardcoded list of keys — hostname, process.cmd_line, parent_process.*, user — taken from the one alert this work was built against, a process_start_event detection. Every entry is a JSONata predicate on an exact key, so a rule emitting anything else matches none of them. A real alert from a second tenant showed what that costs. It is a custom ingress_auth rule, and its keys are: account locuz_hma@totogroup.net source_ip 183.82.103.101 geoip_country_code IN result SUCCESS Not one appears in the hardcoded list. The alert was fetched, inlined and mapped, and produced an incident as empty as the ones this integration set out to fix — with the account and the source IP sitting in the payload, unread. Two changes: - The whole list is flattened verbatim into `detection`, and rule_matching_keys into `detection_match`. Keys nobody anticipated now arrive intact, and they reach the AI analysis, which reads the mapped context and never `raw`. - account, source_ip, destination_ip, geoip_country_code and result gain explicit OCSF entries. The explicit mappings are not redundant with the flattening: a value under `detection.` is only ever seen by the IOC regex, so an account there would be indexed as an email, or not at all, rather than as a user. Typing an entity correctly still requires naming its field. evidences[] carries the flattened keys too, so the same holds for the second and third alert of an investigation, not only the first. Verified against both rule shapes end to end. The process rule yields the two hosts it did before; the authentication rule now yields the source IP and the account, where it previously yielded nothing. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
f3nris
|
841ae1199f |
feat(rapid7-insightidr): the investigation, with its alerts inlined (v1.4.0)
The previous command returned the alerts as its result rows, which quietly changed the unit: an investigation holding three alerts would have become three incidents. The investigation is what an analyst opens and what the SOC tracks — it stays the incident, and the alerts are its content. get_investigations_full walks the same three hops and inverts the shape: investigations under `data`, each carrying its alerts under `alerts` plus an alert_count. results_path=data and dedup_key=rrn, so it is a drop-in replacement for list_investigations in an existing ingestion rule — only the command name and the mapper change. A few first-alert values are also lifted to the top level, and that is not cosmetic. MapFields resolves dotted paths only and cannot index an array, so `alerts[0].ai_suggested_disposition` in a field_mappings entry reads as nothing — silently, leaving the custom field empty with no error anywhere. Three custom fields were empty for exactly that reason until an end-to-end run showed it. The ocsf section has no such limit: it is evaluated by full JSONata and indexes freely, which is why the host and process mappings worked while the custom fields did not. The mapper maps the investigation first and the first alert after, so the alert wins wherever the container says nothing. It also lands every alert — not just alerts[0] — as OCSF-shaped objects under `evidences`, because the extractor walks arrays and classifies on the path suffix: an investigation aggregating two alerts on two machines now indexes both hosts, where reading alerts[0] alone would have indexed one and silently dropped the other. Verified end to end against the investigation that started this, with a second alert added on another host: field mapping, OCSF mapping and entity extraction. Both hosts are indexed. With the incident type's extraction policy set to selected/host,ip,user, the entity set is exactly those two — the alert RRN's 32-hex suffix, which the IOC regex cannot tell from an MD5, is filtered out there. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |