docs: document bundled mappers, jsonata: expressions and the reshape-in-the-mapper convention
The README never described mappers/ or incident-types/ although the sync collects both. Documents the mapper file format, the jsonata: prefix with $parseJson/$toTable, the singleton [] trap, and the convention that new integrations reshape in the mapper rather than pre-building display structures in scripts. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01L1fWBoEKvAw3k78XDkpRQo
This commit is contained in:
@@ -26,8 +26,12 @@ and walks the whole tree looking for files named exactly **`manifest.yaml`**.
|
|||||||
integrations/
|
integrations/
|
||||||
└── <integration-id>/
|
└── <integration-id>/
|
||||||
├── manifest.yaml # required — the integration definition
|
├── manifest.yaml # required — the integration definition
|
||||||
└── scripts/ # optional — only for script-based commands
|
├── scripts/ # optional — only for script-based commands
|
||||||
└── <command-id>.py
|
│ └── <command-id>.py
|
||||||
|
├── mappers/ # optional — bundled mapper templates, one per command
|
||||||
|
│ └── <command-id>.yaml
|
||||||
|
└── incident-types/ # optional — bundled incident types
|
||||||
|
└── <anything>.yaml
|
||||||
```
|
```
|
||||||
|
|
||||||
---
|
---
|
||||||
@@ -132,6 +136,72 @@ Two rules the catalog applies consistently, because they are not obvious:
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
## Bundled mappers (`mappers/<command-id>.yaml`)
|
||||||
|
|
||||||
|
A mapper turns one raw element of a command's output into an incident: the
|
||||||
|
fields Riposte shows and routes on, and an OCSF projection for entity
|
||||||
|
indexing. Bundled mappers are registered as **read-only library templates**
|
||||||
|
on install and **replaced on every update** — an operator's clones are never
|
||||||
|
touched, which also means a clone never receives your fixes. Ship correct
|
||||||
|
templates; let operators clone only to diverge on purpose.
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
name: "My Product Investigations → OCSF" # required — shown in the mapper library
|
||||||
|
description: |
|
||||||
|
Free text. Say what one element looks like and what the mapper reads.
|
||||||
|
|
||||||
|
# Incident fields. Target: title, severity, source, description, custom:<name>.
|
||||||
|
field_mappings:
|
||||||
|
title: "title" # dotted path into the element
|
||||||
|
severity: "priority" # values run through toSeverity (CRITICAL→5 … LOW→2)
|
||||||
|
"custom:vendor_status": "status" # lands in the declared custom field <name>
|
||||||
|
|
||||||
|
# OCSF projection. source_path is full JSONata; ocsf_field is the destination.
|
||||||
|
ocsf:
|
||||||
|
- { source_path: "rrn", ocsf_field: "finding_info.uid" }
|
||||||
|
- { source_path: "created_time", ocsf_field: "finding_info.created_time" }
|
||||||
|
```
|
||||||
|
|
||||||
|
### Expressions: dotted paths and `jsonata:`
|
||||||
|
|
||||||
|
A `field_mappings` expression is a dotted path (with an optional
|
||||||
|
`cond ? then : else` ternary). Prefixed with **`jsonata:`**, it is evaluated
|
||||||
|
by the full JSONata engine instead — the same one `ocsf.source_path` always
|
||||||
|
uses — which can index arrays, filter, and reshape:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
field_mappings:
|
||||||
|
"custom:evidence": "jsonata: $toTable(alerts.evidences.fields[])"
|
||||||
|
"custom:alert_name": "jsonata: $parseJson(alerts[0].evidences[0].data).source_json.AlertDisplayName"
|
||||||
|
severity: "jsonata: priority = 'HIGH' ? 4 : 2"
|
||||||
|
```
|
||||||
|
|
||||||
|
Two Riposte extensions exist alongside the standard JSONata functions:
|
||||||
|
|
||||||
|
| Function | What it does |
|
||||||
|
|----------|--------------|
|
||||||
|
| `$parseJson(str)` | Opens a JSON-encoded string — the original log line most SIEMs bury inside a JSON string field. |
|
||||||
|
| `$toTable(x [, cols])` | Turns an array of objects into rows for a `table` custom field (a single object becomes key/value rows). `cols` keeps only the named columns. |
|
||||||
|
|
||||||
|
Mind JSONata's singleton semantics: a path that finds **one** element unwraps
|
||||||
|
it from its array. End the path with `[]` (`alerts.evidences.fields[]`) so a
|
||||||
|
one-alert investigation still produces table rows instead of key/value pairs.
|
||||||
|
|
||||||
|
**Convention: reshape in the mapper, not in the script.** A command script
|
||||||
|
should return what the API gives, plus at most the flattening the API forces
|
||||||
|
on it. Building display structures in Python (pre-joined rows, markdown
|
||||||
|
summaries) welds them to the integration version — as expressions in the
|
||||||
|
mapper they are visible, testable against the rule's captured samples
|
||||||
|
(editor → Test), and an operator can adapt them without a package release.
|
||||||
|
Pre-1.11 integrations that ship pre-built fields keep working; don't extend
|
||||||
|
the pattern.
|
||||||
|
|
||||||
|
An expression that fails costs its own field and never the alert — silently,
|
||||||
|
by design. The mapper editor's **Test** button is where those errors surface,
|
||||||
|
named by field, evaluated against the rule's last captured samples.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
## Adding this catalog to Riposte
|
## Adding this catalog to Riposte
|
||||||
|
|
||||||
In Riposte → **Integrations → Marketplace → Add source**:
|
In Riposte → **Integrations → Marketplace → Add source**:
|
||||||
@@ -152,9 +222,11 @@ Then **Sync**. Discovered integrations appear in the marketplace, ready to insta
|
|||||||
|
|
||||||
1. Create `integrations/<id>/manifest.yaml` (one directory per integration).
|
1. Create `integrations/<id>/manifest.yaml` (one directory per integration).
|
||||||
2. Add `scripts/<command-id>.py` only for script-based commands.
|
2. Add `scripts/<command-id>.py` only for script-based commands.
|
||||||
3. Bump `version` (semver) on every change — Riposte tracks versions per source.
|
3. Add `mappers/<command-id>.yaml` for every ingestion command — reshape with
|
||||||
4. Validate the YAML parses and `id`/`name` are set.
|
`jsonata:` expressions there rather than in the script (see above).
|
||||||
5. Open a merge request.
|
4. Bump `version` (semver) on every change — Riposte tracks versions per source.
|
||||||
|
5. Validate the YAML parses and `id`/`name` are set.
|
||||||
|
6. Open a merge request.
|
||||||
|
|
||||||
See [`templates/manifest.example.yaml`](templates/manifest.example.yaml) for a
|
See [`templates/manifest.example.yaml`](templates/manifest.example.yaml) for a
|
||||||
fully-commented starting point (that file is intentionally **not** ingested).
|
fully-commented starting point (that file is intentionally **not** ingested).
|
||||||
|
|||||||
Reference in New Issue
Block a user