feat(cortex-xdr): ingest alerts, not only incidents (v1.3.0)
An XDR incident is an aggregate; the SOC works the detections under it. The alerts endpoint was already exposed as a read command, but nothing could feed an alert rule with it — no results path, no dedup key, no watermark, no mapper, no incident type. All five are here now, so an alert rule can be pointed at reply.alerts the same way it is pointed at reply.incidents. get_alerts pages past the API's 100-results-per-call ceiling: an alert feed carries far more than a hundred detections between two polls, and whatever a single page leaves behind is never fetched again, because the next run's watermark has already moved past it. On an incremental fetch it also sorts oldest first, so a window larger than the limit drops its most recent alerts — the only ones the next poll can still see — and says so via `truncated`. Two fixes to the incident side while in the same files: - The severity expression compared strings, which the mapping engine cannot do (it reads numeric comparisons only). Every test read as false, so every ingested incident silently took the alert rule's default severity. The bare field works: Riposte maps critical/high/medium/low onto 1-5 itself. - The incident mapper carried no `time`, so the detection anchor was missing and MTTD stayed empty for the whole feed. creation_time fills it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,3 @@
|
||||
name: "Cortex XDR Alert"
|
||||
color: "#ef8354"
|
||||
icon: "alert"
|
||||
@@ -1,8 +1,8 @@
|
||||
id: cortex_xdr
|
||||
name: Cortex XDR
|
||||
version: 1.2.1
|
||||
description: "Palo Alto Cortex XDR (public API v1) — incident ingestion + write-back, endpoint isolation/scan/delete/tagging, RTR scripts, hash block/allow lists, file quarantine/restore/retrieval, alert exclusions, external alert push (parsed/CEF), device-control violations, audits, distributions and RBAC/risk."
|
||||
changelog: "1.2.1 — Connection troubleshooting: the URL is normalised to the tenant host (a pasted /public_api/v1 or console path no longer breaks the call), a non-JSON reply reports the status, content type and body instead of a bare JSON parse error, missing key/key ID is caught up front, nonce and timestamp are sent in both auth modes as the reference client does, and test_connection now probes get_incidents. 1.2.0 — Incident write-back (update_incident: status/severity/assignment/resolve comment) and external alert push (insert_parsed_alerts, insert_cef_alerts). 1.1.0 — Full command coverage: added delete/alias/tag endpoints, abort scan, original alerts, script metadata/code/snippet/exec-status, file retrieval (+details), alert exclusions, device-control violations, audits, distribution url/status/create and RBAC (users, roles, groups, risk score, risky users/hosts). 1.0.0 — Initial release: incident ingestion (get_incidents) with OCSF mapper, endpoints, isolate/unisolate, scan, hash blocklist/allowlist, quarantine/restore, run script + results, alerts retrieval, distributions and action status. Standard or Advanced API authentication."
|
||||
version: 1.3.0
|
||||
description: "Palo Alto Cortex XDR (public API v1) — incident and alert ingestion + write-back, endpoint isolation/scan/delete/tagging, RTR scripts, hash block/allow lists, file quarantine/restore/retrieval, alert exclusions, external alert push (parsed/CEF), device-control violations, audits, distributions and RBAC/risk."
|
||||
changelog: "1.3.0 — Alert ingestion: get_alerts is now a fetch command (results path reply.alerts, dedup on alert_id, incremental on source_insert_ts) with a bundled OCSF mapper and a Cortex XDR Alert incident type, so detections can be ingested alongside — or instead of — incidents. The incident mapper is fixed on the way past: its severity expression compared strings, which the mapping engine cannot do, so every ingested incident silently took the rule's default severity; it also now carries a detection anchor so MTTD is measurable. It pages past the API's 100-results-per-call ceiling, and sorts oldest-first on an incremental fetch so a truncated window drops the alerts the next poll can still see. 1.2.1 — Connection troubleshooting: the URL is normalised to the tenant host (a pasted /public_api/v1 or console path no longer breaks the call), a non-JSON reply reports the status, content type and body instead of a bare JSON parse error, missing key/key ID is caught up front, nonce and timestamp are sent in both auth modes as the reference client does, and test_connection now probes get_incidents. 1.2.0 — Incident write-back (update_incident: status/severity/assignment/resolve comment) and external alert push (insert_parsed_alerts, insert_cef_alerts). 1.1.0 — Full command coverage: added delete/alias/tag endpoints, abort scan, original alerts, script metadata/code/snippet/exec-status, file retrieval (+details), alert exclusions, device-control violations, audits, distribution url/status/create and RBAC (users, roles, groups, risk score, risky users/hosts). 1.0.0 — Initial release: incident ingestion (get_incidents) with OCSF mapper, endpoints, isolate/unisolate, scan, hash blocklist/allowlist, quarantine/restore, run script + results, alerts retrieval, distributions and action status. Standard or Advanced API authentication."
|
||||
category: endpoint
|
||||
|
||||
# Per-instance configuration. The base URL is the tenant API root, e.g.
|
||||
@@ -83,15 +83,19 @@ commands:
|
||||
outputs_schema: { properties: {} }
|
||||
- id: get_alerts
|
||||
name: cortex-xdr-get-alerts
|
||||
description: "Retrieve alerts using a custom filter (get_alerts_by_filter_data). Returns rich alert objects."
|
||||
description: "Fetch Cortex XDR alerts for ingestion (get_alerts_by_filter_data). Returns {reply:{alerts:[...]}}; use reply.alerts as the alert rule results path. Alerts are the detection layer under incidents: ingest them alongside get_incidents when the SOC works detections, not only aggregates."
|
||||
risk: read
|
||||
inputs_schema:
|
||||
properties:
|
||||
severity: { type: string, description: "Comma-separated severities (low, medium, high, critical)" }
|
||||
created_after: { type: string, description: "Lower bound on alert source_insert_ts, epoch ms" }
|
||||
limit: { type: number, description: "Maximum alerts to fetch (default 100)" }
|
||||
severity: { type: string, description: "Comma-separated severities (informational, low, medium, high, critical)" }
|
||||
created_after: { type: string, description: "Lower bound on alert source_insert_ts, ISO8601 or epoch ms (incremental fetch watermark)" }
|
||||
limit: { type: number, description: "Maximum alerts to fetch (default 100). The API serves 100 per call at most; above that the script pages until the limit is reached." }
|
||||
required: []
|
||||
outputs_schema: { properties: {} }
|
||||
ingest:
|
||||
results_path: reply.alerts
|
||||
dedup_key: alert_id
|
||||
incremental_field: created_after
|
||||
- id: insert_parsed_alerts
|
||||
name: cortex-xdr-insert-parsed-alerts
|
||||
description: "Push external alerts (parsed JSON objects) into Cortex XDR for correlation."
|
||||
|
||||
@@ -0,0 +1,76 @@
|
||||
name: "Cortex XDR Alerts → OCSF"
|
||||
description: "Maps a Cortex XDR alert (alerts/get_alerts_by_filter_data/, results_path = reply.alerts) to OCSF Detection Finding fields. Alerts are the per-detection layer under incidents: one incident aggregates many alerts, so a tenant ingesting both feeds will hold the same detection twice, once as an aggregate and once on its own."
|
||||
field_mappings:
|
||||
title: "name"
|
||||
description: "description"
|
||||
# The raw string is enough: Riposte reads critical/high/medium/low/informational
|
||||
# onto its 1-5 scale itself, and the mapping engine cannot compare strings.
|
||||
severity: "severity"
|
||||
# Which sensor fired: "XDR Agent", "PAN NGFW", "XDR Analytics"…
|
||||
source: "source"
|
||||
# results_path = reply.alerts; source_path is JSONata over ONE alert object.
|
||||
# Paths absent from a given alert are skipped at ingestion, so entries for fields
|
||||
# a tenant never emits are safe. Where two entries target the same OCSF field,
|
||||
# the LAST non-empty one wins — that is how the fallbacks below are ordered.
|
||||
ocsf:
|
||||
# ── Finding ───────────────────────────────────────────────────────
|
||||
- { source_path: "alert_id", ocsf_field: "finding_info.uid" }
|
||||
- { source_path: "external_id", ocsf_field: "finding_info.uid_alt" }
|
||||
- { source_path: "name", ocsf_field: "finding_info.title" }
|
||||
- { source_path: "description", ocsf_field: "finding_info.desc" }
|
||||
- { source_path: "detection_timestamp", ocsf_field: "finding_info.created_time" }
|
||||
- { source_path: "local_insert_ts", ocsf_field: "finding_info.modified_time" }
|
||||
- { source_path: "category", ocsf_field: "finding_info.analytic.category" }
|
||||
- { source_path: "name", ocsf_field: "finding_info.analytic.name" }
|
||||
- { source_path: "filter_rule_id", ocsf_field: "finding_info.analytic.uid" }
|
||||
# ── Detection time: the MTTD anchor ───────────────────────────────
|
||||
# `time` is what Riposte measures detection-to-ingestion against. Ingestion
|
||||
# time (source_insert_ts) is the honest fallback when the sensor sent no
|
||||
# detection timestamp of its own — it still beats leaving MTTD empty.
|
||||
- { source_path: "source_insert_ts", ocsf_field: "time" }
|
||||
- { source_path: "detection_timestamp", ocsf_field: "time" }
|
||||
# ── Alert state ───────────────────────────────────────────────────
|
||||
- { source_path: "severity", ocsf_field: "severity" }
|
||||
- { source_path: "alert_type", ocsf_field: "activity_name" }
|
||||
- { source_path: "action_pretty", ocsf_field: "action" }
|
||||
- { source_path: "action", ocsf_field: "action" }
|
||||
- { source_path: "resolution_status", ocsf_field: "status" }
|
||||
- { source_path: "resolution_comment", ocsf_field: "status_detail" }
|
||||
# ── Product identity ──────────────────────────────────────────────
|
||||
- { source_path: "'Cortex XDR'", ocsf_field: "metadata.product.name" }
|
||||
- { source_path: "'Palo Alto Networks'", ocsf_field: "metadata.product.vendor_name" }
|
||||
- { source_path: "source", ocsf_field: "metadata.log_source" }
|
||||
# ── MITRE ATT&CK ──────────────────────────────────────────────────
|
||||
# Both fields arrive as a list on most tenants and as a bare string on some;
|
||||
# [0] reads the first element either way.
|
||||
- { source_path: "mitre_tactic_id_and_name[0]", ocsf_field: "attacks.tactic.name" }
|
||||
- { source_path: "mitre_technique_id_and_name[0]", ocsf_field: "attacks.technique.name" }
|
||||
# ── Affected endpoint ─────────────────────────────────────────────
|
||||
- { source_path: "host_name", ocsf_field: "device.hostname" }
|
||||
- { source_path: "host_ip[0]", ocsf_field: "device.ip" }
|
||||
- { source_path: "mac", ocsf_field: "device.mac" }
|
||||
- { source_path: "endpoint_id", ocsf_field: "device.uid" }
|
||||
- { source_path: "agent_device_domain", ocsf_field: "device.domain" }
|
||||
- { source_path: "agent_os_type", ocsf_field: "device.os.type" }
|
||||
- { source_path: "agent_os_sub_type", ocsf_field: "device.os.name" }
|
||||
# Mirrored onto src_endpoint so routers and pre-processing rules written for
|
||||
# the incident feed (which maps hosts there) match alerts unchanged.
|
||||
- { source_path: "host_name", ocsf_field: "src_endpoint.hostname" }
|
||||
- { source_path: "host_ip[0]", ocsf_field: "src_endpoint.ip" }
|
||||
- { source_path: "user_name", ocsf_field: "user.name" }
|
||||
# ── First triggering event (events[] carries the per-event detail) ─
|
||||
- { source_path: "events[0].actor_process_image_name", ocsf_field: "process.name" }
|
||||
- { source_path: "events[0].actor_process_command_line", ocsf_field: "process.cmd_line" }
|
||||
- { source_path: "events[0].actor_process_image_path", ocsf_field: "process.path" }
|
||||
- { source_path: "events[0].actor_process_os_pid", ocsf_field: "process.pid" }
|
||||
- { source_path: "events[0].actor_process_image_sha256", ocsf_field: "process.file.hashes.sha256" }
|
||||
- { source_path: "events[0].causality_actor_process_image_name", ocsf_field: "process.parent_process.name" }
|
||||
- { source_path: "events[0].causality_actor_process_command_line", ocsf_field: "process.parent_process.cmd_line" }
|
||||
- { source_path: "events[0].action_file_path", ocsf_field: "file.path" }
|
||||
- { source_path: "events[0].action_file_sha256", ocsf_field: "file.hashes.sha256" }
|
||||
- { source_path: "events[0].action_file_md5", ocsf_field: "file.hashes.md5" }
|
||||
- { source_path: "events[0].action_remote_ip", ocsf_field: "dst_endpoint.ip" }
|
||||
- { source_path: "events[0].action_remote_port", ocsf_field: "dst_endpoint.port" }
|
||||
- { source_path: "events[0].action_external_hostname", ocsf_field: "dst_endpoint.hostname" }
|
||||
- { source_path: "events[0].action_local_ip", ocsf_field: "src_endpoint.ip" }
|
||||
- { source_path: "events[0].action_local_port", ocsf_field: "src_endpoint.port" }
|
||||
@@ -2,7 +2,10 @@ name: "Cortex XDR Incidents → OCSF"
|
||||
description: "Maps a Cortex XDR incident (incidents/get_incidents/, results_path = reply.incidents) to OCSF finding fields. Incidents are aggregates; use get_incident_extra_data for per-alert detail."
|
||||
field_mappings:
|
||||
title: "incident_name"
|
||||
severity: "severity = 'critical' ? 5 : (severity = 'high' ? 4 : (severity = 'medium' ? 3 : 2))"
|
||||
# The raw string, not a ternary: the mapping engine compares numbers only, so
|
||||
# every string test read as false and every incident landed on the rule's
|
||||
# default severity. Riposte reads critical/high/medium/low onto 1-5 itself.
|
||||
severity: "severity"
|
||||
description: "description"
|
||||
# results_path = reply.incidents; source_path is JSONata over ONE incident object.
|
||||
# Paths absent from a given incident are skipped at ingestion, so extra entries are safe.
|
||||
@@ -15,6 +18,9 @@ ocsf:
|
||||
- { source_path: "modification_time", ocsf_field: "finding_info.modified_time" }
|
||||
- { source_path: "xdr_url", ocsf_field: "finding_info.src_url" }
|
||||
- { source_path: "status", ocsf_field: "status" }
|
||||
# `time` is the MTTD anchor — when XDR opened the incident, as opposed to when
|
||||
# Riposte ingested it. Without it the detection delay column stays empty.
|
||||
- { source_path: "creation_time", ocsf_field: "time" }
|
||||
- { source_path: "alert_count", ocsf_field: "count" }
|
||||
# ── MITRE ATT&CK (first aggregated tactic/technique) ──────────────
|
||||
- { source_path: "mitre_tactics_ids_and_names[0]", ocsf_field: "attacks.tactic.name" }
|
||||
|
||||
@@ -76,19 +76,52 @@ def to_ms(v):
|
||||
return None
|
||||
|
||||
|
||||
# The API serves at most 100 alerts per call. Ingestion is what makes that a
|
||||
# real ceiling: an alert feed easily carries more than a hundred detections
|
||||
# between two polls, and anything a single page leaves behind is never fetched
|
||||
# again — the next run's watermark has already moved past it.
|
||||
PAGE = 100
|
||||
|
||||
|
||||
def main():
|
||||
inputs = json.loads(os.environ.get("INTEGRATION_INPUTS", "{}"))
|
||||
limit = int(inputs.get("limit") or 100)
|
||||
limit = max(1, int(inputs.get("limit") or 100))
|
||||
filters = []
|
||||
if inputs.get("severity"):
|
||||
filters.append({"field": "severity", "operator": "in", "value": csv(inputs["severity"])})
|
||||
created_ms = to_ms(inputs.get("created_after"))
|
||||
if created_ms is not None:
|
||||
filters.append({"field": "source_insert_ts", "operator": "gte", "value": created_ms})
|
||||
rd = {"search_from": 0, "search_to": limit, "sort": {"field": "source_insert_ts", "keyword": "desc"}}
|
||||
if filters:
|
||||
rd["filters"] = filters
|
||||
print(json.dumps(post("/alerts/get_alerts_by_filter_data/", rd)))
|
||||
# Oldest first on an incremental fetch, so that a window holding more alerts
|
||||
# than `limit` drops its most RECENT ones — the only ones the next poll can
|
||||
# still see. Newest first otherwise, which is what an operator running the
|
||||
# command by hand is asking for.
|
||||
keyword = "asc" if created_ms is not None else "desc"
|
||||
|
||||
alerts, total = [], None
|
||||
while len(alerts) < limit:
|
||||
rd = {
|
||||
"search_from": len(alerts),
|
||||
"search_to": min(len(alerts) + PAGE, limit),
|
||||
"sort": {"field": "source_insert_ts", "keyword": keyword},
|
||||
}
|
||||
if filters:
|
||||
rd["filters"] = filters
|
||||
reply = (post("/alerts/get_alerts_by_filter_data/", rd) or {}).get("reply") or {}
|
||||
page = reply.get("alerts") or []
|
||||
if total is None:
|
||||
total = reply.get("total_count")
|
||||
alerts.extend(page)
|
||||
if len(page) < PAGE:
|
||||
break
|
||||
|
||||
out = {"result_count": len(alerts), "alerts": alerts}
|
||||
if total is not None:
|
||||
out["total_count"] = total
|
||||
# Say it when the window was larger than the limit: those alerts are not
|
||||
# coming back on the next poll, and a silent cap reads like a quiet feed.
|
||||
out["truncated"] = total > len(alerts)
|
||||
print(json.dumps({"reply": out}))
|
||||
|
||||
|
||||
try:
|
||||
|
||||
Reference in New Issue
Block a user