feat(cortex-xdr): fetch incidents with their alerts, not a 21-field summary

incidents/get_incidents/ answers with a summary and nothing else: 21 fields,
no hosts, no users, no MITRE, no tags, and not one of the alerts the incident
aggregates. Ingesting through it leaves an incident whose raw payload says
almost nothing about what happened — and the shipped mapper had been written
for a richer shape than the endpoint ever returns, mapping hosts[0], users[0]
and mitre_* that simply are not in that response.

incidents/get_multiple_incidents_extra_data/ returns the same incidents with
39 fields, every alert in full — 156 fields each — and the file and network
artifacts. It is what the reference client fetches through (demisto/content,
CortexXDRIR.get_multiple_incidents_extra_data), and full_alert_fields must be
set or the nested alerts come back trimmed to a handful of fields.

Records arrive as {incident, alerts, network_artifacts, file_artifacts} with
each nested block wrapped as {total_count, data}. The script flattens them, so
every expression written against get_incidents keeps working — the summary's
21 fields are a subset of these 39 — while the alerts and artifacts land beside
them as plain lists, and their total_count says when a list is a sample rather
than the whole set. incident_sources is lifted into a scalar for the same
reason severity was on the alerts side: the incident-field mapper reads dotted
paths and cannot index a list.

get_incidents stays, for cheap polling, and now says in its description what it
does and does not carry.

The mapper maps the aggregate first and the first alert last, so the alert
fills in whatever the aggregate leaves silent — including the detection anchor,
since an XDR incident's detection_time is usually null while its alerts carry
theirs. Verified against the vendor's recorded response
(test_data/get_multiple_incidents_extra_data.json): 33 of 52 entries resolve,
severity critical lands on 5, source reads "XDR Agent", and the anchor falls
through to the alert's detection timestamp.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-08-18 23:41:27 +02:00
parent d06f8ea413
commit a6e1245c79
3 changed files with 275 additions and 2 deletions
+18 -2
View File
@@ -2,7 +2,7 @@ id: cortex_xdr
name: Cortex XDR
version: 1.3.0
description: "Palo Alto Cortex XDR (public API v1) — incident and alert ingestion + write-back, endpoint isolation/scan/delete/tagging, RTR scripts, hash block/allow lists, file quarantine/restore/retrieval, alert exclusions, external alert push (parsed/CEF), device-control violations, audits, distributions and RBAC/risk."
changelog: "1.3.0 — Alert ingestion, and the alerts endpoint answers at last: get_alerts was sending the incidents dialect ({filters, search_from, search_to, sort}) to a grid endpoint that speaks request_data.filter_data (SEARCH_FIELD/SEARCH_TYPE/SEARCH_VALUE blocks, paging.from/to, sort as a list), and every call came back HTTP 500. Body rebuilt from the reference client, rows unwrapped out of their alert_fields envelope, severity code and the dotted status.progress key derived into readable fields. Alert ingestion: get_alerts is now a fetch command (results path reply.alerts, dedup on alert_id, incremental on source_insert_ts) with a bundled OCSF mapper and a Cortex XDR Alert incident type, so detections can be ingested alongside — or instead of — incidents. The incident mapper is fixed on the way past: its severity expression compared strings, which the mapping engine cannot do, so every ingested incident silently took the rule's default severity; it also now carries a detection anchor so MTTD is measurable. It pages past the API's 100-results-per-call ceiling, and sorts oldest-first on an incremental fetch so a truncated window drops the alerts the next poll can still see. 1.2.1 — Connection troubleshooting: the URL is normalised to the tenant host (a pasted /public_api/v1 or console path no longer breaks the call), a non-JSON reply reports the status, content type and body instead of a bare JSON parse error, missing key/key ID is caught up front, nonce and timestamp are sent in both auth modes as the reference client does, and test_connection now probes get_incidents. 1.2.0 — Incident write-back (update_incident: status/severity/assignment/resolve comment) and external alert push (insert_parsed_alerts, insert_cef_alerts). 1.1.0 — Full command coverage: added delete/alias/tag endpoints, abort scan, original alerts, script metadata/code/snippet/exec-status, file retrieval (+details), alert exclusions, device-control violations, audits, distribution url/status/create and RBAC (users, roles, groups, risk score, risky users/hosts). 1.0.0 — Initial release: incident ingestion (get_incidents) with OCSF mapper, endpoints, isolate/unisolate, scan, hash blocklist/allowlist, quarantine/restore, run script + results, alerts retrieval, distributions and action status. Standard or Advanced API authentication."
changelog: "1.3.0 — Richer incident ingestion (fetch_incidents, on get_multiple_incidents_extra_data): incidents now arrive with their alerts in full and their file/network artifacts, where get_incidents only ever answered a 21-field summary carrying neither hosts, users, MITRE nor a single alert. Alert ingestion, and the alerts endpoint answers at last: get_alerts was sending the incidents dialect ({filters, search_from, search_to, sort}) to a grid endpoint that speaks request_data.filter_data (SEARCH_FIELD/SEARCH_TYPE/SEARCH_VALUE blocks, paging.from/to, sort as a list), and every call came back HTTP 500. Body rebuilt from the reference client, rows unwrapped out of their alert_fields envelope, severity code and the dotted status.progress key derived into readable fields. Alert ingestion: get_alerts is now a fetch command (results path reply.alerts, dedup on alert_id, incremental on source_insert_ts) with a bundled OCSF mapper and a Cortex XDR Alert incident type, so detections can be ingested alongside — or instead of — incidents. The incident mapper is fixed on the way past: its severity expression compared strings, which the mapping engine cannot do, so every ingested incident silently took the rule's default severity; it also now carries a detection anchor so MTTD is measurable. It pages past the API's 100-results-per-call ceiling, and sorts oldest-first on an incremental fetch so a truncated window drops the alerts the next poll can still see. 1.2.1 — Connection troubleshooting: the URL is normalised to the tenant host (a pasted /public_api/v1 or console path no longer breaks the call), a non-JSON reply reports the status, content type and body instead of a bare JSON parse error, missing key/key ID is caught up front, nonce and timestamp are sent in both auth modes as the reference client does, and test_connection now probes get_incidents. 1.2.0 — Incident write-back (update_incident: status/severity/assignment/resolve comment) and external alert push (insert_parsed_alerts, insert_cef_alerts). 1.1.0 — Full command coverage: added delete/alias/tag endpoints, abort scan, original alerts, script metadata/code/snippet/exec-status, file retrieval (+details), alert exclusions, device-control violations, audits, distribution url/status/create and RBAC (users, roles, groups, risk score, risky users/hosts). 1.0.0 — Initial release: incident ingestion (get_incidents) with OCSF mapper, endpoints, isolate/unisolate, scan, hash blocklist/allowlist, quarantine/restore, run script + results, alerts retrieval, distributions and action status. Standard or Advanced API authentication."
category: endpoint
# Per-instance configuration. The base URL is the tenant API root, e.g.
@@ -44,7 +44,7 @@ commands:
# ── Ingestion ─────────────────────────────────────────────────────────────
- id: get_incidents
name: cortex-xdr-get-incidents
description: "Fetch Cortex XDR incidents for ingestion. Returns {reply:{incidents:[...]}}; use reply.incidents as the alert rule results path."
description: "List Cortex XDR incidents as a 21-field summary (no hosts, no users, no MITRE, no alerts). Cheap to poll, but for ingestion prefer cortex-xdr-fetch-incidents, which returns the same incidents with their alerts and artifacts. Returns {reply:{incidents:[...]}}."
risk: read
inputs_schema:
properties:
@@ -57,6 +57,22 @@ commands:
results_path: reply.incidents
dedup_key: incident_id
incremental_field: created_after
- id: fetch_incidents
name: cortex-xdr-fetch-incidents
description: "Fetch incidents WITH their alerts and artifacts (get_multiple_incidents_extra_data) — the ingestion command to prefer. get_incidents answers with a 21-field summary carrying no hosts, no users, no MITRE and none of the alerts; this one returns 39 incident fields, every alert in full (156 fields each) and the file/network artifacts. Records are flattened, so mapping expressions written against get_incidents keep working and alerts[], file_artifacts[], network_artifacts[] sit beside them. Returns {reply:{incidents:[...]}}."
risk: read
inputs_schema:
properties:
status: { type: string, description: "Comma-separated statuses to keep (new, under_investigation, resolved_threat_handled…)" }
created_after: { type: string, description: "Lower bound on creation_time, ISO8601 or epoch ms (incremental fetch watermark)" }
limit: { type: number, description: "Maximum incidents to fetch (default 50, paged 50 at a time). A full incident weighs a few KB and up to a few hundred with its alerts, so raise this knowingly." }
exclude_artifacts: { type: boolean, description: "Drop the file and network artifact blocks, keeping the alerts (lighter payload)" }
required: []
outputs_schema: { properties: {} }
ingest:
results_path: reply.incidents
dedup_key: incident_id
incremental_field: created_after
- id: get_incident_extra_data
name: cortex-xdr-get-incident-extra-data
description: "Get full incident data including its alerts and network artifacts by incident ID."
@@ -0,0 +1,83 @@
name: "Cortex XDR Incidents (full) → OCSF"
description: "Maps one Cortex XDR incident fetched with its alerts and artifacts (incidents/get_multiple_incidents_extra_data/, results_path = reply.incidents) to OCSF finding fields. The script flattens the record, so incident fields sit at the top level — every expression written against get_incidents keeps working — while alerts[], file_artifacts[] and network_artifacts[] are plain lists beside them. Incident-level values are mapped first and the first alert's equivalents last, so the alert wins wherever the aggregate says nothing."
field_mappings:
# incident_name is null on most tenants (it is only set when someone renames
# the incident), and a mapping that resolves to nothing leaves the title to
# the incident type's fallback. description is the sentence XDR itself shows.
title: "description"
description: "description"
severity: "severity"
# incident_source, not incident_sources[0]: this mapper reads dotted paths and
# cannot index a list, so the script lifts the first sensor out for it.
source: "incident_source"
# results_path = reply.incidents; source_path is JSONata over ONE flattened
# incident. Paths absent from a given incident are skipped at ingestion, so
# entries for fields a tenant never emits are safe. Where two entries target the
# same OCSF field, the LAST non-empty one wins.
ocsf:
# ── Finding ───────────────────────────────────────────────────────
- { source_path: "incident_id", ocsf_field: "finding_info.uid" }
- { source_path: "incident_name ? incident_name : description", ocsf_field: "finding_info.title" }
- { source_path: "description", ocsf_field: "finding_info.desc" }
- { source_path: "creation_time", ocsf_field: "finding_info.created_time" }
- { source_path: "modification_time", ocsf_field: "finding_info.modified_time" }
- { source_path: "xdr_url", ocsf_field: "finding_info.src_url" }
- { source_path: "alert_categories[0]", ocsf_field: "finding_info.analytic.category" }
- { source_path: "alerts[0].name", ocsf_field: "finding_info.analytic.name" }
# ── Detection time: the MTTD anchor ───────────────────────────────
# Weakest first, strongest last. detection_time is often null on an XDR
# incident, and then the first alert's own detection timestamp is the honest
# anchor; incident creation is the last resort.
- { source_path: "creation_time", ocsf_field: "time" }
- { source_path: "alerts[0].detection_timestamp", ocsf_field: "time" }
- { source_path: "detection_time", ocsf_field: "time" }
# ── Incident state ────────────────────────────────────────────────
- { source_path: "severity", ocsf_field: "severity" }
- { source_path: "status", ocsf_field: "status" }
- { source_path: "resolve_comment", ocsf_field: "status_detail" }
- { source_path: "alert_count", ocsf_field: "count" }
- { source_path: "aggregated_score", ocsf_field: "risk_score" }
- { source_path: "tags", ocsf_field: "metadata.labels" }
- { source_path: "alerts[0].action_pretty", ocsf_field: "action" }
# ── Product identity ──────────────────────────────────────────────
- { source_path: "'Cortex XDR'", ocsf_field: "metadata.product.name" }
- { source_path: "'Palo Alto Networks'", ocsf_field: "metadata.product.vendor_name" }
- { source_path: "incident_sources[0]", ocsf_field: "metadata.log_source" }
# ── MITRE ATT&CK: the aggregate, else the first alert ─────────────
- { source_path: "mitre_tactics_ids_and_names[0]", ocsf_field: "attacks.tactic.name" }
- { source_path: "mitre_techniques_ids_and_names[0]", ocsf_field: "attacks.technique.name" }
- { source_path: "alerts[0].mitre_tactic_id_and_name[0]", ocsf_field: "attacks.tactic.name" }
- { source_path: "alerts[0].mitre_technique_id_and_name[0]", ocsf_field: "attacks.technique.name" }
# ── Affected host / user ──────────────────────────────────────────
# An incident's hosts are 'hostname:agent_id' strings; an alert names them plainly.
- { source_path: "$split(hosts[0], ':')[0]", ocsf_field: "src_endpoint.hostname" }
- { source_path: "$split(hosts[0], ':')[0]", ocsf_field: "device.hostname" }
- { source_path: "alerts[0].host_name", ocsf_field: "src_endpoint.hostname" }
- { source_path: "alerts[0].host_name", ocsf_field: "device.hostname" }
- { source_path: "alerts[0].host_ip[0]", ocsf_field: "device.ip" }
- { source_path: "alerts[0].host_ip[0]", ocsf_field: "src_endpoint.ip" }
- { source_path: "alerts[0].endpoint_id", ocsf_field: "device.uid" }
- { source_path: "alerts[0].agent_os_type", ocsf_field: "device.os.type" }
- { source_path: "users[0]", ocsf_field: "user.name" }
- { source_path: "alerts[0].user_name", ocsf_field: "user.name" }
# ── What the first alert actually saw ─────────────────────────────
- { source_path: "alerts[0].actor_process_image_name", ocsf_field: "process.name" }
- { source_path: "alerts[0].actor_process_command_line", ocsf_field: "process.cmd_line" }
- { source_path: "alerts[0].actor_process_image_path", ocsf_field: "process.path" }
- { source_path: "alerts[0].actor_process_image_sha256", ocsf_field: "process.file.hashes.sha256" }
- { source_path: "alerts[0].causality_actor_process_image_name", ocsf_field: "process.parent_process.name" }
- { source_path: "alerts[0].action_file_path", ocsf_field: "file.path" }
- { source_path: "alerts[0].action_file_name", ocsf_field: "file.name" }
- { source_path: "alerts[0].action_file_sha256", ocsf_field: "file.hashes.sha256" }
- { source_path: "alerts[0].action_file_md5", ocsf_field: "file.hashes.md5" }
- { source_path: "alerts[0].action_remote_ip", ocsf_field: "dst_endpoint.ip" }
- { source_path: "alerts[0].action_remote_port", ocsf_field: "dst_endpoint.port" }
- { source_path: "alerts[0].action_external_hostname", ocsf_field: "dst_endpoint.hostname" }
# ── The artifact the incident is really about ─────────────────────
# Last, because a file artifact is the incident's verdict on the file, where
# the alert only reports what one detection touched.
- { source_path: "file_artifacts[0].file_name", ocsf_field: "file.name" }
- { source_path: "file_artifacts[0].file_sha256", ocsf_field: "file.hashes.sha256" }
- { source_path: "file_artifacts[0].file_wildfire_verdict", ocsf_field: "malware.classifications" }
- { source_path: "network_artifacts[0].network_remote_ip", ocsf_field: "dst_endpoint.ip" }
- { source_path: "network_artifacts[0].network_domain", ocsf_field: "dst_endpoint.hostname" }
@@ -0,0 +1,174 @@
import json, os, sys, time, hashlib, secrets, string, urllib.request, urllib.error
from datetime import datetime
def _client():
s = json.loads(os.environ.get("INTEGRATION_SECRETS") or "{}")
raw = str(s.get("url") or "").strip().rstrip("/")
if not raw:
raise ValueError("no url configured — paste the tenant API URL (Cortex XDR > Settings > Configurations > API Keys > Copy URL)")
if "://" not in raw:
raw = "https://" + raw
# The tenant URL is a bare host. Drop whatever was pasted after it (a stray
# /public_api/v1, a console path) so the API root is built exactly once.
scheme, _, rest = raw.partition("://")
base = scheme + "://" + rest.split("/")[0] + "/public_api/v1"
key = s.get("api_key", "")
kid = str(s.get("api_key_id", ""))
if not key or not kid:
raise ValueError("api_key and api_key_id are both required")
# Nonce and timestamp ride along in both modes, as the reference client does.
# A standard key travels as-is; an advanced one as sha256(key + nonce + ts).
nonce = "".join(secrets.choice(string.ascii_letters + string.digits) for _ in range(64))
ts = str(int(time.time()) * 1000)
headers = {
"x-xdr-auth-id": kid,
"x-xdr-nonce": nonce,
"x-xdr-timestamp": ts,
"Content-Type": "application/json",
"Accept": "application/json",
}
if str(s.get("auth_type") or "standard").lower() == "advanced":
headers["Authorization"] = hashlib.sha256((key + nonce + ts).encode("utf-8")).hexdigest()
else:
headers["Authorization"] = key
return base, headers
def _not_json(r, raw):
"""A 2xx that is not JSON means we are not talking to the XDR API at all."""
ctype = (r.headers.get("Content-Type") or "unknown").split(";")[0].strip()
head = raw[:160].decode("utf-8", "replace").replace("\n", " ").strip()
return (
"expected JSON from " + r.geturl() + ", got " + ctype + " (HTTP " + str(r.status) + "): " + head
+ " — check the configured url is the tenant API host"
+ " (https://api-<tenant>.xdr.<region>.paloaltonetworks.com), not the console URL"
)
def post(path, request_data):
base, headers = _client()
data = json.dumps({"request_data": request_data}).encode("utf-8")
req = urllib.request.Request(base + path, data=data, headers=headers, method="POST")
with urllib.request.urlopen(req, timeout=90) as r:
raw = r.read()
if not raw:
return {}
try:
return json.loads(raw)
except ValueError:
raise ValueError(_not_json(r, raw))
def to_ms(v):
if v in (None, ""):
return None
s = str(v)
if s.isdigit():
return int(s)
try:
return int(datetime.fromisoformat(s.replace("Z", "+00:00")).timestamp() * 1000)
except Exception:
return None
def csv(v):
return [x.strip() for x in str(v or "").split(",") if x.strip()]
# incidents/get_incidents/ answers with a 21-field summary: no hosts, no users,
# no MITRE, and above all not one of the alerts the incident aggregates. This
# endpoint returns the same incident with 39 fields, its alerts in full (156
# fields each) and its file/network artifacts — which is why the reference
# client fetches through it and not through get_incidents (demisto/content,
# CortexXDRIR.get_multiple_incidents_extra_data).
PAGE = 50
# Artifacts are dropped by name, not by omission — the API only understands
# being told which blocks to leave out.
ARTIFACT_BLOCKS = ["network_artifacts", "file_artifacts"]
def flatten(item):
"""One record -> one incident.
Records arrive as {incident, alerts, network_artifacts, file_artifacts},
each nested block wrapped as {total_count, data}. Flattening the incident to
the top level keeps every expression written against get_incidents working
unchanged — the summary's 21 fields are a subset of these 39 — while the
alerts and artifacts land beside them as plain lists.
"""
incident = dict(item.get("incident") or {})
for key in ("alerts", "network_artifacts", "file_artifacts"):
block = item.get(key)
if not isinstance(block, dict):
continue
incident[key] = block.get("data") or []
if block.get("total_count") is not None:
# The tenant caps alerts per incident (50 by default), so the count
# says when the list is a sample rather than the whole set.
incident[key + "_total_count"] = block["total_count"]
# The producing sensor is a list here, and the incident-field mapper reads
# dotted paths only — no array indexing — so the first source is lifted out
# for it. The list itself stays, for expressions that can index.
sources = incident.get("incident_sources")
if isinstance(sources, list) and sources:
incident["incident_source"] = sources[0]
return incident
def main():
inputs = json.loads(os.environ.get("INTEGRATION_INPUTS", "{}"))
limit = max(1, int(inputs.get("limit") or 50))
filters = []
if inputs.get("status"):
statuses = csv(inputs["status"])
filters.append({"field": "status", "operator": "in", "value": statuses})
created_ms = to_ms(inputs.get("created_after"))
if created_ms is not None:
filters.append({"field": "creation_time", "operator": "gte", "value": created_ms})
# Oldest first on an incremental fetch, so that a window holding more
# incidents than `limit` drops its most RECENT ones — the only ones the next
# poll can still see. Newest first otherwise, for a hand-run command.
keyword = "asc" if created_ms is not None else "desc"
exclude = str(inputs.get("exclude_artifacts") or "").lower() in ("1", "true", "yes")
incidents, total = [], None
while len(incidents) < limit:
rd = {
"search_from": len(incidents),
"search_to": min(len(incidents) + PAGE, limit),
"sort": {"field": "creation_time", "keyword": keyword},
# Without this the nested alerts come back trimmed to a handful of
# fields — the very thing this command exists to avoid.
"full_alert_fields": True,
}
if filters:
rd["filters"] = filters
if exclude:
rd["fields_to_exclude"] = ARTIFACT_BLOCKS
reply = (post("/incidents/get_multiple_incidents_extra_data/", rd) or {}).get("reply") or {}
page = reply.get("incidents") or []
if total is None:
total = reply.get("total_count")
incidents.extend(flatten(i) for i in page)
if len(page) < PAGE:
break
out = {"result_count": len(incidents), "incidents": incidents}
if total is not None:
out["total_count"] = total
# Say it when the window held more than the limit: those incidents are
# not coming back on the next poll.
out["truncated"] = total > len(incidents)
print(json.dumps({"reply": out}))
try:
main()
except urllib.error.HTTPError as e:
print(json.dumps({"error": "HTTP " + str(e.code), "detail": e.read().decode("utf-8", "replace")}))
sys.exit(1)
except Exception as e:
print(json.dumps({"error": str(e)}))
sys.exit(1)