diff --git a/integrations/cortex-xdr/manifest.yaml b/integrations/cortex-xdr/manifest.yaml index 0084e09..d0f2433 100644 --- a/integrations/cortex-xdr/manifest.yaml +++ b/integrations/cortex-xdr/manifest.yaml @@ -2,7 +2,7 @@ id: cortex_xdr name: Cortex XDR version: 1.3.1 description: "Palo Alto Cortex XDR (public API v1) — incident and alert ingestion + write-back, endpoint isolation/scan/delete/tagging, RTR scripts, hash block/allow lists, file quarantine/restore/retrieval, alert exclusions, external alert push (parsed/CEF), device-control violations, audits, distributions and RBAC/risk." -changelog: "1.3.1 — The full-incident fetch command is named cortex-xdr-get-incidents-full (id get_incidents_full), not cortex-xdr-fetch-incidents: sitting next to cortex-xdr-get-incidents in the command list, it now reads as what it is — the same call, everything included. A rule created against the old id must be pointed at the new one. 1.3.0 — Richer incident ingestion (get_incidents_full, on get_multiple_incidents_extra_data): incidents now arrive with their alerts in full and their file/network artifacts, where get_incidents only ever answered a 21-field summary carrying neither hosts, users, MITRE nor a single alert. Alert ingestion, and the alerts endpoint answers at last: get_alerts was sending the incidents dialect ({filters, search_from, search_to, sort}) to a grid endpoint that speaks request_data.filter_data (SEARCH_FIELD/SEARCH_TYPE/SEARCH_VALUE blocks, paging.from/to, sort as a list), and every call came back HTTP 500. Body rebuilt from the reference client, rows unwrapped out of their alert_fields envelope, severity code and the dotted status.progress key derived into readable fields. Alert ingestion: get_alerts is now a fetch command (results path reply.alerts, dedup on alert_id, incremental on source_insert_ts) with a bundled OCSF mapper and a Cortex XDR Alert incident type, so detections can be ingested alongside — or instead of — incidents. The incident mapper is fixed on the way past: its severity expression compared strings, which the mapping engine cannot do, so every ingested incident silently took the rule's default severity; it also now carries a detection anchor so MTTD is measurable. It pages past the API's 100-results-per-call ceiling, and sorts oldest-first on an incremental fetch so a truncated window drops the alerts the next poll can still see. 1.2.1 — Connection troubleshooting: the URL is normalised to the tenant host (a pasted /public_api/v1 or console path no longer breaks the call), a non-JSON reply reports the status, content type and body instead of a bare JSON parse error, missing key/key ID is caught up front, nonce and timestamp are sent in both auth modes as the reference client does, and test_connection now probes get_incidents. 1.2.0 — Incident write-back (update_incident: status/severity/assignment/resolve comment) and external alert push (insert_parsed_alerts, insert_cef_alerts). 1.1.0 — Full command coverage: added delete/alias/tag endpoints, abort scan, original alerts, script metadata/code/snippet/exec-status, file retrieval (+details), alert exclusions, device-control violations, audits, distribution url/status/create and RBAC (users, roles, groups, risk score, risky users/hosts). 1.0.0 — Initial release: incident ingestion (get_incidents) with OCSF mapper, endpoints, isolate/unisolate, scan, hash blocklist/allowlist, quarantine/restore, run script + results, alerts retrieval, distributions and action status. Standard or Advanced API authentication." +changelog: "1.3.1 — get_incidents_full can watermark on modification_time (modified_after), which is what ingestion wants: an XDR incident keeps growing after creation, and a creation_time watermark fetches it once and never looks again, so every alert that joins it afterwards is lost. The full-incident fetch command is named cortex-xdr-get-incidents-full (id get_incidents_full), not cortex-xdr-fetch-incidents: sitting next to cortex-xdr-get-incidents in the command list, it now reads as what it is — the same call, everything included. A rule created against the old id must be pointed at the new one. 1.3.0 — Richer incident ingestion (get_incidents_full, on get_multiple_incidents_extra_data): incidents now arrive with their alerts in full and their file/network artifacts, where get_incidents only ever answered a 21-field summary carrying neither hosts, users, MITRE nor a single alert. Alert ingestion, and the alerts endpoint answers at last: get_alerts was sending the incidents dialect ({filters, search_from, search_to, sort}) to a grid endpoint that speaks request_data.filter_data (SEARCH_FIELD/SEARCH_TYPE/SEARCH_VALUE blocks, paging.from/to, sort as a list), and every call came back HTTP 500. Body rebuilt from the reference client, rows unwrapped out of their alert_fields envelope, severity code and the dotted status.progress key derived into readable fields. Alert ingestion: get_alerts is now a fetch command (results path reply.alerts, dedup on alert_id, incremental on source_insert_ts) with a bundled OCSF mapper and a Cortex XDR Alert incident type, so detections can be ingested alongside — or instead of — incidents. The incident mapper is fixed on the way past: its severity expression compared strings, which the mapping engine cannot do, so every ingested incident silently took the rule's default severity; it also now carries a detection anchor so MTTD is measurable. It pages past the API's 100-results-per-call ceiling, and sorts oldest-first on an incremental fetch so a truncated window drops the alerts the next poll can still see. 1.2.1 — Connection troubleshooting: the URL is normalised to the tenant host (a pasted /public_api/v1 or console path no longer breaks the call), a non-JSON reply reports the status, content type and body instead of a bare JSON parse error, missing key/key ID is caught up front, nonce and timestamp are sent in both auth modes as the reference client does, and test_connection now probes get_incidents. 1.2.0 — Incident write-back (update_incident: status/severity/assignment/resolve comment) and external alert push (insert_parsed_alerts, insert_cef_alerts). 1.1.0 — Full command coverage: added delete/alias/tag endpoints, abort scan, original alerts, script metadata/code/snippet/exec-status, file retrieval (+details), alert exclusions, device-control violations, audits, distribution url/status/create and RBAC (users, roles, groups, risk score, risky users/hosts). 1.0.0 — Initial release: incident ingestion (get_incidents) with OCSF mapper, endpoints, isolate/unisolate, scan, hash blocklist/allowlist, quarantine/restore, run script + results, alerts retrieval, distributions and action status. Standard or Advanced API authentication." category: endpoint # Per-instance configuration. The base URL is the tenant API root, e.g. @@ -64,7 +64,8 @@ commands: inputs_schema: properties: status: { type: string, description: "Comma-separated statuses to keep (new, under_investigation, resolved_threat_handled…)" } - created_after: { type: string, description: "Lower bound on creation_time, ISO8601 or epoch ms (incremental fetch watermark)" } + created_after: { type: string, description: "Lower bound on creation_time, ISO8601 or epoch ms. Watermarking on this fetches each incident once and never revisits it — alerts joining it later never arrive." } + modified_after: { type: string, description: "Lower bound on modification_time, ISO8601 or epoch ms. The watermark to prefer for ingestion: an incident comes back whenever it changes, and dedup on incident_id turns the second visit into an enrichment." } limit: { type: number, description: "Maximum incidents to fetch (default 50, paged 50 at a time). A full incident weighs a few KB and up to a few hundred with its alerts, so raise this knowingly." } exclude_artifacts: { type: boolean, description: "Drop the file and network artifact blocks, keeping the alerts (lighter payload)" } required: [] @@ -72,7 +73,7 @@ commands: ingest: results_path: reply.incidents dedup_key: incident_id - incremental_field: created_after + incremental_field: modified_after - id: get_incident_extra_data name: cortex-xdr-get-incident-extra-data description: "Get full incident data including its alerts and network artifacts by incident ID." diff --git a/integrations/cortex-xdr/scripts/get_incidents_full.py b/integrations/cortex-xdr/scripts/get_incidents_full.py index ec520e7..b013c8c 100644 --- a/integrations/cortex-xdr/scripts/get_incidents_full.py +++ b/integrations/cortex-xdr/scripts/get_incidents_full.py @@ -127,10 +127,20 @@ def main(): created_ms = to_ms(inputs.get("created_after")) if created_ms is not None: filters.append({"field": "creation_time", "operator": "gte", "value": created_ms}) + # An XDR incident keeps growing after it is created: alerts join it, an + # analyst changes its status. Watermarking on creation_time fetches it once + # and never looks again, so everything that happened afterwards is lost. + # Watermarking on modification_time brings it back on every change, where + # dedup on incident_id turns the second visit into an enrichment. + modified_ms = to_ms(inputs.get("modified_after")) + if modified_ms is not None: + filters.append({"field": "modification_time", "operator": "gte", "value": modified_ms}) + incremental = created_ms is not None or modified_ms is not None # Oldest first on an incremental fetch, so that a window holding more # incidents than `limit` drops its most RECENT ones — the only ones the next # poll can still see. Newest first otherwise, for a hand-run command. - keyword = "asc" if created_ms is not None else "desc" + sort_field = "modification_time" if modified_ms is not None else "creation_time" + keyword = "asc" if incremental else "desc" exclude = str(inputs.get("exclude_artifacts") or "").lower() in ("1", "true", "yes") incidents, total = [], None @@ -138,7 +148,7 @@ def main(): rd = { "search_from": len(incidents), "search_to": min(len(incidents) + PAGE, limit), - "sort": {"field": "creation_time", "keyword": keyword}, + "sort": {"field": sort_field, "keyword": keyword}, # Without this the nested alerts come back trimmed to a handful of # fields — the very thing this command exists to avoid. "full_alert_fields": True,