feat(cortex-xdr): watermark incidents on modification_time, not creation

An XDR incident is not finished when it is created. Alerts keep joining it, an
analyst changes its status, its severity is raised. A creation_time watermark
fetches it once, the watermark moves past it, and nothing that happens
afterwards ever reaches Riposte — which is precisely the content the full fetch
exists to bring in.

modified_after filters and sorts on modification_time instead, so an incident
comes back on every change and dedup on incident_id turns the second visit into
an enrichment of the incident already there. It is now what the ingest hint
prefills; created_after stays for a one-shot backfill.

Worth knowing about that enrichment: it merges context and can fill a detection
anchor that was missing, but it does not restate the incident's severity or
status. An incident XDR later raises to critical stays at the severity it was
ingested with.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

integrations/cortex-xdr/manifest.yaml

@@ -1,8 +1,8 @@
 id: cortex_xdr
 name: Cortex XDR
-version: 1.3.0
+version: 1.3.1
 description: "Palo Alto Cortex XDR (public API v1) — incident and alert ingestion + write-back, endpoint isolation/scan/delete/tagging, RTR scripts, hash block/allow lists, file quarantine/restore/retrieval, alert exclusions, external alert push (parsed/CEF), device-control violations, audits, distributions and RBAC/risk."
-changelog: "1.3.0 — Richer incident ingestion (fetch_incidents, on get_multiple_incidents_extra_data): incidents now arrive with their alerts in full and their file/network artifacts, where get_incidents only ever answered a 21-field summary carrying neither hosts, users, MITRE nor a single alert. Alert ingestion, and the alerts endpoint answers at last: get_alerts was sending the incidents dialect ({filters, search_from, search_to, sort}) to a grid endpoint that speaks request_data.filter_data (SEARCH_FIELD/SEARCH_TYPE/SEARCH_VALUE blocks, paging.from/to, sort as a list), and every call came back HTTP 500. Body rebuilt from the reference client, rows unwrapped out of their alert_fields envelope, severity code and the dotted status.progress key derived into readable fields. Alert ingestion: get_alerts is now a fetch command (results path reply.alerts, dedup on alert_id, incremental on source_insert_ts) with a bundled OCSF mapper and a Cortex XDR Alert incident type, so detections can be ingested alongside — or instead of — incidents. The incident mapper is fixed on the way past: its severity expression compared strings, which the mapping engine cannot do, so every ingested incident silently took the rule's default severity; it also now carries a detection anchor so MTTD is measurable. It pages past the API's 100-results-per-call ceiling, and sorts oldest-first on an incremental fetch so a truncated window drops the alerts the next poll can still see. 1.2.1 — Connection troubleshooting: the URL is normalised to the tenant host (a pasted /public_api/v1 or console path no longer breaks the call), a non-JSON reply reports the status, content type and body instead of a bare JSON parse error, missing key/key ID is caught up front, nonce and timestamp are sent in both auth modes as the reference client does, and test_connection now probes get_incidents. 1.2.0 — Incident write-back (update_incident: status/severity/assignment/resolve comment) and external alert push (insert_parsed_alerts, insert_cef_alerts). 1.1.0 — Full command coverage: added delete/alias/tag endpoints, abort scan, original alerts, script metadata/code/snippet/exec-status, file retrieval (+details), alert exclusions, device-control violations, audits, distribution url/status/create and RBAC (users, roles, groups, risk score, risky users/hosts). 1.0.0 — Initial release: incident ingestion (get_incidents) with OCSF mapper, endpoints, isolate/unisolate, scan, hash blocklist/allowlist, quarantine/restore, run script + results, alerts retrieval, distributions and action status. Standard or Advanced API authentication."
+changelog: "1.3.1 — get_incidents_full can watermark on modification_time (modified_after), which is what ingestion wants: an XDR incident keeps growing after creation, and a creation_time watermark fetches it once and never looks again, so every alert that joins it afterwards is lost. The full-incident fetch command is named cortex-xdr-get-incidents-full (id get_incidents_full), not cortex-xdr-fetch-incidents: sitting next to cortex-xdr-get-incidents in the command list, it now reads as what it is — the same call, everything included. A rule created against the old id must be pointed at the new one. 1.3.0 — Richer incident ingestion (get_incidents_full, on get_multiple_incidents_extra_data): incidents now arrive with their alerts in full and their file/network artifacts, where get_incidents only ever answered a 21-field summary carrying neither hosts, users, MITRE nor a single alert. Alert ingestion, and the alerts endpoint answers at last: get_alerts was sending the incidents dialect ({filters, search_from, search_to, sort}) to a grid endpoint that speaks request_data.filter_data (SEARCH_FIELD/SEARCH_TYPE/SEARCH_VALUE blocks, paging.from/to, sort as a list), and every call came back HTTP 500. Body rebuilt from the reference client, rows unwrapped out of their alert_fields envelope, severity code and the dotted status.progress key derived into readable fields. Alert ingestion: get_alerts is now a fetch command (results path reply.alerts, dedup on alert_id, incremental on source_insert_ts) with a bundled OCSF mapper and a Cortex XDR Alert incident type, so detections can be ingested alongside — or instead of — incidents. The incident mapper is fixed on the way past: its severity expression compared strings, which the mapping engine cannot do, so every ingested incident silently took the rule's default severity; it also now carries a detection anchor so MTTD is measurable. It pages past the API's 100-results-per-call ceiling, and sorts oldest-first on an incremental fetch so a truncated window drops the alerts the next poll can still see. 1.2.1 — Connection troubleshooting: the URL is normalised to the tenant host (a pasted /public_api/v1 or console path no longer breaks the call), a non-JSON reply reports the status, content type and body instead of a bare JSON parse error, missing key/key ID is caught up front, nonce and timestamp are sent in both auth modes as the reference client does, and test_connection now probes get_incidents. 1.2.0 — Incident write-back (update_incident: status/severity/assignment/resolve comment) and external alert push (insert_parsed_alerts, insert_cef_alerts). 1.1.0 — Full command coverage: added delete/alias/tag endpoints, abort scan, original alerts, script metadata/code/snippet/exec-status, file retrieval (+details), alert exclusions, device-control violations, audits, distribution url/status/create and RBAC (users, roles, groups, risk score, risky users/hosts). 1.0.0 — Initial release: incident ingestion (get_incidents) with OCSF mapper, endpoints, isolate/unisolate, scan, hash blocklist/allowlist, quarantine/restore, run script + results, alerts retrieval, distributions and action status. Standard or Advanced API authentication."
 category: endpoint
 
 # Per-instance configuration. The base URL is the tenant API root, e.g.
@@ -44,7 +44,7 @@ commands:
   # ── Ingestion ─────────────────────────────────────────────────────────────
   - id: get_incidents
     name: cortex-xdr-get-incidents
-    description: "List Cortex XDR incidents as a 21-field summary (no hosts, no users, no MITRE, no alerts). Cheap to poll, but for ingestion prefer cortex-xdr-fetch-incidents, which returns the same incidents with their alerts and artifacts. Returns {reply:{incidents:[...]}}."
+    description: "List Cortex XDR incidents as a 21-field summary (no hosts, no users, no MITRE, no alerts). Cheap to poll, but for ingestion prefer cortex-xdr-get-incidents-full, which returns the same incidents with their alerts and artifacts. Returns {reply:{incidents:[...]}}."
     risk: read
     inputs_schema:
       properties:
@@ -57,14 +57,15 @@ commands:
       results_path: reply.incidents
       dedup_key: incident_id
       incremental_field: created_after
-  - id: fetch_incidents
+  - id: get_incidents_full
-    name: cortex-xdr-fetch-incidents
+    name: cortex-xdr-get-incidents-full
     description: "Fetch incidents WITH their alerts and artifacts (get_multiple_incidents_extra_data) — the ingestion command to prefer. get_incidents answers with a 21-field summary carrying no hosts, no users, no MITRE and none of the alerts; this one returns 39 incident fields, every alert in full (156 fields each) and the file/network artifacts. Records are flattened, so mapping expressions written against get_incidents keep working and alerts[], file_artifacts[], network_artifacts[] sit beside them. Returns {reply:{incidents:[...]}}."
     risk: read
     inputs_schema:
       properties:
         status: { type: string, description: "Comma-separated statuses to keep (new, under_investigation, resolved_threat_handled…)" }
-        created_after: { type: string, description: "Lower bound on creation_time, ISO8601 or epoch ms (incremental fetch watermark)" }
+        created_after: { type: string, description: "Lower bound on creation_time, ISO8601 or epoch ms. Watermarking on this fetches each incident once and never revisits it — alerts joining it later never arrive." }
+        modified_after: { type: string, description: "Lower bound on modification_time, ISO8601 or epoch ms. The watermark to prefer for ingestion: an incident comes back whenever it changes, and dedup on incident_id turns the second visit into an enrichment." }
         limit: { type: number, description: "Maximum incidents to fetch (default 50, paged 50 at a time). A full incident weighs a few KB and up to a few hundred with its alerts, so raise this knowingly." }
         exclude_artifacts: { type: boolean, description: "Drop the file and network artifact blocks, keeping the alerts (lighter payload)" }
       required: []
@@ -72,7 +73,7 @@ commands:
     ingest:
       results_path: reply.incidents
       dedup_key: incident_id
-      incremental_field: created_after
+      incremental_field: modified_after
   - id: get_incident_extra_data
     name: cortex-xdr-get-incident-extra-data
     description: "Get full incident data including its alerts and network artifacts by incident ID."

integrations/cortex-xdr/scripts/get_incidents_full.py

@@ -127,10 +127,20 @@ def main():
     created_ms = to_ms(inputs.get("created_after"))
     if created_ms is not None:
         filters.append({"field": "creation_time", "operator": "gte", "value": created_ms})
+    # An XDR incident keeps growing after it is created: alerts join it, an
+    # analyst changes its status. Watermarking on creation_time fetches it once
+    # and never looks again, so everything that happened afterwards is lost.
+    # Watermarking on modification_time brings it back on every change, where
+    # dedup on incident_id turns the second visit into an enrichment.
+    modified_ms = to_ms(inputs.get("modified_after"))
+    if modified_ms is not None:
+        filters.append({"field": "modification_time", "operator": "gte", "value": modified_ms})
+    incremental = created_ms is not None or modified_ms is not None
     # Oldest first on an incremental fetch, so that a window holding more
     # incidents than `limit` drops its most RECENT ones — the only ones the next
     # poll can still see. Newest first otherwise, for a hand-run command.
-    keyword = "asc" if created_ms is not None else "desc"
+    sort_field = "modification_time" if modified_ms is not None else "creation_time"
+    keyword = "asc" if incremental else "desc"
     exclude = str(inputs.get("exclude_artifacts") or "").lower() in ("1", "true", "yes")
 
     incidents, total = [], None
@@ -138,7 +148,7 @@ def main():
         rd = {
             "search_from": len(incidents),
             "search_to": min(len(incidents) + PAGE, limit),
-            "sort": {"field": "creation_time", "keyword": keyword},
+            "sort": {"field": sort_field, "keyword": keyword},
             # Without this the nested alerts come back trimmed to a handful of
             # fields — the very thing this command exists to avoid.
             "full_alert_fields": True,