diff --git a/integrations/cortex-xdr/manifest.yaml b/integrations/cortex-xdr/manifest.yaml index 2f292d7..c3f29e5 100644 --- a/integrations/cortex-xdr/manifest.yaml +++ b/integrations/cortex-xdr/manifest.yaml @@ -1,8 +1,8 @@ id: cortex_xdr name: Cortex XDR -version: 1.4.0 +version: 1.5.0 description: "Palo Alto Cortex XDR (public API v1) — incident and alert ingestion + write-back, endpoint isolation/scan/delete/tagging, XQL queries against the data lake, RTR scripts, hash block/allow lists, file quarantine/restore/retrieval, alert exclusions, external alert push (parsed/CEF), device-control violations, audits, distributions and RBAC/risk." -changelog: "1.4.0 — XQL. Four commands reach the data lake: cortex-xdr-xql-query runs a query and waits for its rows, cortex-xdr-xql-start-query and cortex-xdr-xql-get-query-results split that work in two when the query outlives a command, and cortex-xdr-xql-get-quota says what the day’s hunting has already cost — an XQL query is billed by the data it scans. Two things the API makes easy to get wrong are handled here. Past 1000 rows it stops answering with results and hands over a stream id instead: gzipped JSON lines, unpacked into rows rather than dropped on the playbook as an archive. And a query carrying no limit stage is served the tenant’s own default, so one is appended (100 rows, configurable) unless the query already has its own. Waiting is a budget, not a promise: a command is killed at 60 seconds, so a query still running comes back PENDING with its execution_id, which the results command collects afterwards — re-running the query would be charged against the quota a second time. 1.3.2 — The mappers fill the Actor group, as the HarfangLab and SentinelOne mappers already did. XDR names the process that acted actor_process_* and its account actor_effective_username, but both landed outside the actor: the process in process.*, the account in user.name — the OCSF slot for the user an event was done TO. So a Cortex tenant answered nothing to a correlation or pre-processing rule written on actor.user.name / actor.process.name, and any rule reading user.name as the victim read the attacker instead. actor_process_* now maps to actor.process.*, causality_actor_process_* to actor.process.parent_process.*, actor_effective_username to actor.user.name; process.* is left free for the process acted upon, which is what it holds in the SentinelOne (targetProcessInfo) and Carbon Black (childproc_*) mappers. On get_incidents_full the incident's impacted users[] and the first alert's user_name both wrote user.name, so the alert's actor overwrote the impacted user; they are now two fields. Rules and columns written against the old Cortex paths need repointing. 1.3.1 — get_incidents_full can watermark on modification_time (modified_after), which is what ingestion wants: an XDR incident keeps growing after creation, and a creation_time watermark fetches it once and never looks again, so every alert that joins it afterwards is lost. The full-incident fetch command is named cortex-xdr-get-incidents-full (id get_incidents_full), not cortex-xdr-fetch-incidents: sitting next to cortex-xdr-get-incidents in the command list, it now reads as what it is — the same call, everything included. A rule created against the old id must be pointed at the new one. 1.3.0 — Richer incident ingestion (get_incidents_full, on get_multiple_incidents_extra_data): incidents now arrive with their alerts in full and their file/network artifacts, where get_incidents only ever answered a 21-field summary carrying neither hosts, users, MITRE nor a single alert. Alert ingestion, and the alerts endpoint answers at last: get_alerts was sending the incidents dialect ({filters, search_from, search_to, sort}) to a grid endpoint that speaks request_data.filter_data (SEARCH_FIELD/SEARCH_TYPE/SEARCH_VALUE blocks, paging.from/to, sort as a list), and every call came back HTTP 500. Body rebuilt from the reference client, rows unwrapped out of their alert_fields envelope, severity code and the dotted status.progress key derived into readable fields. Alert ingestion: get_alerts is now a fetch command (results path reply.alerts, dedup on alert_id, incremental on source_insert_ts) with a bundled OCSF mapper and a Cortex XDR Alert incident type, so detections can be ingested alongside — or instead of — incidents. The incident mapper is fixed on the way past: its severity expression compared strings, which the mapping engine cannot do, so every ingested incident silently took the rule's default severity; it also now carries a detection anchor so MTTD is measurable. It pages past the API's 100-results-per-call ceiling, and sorts oldest-first on an incremental fetch so a truncated window drops the alerts the next poll can still see. 1.2.1 — Connection troubleshooting: the URL is normalised to the tenant host (a pasted /public_api/v1 or console path no longer breaks the call), a non-JSON reply reports the status, content type and body instead of a bare JSON parse error, missing key/key ID is caught up front, nonce and timestamp are sent in both auth modes as the reference client does, and test_connection now probes get_incidents. 1.2.0 — Incident write-back (update_incident: status/severity/assignment/resolve comment) and external alert push (insert_parsed_alerts, insert_cef_alerts). 1.1.0 — Full command coverage: added delete/alias/tag endpoints, abort scan, original alerts, script metadata/code/snippet/exec-status, file retrieval (+details), alert exclusions, device-control violations, audits, distribution url/status/create and RBAC (users, roles, groups, risk score, risky users/hosts). 1.0.0 — Initial release: incident ingestion (get_incidents) with OCSF mapper, endpoints, isolate/unisolate, scan, hash blocklist/allowlist, quarantine/restore, run script + results, alerts retrieval, distributions and action status. Standard or Advanced API authentication." +changelog: "1.5.0 — The full-incident mapper now carries the investigation. XDR names action_process_image_* the process an alert is actually about — the child a prevention module blocked — and it was mapped nowhere, so the single most investigable string in the payload, that process’s command line, existed only inside raw: on a Child Process Protection alert, a WScript.exe invocation on a .js file sitting in the user’s Downloads folder. It fills process.* now, the group the SentinelOne (targetProcessInfo) and Carbon Black (childproc_*) mappers already reserve for the process acted upon. The chain above it lands in process.lineage as OCSF Process objects, root-first and ancestry only, which is what lineage means in OCSF; a generation XDR leaves entirely null — os_actor on an agent-side prevention — drops out rather than showing as an empty level. Every alert’s chain also travels under a process_tree context key, deliberately not an OCSF field: process.lineage describes the ancestry of ONE process, and an XDR incident groups N alerts, each with its own chain and often a shared ancestor, which OCSF has no shape for. Nodes there carry their role because the chain is positional — nothing in the payload links a child to a parent, action_process_instance_id is null, and the *_causality_id fields name the causality GROUP rather than the node, so merging chains on a causality id fuses two distinct processes. Also mapped, all for the same reason: the actor’s pid and MD5, the causality actor’s command line (mapped in the alerts feed but not here), the endpoint’s FQDN, AD domain, MAC and OS build, the protection module that fired and the agent version, the firing alert’s id, the incident id as the correlation key both feeds join on, the activity kind, and the registry, DNS and user-agent fields the other alert kinds in this feed carry. action_country is dropped when it is the literal UNKNOWN, which XDR writes on alerts that have no network dimension at all: mapping it put a destination country on process-execution incidents that never had a destination. 1.4.0 — XQL. Four commands reach the data lake: cortex-xdr-xql-query runs a query and waits for its rows, cortex-xdr-xql-start-query and cortex-xdr-xql-get-query-results split that work in two when the query outlives a command, and cortex-xdr-xql-get-quota says what the day’s hunting has already cost — an XQL query is billed by the data it scans. Two things the API makes easy to get wrong are handled here. Past 1000 rows it stops answering with results and hands over a stream id instead: gzipped JSON lines, unpacked into rows rather than dropped on the playbook as an archive. And a query carrying no limit stage is served the tenant’s own default, so one is appended (100 rows, configurable) unless the query already has its own. Waiting is a budget, not a promise: a command is killed at 60 seconds, so a query still running comes back PENDING with its execution_id, which the results command collects afterwards — re-running the query would be charged against the quota a second time. 1.3.2 — The mappers fill the Actor group, as the HarfangLab and SentinelOne mappers already did. XDR names the process that acted actor_process_* and its account actor_effective_username, but both landed outside the actor: the process in process.*, the account in user.name — the OCSF slot for the user an event was done TO. So a Cortex tenant answered nothing to a correlation or pre-processing rule written on actor.user.name / actor.process.name, and any rule reading user.name as the victim read the attacker instead. actor_process_* now maps to actor.process.*, causality_actor_process_* to actor.process.parent_process.*, actor_effective_username to actor.user.name; process.* is left free for the process acted upon, which is what it holds in the SentinelOne (targetProcessInfo) and Carbon Black (childproc_*) mappers. On get_incidents_full the incident's impacted users[] and the first alert's user_name both wrote user.name, so the alert's actor overwrote the impacted user; they are now two fields. Rules and columns written against the old Cortex paths need repointing. 1.3.1 — get_incidents_full can watermark on modification_time (modified_after), which is what ingestion wants: an XDR incident keeps growing after creation, and a creation_time watermark fetches it once and never looks again, so every alert that joins it afterwards is lost. The full-incident fetch command is named cortex-xdr-get-incidents-full (id get_incidents_full), not cortex-xdr-fetch-incidents: sitting next to cortex-xdr-get-incidents in the command list, it now reads as what it is — the same call, everything included. A rule created against the old id must be pointed at the new one. 1.3.0 — Richer incident ingestion (get_incidents_full, on get_multiple_incidents_extra_data): incidents now arrive with their alerts in full and their file/network artifacts, where get_incidents only ever answered a 21-field summary carrying neither hosts, users, MITRE nor a single alert. Alert ingestion, and the alerts endpoint answers at last: get_alerts was sending the incidents dialect ({filters, search_from, search_to, sort}) to a grid endpoint that speaks request_data.filter_data (SEARCH_FIELD/SEARCH_TYPE/SEARCH_VALUE blocks, paging.from/to, sort as a list), and every call came back HTTP 500. Body rebuilt from the reference client, rows unwrapped out of their alert_fields envelope, severity code and the dotted status.progress key derived into readable fields. Alert ingestion: get_alerts is now a fetch command (results path reply.alerts, dedup on alert_id, incremental on source_insert_ts) with a bundled OCSF mapper and a Cortex XDR Alert incident type, so detections can be ingested alongside — or instead of — incidents. The incident mapper is fixed on the way past: its severity expression compared strings, which the mapping engine cannot do, so every ingested incident silently took the rule's default severity; it also now carries a detection anchor so MTTD is measurable. It pages past the API's 100-results-per-call ceiling, and sorts oldest-first on an incremental fetch so a truncated window drops the alerts the next poll can still see. 1.2.1 — Connection troubleshooting: the URL is normalised to the tenant host (a pasted /public_api/v1 or console path no longer breaks the call), a non-JSON reply reports the status, content type and body instead of a bare JSON parse error, missing key/key ID is caught up front, nonce and timestamp are sent in both auth modes as the reference client does, and test_connection now probes get_incidents. 1.2.0 — Incident write-back (update_incident: status/severity/assignment/resolve comment) and external alert push (insert_parsed_alerts, insert_cef_alerts). 1.1.0 — Full command coverage: added delete/alias/tag endpoints, abort scan, original alerts, script metadata/code/snippet/exec-status, file retrieval (+details), alert exclusions, device-control violations, audits, distribution url/status/create and RBAC (users, roles, groups, risk score, risky users/hosts). 1.0.0 — Initial release: incident ingestion (get_incidents) with OCSF mapper, endpoints, isolate/unisolate, scan, hash blocklist/allowlist, quarantine/restore, run script + results, alerts retrieval, distributions and action status. Standard or Advanced API authentication." category: endpoint # Per-instance configuration. The base URL is the tenant API root, e.g. diff --git a/integrations/cortex-xdr/mappers/get_incidents_full.yaml b/integrations/cortex-xdr/mappers/get_incidents_full.yaml index 2a6d81f..f6fa5c5 100644 --- a/integrations/cortex-xdr/mappers/get_incidents_full.yaml +++ b/integrations/cortex-xdr/mappers/get_incidents_full.yaml @@ -24,6 +24,9 @@ ocsf: - { source_path: "xdr_url", ocsf_field: "finding_info.src_url" } - { source_path: "alert_categories[0]", ocsf_field: "finding_info.analytic.category" } - { source_path: "alerts[0].name", ocsf_field: "finding_info.analytic.name" } + # The detection that opened the incident, so an analyst reading the aggregate + # can still pivot to the single alert in the XDR console. + - { source_path: "alerts[0].alert_id", ocsf_field: "finding_info.uid_alt" } # ── Detection time: the MTTD anchor ─────────────────────────────── # Weakest first, strongest last. detection_time is often null on an XDR # incident, and then the first alert's own detection timestamp is the honest @@ -39,10 +42,22 @@ ocsf: - { source_path: "aggregated_score", ocsf_field: "risk_score" } - { source_path: "tags", ocsf_field: "metadata.labels" } - { source_path: "alerts[0].action_pretty", ocsf_field: "action" } + # What kind of activity the sensor saw ("Process Execution", "Network + # Connection"…). It is what tells an analyst, before opening anything, whether + # this incident is about a process, a file or a flow. + - { source_path: "alerts[0].event_type", ocsf_field: "activity_name" } # ── Product identity ────────────────────────────────────────────── - { source_path: "'Cortex XDR'", ocsf_field: "metadata.product.name" } - { source_path: "'Palo Alto Networks'", ocsf_field: "metadata.product.vendor_name" } - { source_path: "incident_sources[0]", ocsf_field: "metadata.log_source" } + # Which protection module fired ("Child Process Protection", "Behavioral + # Threat Protection"…) — the closest thing XDR gives to a rule name when + # matching_service_rule_id is null, which it is on agent-side preventions. + - { source_path: "alerts[0].module_id", ocsf_field: "metadata.product.feature.name" } + - { source_path: "alerts[0].agent_version", ocsf_field: "metadata.product.version" } + # The XDR case id, on the same OCSF field the alerts mapper writes it to, so + # an incident and the alerts folded into it join on one value from both feeds. + - { source_path: "incident_id", ocsf_field: "metadata.correlation_uid" } # ── MITRE ATT&CK: the aggregate, else the first alert ───────────── - { source_path: "mitre_tactics_ids_and_names[0]", ocsf_field: "attacks.tactic.name" } - { source_path: "mitre_techniques_ids_and_names[0]", ocsf_field: "attacks.technique.name" } @@ -58,32 +73,165 @@ ocsf: - { source_path: "alerts[0].host_ip[0]", ocsf_field: "src_endpoint.ip" } - { source_path: "alerts[0].endpoint_id", ocsf_field: "device.uid" } - { source_path: "alerts[0].agent_os_type", ocsf_field: "device.os.type" } + # The identity an analyst needs to find the machine again: its FQDN, its AD + # domain, its MAC, and the OS build the exploit ran against. + - { source_path: "alerts[0].agent_fqdn", ocsf_field: "device.name" } + - { source_path: "alerts[0].agent_device_domain", ocsf_field: "device.domain" } + - { source_path: "alerts[0].mac", ocsf_field: "device.mac" } + - { source_path: "alerts[0].agent_os_sub_type", ocsf_field: "device.os.name" } # users[] is who the incident happened TO; alerts[0].user_name is the account # the acting process ran under. Both used to write user.name, so the alert's # actor silently overwrote the impacted user (last non-empty wins). - { source_path: "users[0]", ocsf_field: "user.name" } - { source_path: "alerts[0].user_name", ocsf_field: "actor.user.name" } + - { source_path: "alerts[0].os_actor_effective_username", ocsf_field: "actor.user.name" } # ── What the first alert actually saw: the actor ────────────────── # XDR's own `actor_*` naming is taken at its word and lands in the Actor # group, as it does in the alerts mapper and in the HarfangLab and # SentinelOne feeds; `process.*` stays reserved for the process acted upon. - { source_path: "alerts[0].actor_process_image_name", ocsf_field: "actor.process.name" } + - { source_path: "alerts[0].actor_process_image_name", ocsf_field: "actor.process.file.name" } - { source_path: "alerts[0].actor_process_command_line", ocsf_field: "actor.process.cmd_line" } - { source_path: "alerts[0].actor_process_image_path", ocsf_field: "actor.process.file.path" } - { source_path: "alerts[0].actor_process_image_sha256", ocsf_field: "actor.process.file.hashes.sha256" } + - { source_path: "alerts[0].actor_process_image_md5", ocsf_field: "actor.process.file.hashes.md5" } + - { source_path: "alerts[0].actor_process_os_pid", ocsf_field: "actor.process.pid" } - { source_path: "alerts[0].causality_actor_process_image_name", ocsf_field: "actor.process.parent_process.name" } + # The causality actor's command line: on a logon-time ancestor it is the only + # thing that says WHICH session spawned the chain, and it was mapped in the + # alerts feed but not here. + - { source_path: "alerts[0].causality_actor_process_command_line", ocsf_field: "actor.process.parent_process.cmd_line" } + # ── The process acted UPON ──────────────────────────────────────── + # XDR's `action_process_image_*` is the process the alert is about — the child + # a prevention module blocked. It was mapped NOWHERE, which meant the single + # most investigable string in the whole payload, the blocked command line, + # existed only inside raw. `process.*` is where the other EDR mappers put the + # acted-upon process (SentinelOne targetProcessInfo, Carbon Black childproc_*), + # so it lands there and nothing else in this file writes that group. + - { source_path: "alerts[0].action_process_image_name", ocsf_field: "process.name" } + - { source_path: "alerts[0].action_process_image_name", ocsf_field: "process.file.name" } + - { source_path: "alerts[0].action_process_image_command_line", ocsf_field: "process.cmd_line" } + - { source_path: "alerts[0].action_process_image_sha256", ocsf_field: "process.file.hashes.sha256" } + - { source_path: "alerts[0].action_process_signature_vendor", ocsf_field: "process.file.company_name" } + - { source_path: "alerts[0].action_file_path", ocsf_field: "process.file.path" } + - { source_path: "alerts[0].event_timestamp", ocsf_field: "process.created_time" } + # Its ancestry, root-first, as OCSF Process objects — this is the chain and + # NOT the process itself, which is what `process.lineage` means in OCSF. + # A level XDR left entirely null (os_actor on an agent-side prevention) drops + # out on the [name] predicate rather than showing as an empty generation. + - source_path: | + alerts[0].( + [ + { + "name": causality_actor_process_image_name, + "cmd_line": causality_actor_process_command_line, + "created_time": causality_actor_process_execution_time, + "file": { "name": causality_actor_process_image_name, "path": causality_actor_process_image_path, + "hashes": { "sha256": causality_actor_process_image_sha256, "md5": causality_actor_process_image_md5 } } + }, + { + "name": os_actor_process_image_name, + "cmd_line": os_actor_process_command_line, + "pid": os_actor_process_os_pid, + "file": { "name": os_actor_process_image_name, "path": os_actor_process_image_path, + "hashes": { "sha256": os_actor_process_image_sha256 } } + }, + { + "name": actor_process_image_name, + "cmd_line": actor_process_command_line, + "pid": actor_process_os_pid, + "file": { "name": actor_process_image_name, "path": actor_process_image_path, + "hashes": { "sha256": actor_process_image_sha256, "md5": actor_process_image_md5 } } + } + ][name] + ) + ocsf_field: "process.lineage" + # ── Every alert's chain, for the process-tree panel ─────────────── + # Not an OCSF field: `process.lineage` describes the ancestry of ONE process, + # and an XDR incident groups N alerts, each with its own chain and often a + # shared ancestor. That shape has no OCSF equivalent, so it travels under its + # own context key rather than being bent into one. + # + # Nodes carry their role because the chain is POSITIONAL: outside actor_*, + # nothing in the payload links a child to a parent — action_process_instance_id + # is null and the *_causality_id fields identify the causality GROUP, not the + # node (here KUsrInit and explorer share one). Merge chains on name+sha256, + # never on a causality id. + - source_path: | + ( + $arts := file_artifacts; + $art := function($sha) { $sha ? $arts[file_sha256 = $sha][0] }; + $node := function($role, $name, $cmd, $path, $sha, $md5, $pid, $ts, $signer, $sigstatus, $act) { + { + "role": $role, "name": $name, "cmd_line": $cmd, "path": $path, + "sha256": $sha, "md5": $md5, "pid": $pid, "started_at": $ts, + "signer": $signer ? $signer : $art($sha).file_signature_vendor_name, + "signature": $sigstatus != "N/A" ? $sigstatus, + "verdict": $art($sha).file_wildfire_verdict, + "action": $act + } + }; + $map(alerts, function($a) { + { + "alert_id": $a.alert_id, + "rule": $a.name, + "module": $a.module_id, + "event_type": $a.event_type, + "action": $a.action_pretty, + "time": $a.event_timestamp, + "host": $a.host_name, + "user": $a.user_name, + "nodes": [ + $node("causality_actor", $a.causality_actor_process_image_name, $a.causality_actor_process_command_line, + $a.causality_actor_process_image_path, $a.causality_actor_process_image_sha256, + $a.causality_actor_process_image_md5, undefined, $a.causality_actor_process_execution_time, + $a.causality_actor_process_signature_vendor, $a.causality_actor_process_signature_status, undefined), + $node("os_actor", $a.os_actor_process_image_name, $a.os_actor_process_command_line, + $a.os_actor_process_image_path, $a.os_actor_process_image_sha256, + undefined, $a.os_actor_process_os_pid, undefined, + $a.os_actor_process_signature_vendor, $a.os_actor_process_signature_status, undefined), + $node("actor", $a.actor_process_image_name, $a.actor_process_command_line, + $a.actor_process_image_path, $a.actor_process_image_sha256, + $a.actor_process_image_md5, $a.actor_process_os_pid, undefined, + $a.actor_process_signature_vendor, $a.actor_process_signature_status, undefined), + $node("action", $a.action_process_image_name, $a.action_process_image_command_line, + $a.action_file_path, $a.action_process_image_sha256, + $a.action_file_md5, undefined, $a.event_timestamp, + $a.action_process_signature_vendor, $a.action_process_signature_status, $a.action_pretty) + ][name] + } + }) + ) + ocsf_field: "process_tree" + # ── What the first alert touched: file, registry, network, DNS ──── + # Most are null on a process-execution alert and are skipped at ingestion; + # they are what makes this same mapper readable on the other alert kinds the + # feed carries. - { source_path: "alerts[0].action_file_path", ocsf_field: "file.path" } - { source_path: "alerts[0].action_file_name", ocsf_field: "file.name" } - { source_path: "alerts[0].action_file_sha256", ocsf_field: "file.hashes.sha256" } - { source_path: "alerts[0].action_file_md5", ocsf_field: "file.hashes.md5" } + - { source_path: "alerts[0].action_registry_full_key", ocsf_field: "reg_key.path" } + - { source_path: "alerts[0].action_registry_value_name", ocsf_field: "reg_value.name" } + - { source_path: "alerts[0].action_registry_data", ocsf_field: "reg_value.data" } + - { source_path: "alerts[0].dns_query_name", ocsf_field: "dns_query.hostname" } + - { source_path: "alerts[0].user_agent", ocsf_field: "http_request.user_agent" } + - { source_path: "alerts[0].action_local_ip", ocsf_field: "src_endpoint.ip" } + - { source_path: "alerts[0].action_local_port", ocsf_field: "src_endpoint.port" } - { source_path: "alerts[0].action_remote_ip", ocsf_field: "dst_endpoint.ip" } - { source_path: "alerts[0].action_remote_port", ocsf_field: "dst_endpoint.port" } - { source_path: "alerts[0].action_external_hostname", ocsf_field: "dst_endpoint.hostname" } + # XDR writes the literal "UNKNOWN" rather than nothing when an alert has no + # network dimension at all, and mapping it would put a destination country on + # a process-execution incident that never had a destination. + - { source_path: 'alerts[0].action_country != ''UNKNOWN'' ? alerts[0].action_country', ocsf_field: "dst_endpoint.location.country" } # ── The artifact the incident is really about ───────────────────── # Last, because a file artifact is the incident's verdict on the file, where # the alert only reports what one detection touched. - { source_path: "file_artifacts[0].file_name", ocsf_field: "file.name" } - { source_path: "file_artifacts[0].file_sha256", ocsf_field: "file.hashes.sha256" } + - { source_path: "file_artifacts[0].file_signature_vendor_name", ocsf_field: "file.company_name" } + - { source_path: "file_artifacts[0].file_signature_status", ocsf_field: "file.signature.value" } - { source_path: "file_artifacts[0].file_wildfire_verdict", ocsf_field: "malware.classifications" } - { source_path: "network_artifacts[0].network_remote_ip", ocsf_field: "dst_endpoint.ip" } - { source_path: "network_artifacts[0].network_domain", ocsf_field: "dst_endpoint.hostname" }