diff --git a/integrations/cortex-xdr/manifest.yaml b/integrations/cortex-xdr/manifest.yaml index acc5a26..7076667 100644 --- a/integrations/cortex-xdr/manifest.yaml +++ b/integrations/cortex-xdr/manifest.yaml @@ -1,8 +1,8 @@ id: cortex_xdr name: Cortex XDR -version: 1.6.0 +version: 1.7.0 description: "Palo Alto Cortex XDR (public API v1) — incident and alert ingestion + write-back, endpoint isolation/scan/delete/tagging, XQL queries against the data lake, RTR scripts, hash block/allow lists, file quarantine/restore/retrieval, alert exclusions, external alert push (parsed/CEF), device-control violations, audits, distributions and RBAC/risk." -changelog: "1.6.0 — Declare the risk tier on the 25 commands that carried none: 10 safe_write, 15 destructive. An unset risk is read as destructive by the agent approval gate, so the 10 that only annotate sat behind the same prompt as the ones that change the estate. | 1.5.0 — The full-incident mapper now carries the investigation. XDR names action_process_image_* the process an alert is actually about — the child a prevention module blocked — and it was mapped nowhere, so the single most investigable string in the payload, that process’s command line, existed only inside raw: on a Child Process Protection alert, a WScript.exe invocation on a .js file sitting in the user’s Downloads folder. It fills process.* now, the group the SentinelOne (targetProcessInfo) and Carbon Black (childproc_*) mappers already reserve for the process acted upon. The chain above it lands in process.lineage as OCSF Process objects, root-first and ancestry only, which is what lineage means in OCSF; a generation XDR leaves entirely null — os_actor on an agent-side prevention — drops out rather than showing as an empty level. Every alert’s chain also travels under a process_tree context key, deliberately not an OCSF field: process.lineage describes the ancestry of ONE process, and an XDR incident groups N alerts, each with its own chain and often a shared ancestor, which OCSF has no shape for. Nodes there carry their role because the chain is positional — nothing in the payload links a child to a parent, action_process_instance_id is null, and the *_causality_id fields name the causality GROUP rather than the node, so merging chains on a causality id fuses two distinct processes. Also mapped, all for the same reason: the actor’s pid and MD5, the causality actor’s command line (mapped in the alerts feed but not here), the endpoint’s FQDN, AD domain, MAC and OS build, the protection module that fired and the agent version, the firing alert’s id, the incident id as the correlation key both feeds join on, the activity kind, and the registry, DNS and user-agent fields the other alert kinds in this feed carry. action_country is dropped when it is the literal UNKNOWN, which XDR writes on alerts that have no network dimension at all: mapping it put a destination country on process-execution incidents that never had a destination. 1.4.0 — XQL. Four commands reach the data lake: cortex-xdr-xql-query runs a query and waits for its rows, cortex-xdr-xql-start-query and cortex-xdr-xql-get-query-results split that work in two when the query outlives a command, and cortex-xdr-xql-get-quota says what the day’s hunting has already cost — an XQL query is billed by the data it scans. Two things the API makes easy to get wrong are handled here. Past 1000 rows it stops answering with results and hands over a stream id instead: gzipped JSON lines, unpacked into rows rather than dropped on the playbook as an archive. And a query carrying no limit stage is served the tenant’s own default, so one is appended (100 rows, configurable) unless the query already has its own. Waiting is a budget, not a promise: a command is killed at 60 seconds, so a query still running comes back PENDING with its execution_id, which the results command collects afterwards — re-running the query would be charged against the quota a second time. 1.3.2 — The mappers fill the Actor group, as the HarfangLab and SentinelOne mappers already did. XDR names the process that acted actor_process_* and its account actor_effective_username, but both landed outside the actor: the process in process.*, the account in user.name — the OCSF slot for the user an event was done TO. So a Cortex tenant answered nothing to a correlation or pre-processing rule written on actor.user.name / actor.process.name, and any rule reading user.name as the victim read the attacker instead. actor_process_* now maps to actor.process.*, causality_actor_process_* to actor.process.parent_process.*, actor_effective_username to actor.user.name; process.* is left free for the process acted upon, which is what it holds in the SentinelOne (targetProcessInfo) and Carbon Black (childproc_*) mappers. On get_incidents_full the incident's impacted users[] and the first alert's user_name both wrote user.name, so the alert's actor overwrote the impacted user; they are now two fields. Rules and columns written against the old Cortex paths need repointing. 1.3.1 — get_incidents_full can watermark on modification_time (modified_after), which is what ingestion wants: an XDR incident keeps growing after creation, and a creation_time watermark fetches it once and never looks again, so every alert that joins it afterwards is lost. The full-incident fetch command is named cortex-xdr-get-incidents-full (id get_incidents_full), not cortex-xdr-fetch-incidents: sitting next to cortex-xdr-get-incidents in the command list, it now reads as what it is — the same call, everything included. A rule created against the old id must be pointed at the new one. 1.3.0 — Richer incident ingestion (get_incidents_full, on get_multiple_incidents_extra_data): incidents now arrive with their alerts in full and their file/network artifacts, where get_incidents only ever answered a 21-field summary carrying neither hosts, users, MITRE nor a single alert. Alert ingestion, and the alerts endpoint answers at last: get_alerts was sending the incidents dialect ({filters, search_from, search_to, sort}) to a grid endpoint that speaks request_data.filter_data (SEARCH_FIELD/SEARCH_TYPE/SEARCH_VALUE blocks, paging.from/to, sort as a list), and every call came back HTTP 500. Body rebuilt from the reference client, rows unwrapped out of their alert_fields envelope, severity code and the dotted status.progress key derived into readable fields. Alert ingestion: get_alerts is now a fetch command (results path reply.alerts, dedup on alert_id, incremental on source_insert_ts) with a bundled OCSF mapper and a Cortex XDR Alert incident type, so detections can be ingested alongside — or instead of — incidents. The incident mapper is fixed on the way past: its severity expression compared strings, which the mapping engine cannot do, so every ingested incident silently took the rule's default severity; it also now carries a detection anchor so MTTD is measurable. It pages past the API's 100-results-per-call ceiling, and sorts oldest-first on an incremental fetch so a truncated window drops the alerts the next poll can still see. 1.2.1 — Connection troubleshooting: the URL is normalised to the tenant host (a pasted /public_api/v1 or console path no longer breaks the call), a non-JSON reply reports the status, content type and body instead of a bare JSON parse error, missing key/key ID is caught up front, nonce and timestamp are sent in both auth modes as the reference client does, and test_connection now probes get_incidents. 1.2.0 — Incident write-back (update_incident: status/severity/assignment/resolve comment) and external alert push (insert_parsed_alerts, insert_cef_alerts). 1.1.0 — Full command coverage: added delete/alias/tag endpoints, abort scan, original alerts, script metadata/code/snippet/exec-status, file retrieval (+details), alert exclusions, device-control violations, audits, distribution url/status/create and RBAC (users, roles, groups, risk score, risky users/hosts). 1.0.0 — Initial release: incident ingestion (get_incidents) with OCSF mapper, endpoints, isolate/unisolate, scan, hash blocklist/allowlist, quarantine/restore, run script + results, alerts retrieval, distributions and action status. Standard or Advanced API authentication." +changelog: "1.7.0 — New endpoint_count command: counts endpoints matching filters without listing them. The endpoints grid already answers with reply.total_count — the number of matches, independent of the page asked for — so endpoint_count asks for a one-record page and reads it, where get-endpoints has to walk the estate 100 records at a time to reach the same figure, and only filters on four fields. Filters cover the twelve list fields the API accepts (hostname, alias, endpoint id, internal and public IP, group, platform, distribution, status, isolation, scan status, username), add first_seen/last_seen bounds taking ISO8601 or epoch ms, and leave a raw JSON `filters` escape hatch for the rest. The estate `total` comes back next to `count` so the figure reads as a proportion, and a filter too broad to mean anything shows as matching everything rather than passing for an answer. | 1.6.0 — Declare the risk tier on the 25 commands that carried none: 10 safe_write, 15 destructive. An unset risk is read as destructive by the agent approval gate, so the 10 that only annotate sat behind the same prompt as the ones that change the estate. | 1.5.0 — The full-incident mapper now carries the investigation. XDR names action_process_image_* the process an alert is actually about — the child a prevention module blocked — and it was mapped nowhere, so the single most investigable string in the payload, that process’s command line, existed only inside raw: on a Child Process Protection alert, a WScript.exe invocation on a .js file sitting in the user’s Downloads folder. It fills process.* now, the group the SentinelOne (targetProcessInfo) and Carbon Black (childproc_*) mappers already reserve for the process acted upon. The chain above it lands in process.lineage as OCSF Process objects, root-first and ancestry only, which is what lineage means in OCSF; a generation XDR leaves entirely null — os_actor on an agent-side prevention — drops out rather than showing as an empty level. Every alert’s chain also travels under a process_tree context key, deliberately not an OCSF field: process.lineage describes the ancestry of ONE process, and an XDR incident groups N alerts, each with its own chain and often a shared ancestor, which OCSF has no shape for. Nodes there carry their role because the chain is positional — nothing in the payload links a child to a parent, action_process_instance_id is null, and the *_causality_id fields name the causality GROUP rather than the node, so merging chains on a causality id fuses two distinct processes. Also mapped, all for the same reason: the actor’s pid and MD5, the causality actor’s command line (mapped in the alerts feed but not here), the endpoint’s FQDN, AD domain, MAC and OS build, the protection module that fired and the agent version, the firing alert’s id, the incident id as the correlation key both feeds join on, the activity kind, and the registry, DNS and user-agent fields the other alert kinds in this feed carry. action_country is dropped when it is the literal UNKNOWN, which XDR writes on alerts that have no network dimension at all: mapping it put a destination country on process-execution incidents that never had a destination. 1.4.0 — XQL. Four commands reach the data lake: cortex-xdr-xql-query runs a query and waits for its rows, cortex-xdr-xql-start-query and cortex-xdr-xql-get-query-results split that work in two when the query outlives a command, and cortex-xdr-xql-get-quota says what the day’s hunting has already cost — an XQL query is billed by the data it scans. Two things the API makes easy to get wrong are handled here. Past 1000 rows it stops answering with results and hands over a stream id instead: gzipped JSON lines, unpacked into rows rather than dropped on the playbook as an archive. And a query carrying no limit stage is served the tenant’s own default, so one is appended (100 rows, configurable) unless the query already has its own. Waiting is a budget, not a promise: a command is killed at 60 seconds, so a query still running comes back PENDING with its execution_id, which the results command collects afterwards — re-running the query would be charged against the quota a second time. 1.3.2 — The mappers fill the Actor group, as the HarfangLab and SentinelOne mappers already did. XDR names the process that acted actor_process_* and its account actor_effective_username, but both landed outside the actor: the process in process.*, the account in user.name — the OCSF slot for the user an event was done TO. So a Cortex tenant answered nothing to a correlation or pre-processing rule written on actor.user.name / actor.process.name, and any rule reading user.name as the victim read the attacker instead. actor_process_* now maps to actor.process.*, causality_actor_process_* to actor.process.parent_process.*, actor_effective_username to actor.user.name; process.* is left free for the process acted upon, which is what it holds in the SentinelOne (targetProcessInfo) and Carbon Black (childproc_*) mappers. On get_incidents_full the incident's impacted users[] and the first alert's user_name both wrote user.name, so the alert's actor overwrote the impacted user; they are now two fields. Rules and columns written against the old Cortex paths need repointing. 1.3.1 — get_incidents_full can watermark on modification_time (modified_after), which is what ingestion wants: an XDR incident keeps growing after creation, and a creation_time watermark fetches it once and never looks again, so every alert that joins it afterwards is lost. The full-incident fetch command is named cortex-xdr-get-incidents-full (id get_incidents_full), not cortex-xdr-fetch-incidents: sitting next to cortex-xdr-get-incidents in the command list, it now reads as what it is — the same call, everything included. A rule created against the old id must be pointed at the new one. 1.3.0 — Richer incident ingestion (get_incidents_full, on get_multiple_incidents_extra_data): incidents now arrive with their alerts in full and their file/network artifacts, where get_incidents only ever answered a 21-field summary carrying neither hosts, users, MITRE nor a single alert. Alert ingestion, and the alerts endpoint answers at last: get_alerts was sending the incidents dialect ({filters, search_from, search_to, sort}) to a grid endpoint that speaks request_data.filter_data (SEARCH_FIELD/SEARCH_TYPE/SEARCH_VALUE blocks, paging.from/to, sort as a list), and every call came back HTTP 500. Body rebuilt from the reference client, rows unwrapped out of their alert_fields envelope, severity code and the dotted status.progress key derived into readable fields. Alert ingestion: get_alerts is now a fetch command (results path reply.alerts, dedup on alert_id, incremental on source_insert_ts) with a bundled OCSF mapper and a Cortex XDR Alert incident type, so detections can be ingested alongside — or instead of — incidents. The incident mapper is fixed on the way past: its severity expression compared strings, which the mapping engine cannot do, so every ingested incident silently took the rule's default severity; it also now carries a detection anchor so MTTD is measurable. It pages past the API's 100-results-per-call ceiling, and sorts oldest-first on an incremental fetch so a truncated window drops the alerts the next poll can still see. 1.2.1 — Connection troubleshooting: the URL is normalised to the tenant host (a pasted /public_api/v1 or console path no longer breaks the call), a non-JSON reply reports the status, content type and body instead of a bare JSON parse error, missing key/key ID is caught up front, nonce and timestamp are sent in both auth modes as the reference client does, and test_connection now probes get_incidents. 1.2.0 — Incident write-back (update_incident: status/severity/assignment/resolve comment) and external alert push (insert_parsed_alerts, insert_cef_alerts). 1.1.0 — Full command coverage: added delete/alias/tag endpoints, abort scan, original alerts, script metadata/code/snippet/exec-status, file retrieval (+details), alert exclusions, device-control violations, audits, distribution url/status/create and RBAC (users, roles, groups, risk score, risky users/hosts). 1.0.0 — Initial release: incident ingestion (get_incidents) with OCSF mapper, endpoints, isolate/unisolate, scan, hash blocklist/allowlist, quarantine/restore, run script + results, alerts retrieval, distributions and action status. Standard or Advanced API authentication." category: endpoint # Per-instance configuration. The base URL is the tenant API root, e.g. @@ -149,6 +149,31 @@ commands: limit: { type: number, description: "Max endpoints (default 30)" } required: [] outputs_schema: { properties: {} } + - id: endpoint_count + name: cortex-xdr-endpoint-count + description: "Count endpoints matching filters, without listing them. The endpoints grid answers with reply.total_count — the number of matches, independent of the page asked for — so one call with a single-record page returns the figure, where cortex-xdr-get-endpoints would page 100 endpoint records at a time to arrive at it. Returns {count, total, filters}: count is the number of matching endpoints, total the whole estate, so the figure reads as a proportion and a filter that matched everything is visible rather than plausible." + risk: read + inputs_schema: + properties: + hostname: { type: string, description: "Comma-separated hostnames" } + alias: { type: string, description: "Comma-separated endpoint aliases" } + endpoint_id: { type: string, description: "Comma-separated endpoint (agent) IDs" } + ip_list: { type: string, description: "Comma-separated internal IPs" } + public_ip_list: { type: string, description: "Comma-separated public IPs" } + group_name: { type: string, description: "Comma-separated endpoint group names" } + platform: { type: string, description: "Comma-separated platforms (windows, linux, macos, android)" } + dist_name: { type: string, description: "Comma-separated distribution (installation package) names" } + status: { type: string, description: "Comma-separated endpoint statuses (connected, disconnected, lost, uninstalled)" } + isolate: { type: string, description: "Isolation state: isolated or unisolated" } + scan_status: { type: string, description: "Comma-separated scan statuses (none, pending, in_progress, canceled, aborted, pending_cancellation, success, error)" } + username: { type: string, description: "Comma-separated usernames logged on the endpoint" } + first_seen_after: { type: string, description: "Lower bound on first_seen, ISO8601 or epoch ms — endpoints enrolled since that instant" } + first_seen_before: { type: string, description: "Upper bound on first_seen, ISO8601 or epoch ms" } + last_seen_after: { type: string, description: "Lower bound on last_seen, ISO8601 or epoch ms — endpoints seen since that instant" } + last_seen_before: { type: string, description: "Upper bound on last_seen, ISO8601 or epoch ms — endpoints silent since that instant" } + filters: { type: string, description: "Any other API filter, as a JSON array of {field, operator, value} objects, appended as-is. Filters are combined with AND; the API has no OR." } + required: [] + outputs_schema: { properties: {} } - id: isolate_endpoint name: cortex-xdr-isolate-endpoint description: "Network-isolate an endpoint by endpoint (agent) ID." diff --git a/integrations/cortex-xdr/scripts/endpoint_count.py b/integrations/cortex-xdr/scripts/endpoint_count.py new file mode 100644 index 0000000..6386c97 --- /dev/null +++ b/integrations/cortex-xdr/scripts/endpoint_count.py @@ -0,0 +1,156 @@ +import json, os, sys, time, hashlib, secrets, string, urllib.request, urllib.error +from datetime import datetime + + +def _client(): + s = json.loads(os.environ.get("INTEGRATION_SECRETS") or "{}") + raw = str(s.get("url") or "").strip().rstrip("/") + if not raw: + raise ValueError("no url configured — paste the tenant API URL (Cortex XDR > Settings > Configurations > API Keys > Copy URL)") + if "://" not in raw: + raw = "https://" + raw + # The tenant URL is a bare host. Drop whatever was pasted after it (a stray + # /public_api/v1, a console path) so the API root is built exactly once. + scheme, _, rest = raw.partition("://") + base = scheme + "://" + rest.split("/")[0] + "/public_api/v1" + key = s.get("api_key", "") + kid = str(s.get("api_key_id", "")) + if not key or not kid: + raise ValueError("api_key and api_key_id are both required") + # Nonce and timestamp ride along in both modes, as the reference client does. + # A standard key travels as-is; an advanced one as sha256(key + nonce + ts). + nonce = "".join(secrets.choice(string.ascii_letters + string.digits) for _ in range(64)) + ts = str(int(time.time()) * 1000) + headers = { + "x-xdr-auth-id": kid, + "x-xdr-nonce": nonce, + "x-xdr-timestamp": ts, + "Content-Type": "application/json", + "Accept": "application/json", + } + if str(s.get("auth_type") or "standard").lower() == "advanced": + headers["Authorization"] = hashlib.sha256((key + nonce + ts).encode("utf-8")).hexdigest() + else: + headers["Authorization"] = key + return base, headers + + +def _not_json(r, raw): + """A 2xx that is not JSON means we are not talking to the XDR API at all.""" + ctype = (r.headers.get("Content-Type") or "unknown").split(";")[0].strip() + head = raw[:160].decode("utf-8", "replace").replace("\n", " ").strip() + return ( + "expected JSON from " + r.geturl() + ", got " + ctype + " (HTTP " + str(r.status) + "): " + head + + " — check the configured url is the tenant API host" + + " (https://api-.xdr..paloaltonetworks.com), not the console URL" + ) + + +def post(path, request_data): + base, headers = _client() + data = json.dumps({"request_data": request_data}).encode("utf-8") + req = urllib.request.Request(base + path, data=data, headers=headers, method="POST") + with urllib.request.urlopen(req, timeout=90) as r: + raw = r.read() + if not raw: + return {} + try: + return json.loads(raw) + except ValueError: + raise ValueError(_not_json(r, raw)) + + +def csv(v): + return [x.strip() for x in str(v or "").split(",") if x.strip()] + + +def to_ms(v): + if v in (None, ""): + return None + s = str(v) + if s.isdigit(): + return int(s) + try: + return int(datetime.fromisoformat(s.replace("Z", "+00:00")).timestamp() * 1000) + except Exception: + return None + + +# input name -> (API filter field, operator). Values are comma-separated lists, +# which is the shape every one of these fields takes in the API. +LIST_FILTERS = [ + ("hostname", "hostname", "in"), + ("alias", "alias", "in"), + ("endpoint_id", "endpoint_id_list", "in"), + ("ip_list", "ip_list", "in"), + ("public_ip_list", "public_ip_list", "in"), + ("group_name", "group_name", "in"), + ("platform", "platform", "in"), + ("dist_name", "dist_name", "in"), + ("status", "endpoint_status", "IN"), + ("isolate", "isolate", "in"), + ("scan_status", "scan_status", "IN"), + ("username", "username", "IN"), +] + +# input name -> (API filter field, operator). Values are epoch-ms bounds. +RANGE_FILTERS = [ + ("first_seen_after", "first_seen", "gte"), + ("first_seen_before", "first_seen", "lte"), + ("last_seen_after", "last_seen", "gte"), + ("last_seen_before", "last_seen", "lte"), +] + + +def build_filters(inputs): + filters = [] + for name, field, operator in LIST_FILTERS: + values = csv(inputs.get(name)) + if values: + filters.append({"field": field, "operator": operator, "value": values}) + for name, field, operator in RANGE_FILTERS: + ms = to_ms(inputs.get(name)) + if ms is not None: + filters.append({"field": field, "operator": operator, "value": ms}) + # Escape hatch for any filter this command does not name explicitly: a JSON + # array of {field, operator, value} blocks, appended as-is. + raw = inputs.get("filters") + if raw: + try: + extra = json.loads(raw) if isinstance(raw, str) else raw + except ValueError: + extra = None + if not isinstance(extra, list) or not all(isinstance(f, dict) for f in extra): + raise ValueError("filters must be a JSON array of {field, operator, value} objects, got: " + str(raw)[:120]) + filters.extend(extra) + return filters + + +def count(filters): + # The endpoints grid answers with reply.total_count: the number of endpoints + # matching the filters, independent of the page asked for. search_to=1 keeps + # the page — and so the payload — to a single endpoint record. + request_data = {"search_from": 0, "search_to": 1, "filters": filters} + reply = post("/endpoints/get_endpoint/", request_data).get("reply") or {} + return int(reply.get("total_count") or 0) + + +def main(): + inputs = json.loads(os.environ.get("INTEGRATION_INPUTS", "{}")) + filters = build_filters(inputs) + result = {"count": count(filters), "filters": filters} + # The fleet total is the denominator the filtered count is read against: it + # turns a bare number into a proportion, and makes a filter that matched the + # whole estate — because it was too broad — visible instead of plausible. + result["total"] = count([]) if filters else result["count"] + print(json.dumps(result)) + + +try: + main() +except urllib.error.HTTPError as e: + print(json.dumps({"error": "HTTP " + str(e.code), "detail": e.read().decode("utf-8", "replace")})) + sys.exit(1) +except Exception as e: + print(json.dumps({"error": str(e)})) + sys.exit(1)