f3nris/riposte-marketplace
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix(cortex-xdr): the alerts endpoint never answered — wrong request dialect

Browse Source 
get_alerts posted the incidents body, {filters, search_from, search_to, sort},
to alerts/get_alerts_by_filter_data/. That endpoint serves the alerts GRID and
speaks another dialect entirely, so every call — before this branch as much as
after it — came back a bare HTTP 500 with no hint as to why.

Shape taken from the reference client (demisto/content, Packs/ApiModules/
Scripts/CoreIRApiModule, get_alerts_by_filter_command):

  request_data.filter_data = {
    sort:   [{FIELD, ORDER}],          # a list, uppercase keys
    paging: {from, to},                # not search_from/search_to
    filter: {AND: [{SEARCH_FIELD, SEARCH_TYPE, SEARCH_VALUE}]},
  }

Severity is an enum there (SEV_040_HIGH), and several severities are OR'd, not
passed as a list. The watermark is a RANGE, since the grid has no gte operator;
its upper bound carries five minutes of slack, because our clock and the
tenant's are not the same clock. A filterless query is bounded to the last
thirty days rather than sent empty — the reference client refuses one outright,
and the grid is not meant to be asked for a whole retention.

The response needed as much work as the request. Rows arrive wrapped as
{alert_fields, incident_fields}, and mapping through that wrapper would put an
alert_fields. prefix on every expression an operator writes, so each row is
unwrapped. Two of its fields cannot be mapped as they stand: severity is the
enum code, and status.progress carries a dot INSIDE the key, which no mapping
path can express. Both are derived into severity_name and status_progress.

The mapper follows the grid's own vocabulary — internal_id, alert_name,
agent_hostname, agent_ip_addresses — and dedup moves to internal_id, since
alert_id belongs to the other API. case_id is kept as the correlation UID: it
is the join back to the incident feed.

Verified end to end against the vendor's own recorded response
(test_data/get_alerts_by_filter_results.json): 33 of 54 OCSF entries resolve on
it, severity lands on 3, the detection anchor is set, and the paging walks
0-100, 100-200, 200-250 with the truncation flag raised only when the ceiling,
not the window, ended the fetch.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Guillaume BOURGEOIS
2026-08-18 23:34:26 +02:00
parent 17318d4225
commit d06f8ea413
3 changed files with 154 additions and 72 deletions
Download Patch File Download Diff File Expand all files Collapse all files
integrations/cortex-xdr/scripts/get_alerts.py
+94 -25
View File
@@ -76,51 +76,120 @@ def to_ms(v):
return None
# The API serves at most 100 alerts per call. Ingestion is what makes that a
# real ceiling: an alert feed easily carries more than a hundred detections
# between two polls, and anything a single page leaves behind is never fetched
# again — the next run's watermark has already moved past it.
# This endpoint speaks the alerts-grid dialect, NOT the incidents one: a body of
# {filters, search_from, search_to, sort} — what incidents/get_incidents/ takes —
# is answered with a bare HTTP 500. It wants request_data.filter_data with
# SEARCH_FIELD/SEARCH_TYPE/SEARCH_VALUE blocks, paging.from/to and a sort LIST.
# Shape taken from the reference client (demisto/content,
# Packs/ApiModules/Scripts/CoreIRApiModule — get_alerts_by_filter_command).
PAGE = 100
# Severity travels as an enum code both ways. Riposte reads plain names onto its
# 1-5 scale, so alerts carry `severity_name` alongside the raw code.
SEVERITY_CODE_TO_NAME = {
"SEV_010_INFO": "informational",
"SEV_020_LOW": "low",
"SEV_030_MEDIUM": "medium",
"SEV_040_HIGH": "high",
"SEV_050_CRITICAL": "critical",
}
SEVERITY_NAME_TO_CODE = dict((v, k) for k, v in SEVERITY_CODE_TO_NAME.items())
SEVERITY_NAME_TO_CODE["info"] = "SEV_010_INFO"
# Our clock and the tenant's are not the same clock. A range that ends exactly
# now silently drops alerts the tenant stamped a few seconds ahead of us.
SKEW_MS = 5 * 60 * 1000
# Window applied when the caller passes no filter at all — see main().
DEFAULT_LOOKBACK_MS = 30 * 24 * 60 * 60 * 1000
def severity_block(value):
"""One EQ block per severity, OR'd together (the reference client's array rule)."""
blocks = [
{"SEARCH_FIELD": "severity", "SEARCH_TYPE": "EQ",
"SEARCH_VALUE": SEVERITY_NAME_TO_CODE.get(s.lower(), s.upper())}
for s in csv(value)
]
if not blocks:
return None
return blocks[0] if len(blocks) == 1 else {"OR": blocks}
def flatten(item):
"""One grid row -> one flat alert.
The API wraps every row as {alert_fields, incident_fields}. Mapping through
that wrapper would put an `alert_fields.` prefix on every expression an
operator writes, so the row is unwrapped here and the two fields Riposte
cannot express are derived: `status.progress` carries a dot INSIDE the key
(unusable as a mapping path) and severity is an enum code.
"""
fields = item.get("alert_fields")
alert = dict(fields) if isinstance(fields, dict) else dict(item)
alert.pop("incident_fields", None)
if "status.progress" in alert:
alert["status_progress"] = alert.pop("status.progress")
name = SEVERITY_CODE_TO_NAME.get(alert.get("severity"))
if name:
alert["severity_name"] = name
incident = item.get("incident_fields")
if isinstance(incident, dict):
alert["incident_fields"] = incident
return alert
def main():
inputs = json.loads(os.environ.get("INTEGRATION_INPUTS", "{}"))
limit = max(1, int(inputs.get("limit") or 100))
filters = []
if inputs.get("severity"):
filters.append({"field": "severity", "operator": "in", "value": csv(inputs["severity"])})
conditions = []
sev = severity_block(inputs.get("severity"))
if sev:
conditions.append(sev)
created_ms = to_ms(inputs.get("created_after"))
if created_ms is not None:
filters.append({"field": "source_insert_ts", "operator": "gte", "value": created_ms})
conditions.append({
"SEARCH_FIELD": "source_insert_ts",
"SEARCH_TYPE": "RANGE",
"SEARCH_VALUE": {"from": created_ms, "to": int(time.time() * 1000) + SKEW_MS},
})
if not conditions:
# The reference client refuses a filterless query outright, and an
# unbounded scan of the whole alerts grid is not what the API is for.
# A recent window is a better default than an empty filter the tenant
# may well answer with a 500.
now_ms = int(time.time() * 1000)
conditions.append({
"SEARCH_FIELD": "source_insert_ts",
"SEARCH_TYPE": "RANGE",
"SEARCH_VALUE": {"from": now_ms - DEFAULT_LOOKBACK_MS, "to": now_ms + SKEW_MS},
})
# Oldest first on an incremental fetch, so that a window holding more alerts
# than `limit` drops its most RECENT ones — the only ones the next poll can
# still see. Newest first otherwise, which is what an operator running the
# command by hand is asking for.
keyword = "asc" if created_ms is not None else "desc"
order = "ASC" if created_ms is not None else "DESC"
alerts, total = [], None
alerts, truncated = [], False
while len(alerts) < limit:
rd = {
"search_from": len(alerts),
"search_to": min(len(alerts) + PAGE, limit),
"sort": {"field": "source_insert_ts", "keyword": keyword},
}
if filters:
rd["filters"] = filters
rd = {"filter_data": {
"sort": [{"FIELD": "source_insert_ts", "ORDER": order}],
"paging": {"from": len(alerts), "to": min(len(alerts) + PAGE, limit)},
"filter": {"AND": conditions},
}}
reply = (post("/alerts/get_alerts_by_filter_data/", rd) or {}).get("reply") or {}
page = reply.get("alerts") or []
if total is None:
total = reply.get("total_count")
alerts.extend(page)
alerts.extend(flatten(a) for a in page)
if len(page) < PAGE:
break
# Stopped on the ceiling rather than on an exhausted window: whatever is
# left is not coming back on the next poll, and a silent cap reads like
# a quiet feed.
truncated = len(alerts) >= limit
out = {"result_count": len(alerts), "alerts": alerts}
if total is not None:
out["total_count"] = total
# Say it when the window was larger than the limit: those alerts are not
# coming back on the next poll, and a silent cap reads like a quiet feed.
out["truncated"] = total > len(alerts)
if truncated:
out["truncated"] = True
print(json.dumps({"reply": out}))