feat: CIRCL hashlookup, RDAP and crt.sh — free IOC lookups, no API key
Three keyless sources for the questions an analyst asks first: - circl-hashlookup 1.0.0: is this hash known software (NSRL, distro packages, Windows updates)? `score` 0 for a known file, 100 when CIRCL flags it KnownMalicious, so an enrichment rule with benign 0 marks it benign; an unknown hash is an HTTP 404, read as "no record". Single and bulk lookups; base_url for a self-hosted instance. - rdap 1.0.0: who registered a domain, IP block or AS, and when — with age_days for the phishing question. Authoritative server from the IANA bootstrap, rdap.org as fallback; a host name walks up to its registered domain; GDPR placeholders are not reported as registrants; a TLD with no RDAP (.ch) is an error, not a fake "unregistered". - crtsh 1.0.0: Certificate Transparency — sub-domains with first/last seen, issuers, recent certificates, and a starts_with mode for look-alikes of a brand (crt.sh no longer searches with a leading wildcard). Retries the 502/503/504 its shared front throws under load. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AmWT9wtWHGtZpMS64Hp4q5
This commit is contained in:
@@ -0,0 +1,60 @@
|
||||
import datetime, json, os, ssl, sys, time, urllib.error, urllib.parse, urllib.request
|
||||
|
||||
DEFAULT_BASE = "https://crt.sh"
|
||||
|
||||
|
||||
def _cfg():
|
||||
return json.loads(os.environ.get("INTEGRATION_SECRETS", "{}"))
|
||||
|
||||
|
||||
def _inputs():
|
||||
return json.loads(os.environ.get("INTEGRATION_INPUTS", "{}"))
|
||||
|
||||
|
||||
def _ctx(cfg):
|
||||
if cfg.get("insecure") in (True, "true", "True", 1, "1"):
|
||||
c = ssl.create_default_context()
|
||||
c.check_hostname = False
|
||||
c.verify_mode = ssl.CERT_NONE
|
||||
return c
|
||||
return None
|
||||
|
||||
|
||||
def search(cfg, q, exclude_expired):
|
||||
base = (str(cfg.get("base_url") or "").strip() or DEFAULT_BASE).rstrip("/")
|
||||
params = {"q": q, "output": "json"}
|
||||
if exclude_expired:
|
||||
params["exclude"] = "expired"
|
||||
req = urllib.request.Request(base + "/?" + urllib.parse.urlencode(params),
|
||||
headers={"Accept": "application/json", "User-Agent": "riposte-soar"})
|
||||
# crt.sh's database is shared by the whole internet and its front answers
|
||||
# 502/503/504 when a query queues too long; the same query usually goes
|
||||
# through a few seconds later.
|
||||
for attempt in range(4):
|
||||
try:
|
||||
with urllib.request.urlopen(req, timeout=120, context=_ctx(cfg)) as r:
|
||||
raw = r.read()
|
||||
return json.loads(raw) if raw.strip() else []
|
||||
except urllib.error.HTTPError as e:
|
||||
if e.code not in (502, 503, 504) or attempt == 3:
|
||||
raise
|
||||
time.sleep(3 * (attempt + 1))
|
||||
|
||||
|
||||
def _run(fn):
|
||||
try:
|
||||
print(json.dumps(fn(_cfg(), _inputs())))
|
||||
except urllib.error.HTTPError as e:
|
||||
print(json.dumps({"error": "HTTP " + str(e.code), "detail": e.read().decode("utf-8", "replace")[:2000]}))
|
||||
sys.exit(1)
|
||||
except Exception as e:
|
||||
print(json.dumps({"error": str(e)}))
|
||||
sys.exit(1)
|
||||
|
||||
|
||||
def main(cfg, inputs):
|
||||
rows = search(cfg, "crt.sh", True)
|
||||
return {"ok": True, "certificates": len(rows)}
|
||||
|
||||
|
||||
_run(main)
|
||||
Reference in New Issue
Block a user