BleedStop

Cost abuse control plane for LLM, SMS and payment APIs (OWASP LLM10). Multi tenant policies, evals, Redis scale, Postgres, Helm/K8s.

GitHub repoRaw resultsEnterprise overview Live demo

Try the live demo

curl -s -X POST https://bleedstop-demo.vercel.app/decide \
  -H "Content-Type: application/json" -H "X-API-Key: demo-public-key" \
  -d '{"actorId":"demo-user-1","action":"llm.call","paid":false,"estimatedTokens":500,"signals":{"deviceFingerprint":"fp-demo-1"}}'
# ALLOW; with "estimatedTokens":400000 the answer is DENY (unpaid token cap 32000)

Public demo only key, tenant "demo". Runs on Vercel Hobby serverless functions with in memory state and no Redis or Postgres, so budgets, velocity windows and rate limits can reset on cold starts or across instances; the $5 cluster ceiling is not reliably enforced here. The measured results below come from the Docker Compose and Kubernetes stack with Redis. Admin endpoints require login.

How it works

Apps call POST /decide before spending. BleedStop returns ALLOW | LIMIT | CHALLENGE | DENY from deterministic rules. Budgets are enforced per identity cluster (device, normalised email), atomically across instances with a Redis Lua check and charge. Agent calls are pre checked for injection, exfiltration, tool loops and runaway recursion. A reconciler compares the app ledger to vendor usage and kills leaked keys (ghost spend).

Your apps (Node SDK, Python SDK, agents)
   | POST /decide (X-API-Key -> tenant)
   v
Load balancer -> BleedStop x N
   rate limit -> validate -> tenant policy (YAML) -> idempotency
   -> identity cluster -> network signals (/24, /64, ASN)
   -> agent guard -> rule engine -> atomic charge
   |                         |                      |
 Redis (ceilings,       Postgres (ledger,      /metrics, OTLP traces,
 identity graph)        audit, policies)       JSON logs
   v  only if ALLOW / LIMIT
LLM / SMS / payments  --usage-->  reconciler -> kill switch

Measured results

All numbers from one 8 vCPU box with a mock LLM upstream, no real provider calls. Raw JSON in results/.

TestResult
3 instances + Redis atomic + LB6,291 decisions/s, p50 9.0 ms, p99 31.9 ms, 0 errors, max overspend $0.000 vs $5 ceiling
Kubernetes rollout restart of 6 pods under load (90 s)287,403 requests, 0 failed, p99 33.2 ms, rollout 18 s
Docker Compose, Postgres durable modemedian 5,433 decisions/s, median p99 36.2 ms, 0 failed of 60,000
Postgres stopped ~7 s mid load86,071 requests, 0 failed; 86,071 of 86,071 ledger rows persisted
Golden set evals, 12,140 labeled requestsprecision 100.0%, benign FP rate 0.0%, recall 48.0% (same result via LB and in K8s)
Attack suite (3 runs, median)token flood 400k chars 100% blocked; farm on one device 97%; rotated signals in one /24 66.7%; retry storm 94%; same idempotency key 99.9% upstream calls prevented; normal users 0% blocked
Agent guarddev set 96.4% detection; held out set 22.5% detection (never tuned on); FP 0.0%; tool sequences 100% / 0% FP
Added latency via SDKp50 1.16 ms, p95 1.43 ms

Architecture decision records

  1. Deterministic rules on the hot path
  2. Budgets per identity cluster
  3. IP and ASN are pool/velocity signals, never identity unions
  4. Redis Lua check and charge and a shared identity graph
  5. OpenTelemetry tracing next to Prometheus
  6. Per tenant policy in YAML, validated at boot
  7. Harden the control plane itself
  8. Deterministic agent guard; held out numbers are the ones we quote
  9. Per action fail policy and graceful Redis degradation

Not a hosted API: run locally with npm install && npm start or the Docker Compose / Helm stack in the repo.