Lakuna · docs

Fine-tuning signal (wall detector)

Escalating to cloud is not a failure, it is a data point. app/lib/wall-detector.ts's computeWallVerdictanswers a question the raw usage numbers don't: is the local tier actually getting better, or has it plateaued despite a growing strategy library? It splits a key's usage history in half by time, compares the escalation rate (cloud / (local + cloud)) of each half, and returns a verdict - never a bare label, always with the real numbers behind it.

With fewer than about 20 total requests, or all of them in one half, it says so plainly instead of guessing. Otherwise: if the newer half's escalation rate hasn't improved by at least 5 percentage points, it flags "diminishing returns"- a signal that the growing strategy library isn't meaningfully improving local-tier success anymore, and this key might be a good candidate for actually fine-tuning a local model on its own traffic (not built yet - see the README roadmap). Otherwise it reports the rate is improving.

wall-detector.ts
computeWallVerdict(apiKey): Promise<WallVerdict>
  { triggered, label, reason, inputsUsed }

older, newer = split(usage.series)
rate = cloud / (local + cloud)

newer.rate - older.rate < 5pp
  -> { triggered: true,  label: "diminishing returns", reason }
  -> { triggered: false, label: "improving",           reason }

inputsUsed

Every verdict ships the raw numbers it was computed from, not just the label:

  • escalation rate in the older half of the key's history
  • escalation rate in the newer half
  • how much the escalation rate improved, in percentage points
  • how many strategies are stored in the shared corpus
  • total request count (gates the verdict until there's enough data)

Served at GET /api/wall-detector (session auth, same pattern as /api/usage) and shown on the dashboard as a "Fine-tuning signal" card.