Why we killed the AI report (and made it free)

We used to generate reports with a large language model. They read beautifully, took minutes, and cost about two dollars each. We replaced them with a deterministic rules engine that runs in seconds and costs nothing. Here’s why — and what we learned.

The three problems money couldn’t fix

1. Confidence without sample. The model would happily compare an ad with 3 clicks to one with 3,000 and declare a winner. You can prompt against this all day; some percentage of the time, eloquence beats arithmetic.

2. Different answers to the same question. Run the same account twice, get two subtly different reports. For a tool whose whole promise is “the truth about your ads”, that’s disqualifying.

3. Cost that scales with your success. Two dollars per report is fine for ten users. At a thousand users generating reports weekly, the economics push you to sample, shorten, and degrade — quietly.

What replaced it

About forty-five explicit criteria, each with a threshold, a reason, and a test: sample-size gates, within-account benchmarks, video retention analysis, trend detection over daily data, audience segments, fraud signatures, cross-checks against Google Analytics. Same input, same report, every time — and each claim ships with the numbers behind it, so you can check us.

Where AI still earns its seat

Language. Ask the report chat “why should I kill the Laktotek ads?” and a model answers — grounded in your already-computed report. Semantics is what these models are for. Arithmetic never was.

The result: reports went from minutes to seconds, from two dollars to free, and — the part we actually care about — from “sounds right” to checkable.