Identical behaviour, proven not asserted
Across three scenarios both runtimes reach the same outcome with the same number of model calls and tool calls. A test asserts it, so any future divergence is a real finding.
The same agent,
written twice
A tool calling loop and a LangGraph state graph, running one refund policy against a deterministic model. Press run and watch both work through the same three scenarios.
Both runtimes execute on the server when you press this. Nothing here is a recording.
Three scenarios are waiting: a refund inside the policy window, one outside it, and one where the payment gateway times out on the first attempt. Press run and both runtimes work through them step by step.
identical behaviour · 97 lines against 50 · benchmark gated in CI
A framework opinion without a number attached is a preference. Run it yourself and the argument is over.
What the measurement actually shows
The graph costs 97 lines against 50
Same policy, same model, same evaluation. The only thing that changed is how control flow is expressed, and that is where the whole difference sits.
Deterministic by construction
The model is scripted, so the benchmark measures the runtime rather than model variance. No API key, no network, and the same numbers on every run.
How it works
- 01
One policy
Look up the order, check it against a 30 day refund window, refund or hand off to a human, and retry exactly once when the payment gateway times out.
- 02
Two runtimes
A loop written by hand with one switch statement, and a LangGraph state graph where every edge is declared up front as data.
- 03
One measurement
Model calls, tool calls, tool failures, characters sent and source size, collected per scenario and enforced in CI as a gate.
Source size
Native loop50 lines
LangGraph state graph97 lines
Comments and blank lines excluded. At this size the loop wins and it is not close. What the graph buys does not appear in this chart: its edges are data, so the topology can be inspected, drawn and replayed without reading any node body. That is worth nothing at three steps and quite a lot at thirty.
Questions
- Why is the model scripted instead of live?
- With a live model the numbers move on every run and the benchmark quietly turns into a measurement of model variance. The scripted model encodes one fixed policy, so the runtime is the only variable left.
- Does this prove LangGraph is slower?
- No. It measures architecture at one size, not throughput. Streaming, token cost under a real model, checkpointing and concurrency are all outside the scope of these three scenarios.
- So when is a graph worth the extra code?
- When branching becomes real: several terminal states, parallel branches, or a human in the loop pause that has to survive a process restart. At that point the implicit control flow of a loop is what breaks first.
- What runs when I press the button?
- A request hits a Node route that executes both runtimes, the real LangGraph graph included, and returns the transcripts. The page then walks through them step by step.
- Can I reproduce these numbers?
- Clone the repository and run npm test and npm run bench. The benchmark prints the same table you see here, and CI fails the build if any runtime stops reaching the expected outcome.
- Who built this?
- Huseyin Kaplan, a full stack and AI engineer in Istanbul who builds production LLM systems in TypeScript and Python.