From Dev.to
Cheapest per token is not cheapest per finished job: what 2,400 runs show
A model with a 33% pass rate turned out to be the cheapest way to finish a task in a recent 10-model benchmark. The same benchmark shows why that result is less of a free lunch than it looks. Here is...