New models tested: Sol 6, Mistral Large 4, MiniMax M3, and Gemini Flash-Lite
Newer does not always mean better. Sol cuts cost and latency, Luna beats MiniMax on quality and cost, and Gemini Flash-Lite leads on speed.
We test models on synthetic tasks and production workloads, then publish what we measure: what each one costs, how fast it is, and whether the output does the job.
Newer does not always mean better. Sol cuts cost and latency, Luna beats MiniMax on quality and cost, and Gemini Flash-Lite leads on speed.
Terra 6.1 improves coding quality and cost. Kimi K3.1 Pro and Highspeed lead on selected tasks, but their cost and latency are hard to justify.
Haiku 5.5 is a useful specialist, DeepSeek 4.1 Flash 1017 is faster but not fast enough, and Gemini 4 Argon is a solid upgrade over Gemini 3.8 Flash.
14 frontier and high-efficiency models measured on six software factory workloads. Frontier models lead on design; on coding, review, CI triage and pull requests, high-efficiency models match them at a fraction of the cost.
13 low-cost models measured on production workloads: quality, cost and latency against the field, and the format failures that rule a model out.
How to choose where to run an LLM. The same open-weight model can cost up to 14x more depending on the provider, and the cheapest endpoint may run a lower-precision copy. Built from 1,317 live provider endpoints.
Five AI models answered the same startup-studio tasks. Pick your preferred responses blindly, then see what each cost. An interactive case study with Pioneer Square Labs.
We used libfx to give 11 models the same business tasks, check their answers, and compare cost per successful task. Here is what the bills showed.
Ten inexpensive models, three production workloads, identical inputs. We scored quality, latency and cost, then ranked each model against its peers on the two ratios that decide a deployment.
