From Dev.to
Ten engines for one box: which local inference engine fits a 100 GB rig with two 16 GB cards?
This blog has published notes on ten inference engines — llama.cpp, Ollama, vLLM, Sonar (formerly Aphrodite Engine), exo, ExLlamaV3 + TabbyAPI, Strata, PolyStrata, Avalon and Xolotl. This is the decis...