From Dev.to
ExLlamaV3 and TabbyAPI - the pair that ships today, and what it asks of your hardware
ExLlamaV3 and TabbyAPI are a pair documented by both projects: exllamav3's repository blurb reads "An optimized quantization and inference library for running LLMs locally on modern consumer-class GPU...