AutoGPTQ is an LLM quantization package implementing the GPTQ algorithm for weight-only quantization. Note that this project is archived; consider GPTQModel for active maintenance and new model support.
Visit github.com →
This repository provides a practical guide and decision tree to help determine if an LLM is truly necessary for a given problem, often indicating that simpler, more efficient solutions are overlooked. It aims to prevent over-engineering with frontier models.
Pocket TTS is a small English text-to-speech model with voice cloning. It's designed for local deployment and runs efficiently on a laptop's CPU.
This ngrok blog provides a solid technical dive into LLM prompt caching, explaining how it significantly reduces token costs and latency for repeated inputs. It's a really useful breakdown of the underlying mechanics.
DINOv3 is a self-supervised vision model from Meta AI that achieves state-of-the-art performance by scaling training to 1.7B images and 7B parameters without labeled data, providing universal vision backbones.
Jan.ai is an easy-to-use open source AI tooling, providing a local desktop client to run various large language models directly on your hardware. This allows for privacy-focused AI interactions without cloud dependency.
Inception Labs is a diffusion-based LLM. I find its ability to generate entire texts at once intriguing, as this approach promises significantly faster output compared to traditional methods.