This category compiles practical AI tools and resources, spanning LLMs, vision, and audio. It features solutions for local deployment, optimizing model efficiency, and robust evaluation. I curate these links for developers and engineers focused on building, deploying, and maintaining real-world AI applications, often leveraging open-source projects to avoid cloud dependencies.
This repository provides a practical guide and decision tree to help determine if an LLM is truly necessary for a given problem, often indicating that simpler, more efficient solutions are overlooked. It aims to prevent over-engineering with frontier models.
Pocket TTS is a small English text-to-speech model with voice cloning. It's designed for local deployment and runs efficiently on a laptop's CPU.
This ngrok blog provides a solid technical dive into LLM prompt caching, explaining how it significantly reduces token costs and latency for repeated inputs. It's a really useful breakdown of the underlying mechanics.
DINOv3 is a self-supervised vision model from Meta AI that achieves state-of-the-art performance by scaling training to 1.7B images and 7B parameters without labeled data, providing universal vision backbones.
Jan.ai is an easy-to-use open source AI tooling, providing a local desktop client to run various large language models directly on your hardware. This allows for privacy-focused AI interactions without cloud dependency.
Inception Labs is a diffusion-based LLM. I find its ability to generate entire texts at once intriguing, as this approach promises significantly faster output compared to traditional methods.
FaceFusion is a powerful, self-hostable face manipulation platform, primarily for swapping faces in videos. Its installation and usage require technical skills and comfort with command-line operations.
DeepFilterNet is an open-source, low-complexity speech enhancement framework for 48kHz full-band audio. I appreciate its real-time noise reduction, pre-compiled binaries, and LADSPA plugin for PipeWire.
This free online book provides a concise, technical introduction to Reinforcement Learning from Human Feedback (RLHF) and post-training methods, primarily focusing on their application in language models.
This CMU course covers the entire lifecycle of building, deploying, assuring, and maintaining production-grade software products with machine-learned models, emphasizing responsible AI and MLOps.
This page details the Ragas evaluation metrics, crucial for objectively measuring and developing RAG systems and other LLM applications. I find this list essential for anyone serious about improving their LLM application's performance.
This is NSFW JS, a library for client-side indecent content checking. It uses TensorFlow.js for efficient, browser-based NSFW detection, which is quite clever.
Opik provides comprehensive LLM tracing and observability, supporting the full lifecycle from development to production. It helps evaluate, test, monitor, and optimize generative AI applications.
PaperQA2 is an agentic RAG tool explicitly designed for high-accuracy performance on scientific literature, including PDFs and text. It aims to deliver superhuman results in tasks like Q&A and summarization, complete with in-text citations.
Marker is a solid AI library for converting PDFs and other documents to markdown, JSON, or HTML. I appreciate its high accuracy, even with complex layouts, and strong performance benchmarks against commercial alternatives.
Flux Gym provides a local web UI for FLUX LoRA training. It's built on Kohya Scripts, offering broad feature support with low VRAM (12GB/16GB/20GB) compatibility, which is a significant advantage for local development.
Pinokio is a local platform for easily installing, running, and managing AI applications and other server-side tools on your machine. I see it as a streamlined installer and manager, simplifying the local deployment of complex AI setups.
BAIOSPHERE is Bavaria's AI network, orchestrating events, research, and practical AI application initiatives across the region. It is significantly integrated with local universities, fostering the entire AI ecosystem from academia to industry.
Fish Speech is an impressive open-source AI model for voice cloning and advanced text-to-speech. It offers fine-grained control and multilingual capabilities, setting a high bar for quality among both open and closed source systems.
This Hugging Face Space is intended to quantize any LLM repository into the GGUF format. However, it's currently failing with a build error, specifically exit code 143.
R2R provides a comprehensive RAG framework for building and deploying scalable Retrieval-Augmented Generation applications. It features multimodal ingestion, hybrid search, and a robust RESTful API.
I've found SAMURAI to be a promising zero-shot video segmentation model, building on SAM 2.1 with motion-aware memory via Kalman filters. It seems to outperform SAM2, offering improved visual tracking capabilities.
ContextCite helps find the exact context snippets for LLM responses, providing a crucial capability for advanced RAG. It addresses the risk of LLMs inventing citations by attributing generated statements directly to source material.
Exo lets you build a local AI cluster using everyday devices like Mac Minis, intelligently connecting them for larger models and faster inference via RDMA over Thunderbolt.
Smollm by Hugging Face is a family of efficient, lightweight AI models. SmolLM3, their 3B language model, performs exceptionally well for English text, even outperforming some 4B alternatives, making it great for on-device usage.
This is a lightweight and easy-to-hack GraphRAG implementation. I find it provides a smaller, faster, and cleaner core GraphRAG functionality, with about 1100 lines of code excluding tests and prompts. It's portable, asynchronous, and fully typed, supporting various backends.
HippoRAG 2 is a graph-based memory framework for LLMs, enhancing long-term memory and complex context handling via personalized PageRank, as described in their research papers. I find its focus on multi-hop retrieval and sense-making crucial.
This is a fast GraphRAG framework focused on entity extraction for interpretable, high-precision retrieval. It significantly reduces costs compared to other GraphRAG solutions and supports dynamic, incrementally updated knowledge graphs.
Artificial Analysis provides independent, data-driven benchmarks for AI models and API providers. I find their Intelligence Index and comparisons for speed and cost particularly useful for evaluating the current landscape.
ScrapeGraphAI is a Python AI website scraper. It leverages LLMs and graph logic to define scraping pipelines, allowing me to extract specific information from web pages and documents with natural language prompts.
This open-source, self-hosted API converts PDFs, images, and Office documents to Markdown or structured JSON. It leverages local OCR and LLMs (like Ollama) for high accuracy, ensuring data privacy without external dependencies.
Thorsten Voice is a free and open-source German Text-to-Speech engine. It provides high-quality AI-driven voices that can be used offline and locally, promoting digital sovereignty without reliance on large platforms.
Midship is an AI tool designed to automate SOX testing by extracting structured data from various documents, allowing agents to follow audit plans and create fully documented work papers. I see this as a way to streamline compliance and reduce manual effort.
Subtitles.org is my go-to for downloading movie and TV series subtitles. It's particularly useful for feeding textual content to LLMs for analysis or experimentation.
I find The AI Timeline useful for tracking new AI research. It consistently posts weekly summaries of significant papers, making it an efficient way to stay updated on the latest developments.
This library, `dom-to-semantic-markdown`, converts HTML DOM into a semantic Markdown format specifically for LLM consumption. It optimizes token usage by preserving crucial semantic structure and extracting metadata, which is essential for accurate LLM processing.
SAM 2 is Facebook AI's latest, extending the Segment Anything Model to both images and video. It's built on a simple transformer with streaming memory, designed for real-time video segmentation.
AirLLM is a Python library that optimizes LLM inference memory, enabling 70B models on a single 4GB GPU without traditional performance-degrading compression techniques. This is a solid approach for resource-constrained environments.
vLLM is an efficient system for LLM inference and serving, designed to run models across multiple GPUs and computers. It achieves high throughput through features like PagedAttention and continuous batching.
This is the Gradio web UI I use for running various Large Language Models locally. It offers robust support for text generation, vision, tool-calling, and even local fine-tuning and image generation.
llamafile is an impressive tool that lets you distribute and run LLMs with a single file. It bundles llama.cpp and Cosmopolitan Libc into one executable, eliminating installation complexity across platforms.
Axolotl is a robust tool I use to streamline the fine-tuning of various AI models, abstracting away much of the complexity for more accessible and efficient custom model training.
Surya is a robust document OCR toolkit with impressive multi-language support (90+). It handles line-level text detection, layout analysis, reading order, and table recognition, often outperforming cloud services.
Texify is an OCR model for converting images and PDFs containing mathematical content into Markdown and LaTeX, rendering it with MathJax. It was designed to handle both block and inline equations.
This tool provides client-side token counting and up-to-date USD price estimations for over 400 LLMs, crucial for cost management in LLM applications. It helps track frequently changing provider pricing.
This is an API for LLMs to programmatically read web content. It converts a URL into LLM-friendly input, useful for agents requiring web access.
LLM Compare provides a visual way to find the best bang for buck in LLMs by comparing token prices against quality, using the LMSYS Chatbot Arena Leaderboard rank. I find it useful for identifying high-performing, cost-effective models.
supervision is an open-source Python package providing core utilities for computer vision. It offers model-agnostic tools for dataset management, data annotation, and result visualization, streamlining common CV workflows.
ImageBind is truly a 'One Embedding Space To Bind Them All' solution. It learns a joint representation across six modalities—images, text, audio, depth, thermal, and IMU data—enabling powerful cross-modal applications.
Tarsier is a library I'd use to prepare web page content for LLM interaction. It generates a simplified, tagged text representation of a page, including interactable elements, which helps even text-only LLMs understand visual structure and perform actions.
This is a free, community-driven Hugging Face course covering computer vision fundamentals to advanced topics like Vision Transformers and multimodal models. It requires Python and ML basics.
DeepLearning.AI offers robust machine learning and AI courses and specializations, often developed by industry leaders like Andrew Ng. It's a valuable resource for anyone seeking to build or advance their practical AI skills.
This HuggingFaceM4 collection features Idefics2, a robust vision-language model. I find it particularly useful for tasks involving visual parsing, like converting screenshots to structured JSON or HTML.
This is a comprehensive, free deep reinforcement learning course, offering a hands-on approach from foundational algorithms like DQN and PPO to advanced topics such as AlphaZero and RLHF, all within an opinionated, Dockerized environment.
This domain, gpudeploy.com, was an "Airbnb for GPUs" where you could rent idle GPUs cheaply. It's a solid concept for accessing affordable compute.
databonsai is a Python library for LLM-powered data cleaning and categorization, including transformation and extraction. Its adaptive batching and retry logic for token savings and error handling are particularly useful.
LlamaIndex is a solid RAG platform for complex document processing. It leverages agentic OCR and provides a ton of ready-made plugins for end-to-end AI workflows and enterprise automation.
This is an Ollama-compatible DPO-aligned merge of German models, scoring 7.11 on the mt-bench-de average, useful for German chat applications. I find it a solid option for local LLM experiments.
Mergekit is the toolkit I'd use for merging pre-trained large language models. It efficiently combines various models, even on resource-constrained hardware, allowing complex merges to create new, optimized single models.
I find LLaMA-Factory to be a robust framework for easily fine-tuning over 100 large language models via CLI or Web UI. It simplifies what can often be a complex process.
LumiOpen hosts European foundation models, specifically from the University of Turku, Finland. They offer models like Llama-Poro-2-Instruct, valuable for regional NLP tasks.
LLaMa2lang offers scripts to finetune foundation models like LLaMa3 for non-English chat. It enables translating existing datasets and applying QLoRA/PEFT, including DPO/ORPO, to adapt models to other languages where English performance is typically poor.
AutoGPTQ is an LLM quantization package implementing the GPTQ algorithm for weight-only quantization. Note that this project is archived; consider GPTQModel for active maintenance and new model support.
This MTEB Leaderboard provides a massive text embedding benchmark, enabling objective comparison of models for various NLP tasks. It's a key resource for selecting performant embedding architectures.
I'm using Upscayl, an impressive open-source and free AI image upscaler. It effectively enhances image resolution up to 16x, making it a solid tool for improving low-res assets.
OpenFace is a Python/Torch library for face recognition using deep neural networks, providing 128-dimensional face embeddings. It's a robust solution for finding face similarity and recognition tasks.
DeepFace is a lightweight Python framework for face recognition and facial attribute analysis (age, gender, emotion, race). It's a hybrid system wrapping multiple state-of-the-art models like FaceNet and VGG-Face, simplifying a complex pipeline into single-line function calls.
This Udemy course focuses on optimizing YOLOv8 models for speed and accuracy. It details architecture modifications, TensorRT, OpenVINO, quantization, and hyperparameter tuning, which are essential techniques for real-world deployment.
DeepSeek-VL is an open-source, general-purpose vision-language model. I see it as a strong contender for applications requiring robust understanding of complex visual and textual information, from diagrams to natural images.
CVAT is an open-source data annotation platform, crucial for fine-tuning object detection models. I find it a solid choice for preparing machine learning datasets.
This is a CLI tool for ridiculously fast on-device Whisper audio transcription. It leverages optimizations like Flash Attention 2 for rapid processing on GPUs and Apple Silicon Macs.
Artbot offers free, hosted Stable Diffusion, leveraging the AI Horde for easy experimentation with generative AI art. No setup or registration needed.
Danswer is an open-source tool for building RAG applications over internal company data. It integrates with various internal tools like Slack, GitHub, and Confluence to provide natural language answers from private sources.
Lexica is a free prompt gallery that helps you discover and generate AI art. I use it to find inspiration for image styles and prompts, then adapt them for my own creations.
This is the definitive Deep Learning book from Goodfellow, Bengio, and Courville. I refer to it for a solid foundation in both theoretical and practical aspects of deep learning.
This is a comprehensive guide for prompt engineering, covering techniques, resources, and best practices for developing and optimizing prompts for large language models.
XTTS by coqui is a fascinating voice cloning model that translates your voice into any language. This is highly interesting for automagically translating podcasts or YouTube videos, preserving the original speaker's vocal characteristics.
This free MIT lecture series (6.5940) focuses on optimizing deep learning models for size and speed. It's essential for deploying efficient models on local or resource-constrained devices.
Bnomial delivers a daily machine learning question. I find this a straightforward and effective method for consistent learning, helping me maintain and sharpen my ML knowledge.
This article breaks down deep learning performance from first principles, explaining the core bottlenecks: compute, memory bandwidth, and overhead. It's a practical guide to understanding why your GPUs might be slow.
This is a Rust library, `fast_vector_similarity`, for high-performance computation of distance and similarity measures between vectors. It's useful for ML and data analysis tasks, offering various algorithms and Python bindings.
PySceneDetect is a Python library I use for detecting scene changes and shot boundaries in videos. It's a robust tool for automatically splitting video into individual clips based on fast-cuts and fades.
This is a solid, free prompt engineering guide, covering techniques for both LLMs like ChatGPT and image generation with Midjourney. It's a practical resource for improving prompt efficacy.
This site is a straightforward resource for finding grants, credits, and investments specifically targeted at AI projects. It offers a curated list of funding opportunities to help identify financial support.
Splink is a Python library for probabilistic record linkage. It's designed to deduplicate and link datasets lacking unique identifiers, leveraging Fellegi-Sunter's model and offering scalable execution.
This blogpost provides a practical guide on building a scaled-down Llama model from scratch, using TinyShakespeare. It offers a structured, iterative approach to implementing complex papers, which I find highly valuable for understanding deep learning architectures.
Another gallery of prompts for major AI APIs
LangChain is a framework I use to standardize interaction with various LLM APIs and orchestrate them into agents. It helps abstract away provider-specific implementations, enabling complex AI application development.
I outline five key learnings from classifying 500k customer messages using LLMs versus traditional ML, offering insights into practical text classification challenges encountered at scale.
This article details how to leverage vector SQL to retrieve factual information from a database, preventing LLM hallucinations. It provides a structured approach for LLMs to query external knowledge sources.
Andrej Karpathy's YouTube channel is a solid resource for diving into deep learning concepts. I find his explanations insightful for understanding modern AI architectures.
This is an excellent technical breakdown by Stephen Wolfram explaining ChatGPT's underlying mechanism, specifically how it generates text by predicting the next token and the critical role of "temperature" in its output's creativity.
This is OpenAI's official cookbook, providing example code and guides to better leverage their API for common tasks. It's essential documentation if you're serious about effective interaction with models like ChatGPT.
This is an in-depth PDF resource I recommend for learning how to effectively craft prompts for DALL-E 2. It provides technical guidance on generating specific image styles and compositions.
PromptBetter offers robust A/B testing for OpenAI prompts, crucial for optimizing LLM performance in production. I appreciate its focus on coders for managing and tracking prompt variations effectively.
Promptbase is a marketplace for buying and selling prompts across various major AI models like Midjourney and ChatGPT. I find it useful as a curated gallery of pre-tested prompts for exploring specific styles or generating ideas.
This Coursera specialization is Andrew Ng's updated ML intro, covering core supervised and unsupervised techniques, neural networks, and practical application. It's a solid starting point for anyone looking to build a foundational understanding of machine learning.
This is arguably the best entry point for practical deep learning. It's a free course from fast.ai, designed for coders to apply deep learning and ML to real-world problems using PyTorch and the fastai library.
Milvus is a high-performance vector database, primarily written in Go and C++. I see this as a solid choice for scalable AI applications needing efficient unstructured data search via vectors.
This GitHub repo is a comprehensive guide to prompt engineering, aggregating papers, learning resources, and practical techniques for effectively using large language models. I find it a solid reference.
This Google-produced series is an amazing high-level introduction to machine learning concepts. It's a great starting point for anyone looking to get a solid conceptual overview.
EleutherAI is a collective of researchers dedicated to open-source AI development. They focus on training and releasing large language models, alongside research into model interpretability and safety.