Stactora - AI Hardware IndexMy Stack0+ Submit Tool
02 Target GPUs36 VERIFIED ENGINES

Best Local AI Models for 16GB VRAM GPUs (RTX 4080, 4060 Ti 16GB & A4000)

16GB VRAM crosses into real-time 24B agentic LLMs — running Mistral Small 3 (24B at Q4_K_M, 15GB) and high-resolution ComfyUI workflows natively.

TARGET VRAM16 GB VRAM Tier
POPULAR CARDSRTX 4080 · 4060 Ti 16GB
TOP AGENT MODELMistral Small 3 24B (15GB)
Verified Engines for /vram/16gb
36 RESULTSClick any model card to open its dedicated /ai/[slug] page.
01
●LOCAL • 86/100
Llama 3.1 8B Instruct
RESEARCH / AI INDEXL8

Meta's ubiquitous 8B foundation workhorse with 128K context, multilingual capabilities, and massive tool-calling ecosystem support.

BASE ARCHITECTURE / Llama 3.1 8B Instruct
SRuns well (98/100) • Q4_K_M (~141 tok/s • 19GB free)
ℹTHE BRUTAL TRUTH

Still the most integrated small model on earth. Every framework, agent library, and tutorial supports Llama 3.1 8B out of the box.

No cardLocalAPIOpen weightsHF
ILLUSTRATIVE PROMPT

Summarize the key events leading to the 1904 Entente Cordiale in three concise paragraphs.

PREVIEW NOTE

A structured historical summary examining colonial rivalries, Fashoda, and diplomatic reconciliation.

980
06
●LOCAL • 82/100
MiniMax H 3
VIDEO / AI INDEXM3
MiniMax H 3
MiniMaxAI (Open Weights)

Offical skills to improve prompt writing: [skills on github](https://github.com/MiniMax-AI/MiniMax-H3/tree/main/skills)

BASE ARCHITECTURE / MiniMax H 3 (Research Flow Architecture)
SRuns well (87/100) • Q2_K (2-bit Extreme) (~59 tok/s • 12GB free)
ℹTHE BRUTAL TRUTH

Audited in Stactora CMS. Architecture verified with 33.1B parameters.

No cardLocalComfyUIComfy-Org RepackAPIOpen weightsHF
ILLUSTRATIVE PROMPT

A cinematic tracking shot through a bioluminescent forest with cascading waterfalls at dusk, 4k ultra-detailed.

PREVIEW NOTE

Fluid, temporally consistent cinematic video sequence with natural lighting dynamics.

Original Author:MiniMaxAI/MiniMax-H3 ↗
Comfy-Org Repack:Comfy-Org/MiniMax-H3 ↗
5576k
12
●LOCAL • 87/100
Hermes 3 (Llama 3.1 8B)
AGENTS / AI INDEXH8
Hermes 3 (Llama 3.1 8B)

State-of-the-art generalist instruction and roleplay model by Nous Research. Superb function calling, code generation, and multi-turn planning.

BASE ARCHITECTURE / Hermes-3-Llama-3.1-8B
SRuns well (98/100) • Q4_K_M (~141 tok/s • 19GB free)
ℹTHE BRUTAL TRUTH

The best community fine-tune of Llama 3.1 8B. Highly steerable, great at structured tool calling, and free of annoying canned refusals.

No cardLocalAPIOpen weightsHF
ILLUSTRATIVE PROMPT

Plan a 5-step automated penetration test checklist for an OAuth 2.0 authorization code flow with PKCE.

PREVIEW NOTE

A tactical security audit checklist testing token leakage, replay attacks, and state CSRF validation.

760
14
●LOCAL • 88/100
Mistral-Nemo-12B-Instruct
AGENTS / AI INDEXMN
Mistral-Nemo-12B-Instruct
Mistral AI & NVIDIA

Co-developed by Mistral and NVIDIA, this 12B model features Tekken tokenization, a 128K context window, and full Apache 2.0 licensing, fitting easily in 8GB–12GB VRAM.

BASE ARCHITECTURE / Mistral-Nemo-Instruct-2407 (12B)
SRuns well (92/100) • Q4_K_M (~88 tok/s • 16GB free)
ℹTHE BRUTAL TRUTH

The best 12B model in existence. Tekken tokenizer encodes multilingual text with 30% fewer tokens, and it carries true Apache 2.0.

No cardLocalAPIOpen weightsHFPermissive
ILLUSTRATIVE PROMPT

Act as a customer support triage agent: categorize this dispute, identify billing errors, and propose an adjustment.

PREVIEW NOTE

A structured agent evaluation with triage severity, account calculation checks, and credit adjustments.

740
15
●LOCAL • 90/100
Codestral-22B
CODING / AI INDEXCd
Codestral-22B
Mistral AI

Mistral's dedicated 22B open-weight code generation and fill-in-the-middle model with 32K context, fluent in over 80 programming languages.

BASE ARCHITECTURE / Codestral-22B-v0.1
ARuns well (77/100) • Q4_K_M (~50 tok/s • 10GB free)
ℹTHE BRUTAL TRUTH

Superb code generation and fill-in-the-middle accuracy, but the MNPL license prevents production commercial SaaS deployment without contacting Mistral.

No cardLocalAPIOpen weightsHF
ILLUSTRATIVE PROMPT

Implement an LRU cache with O(1) get and put in C++ using std::list and std::unordered_map.

PREVIEW NOTE

An optimized C++17 LRU cache class with lock guards and iterator eviction.

730
16
●LOCAL • 87/100
Stable Diffusion 3.5 Medium (2.5B)
IMAGE / AI INDEXSM

A 2.5B parameter consumer-friendly DiT designed to run on 8GB–10GB consumer GPUs with fast generation times and solid composition.

BASE ARCHITECTURE / Stable Diffusion 3.5 Medium
SRuns well (95/100) • GGUF Q4 (~101 tok/s • 17GB free)
ℹTHE BRUTAL TRUTH

The 8GB sweet spot for SD3.5. Fast and competent, though FLUX.1 Schnell at FP8 still edges it out in overall aesthetic appeal.

No cardLocalAPIOpen weightsHF
ILLUSTRATIVE PROMPT

A retro-futuristic camera sitting on an oak table in warm morning sunlight.

PREVIEW NOTE

A clean 1024x1024 photographic composition with tactile wooden grain textures and brass dials.

720
18
●LOCAL • 88/100
LTX-Video (2B Realtime)
VIDEO / AI INDEXLX

Real-time 2B spatial-temporal video DiT capable of generating 24fps high-resolution video in under 15 seconds on consumer GPUs under Apache 2.0.

BASE ARCHITECTURE / LTX-Video 0.9.1 (2B)
ARuns well (79/100) • BF16 (~78 tok/s • 15GB free)
ℹTHE BRUTAL TRUTH

Generates 24fps video in seconds instead of minutes. On an 8GB card, it feels like magic compared to the painful wait times of older video models.

No cardLocalAPIOpen weightsHFPermissive
ILLUSTRATIVE PROMPT

A golden retriever running through autumn leaves in slow motion.

PREVIEW NOTE

A 24fps video sequence rendered in under 18 seconds with smooth fur movement and flying leaves.

710
19
●LOCAL • 89/100
Ministral-8B-Instruct
AGENTS / AI INDEXM8
Ministral-8B-Instruct

Mistral's edge-computing champion with a 128K context window and interleaved sliding window attention, designed for sub-second agentic decision loops.

BASE ARCHITECTURE / Ministral-8B-Instruct-2410
SRuns well (98/100) • Q4_K_M (~141 tok/s • 19GB free)
ℹTHE BRUTAL TRUTH

Phenomenal edge latency and tool calling accuracy, but restricted by Mistral's research license for production commercial SaaS.

No cardLocalAPIOpen weightsHF
ILLUSTRATIVE PROMPT

Decide whether to execute tool get_weather or search_flights based on query: 'Flight to Boston tomorrow morning'.

PREVIEW NOTE

A structured tool-call payload targeting search_flights with destination 'BOS' and departure date.

705
20
●LOCAL • 86/100
Mistral Small 3
AGENTS / AI INDEXMS
Mistral Small 3
Mistral AI

A 24B Apache-2.0 model tuned for speed and local agentic loops, fitting in 16 GB at 4-bit for real-time workflows.

BASE ARCHITECTURE / Mistral Small 3 (24B)
ARuns well (78/100) • Q4_K_M (~47 tok/s • 9GB free)
ℹTHE BRUTAL TRUTH

The only sub-16GB model on this list that runs real-time agent loops. Trade: weaker reasoning than 27B+ models.

No cardLocalAPIOpen weightsHFPermissive
ILLUSTRATIVE PROMPT

When a new lead arrives, enrich the record and route it to the right owner.

PREVIEW NOTE

Illustrative automation prompt; credentials are not connected.

699
22
●LOCAL • 84/100
Kokoro-82M
AUDIO / AI INDEXKo
Kokoro-82M
Kokoro / community

A compact 82M text-to-speech model suited to local audio pipelines, CPU inference, and lightweight browser prototypes.

BASE ARCHITECTURE / Kokoro-82M
SRuns well (95/100) • ONNX CPU (~220 tok/s • 22GB free)
ℹTHE BRUTAL TRUTH

Runs on CPU, tiny footprint, Apache 2.0 — the only reason not to use it is voice quality. If you need premium voices, pay for ElevenLabs.

No cardLocalOpen weightsHFPermissive
ILLUSTRATIVE PROMPT

Welcome back. Your next idea starts here.

PREVIEW NOTE

Illustrative speech text; no audio is generated by this page.

632
23
●LOCAL • 86/100
InternLM2.5-20B-Chat
AGENTS / AI INDEXIN
InternLM2.5-20B-Chat
Shanghai AI Lab

A 20B parameter model featuring an astounding 1-million-token native context window. Designed for book-length document reasoning and complex agent tools.

BASE ARCHITECTURE / InternLM2.5-20B-Chat
ARuns well (77/100) • Q4_K_M (~54 tok/s • 11GB free)
ℹTHE BRUTAL TRUTH

The only sub-24GB model with a verified 1M context window that doesn't hallucinate past 64K. Watch your KV-cache memory at full lengths.

No cardLocalAPIOpen weightsHFPermissive
ILLUSTRATIVE PROMPT

Search a 500-page codebase transcript for all references to deprecated API route /v1/auth/legacy.

PREVIEW NOTE

A structured cross-reference table indexing timestamps, filenames, and replacement routes.

620
24
●LOCAL • 86/100
StarCoder2-15B
CODING / AI INDEXSC
StarCoder2-15B
BigCode / ServiceNow

Trained on 4+ trillion tokens across 600+ programming languages from The Stack v2. Transparent, permissively trained code generation workhorse.

BASE ARCHITECTURE / StarCoder2-15B (dense)
SRuns well (85/100) • Q4_K_M (~71 tok/s • 14GB free)
ℹTHE BRUTAL TRUTH

Enterprise legal teams love the clean training dataset provenance. Good autocomplete, but shorter context than modern models.

No cardLocalAPIOpen weightsHF
ILLUSTRATIVE PROMPT

Write a Python data validator using Pydantic v2 with custom field serializers.

PREVIEW NOTE

A typed Pydantic BaseModel with @field_validator and model_serializer functions.

612
25
●LOCAL • 86/100
Kolors
IMAGE / AI INDEXKL
Kolors
Kuaishou Technology

Diffusion model based on SDXL and ChatGLM3 text encoding, supporting both English and Chinese text rendering with realistic skin textures and lighting.

BASE ARCHITECTURE / Kolors (2.7B)
ARuns well (79/100) • FP8 (~71 tok/s • 14GB free)
ℹTHE BRUTAL TRUTH

Incredible portrait fidelity and bilingual typography. ChatGLM3 text encoder handles nuances that CLIP misses completely.

No cardLocalAPIOpen weightsHFPermissive
ILLUSTRATIVE PROMPT

A hyper-detailed traditional porcelain craftsman shaping clay in a sunlit Beijing workshop.

PREVIEW NOTE

A lifelike portrait showing clay splatters on artisan fingers and delicate natural ambient lighting.

610
26
●LOCAL • 85/100
AuraFlow-v0.3
IMAGE / AI INDEXAF
AuraFlow-v0.3

An open-source 6.8B parameter text-to-image diffusion transformer model by fal.ai, released under Apache 2.0 with linear attention and fast inference.

BASE ARCHITECTURE / AuraFlow-v0.3 (6.8B)
ARuns well (82/100) • FP8 (~59 tok/s • 12GB free)
ℹTHE BRUTAL TRUTH

A very clean, royalty-free Apache 2.0 DiT. If you dislike Stability's license terms and need an alternative to FLUX, AuraFlow is a real option.

No cardLocalAPIOpen weightsHFPermissive
ILLUSTRATIVE PROMPT

Abstract architectural composition of intersecting glass planes in desert haze.

PREVIEW NOTE

A sharp geometric rendering featuring realistic atmospheric refraction and minimalist shadows.

580
27
●LOCAL • 88/100
MOSS Image Base
IMAGE / AI INDEXMO
MOSS Image Base
OpenMOSS Team

User guide   ·   中文用户指南   ·   ComfyUI

BASE ARCHITECTURE / MOSS Image Base (5.1B Flow-Matching Diffusion Transformer)
SRuns well (97/100) • Q2_K (2-bit Extreme) (~220 tok/s • 22GB free)
ℹTHE BRUTAL TRUTH

Audited in Stactora CMS. Architecture verified with 5.1B parameters.

No cardLocalComfyUIComfy-Org RepackOpen weightsHFPermissive
ILLUSTRATIVE PROMPT

A wide view of a refrigerator beside a kitchen counter with a fruit bowl and a sticky note.

PREVIEW NOTE

Photorealistic generative composition with clear embedded typography.

168k
29
●LOCAL • 85/100
Granite-3.0-8B-Instruct
RESEARCH / AI INDEXIB
Granite-3.0-8B-Instruct

IBM's flagship enterprise open-weight model with 128K context, trained under strict copyright curation and released with full Apache 2.0 licensing and IP indemnity.

BASE ARCHITECTURE / Granite-3.0-8B-Instruct
SRuns well (98/100) • Q4_K_M (~141 tok/s • 19GB free)
ℹTHE BRUTAL TRUTH

Built specifically for enterprise legal departments terrified of training data copyright lawsuits. Solid RAG performance with true Apache 2.0.

No cardLocalAPIOpen weightsHFPermissive
ILLUSTRATIVE PROMPT

Draft an enterprise compliance policy for data classification across Confidential, Internal, and Public.

PREVIEW NOTE

A formal compliance document with handling procedures, access controls, and retention schedules.

530
34
●LOCAL • 91/100
Apex 35B V1.0
RESEARCH / AI INDEXAP
Apex 35B V1.0
Apex Intelligence

Apex Intelligence's 35B MoE language model trained via recursive automated AI research agents. Specialized in computational science, robotics simulation, Claude Code agent workflows, and deep scientific reasoning.

BASE ARCHITECTURE / Apex-35B-V1.0 (Automated Research MoE Architecture)
ARuns well (79/100) • Q4_K_M (~47 tok/s • 9GB free)
ℹTHE BRUTAL TRUTH

Trained by AI agents to perform AI and scientific research. Because only 3B parameters are active per token, it runs like a 3B model while delivering the deep reasoning depth of a 35B architecture on 16GB GPUs.

No cardLocalAPIOpen weightsHFPermissive
ILLUSTRATIVE PROMPT

Formulate a robotics trajectory optimization algorithm with obstacle avoidance constraints.

PREVIEW NOTE

Rigorous mathematical formulation with typed Python simulation code.

390
35
●LOCAL • 84/100
Krea 2 Raw
IMAGE / AI INDEXKR
Krea 2 Raw
Krea (Open Weights)

Access to Krea 2 Raw is gated. Please provide the information below and agree to the license to request access.

BASE ARCHITECTURE / Krea 2 Raw (Image Flow Architecture)
SRuns well (99/100) • Q2_K (2-bit Extreme) (~141 tok/s • 19GB free)
ℹTHE BRUTAL TRUTH

Audited in Stactora CMS. Architecture verified with 12.8B parameters.

LocalComfyUIHF
ILLUSTRATIVE PROMPT

PREVIEW NOTE

88k
36
●LOCAL • 85/100
Fibo 2.0 dev
IMAGE / AI INDEXFI
Fibo 2.0 dev
briaai (Open Weights)

Open-weight foundation model by briaai on Hugging Face Hub with 1,554 30-day downloads and 186 likes.

BASE ARCHITECTURE / Fibo-2.0-dev (8.1B Parameters)
SRuns well (98/100) • Q2_K (2-bit Extreme) (~202 tok/s • 21GB free)
ℹTHE BRUTAL TRUTH

Audited in Stactora CMS. Architecture verified with 8.1B parameters.

LocalHF
ILLUSTRATIVE PROMPT

PREVIEW NOTE

4,200

Frequently Asked Questions · /vram/16gb

What is the best LLM for a 16GB VRAM GPU?
Mistral Small 3 (24B) requires 15GB at Q4_K_M, making it the strongest Apache-2.0 model that fits 100% inside a 16GB card.
Can 16GB VRAM run Qwen 2.5 14B (17GB)?
Yes, with ~1–2 GB of system RAM offload or an IQ4_XS quantization, 27B models run very well on 16GB cards.