Stactora - AI Hardware IndexMy Stack0+ Submit Tool
02 Target GPUs44 VERIFIED ENGINES

Best Local AI Models for 32GB VRAM GPUs (RTX 5090 & 32GB Unified Memory)

Next-gen flagship memory tier. Run 30B–32B models at near-lossless Q8_0 precision or large 24B–30B models with huge 64K+ context caches.

TARGET VRAM32 GB Next-Gen Tier
REPRESENTATIVE GPUSRTX 5090 · 32GB Mac
SWEET SPOTQ8_0 32B Near-Lossless
Verified Engines for /vram/32gb
44 RESULTSClick any model card to open its dedicated /ai/[slug] page.
02
●LOCAL • 86/100
Llama 3.1 8B Instruct
RESEARCH / AI INDEXL8
Llama 3.1 8B Instruct

Meta's ubiquitous 8B foundation workhorse with 128K context, multilingual capabilities, and massive tool-calling ecosystem support.

BASE ARCHITECTURE / Llama 3.1 8B Instruct
SRuns well (98/100) • Q4_K_M (~141 tok/s • 19GB free)
ℹTHE BRUTAL TRUTH

Still the most integrated small model on earth. Every framework, agent library, and tutorial supports Llama 3.1 8B out of the box.

No cardLocalAPIOpen weightsHF
ILLUSTRATIVE PROMPT

Summarize the key events leading to the 1904 Entente Cordiale in three concise paragraphs.

PREVIEW NOTE

A structured historical summary examining colonial rivalries, Fashoda, and diplomatic reconciliation.

980
07
●LOCAL • 82/100
MiniMax H 3
VIDEO / AI INDEXM3
MiniMax H 3
MiniMaxAI (Open Weights)

Offical skills to improve prompt writing: [skills on github](https://github.com/MiniMax-AI/MiniMax-H3/tree/main/skills)

BASE ARCHITECTURE / MiniMax H 3 (Research Flow Architecture)
SRuns well (87/100) • Q2_K (2-bit Extreme) (~59 tok/s • 12GB free)
ℹTHE BRUTAL TRUTH

Audited in Stactora CMS. Architecture verified with 33.1B parameters.

No cardLocalComfyUIComfy-Org RepackAPIOpen weightsHF
ILLUSTRATIVE PROMPT

A cinematic tracking shot through a bioluminescent forest with cascading waterfalls at dusk, 4k ultra-detailed.

PREVIEW NOTE

Fluid, temporally consistent cinematic video sequence with natural lighting dynamics.

Original Author:MiniMaxAI/MiniMax-H3 ↗
Comfy-Org Repack:Comfy-Org/MiniMax-H3 ↗
5576k
08
●LOCAL • 91/100
Gemma 2 27B
CODING / AI INDEXG2
Gemma 2 27B
Google DeepMind

Google DeepMind's flagship 27.2B open model featuring alternating local/global attention and post-training alignment, punching above its weight class on single-GPU hardware.

BASE ARCHITECTURE / Google Gemma 2 27B (Instruct)
BRuns well (60/100) • Q4_K_M (~39 tok/s • 6GB free)
ℹTHE BRUTAL TRUTH

Google's 27B architecture punches way above its parameter weight. If 8K context is sufficient for your workload, it is an outstanding 24GB driver.

No cardLocalAPIOpen weightsHF
ILLUSTRATIVE PROMPT

Implement a rate-limited API endpoint with structured errors.

PREVIEW NOTE

Robust implementation with structured error handling.

890
15
●LOCAL • 87/100
Hermes 3 (Llama 3.1 8B)
AGENTS / AI INDEXH8
Hermes 3 (Llama 3.1 8B)
Nous Research

State-of-the-art generalist instruction and roleplay model by Nous Research. Superb function calling, code generation, and multi-turn planning.

BASE ARCHITECTURE / Hermes-3-Llama-3.1-8B
SRuns well (98/100) • Q4_K_M (~141 tok/s • 19GB free)
ℹTHE BRUTAL TRUTH

The best community fine-tune of Llama 3.1 8B. Highly steerable, great at structured tool calling, and free of annoying canned refusals.

No cardLocalAPIOpen weightsHF
ILLUSTRATIVE PROMPT

Plan a 5-step automated penetration test checklist for an OAuth 2.0 authorization code flow with PKCE.

PREVIEW NOTE

A tactical security audit checklist testing token leakage, replay attacks, and state CSRF validation.

760
17
●LOCAL • 85/100
DeepSeek-R1-Distill-32B
RESEARCH / AI INDEXDS
DeepSeek-R1-Distill-32B

A distilled reasoning model that preserves chain-of-thought behaviour in a size that fits on a 24 GB GPU at 4-bit.

BASE ARCHITECTURE / DeepSeek-R1-Distill 32B
BRuns well (61/100) • Q4_K_M (~35 tok/s • 4GB free)
ℹTHE BRUTAL TRUTH

MIT license + real reasoning traces + fits in 24GB is a rare combo. Watch token burn — reasoning uses 3-5x more output tokens than a normal model.

No cardLocalAPIOpen weightsHFPermissive
ILLUSTRATIVE PROMPT

Explain the trade-offs between RAG and long-context retrieval for a 200K-token corpus.

PREVIEW NOTE

Illustrative reasoning prompt; not executed here.

741
18
●LOCAL • 88/100
Mistral-Nemo-12B-Instruct
AGENTS / AI INDEXMN
Mistral-Nemo-12B-Instruct
Mistral AI & NVIDIA

Co-developed by Mistral and NVIDIA, this 12B model features Tekken tokenization, a 128K context window, and full Apache 2.0 licensing, fitting easily in 8GB–12GB VRAM.

BASE ARCHITECTURE / Mistral-Nemo-Instruct-2407 (12B)
SRuns well (92/100) • Q4_K_M (~88 tok/s • 16GB free)
ℹTHE BRUTAL TRUTH

The best 12B model in existence. Tekken tokenizer encodes multilingual text with 30% fewer tokens, and it carries true Apache 2.0.

No cardLocalAPIOpen weightsHFPermissive
ILLUSTRATIVE PROMPT

Act as a customer support triage agent: categorize this dispute, identify billing errors, and propose an adjustment.

PREVIEW NOTE

A structured agent evaluation with triage severity, account calculation checks, and credit adjustments.

740
19
●LOCAL • 90/100
Codestral-22B
CODING / AI INDEXCd
Codestral-22B
Mistral AI

Mistral's dedicated 22B open-weight code generation and fill-in-the-middle model with 32K context, fluent in over 80 programming languages.

BASE ARCHITECTURE / Codestral-22B-v0.1
ARuns well (77/100) • Q4_K_M (~50 tok/s • 10GB free)
ℹTHE BRUTAL TRUTH

Superb code generation and fill-in-the-middle accuracy, but the MNPL license prevents production commercial SaaS deployment without contacting Mistral.

No cardLocalAPIOpen weightsHF
ILLUSTRATIVE PROMPT

Implement an LRU cache with O(1) get and put in C++ using std::list and std::unordered_map.

PREVIEW NOTE

An optimized C++17 LRU cache class with lock guards and iterator eviction.

730
20
●LOCAL • 87/100
Stable Diffusion 3.5 Medium (2.5B)
IMAGE / AI INDEXSM

A 2.5B parameter consumer-friendly DiT designed to run on 8GB–10GB consumer GPUs with fast generation times and solid composition.

BASE ARCHITECTURE / Stable Diffusion 3.5 Medium
SRuns well (95/100) • GGUF Q4 (~101 tok/s • 17GB free)
ℹTHE BRUTAL TRUTH

The 8GB sweet spot for SD3.5. Fast and competent, though FLUX.1 Schnell at FP8 still edges it out in overall aesthetic appeal.

No cardLocalAPIOpen weightsHF
ILLUSTRATIVE PROMPT

A retro-futuristic camera sitting on an oak table in warm morning sunlight.

PREVIEW NOTE

A clean 1024x1024 photographic composition with tactile wooden grain textures and brass dials.

720
22
●LOCAL • 88/100
LTX-Video (2B Realtime)
VIDEO / AI INDEXLX
LTX-Video (2B Realtime)
Lightricks

Real-time 2B spatial-temporal video DiT capable of generating 24fps high-resolution video in under 15 seconds on consumer GPUs under Apache 2.0.

BASE ARCHITECTURE / LTX-Video 0.9.1 (2B)
ARuns well (79/100) • BF16 (~78 tok/s • 15GB free)
ℹTHE BRUTAL TRUTH

Generates 24fps video in seconds instead of minutes. On an 8GB card, it feels like magic compared to the painful wait times of older video models.

No cardLocalAPIOpen weightsHFPermissive
ILLUSTRATIVE PROMPT

A golden retriever running through autumn leaves in slow motion.

PREVIEW NOTE

A 24fps video sequence rendered in under 18 seconds with smooth fur movement and flying leaves.

710
23
●LOCAL • 89/100
Ministral-8B-Instruct
AGENTS / AI INDEXM8
Ministral-8B-Instruct
Mistral AI

Mistral's edge-computing champion with a 128K context window and interleaved sliding window attention, designed for sub-second agentic decision loops.

BASE ARCHITECTURE / Ministral-8B-Instruct-2410
SRuns well (98/100) • Q4_K_M (~141 tok/s • 19GB free)
ℹTHE BRUTAL TRUTH

Phenomenal edge latency and tool calling accuracy, but restricted by Mistral's research license for production commercial SaaS.

No cardLocalAPIOpen weightsHF
ILLUSTRATIVE PROMPT

Decide whether to execute tool get_weather or search_flights based on query: 'Flight to Boston tomorrow morning'.

PREVIEW NOTE

A structured tool-call payload targeting search_flights with destination 'BOS' and departure date.

705
24
●LOCAL • 86/100
Mistral Small 3
AGENTS / AI INDEXMS
Mistral Small 3
Mistral AI

A 24B Apache-2.0 model tuned for speed and local agentic loops, fitting in 16 GB at 4-bit for real-time workflows.

BASE ARCHITECTURE / Mistral Small 3 (24B)
ARuns well (78/100) • Q4_K_M (~47 tok/s • 9GB free)
ℹTHE BRUTAL TRUTH

The only sub-16GB model on this list that runs real-time agent loops. Trade: weaker reasoning than 27B+ models.

No cardLocalAPIOpen weightsHFPermissive
ILLUSTRATIVE PROMPT

When a new lead arrives, enrich the record and route it to the right owner.

PREVIEW NOTE

Illustrative automation prompt; credentials are not connected.

699
26
●LOCAL • 87/100
Command R (35B)
AGENTS / AI INDEXCR
Command R (35B)
Cohere

Cohere's dedicated 35B open-weight model engineered specifically for retrieval-augmented generation (RAG) and grounded enterprise tool calling across 128K context.

BASE ARCHITECTURE / Command-R-v01 (35B)
BRuns well (61/100) • Q4_K_M (~35 tok/s • 4GB free)
ℹTHE BRUTAL TRUTH

The citation format is brilliant for enterprise document QA, but CC-BY-NC licensing means you cannot commercialize it without Cohere's commercial agreement.

No cardLocalAPIOpen weightsHF
ILLUSTRATIVE PROMPT

Extract claims from this legal deposition, cite line numbers, and flag contradictions.

PREVIEW NOTE

A structured claims matrix with verifiable inline source bracket citations.

640
27
●LOCAL • 84/100
Kokoro-82M
AUDIO / AI INDEXKo
Kokoro-82M
Kokoro / community

A compact 82M text-to-speech model suited to local audio pipelines, CPU inference, and lightweight browser prototypes.

BASE ARCHITECTURE / Kokoro-82M
SRuns well (95/100) • ONNX CPU (~220 tok/s • 22GB free)
ℹTHE BRUTAL TRUTH

Runs on CPU, tiny footprint, Apache 2.0 — the only reason not to use it is voice quality. If you need premium voices, pay for ElevenLabs.

No cardLocalOpen weightsHFPermissive
ILLUSTRATIVE PROMPT

Welcome back. Your next idea starts here.

PREVIEW NOTE

Illustrative speech text; no audio is generated by this page.

632
28
●LOCAL • 86/100
InternLM2.5-20B-Chat
AGENTS / AI INDEXIN
InternLM2.5-20B-Chat
Shanghai AI Lab

A 20B parameter model featuring an astounding 1-million-token native context window. Designed for book-length document reasoning and complex agent tools.

BASE ARCHITECTURE / InternLM2.5-20B-Chat
ARuns well (77/100) • Q4_K_M (~54 tok/s • 11GB free)
ℹTHE BRUTAL TRUTH

The only sub-24GB model with a verified 1M context window that doesn't hallucinate past 64K. Watch your KV-cache memory at full lengths.

No cardLocalAPIOpen weightsHFPermissive
ILLUSTRATIVE PROMPT

Search a 500-page codebase transcript for all references to deprecated API route /v1/auth/legacy.

PREVIEW NOTE

A structured cross-reference table indexing timestamps, filenames, and replacement routes.

620
29
●LOCAL • 86/100
StarCoder2-15B
CODING / AI INDEXSC
StarCoder2-15B
BigCode / ServiceNow

Trained on 4+ trillion tokens across 600+ programming languages from The Stack v2. Transparent, permissively trained code generation workhorse.

BASE ARCHITECTURE / StarCoder2-15B (dense)
SRuns well (85/100) • Q4_K_M (~71 tok/s • 14GB free)
ℹTHE BRUTAL TRUTH

Enterprise legal teams love the clean training dataset provenance. Good autocomplete, but shorter context than modern models.

No cardLocalAPIOpen weightsHF
ILLUSTRATIVE PROMPT

Write a Python data validator using Pydantic v2 with custom field serializers.

PREVIEW NOTE

A typed Pydantic BaseModel with @field_validator and model_serializer functions.

612
30
●LOCAL • 86/100
Kolors
IMAGE / AI INDEXKL
Kolors
KolorsLISTED
Kuaishou Technology

Diffusion model based on SDXL and ChatGLM3 text encoding, supporting both English and Chinese text rendering with realistic skin textures and lighting.

BASE ARCHITECTURE / Kolors (2.7B)
ARuns well (79/100) • FP8 (~71 tok/s • 14GB free)
ℹTHE BRUTAL TRUTH

Incredible portrait fidelity and bilingual typography. ChatGLM3 text encoder handles nuances that CLIP misses completely.

No cardLocalAPIOpen weightsHFPermissive
ILLUSTRATIVE PROMPT

A hyper-detailed traditional porcelain craftsman shaping clay in a sunlit Beijing workshop.

PREVIEW NOTE

A lifelike portrait showing clay splatters on artisan fingers and delicate natural ambient lighting.

610
31
●LOCAL • 82/100
Code Llama 34B Instruct
CODING / AI INDEXCL

Meta's 34B parameter code-specialized model with infilling capabilities and large context. Battle-tested base for enterprise fine-tuning.

BASE ARCHITECTURE / Code Llama 34B Instruct
BRuns well (61/100) • Q4_K_M (~35 tok/s • 4GB free)
ℹTHE BRUTAL TRUTH

Historic milestone, but newer models like Qwen 3.6 27B and Qwen3-Coder beat it soundly while using fewer resources. Only use if tied to existing fine-tunes.

No cardLocalAPIOpen weightsHF
ILLUSTRATIVE PROMPT

Create a Bash script with getopt that parses flags, logs errors to stderr, and verifies disk space.

PREVIEW NOTE

A robust POSIX-compliant shell script with trap cleanup handlers and error codes.

590
32
●LOCAL • 85/100
AuraFlow-v0.3
IMAGE / AI INDEXAF
AuraFlow-v0.3
fal.ai

An open-source 6.8B parameter text-to-image diffusion transformer model by fal.ai, released under Apache 2.0 with linear attention and fast inference.

BASE ARCHITECTURE / AuraFlow-v0.3 (6.8B)
ARuns well (82/100) • FP8 (~59 tok/s • 12GB free)
ℹTHE BRUTAL TRUTH

A very clean, royalty-free Apache 2.0 DiT. If you dislike Stability's license terms and need an alternative to FLUX, AuraFlow is a real option.

No cardLocalAPIOpen weightsHFPermissive
ILLUSTRATIVE PROMPT

Abstract architectural composition of intersecting glass planes in desert haze.

PREVIEW NOTE

A sharp geometric rendering featuring realistic atmospheric refraction and minimalist shadows.

580
33
●LOCAL • 88/100
MOSS Image Base
IMAGE / AI INDEXMO
MOSS Image Base
OpenMOSS Team

User guide   ·   中文用户指南   ·   ComfyUI

BASE ARCHITECTURE / MOSS Image Base (5.1B Flow-Matching Diffusion Transformer)
SRuns well (97/100) • Q2_K (2-bit Extreme) (~220 tok/s • 22GB free)
ℹTHE BRUTAL TRUTH

Audited in Stactora CMS. Architecture verified with 5.1B parameters.

No cardLocalComfyUIComfy-Org RepackOpen weightsHFPermissive
ILLUSTRATIVE PROMPT

A wide view of a refrigerator beside a kitchen counter with a fruit bowl and a sticky note.

PREVIEW NOTE

Photorealistic generative composition with clear embedded typography.

168k
35
●LOCAL • 85/100
Granite-3.0-8B-Instruct
RESEARCH / AI INDEXIB
Granite-3.0-8B-Instruct
IBM Research

IBM's flagship enterprise open-weight model with 128K context, trained under strict copyright curation and released with full Apache 2.0 licensing and IP indemnity.

BASE ARCHITECTURE / Granite-3.0-8B-Instruct
SRuns well (98/100) • Q4_K_M (~141 tok/s • 19GB free)
ℹTHE BRUTAL TRUTH

Built specifically for enterprise legal departments terrified of training data copyright lawsuits. Solid RAG performance with true Apache 2.0.

No cardLocalAPIOpen weightsHFPermissive
ILLUSTRATIVE PROMPT

Draft an enterprise compliance policy for data classification across Confidential, Internal, and Public.

PREVIEW NOTE

A formal compliance document with handling procedures, access controls, and retention schedules.

530
41
●LOCAL • 92/100
Nanbeige 4.3 33B
CODING / AI INDEXNB
Nanbeige 4.3 33B
Nanbeige LLM Lab

Nanbeige's flagship 33B dense language and coding model with a 512K context window. Engineered specifically for agentic terminal workflows, SWE-bench engineering, and professional enterprise co-work under Apache 2.0.

BASE ARCHITECTURE / Nanbeige4.3-33B (Dense Teacher Architecture)
DDecent (36/100) • Q4_K_M (~21 tok/s • 3GB free)
ℹTHE BRUTAL TRUTH

A serious alternative to Qwen2.5-Coder-32B built specifically for autonomous agents that run terminal commands and refactor full repos. If you run SWE-bench agents on a 24GB card, this is a top-tier driver.

No cardLocalAPIOpen weightsHFPermissive
ILLUSTRATIVE PROMPT

Diagnose this failing CI pipeline across 4 microservices and execute terminal fixes.

PREVIEW NOTE

Step-by-step reasoning trace with verified terminal commands and patch files.

410
42
●LOCAL • 91/100
Apex 35B V1.0
RESEARCH / AI INDEXAP
Apex 35B V1.0
Apex Intelligence

Apex Intelligence's 35B MoE language model trained via recursive automated AI research agents. Specialized in computational science, robotics simulation, Claude Code agent workflows, and deep scientific reasoning.

BASE ARCHITECTURE / Apex-35B-V1.0 (Automated Research MoE Architecture)
ARuns well (79/100) • Q4_K_M (~47 tok/s • 9GB free)
ℹTHE BRUTAL TRUTH

Trained by AI agents to perform AI and scientific research. Because only 3B parameters are active per token, it runs like a 3B model while delivering the deep reasoning depth of a 35B architecture on 16GB GPUs.

No cardLocalAPIOpen weightsHFPermissive
ILLUSTRATIVE PROMPT

Formulate a robotics trajectory optimization algorithm with obstacle avoidance constraints.

PREVIEW NOTE

Rigorous mathematical formulation with typed Python simulation code.

390
43
●LOCAL • 84/100
Krea 2 Raw
IMAGE / AI INDEXKR
Krea 2 Raw
Krea (Open Weights)

Access to Krea 2 Raw is gated. Please provide the information below and agree to the license to request access.

BASE ARCHITECTURE / Krea 2 Raw (Image Flow Architecture)
SRuns well (99/100) • Q2_K (2-bit Extreme) (~141 tok/s • 19GB free)
ℹTHE BRUTAL TRUTH

Audited in Stactora CMS. Architecture verified with 12.8B parameters.

LocalComfyUIHF
ILLUSTRATIVE PROMPT

PREVIEW NOTE

88k
44
●LOCAL • 85/100
Fibo 2.0 dev
IMAGE / AI INDEXFI
Fibo 2.0 dev
briaai (Open Weights)

Open-weight foundation model by briaai on Hugging Face Hub with 1,554 30-day downloads and 186 likes.

BASE ARCHITECTURE / Fibo-2.0-dev (8.1B Parameters)
SRuns well (98/100) • Q2_K (2-bit Extreme) (~202 tok/s • 21GB free)
ℹTHE BRUTAL TRUTH

Audited in Stactora CMS. Architecture verified with 8.1B parameters.

LocalHF
ILLUSTRATIVE PROMPT

PREVIEW NOTE

4,200

Frequently Asked Questions · /vram/32gb

What does 32GB VRAM unlock over a 24GB GPU?
32GB lets you run 30B–32B models at Q8_0 (near-lossless precision) or comfortably maintain long 64K–128K context KV-caches.
Can 32GB run Llama 3.3 70B?
At 3-bit quantization (Q3_K_M, ~33GB), it fits with minor offload, or at Q4 with 8GB system RAM offload.