Stactora - AI Hardware IndexMy Stack0+ Submit Tool
02 Target GPUs49 VERIFIED ENGINES

Best Local AI Models for 48GB VRAM (RTX A6000, Dual 24GB & MacBook M3 Max)

Workstation-grade memory unlocks Meta's Llama 3.3 70B Instruct at Q4_K_M (40GB) and Q5_K_M (48GB), plus near-lossless Q8_0 builds of all 27B–32B models.

TARGET VRAM48 GB Workstation
HARDWARERTX A6000 · M3/M4 Max
UNLOCKED CLASSLlama 3.3 70B & Q8_0 32B
Verified Engines for /vram/48gb
49 RESULTSClick any model card to open its dedicated /ai/[slug] page.
03
●LOCAL • 86/100
Llama 3.1 8B Instruct
RESEARCH / AI INDEXL8

Meta's ubiquitous 8B foundation workhorse with 128K context, multilingual capabilities, and massive tool-calling ecosystem support.

BASE ARCHITECTURE / Llama 3.1 8B Instruct
SRuns well (98/100) • Q4_K_M (~141 tok/s • 19GB free)
ℹTHE BRUTAL TRUTH

Still the most integrated small model on earth. Every framework, agent library, and tutorial supports Llama 3.1 8B out of the box.

No cardLocalAPIOpen weightsHF
ILLUSTRATIVE PROMPT

Summarize the key events leading to the 1904 Entente Cordiale in three concise paragraphs.

PREVIEW NOTE

A structured historical summary examining colonial rivalries, Fashoda, and diplomatic reconciliation.

980
09
●LOCAL • 82/100
MiniMax H 3
VIDEO / AI INDEXM3
MiniMax H 3
MiniMaxAI (Open Weights)

Offical skills to improve prompt writing: [skills on github](https://github.com/MiniMax-AI/MiniMax-H3/tree/main/skills)

BASE ARCHITECTURE / MiniMax H 3 (Research Flow Architecture)
SRuns well (87/100) • Q2_K (2-bit Extreme) (~59 tok/s • 12GB free)
ℹTHE BRUTAL TRUTH

Audited in Stactora CMS. Architecture verified with 33.1B parameters.

No cardLocalComfyUIComfy-Org RepackAPIOpen weightsHF
ILLUSTRATIVE PROMPT

A cinematic tracking shot through a bioluminescent forest with cascading waterfalls at dusk, 4k ultra-detailed.

PREVIEW NOTE

Fluid, temporally consistent cinematic video sequence with natural lighting dynamics.

Original Author:MiniMaxAI/MiniMax-H3 ↗
Comfy-Org Repack:Comfy-Org/MiniMax-H3 ↗
5576k
10
●LOCAL • 91/100
Gemma 2 27B
CODING / AI INDEXG2
Gemma 2 27B
Google DeepMind

Google DeepMind's flagship 27.2B open model featuring alternating local/global attention and post-training alignment, punching above its weight class on single-GPU hardware.

BASE ARCHITECTURE / Google Gemma 2 27B (Instruct)
BRuns well (60/100) • Q4_K_M (~39 tok/s • 6GB free)
ℹTHE BRUTAL TRUTH

Google's 27B architecture punches way above its parameter weight. If 8K context is sufficient for your workload, it is an outstanding 24GB driver.

No cardLocalAPIOpen weightsHF
ILLUSTRATIVE PROMPT

Implement a rate-limited API endpoint with structured errors.

PREVIEW NOTE

Robust implementation with structured error handling.

890
16
●LOCAL • 87/100
Llama 3.3 70B Instruct
AGENTS / AI INDEXL3

Meta's reliable 70B dense model for serious reasoning, long-form work, and agentic pipelines that need dependable instruction following.

BASE ARCHITECTURE / Llama 3.3 70B Instruct
FToo heavy (13/100) • Needs 40GB at Q4_K_M
Rent a Cloud GPU•$0.14/hr›
ℹTHE BRUTAL TRUTH

70B capability at 40GB VRAM cost — only worth it if you already own two 24GB cards or a 48GB workstation. Skip on single-GPU rigs.

No cardLocalAPIOpen weightsHF
ILLUSTRATIVE PROMPT

Design a multi-step research workflow with verification checkpoints.

PREVIEW NOTE

Illustrative agent workflow prompt; not executed here.

803
19
●LOCAL • 87/100
Hermes 3 (Llama 3.1 8B)
AGENTS / AI INDEXH8
Hermes 3 (Llama 3.1 8B)

State-of-the-art generalist instruction and roleplay model by Nous Research. Superb function calling, code generation, and multi-turn planning.

BASE ARCHITECTURE / Hermes-3-Llama-3.1-8B
SRuns well (98/100) • Q4_K_M (~141 tok/s • 19GB free)
ℹTHE BRUTAL TRUTH

The best community fine-tune of Llama 3.1 8B. Highly steerable, great at structured tool calling, and free of annoying canned refusals.

No cardLocalAPIOpen weightsHF
ILLUSTRATIVE PROMPT

Plan a 5-step automated penetration test checklist for an OAuth 2.0 authorization code flow with PKCE.

PREVIEW NOTE

A tactical security audit checklist testing token leakage, replay attacks, and state CSRF validation.

760
21
●LOCAL • 85/100
DeepSeek-R1-Distill-32B
RESEARCH / AI INDEXDS

A distilled reasoning model that preserves chain-of-thought behaviour in a size that fits on a 24 GB GPU at 4-bit.

BASE ARCHITECTURE / DeepSeek-R1-Distill 32B
BRuns well (61/100) • Q4_K_M (~35 tok/s • 4GB free)
ℹTHE BRUTAL TRUTH

MIT license + real reasoning traces + fits in 24GB is a rare combo. Watch token burn — reasoning uses 3-5x more output tokens than a normal model.

No cardLocalAPIOpen weightsHFPermissive
ILLUSTRATIVE PROMPT

Explain the trade-offs between RAG and long-context retrieval for a 200K-token corpus.

PREVIEW NOTE

Illustrative reasoning prompt; not executed here.

741
22
●LOCAL • 88/100
Mistral-Nemo-12B-Instruct
AGENTS / AI INDEXMN
Mistral-Nemo-12B-Instruct
Mistral AI & NVIDIA

Co-developed by Mistral and NVIDIA, this 12B model features Tekken tokenization, a 128K context window, and full Apache 2.0 licensing, fitting easily in 8GB–12GB VRAM.

BASE ARCHITECTURE / Mistral-Nemo-Instruct-2407 (12B)
SRuns well (92/100) • Q4_K_M (~88 tok/s • 16GB free)
ℹTHE BRUTAL TRUTH

The best 12B model in existence. Tekken tokenizer encodes multilingual text with 30% fewer tokens, and it carries true Apache 2.0.

No cardLocalAPIOpen weightsHFPermissive
ILLUSTRATIVE PROMPT

Act as a customer support triage agent: categorize this dispute, identify billing errors, and propose an adjustment.

PREVIEW NOTE

A structured agent evaluation with triage severity, account calculation checks, and credit adjustments.

740
23
●LOCAL • 90/100
Codestral-22B
CODING / AI INDEXCd
Codestral-22B
Mistral AI

Mistral's dedicated 22B open-weight code generation and fill-in-the-middle model with 32K context, fluent in over 80 programming languages.

BASE ARCHITECTURE / Codestral-22B-v0.1
ARuns well (77/100) • Q4_K_M (~50 tok/s • 10GB free)
ℹTHE BRUTAL TRUTH

Superb code generation and fill-in-the-middle accuracy, but the MNPL license prevents production commercial SaaS deployment without contacting Mistral.

No cardLocalAPIOpen weightsHF
ILLUSTRATIVE PROMPT

Implement an LRU cache with O(1) get and put in C++ using std::list and std::unordered_map.

PREVIEW NOTE

An optimized C++17 LRU cache class with lock guards and iterator eviction.

730
24
●LOCAL • 87/100
Stable Diffusion 3.5 Medium (2.5B)
IMAGE / AI INDEXSM

A 2.5B parameter consumer-friendly DiT designed to run on 8GB–10GB consumer GPUs with fast generation times and solid composition.

BASE ARCHITECTURE / Stable Diffusion 3.5 Medium
SRuns well (95/100) • GGUF Q4 (~101 tok/s • 17GB free)
ℹTHE BRUTAL TRUTH

The 8GB sweet spot for SD3.5. Fast and competent, though FLUX.1 Schnell at FP8 still edges it out in overall aesthetic appeal.

No cardLocalAPIOpen weightsHF
ILLUSTRATIVE PROMPT

A retro-futuristic camera sitting on an oak table in warm morning sunlight.

PREVIEW NOTE

A clean 1024x1024 photographic composition with tactile wooden grain textures and brass dials.

720
26
●LOCAL • 88/100
LTX-Video (2B Realtime)
VIDEO / AI INDEXLX

Real-time 2B spatial-temporal video DiT capable of generating 24fps high-resolution video in under 15 seconds on consumer GPUs under Apache 2.0.

BASE ARCHITECTURE / LTX-Video 0.9.1 (2B)
ARuns well (79/100) • BF16 (~78 tok/s • 15GB free)
ℹTHE BRUTAL TRUTH

Generates 24fps video in seconds instead of minutes. On an 8GB card, it feels like magic compared to the painful wait times of older video models.

No cardLocalAPIOpen weightsHFPermissive
ILLUSTRATIVE PROMPT

A golden retriever running through autumn leaves in slow motion.

PREVIEW NOTE

A 24fps video sequence rendered in under 18 seconds with smooth fur movement and flying leaves.

710
27
●LOCAL • 89/100
Ministral-8B-Instruct
AGENTS / AI INDEXM8
Ministral-8B-Instruct

Mistral's edge-computing champion with a 128K context window and interleaved sliding window attention, designed for sub-second agentic decision loops.

BASE ARCHITECTURE / Ministral-8B-Instruct-2410
SRuns well (98/100) • Q4_K_M (~141 tok/s • 19GB free)
ℹTHE BRUTAL TRUTH

Phenomenal edge latency and tool calling accuracy, but restricted by Mistral's research license for production commercial SaaS.

No cardLocalAPIOpen weightsHF
ILLUSTRATIVE PROMPT

Decide whether to execute tool get_weather or search_flights based on query: 'Flight to Boston tomorrow morning'.

PREVIEW NOTE

A structured tool-call payload targeting search_flights with destination 'BOS' and departure date.

705
28
●LOCAL • 86/100
Mistral Small 3
AGENTS / AI INDEXMS
Mistral Small 3
Mistral AI

A 24B Apache-2.0 model tuned for speed and local agentic loops, fitting in 16 GB at 4-bit for real-time workflows.

BASE ARCHITECTURE / Mistral Small 3 (24B)
ARuns well (78/100) • Q4_K_M (~47 tok/s • 9GB free)
ℹTHE BRUTAL TRUTH

The only sub-16GB model on this list that runs real-time agent loops. Trade: weaker reasoning than 27B+ models.

No cardLocalAPIOpen weightsHFPermissive
ILLUSTRATIVE PROMPT

When a new lead arrives, enrich the record and route it to the right owner.

PREVIEW NOTE

Illustrative automation prompt; credentials are not connected.

699
29
●LOCAL • 92/100
Qwen2.5-Math-72B-Instruct
RESEARCH / AI INDEXQM
Qwen2.5-Math-72B-Instruct

World-leading open mathematical reasoning model, scoring over 85% on MATH benchmarks. Specialized for scientific computation, equations, and formal proofs.

BASE ARCHITECTURE / Qwen2.5-Math-72B-Instruct
FToo heavy (13/100) • Needs 42GB at Q4_K_M
Rent a Cloud GPU•$0.14/hr›
ℹTHE BRUTAL TRUTH

The absolute gold standard for mathematical precision in the open ecosystem. Don't use it for chat — use it for proofs and numeric analysis.

No cardLocalAPIOpen weightsHFPermissive
ILLUSTRATIVE PROMPT

Find all integers n such that n^4 + 4 is a prime number, with full proof.

PREVIEW NOTE

A formal Sophie Germain identity factorization showing n=1 is the only valid solution.

680
31
●LOCAL • 87/100
Command R (35B)
AGENTS / AI INDEXCR
Command R (35B)

Cohere's dedicated 35B open-weight model engineered specifically for retrieval-augmented generation (RAG) and grounded enterprise tool calling across 128K context.

BASE ARCHITECTURE / Command-R-v01 (35B)
BRuns well (61/100) • Q4_K_M (~35 tok/s • 4GB free)
ℹTHE BRUTAL TRUTH

The citation format is brilliant for enterprise document QA, but CC-BY-NC licensing means you cannot commercialize it without Cohere's commercial agreement.

No cardLocalAPIOpen weightsHF
ILLUSTRATIVE PROMPT

Extract claims from this legal deposition, cite line numbers, and flag contradictions.

PREVIEW NOTE

A structured claims matrix with verifiable inline source bracket citations.

640
32
●LOCAL • 84/100
Kokoro-82M
AUDIO / AI INDEXKo
Kokoro-82M
Kokoro / community

A compact 82M text-to-speech model suited to local audio pipelines, CPU inference, and lightweight browser prototypes.

BASE ARCHITECTURE / Kokoro-82M
SRuns well (95/100) • ONNX CPU (~220 tok/s • 22GB free)
ℹTHE BRUTAL TRUTH

Runs on CPU, tiny footprint, Apache 2.0 — the only reason not to use it is voice quality. If you need premium voices, pay for ElevenLabs.

No cardLocalOpen weightsHFPermissive
ILLUSTRATIVE PROMPT

Welcome back. Your next idea starts here.

PREVIEW NOTE

Illustrative speech text; no audio is generated by this page.

632
33
●LOCAL • 86/100
InternLM2.5-20B-Chat
AGENTS / AI INDEXIN
InternLM2.5-20B-Chat
Shanghai AI Lab

A 20B parameter model featuring an astounding 1-million-token native context window. Designed for book-length document reasoning and complex agent tools.

BASE ARCHITECTURE / InternLM2.5-20B-Chat
ARuns well (77/100) • Q4_K_M (~54 tok/s • 11GB free)
ℹTHE BRUTAL TRUTH

The only sub-24GB model with a verified 1M context window that doesn't hallucinate past 64K. Watch your KV-cache memory at full lengths.

No cardLocalAPIOpen weightsHFPermissive
ILLUSTRATIVE PROMPT

Search a 500-page codebase transcript for all references to deprecated API route /v1/auth/legacy.

PREVIEW NOTE

A structured cross-reference table indexing timestamps, filenames, and replacement routes.

620
34
●LOCAL • 86/100
StarCoder2-15B
CODING / AI INDEXSC
StarCoder2-15B
BigCode / ServiceNow

Trained on 4+ trillion tokens across 600+ programming languages from The Stack v2. Transparent, permissively trained code generation workhorse.

BASE ARCHITECTURE / StarCoder2-15B (dense)
SRuns well (85/100) • Q4_K_M (~71 tok/s • 14GB free)
ℹTHE BRUTAL TRUTH

Enterprise legal teams love the clean training dataset provenance. Good autocomplete, but shorter context than modern models.

No cardLocalAPIOpen weightsHF
ILLUSTRATIVE PROMPT

Write a Python data validator using Pydantic v2 with custom field serializers.

PREVIEW NOTE

A typed Pydantic BaseModel with @field_validator and model_serializer functions.

612
35
●LOCAL • 86/100
Kolors
IMAGE / AI INDEXKL
Kolors
Kuaishou Technology

Diffusion model based on SDXL and ChatGLM3 text encoding, supporting both English and Chinese text rendering with realistic skin textures and lighting.

BASE ARCHITECTURE / Kolors (2.7B)
ARuns well (79/100) • FP8 (~71 tok/s • 14GB free)
ℹTHE BRUTAL TRUTH

Incredible portrait fidelity and bilingual typography. ChatGLM3 text encoder handles nuances that CLIP misses completely.

No cardLocalAPIOpen weightsHFPermissive
ILLUSTRATIVE PROMPT

A hyper-detailed traditional porcelain craftsman shaping clay in a sunlit Beijing workshop.

PREVIEW NOTE

A lifelike portrait showing clay splatters on artisan fingers and delicate natural ambient lighting.

610
36
●LOCAL • 82/100
Code Llama 34B Instruct
CODING / AI INDEXCL

Meta's 34B parameter code-specialized model with infilling capabilities and large context. Battle-tested base for enterprise fine-tuning.

BASE ARCHITECTURE / Code Llama 34B Instruct
BRuns well (61/100) • Q4_K_M (~35 tok/s • 4GB free)
ℹTHE BRUTAL TRUTH

Historic milestone, but newer models like Qwen 3.6 27B and Qwen3-Coder beat it soundly while using fewer resources. Only use if tied to existing fine-tunes.

No cardLocalAPIOpen weightsHF
ILLUSTRATIVE PROMPT

Create a Bash script with getopt that parses flags, logs errors to stderr, and verifies disk space.

PREVIEW NOTE

A robust POSIX-compliant shell script with trap cleanup handlers and error codes.

590
37
●LOCAL • 85/100
AuraFlow-v0.3
IMAGE / AI INDEXAF
AuraFlow-v0.3

An open-source 6.8B parameter text-to-image diffusion transformer model by fal.ai, released under Apache 2.0 with linear attention and fast inference.

BASE ARCHITECTURE / AuraFlow-v0.3 (6.8B)
ARuns well (82/100) • FP8 (~59 tok/s • 12GB free)
ℹTHE BRUTAL TRUTH

A very clean, royalty-free Apache 2.0 DiT. If you dislike Stability's license terms and need an alternative to FLUX, AuraFlow is a real option.

No cardLocalAPIOpen weightsHFPermissive
ILLUSTRATIVE PROMPT

Abstract architectural composition of intersecting glass planes in desert haze.

PREVIEW NOTE

A sharp geometric rendering featuring realistic atmospheric refraction and minimalist shadows.

580
38
●LOCAL • 88/100
MOSS Image Base
IMAGE / AI INDEXMO
MOSS Image Base
OpenMOSS Team

User guide   ·   中文用户指南   ·   ComfyUI

BASE ARCHITECTURE / MOSS Image Base (5.1B Flow-Matching Diffusion Transformer)
SRuns well (97/100) • Q2_K (2-bit Extreme) (~220 tok/s • 22GB free)
ℹTHE BRUTAL TRUTH

Audited in Stactora CMS. Architecture verified with 5.1B parameters.

No cardLocalComfyUIComfy-Org RepackOpen weightsHFPermissive
ILLUSTRATIVE PROMPT

A wide view of a refrigerator beside a kitchen counter with a fruit bowl and a sticky note.

PREVIEW NOTE

Photorealistic generative composition with clear embedded typography.

168k
40
●LOCAL • 85/100
Granite-3.0-8B-Instruct
RESEARCH / AI INDEXIB
Granite-3.0-8B-Instruct

IBM's flagship enterprise open-weight model with 128K context, trained under strict copyright curation and released with full Apache 2.0 licensing and IP indemnity.

BASE ARCHITECTURE / Granite-3.0-8B-Instruct
SRuns well (98/100) • Q4_K_M (~141 tok/s • 19GB free)
ℹTHE BRUTAL TRUTH

Built specifically for enterprise legal departments terrified of training data copyright lawsuits. Solid RAG performance with true Apache 2.0.

No cardLocalAPIOpen weightsHFPermissive
ILLUSTRATIVE PROMPT

Draft an enterprise compliance policy for data classification across Confidential, Internal, and Public.

PREVIEW NOTE

A formal compliance document with handling procedures, access controls, and retention schedules.

530
46
●LOCAL • 92/100
Nanbeige 4.3 33B
CODING / AI INDEXNB
Nanbeige 4.3 33B
Nanbeige LLM Lab

Nanbeige's flagship 33B dense language and coding model with a 512K context window. Engineered specifically for agentic terminal workflows, SWE-bench engineering, and professional enterprise co-work under Apache 2.0.

BASE ARCHITECTURE / Nanbeige4.3-33B (Dense Teacher Architecture)
DDecent (36/100) • Q4_K_M (~21 tok/s • 3GB free)
ℹTHE BRUTAL TRUTH

A serious alternative to Qwen2.5-Coder-32B built specifically for autonomous agents that run terminal commands and refactor full repos. If you run SWE-bench agents on a 24GB card, this is a top-tier driver.

No cardLocalAPIOpen weightsHFPermissive
ILLUSTRATIVE PROMPT

Diagnose this failing CI pipeline across 4 microservices and execute terminal fixes.

PREVIEW NOTE

Step-by-step reasoning trace with verified terminal commands and patch files.

410
47
●LOCAL • 91/100
Apex 35B V1.0
RESEARCH / AI INDEXAP
Apex 35B V1.0
Apex Intelligence

Apex Intelligence's 35B MoE language model trained via recursive automated AI research agents. Specialized in computational science, robotics simulation, Claude Code agent workflows, and deep scientific reasoning.

BASE ARCHITECTURE / Apex-35B-V1.0 (Automated Research MoE Architecture)
ARuns well (79/100) • Q4_K_M (~47 tok/s • 9GB free)
ℹTHE BRUTAL TRUTH

Trained by AI agents to perform AI and scientific research. Because only 3B parameters are active per token, it runs like a 3B model while delivering the deep reasoning depth of a 35B architecture on 16GB GPUs.

No cardLocalAPIOpen weightsHFPermissive
ILLUSTRATIVE PROMPT

Formulate a robotics trajectory optimization algorithm with obstacle avoidance constraints.

PREVIEW NOTE

Rigorous mathematical formulation with typed Python simulation code.

390
48
●LOCAL • 84/100
Krea 2 Raw
IMAGE / AI INDEXKR
Krea 2 Raw
Krea (Open Weights)

Access to Krea 2 Raw is gated. Please provide the information below and agree to the license to request access.

BASE ARCHITECTURE / Krea 2 Raw (Image Flow Architecture)
SRuns well (99/100) • Q2_K (2-bit Extreme) (~141 tok/s • 19GB free)
ℹTHE BRUTAL TRUTH

Audited in Stactora CMS. Architecture verified with 12.8B parameters.

LocalComfyUIHF
ILLUSTRATIVE PROMPT

PREVIEW NOTE

88k
49
●LOCAL • 85/100
Fibo 2.0 dev
IMAGE / AI INDEXFI
Fibo 2.0 dev
briaai (Open Weights)

Open-weight foundation model by briaai on Hugging Face Hub with 1,554 30-day downloads and 186 likes.

BASE ARCHITECTURE / Fibo-2.0-dev (8.1B Parameters)
SRuns well (98/100) • Q2_K (2-bit Extreme) (~202 tok/s • 21GB free)
ℹTHE BRUTAL TRUTH

Audited in Stactora CMS. Architecture verified with 8.1B parameters.

LocalHF
ILLUSTRATIVE PROMPT

PREVIEW NOTE

4,200

Frequently Asked Questions · /vram/48gb

Can a 48GB MacBook Pro M3/M4 Max run Llama 3.3 70B?
Yes — at Q4_K_M, Llama 3.3 70B uses ~40GB, leaving 8GB for macOS and KV context cache.
Is it better to run 70B at Q4 or 32B at Q8_0 on 48GB VRAM?
For coding and math, a 30B–32B model at near-lossless Q8_0 (29–35GB) with a huge 64K+ context cache often beats a 70B model squeezed into Q4.