SmolLM2-1.7B-Instruct
Ultra-compact on-device language model trained on 11 trillion tokens by Hugging Face. Runs at 120+ tokens/sec on CPU or 2GB VRAM with Apache 2.0.
The brutal truth
The best sub-2B model in the world. Ideal for edge IoT, Raspberry Pis, and instant regex/JSON extraction at 120+ tok/s on CPU.
Files and Versions
Direct inspected tree artifacts from Hugging Face. Verified filenames, exact byte sizes, model weight shards, and configuration definitions.
Files and versions are hosted on Hugging Face. You can view all branches, commits, model weights, and tokenizer artifacts directly.
Inspect HuggingFaceTB/SmolLM2-1.7B-Instruct /tree/main ↗Inspect /tree/main Repository ↗Decision matrix
✓ RUN / USE THIS IF…
- •You are deploying on edge hardware, mobile, or CPU
- •You want Apache 2.0
- •Your 24GB GPU comfortably fits the model weights (~2GB VRAM) with dedicated headroom for inference.
- •You require 100% offline data privacy with zero third-party API logging, subscriptions, or prompt retention.
✕ WALK AWAY IF…
- •You need deep multi-step reasoning or complex coding
- •Your GPU has less than 2GB VRAM and you cannot tolerate slow CPU/system RAM offload paging.
- •You prefer a plug-and-play web app interface and don't want to manage local model checkpoints, drivers, or runtimes.
- •You need specialized frontier reasoning that exceeds this model's parameter envelope.
True cost of ownership
About
Ultra-compact on-device language model trained on 11 trillion tokens by Hugging Face. Runs at 120+ tokens/sec on CPU or 2GB VRAM with Apache 2.0.
What it does
Download the weights from Hugging Face, pick a quantisation that matches your GPU, and run it locally with Ollama, llama.cpp, vLLM, or a compatible runtime. Ultra-fast on-device extraction & mobile edge AI runs on your own hardware — no per-token bill, no third-party data handling.
Key features
Runs on low-power devices, Raspberry Pi, and phones
Over 120 tokens/sec on modern laptop CPUs
Permissive Apache 2.0 license
Use cases
Synthesise dozens of sources with citations you can check.
Build structured comparisons from live web results.
Summarise filings and identify inconsistencies.
Turn scattered notes into a coherent argument.
Quantization & Hardware Fit
VRAM Hardware Compatibility & Memory Allocation
| Quant | VRAM | Quality / Fit on 24GB | Install / Run Command (OLLAMA) | |
|---|---|---|---|---|
| Q4_K_M● ACTIVE | 2 GB | Standard · ✓ Fits (22GB spare) | ollama run smollm2:1.7b | |
| Q8_0 | 3 GB | Standard · ✓ Fits (21GB spare) | ollama run smollm2:1.7b | |
| FP16 | 4 GB | Standard · ✓ Fits (20GB spare) | ollama run smollm2:1.7b |
Buyer doubts, cleared
Can I actually run SmolLM2-1.7B-Instruct on my 24GB GPU?
What's the fine print on the license?
Does the 8K tokens actually hold up in practice?
User reviews
Reasoning quality is close to the bigger models for structured tasks. Open-ended planning still needs a frontier model.
Switched our internal workflow to SmolLM2-1.7B-Instruct after the last pricing change. Nobody noticed a quality difference, and the bill dropped by half.
We evaluated SmolLM2-1.7B-Instruct against two hosted alternatives. It won on cost-per-output but lost on latency. For batch work it is a clear yes.
Solid for ultra-fast on-device extraction & mobile edge ai. The learning curve is real but the payoff is there. Start with the smaller build first.
