Back to README.md
DeepSeek V4 Flash, a file of 91 GB, most of which stays in RAM too. Its memory of a conversation is small: a 128K context takes 0.7 GB more VRAM than 32K.
Measured for PolyStrata on the same workstation as GLM 5.3 Flash, with a short chat and a 30K-token prompt.
| NVIDIA: RTX 5090 (32 GB), Xeon Platinum 8470Q (25 vCPU), 120 GB RAM | ||||||
|---|---|---|---|---|---|---|
|
On the same PC llama.cpp wrote 25 to 28 tokens per second and read a prompt of 3,384 tokens at 96. A prompt of 116,473 tokens in a 128K context was read and answered in 174 seconds, one of 487,529 tokens in a 512K context in 32 minutes, and both times the three facts hidden in it were found. What was measured and how: docs/DEEPSEEK.md.
| Graphics card | One NVIDIA GeForce RTX 20 series card or newer. It has run on 32 GB of VRAM only. About 11.5 GB are taken before any expert, so a 12 GB card cannot start it. |
| RAM | Beside a 32 GB card, 72 GB or more, the installer's estimate. A smaller card needs more RAM (how much). It has run with 120 GB. |
| Disk | 91 GB for the model's three files, on an SSD. |
| System | Linux, and a current NVIDIA driver. |
The installer sets up everything else, as it does for GLM 5.3 Flash. Not tried yet: AMD cards, two cards. It reads text only. A user runs it on Windows with one 24 GB card and 128 GB of RAM (report).
The full list: docs/DEEPSEEK.md.
One size so far.
| Size | What it is | Download | RAM beside a 32 GB card |
|---|---|---|---|
| UD-IQ2_XXS | the whole model at about 2 bits (Unsloth) | 91 GB | 72 GB |
As with GLM 5.3 Flash, the experts the card does not hold stay in RAM: about 80 GB beside a 24 GB card and 89 GB beside a 16 GB card, by the installer's estimate, and a 12 GB card cannot start it. Only the 32 GB card has been tried.
The diagram and how it works: HOW_IT_WORKS_MODELS.md.