How much VRAM do you need to run an LLM locally?
Work out the GPU memory a local LLM needs from its parameter count, quantisation and context length, with worked examples from official docs.
Read articleCategory
Local AI
Articles about local ai, with links to the sources behind each claim.
RSS feed1 article