Local AI Glossary

The on-GPU memory that stores model weights. Determines which AI models you can run.

Compressing model weights from 16-bit to 4-bit precision to massively reduce VRAM usage.

Next-generation precision formats that dramatically accelerate inference on NVIDIA Blackwell and Ada GPUs.

The universal speed metric for LLMs — how many words (tokens) your GPU generates per second.

How fast your GPU can read model weights from VRAM — the real determinant of inference speed.

The memory that stores the AI's 'conversation history' during generation — it lives in VRAM.

How much text the AI can 'remember' and process at once — directly tied to VRAM through the KV cache.

The universal file format for running quantized LLMs locally via llama.cpp and Ollama.

The easiest way to run open-source LLMs locally with one command — like Docker for AI models.

A polished desktop GUI for discovering, downloading, and chatting with local AI models.

The maximum sustained power your GPU draws under full load — critical for PSU selection.

The AI models like Llama, Mistral, and DeepSeek that generate human-like text — the software your GPU runs.

NVIDIA's proprietary parallel computing platform — the secret behind why NVIDIA GPUs dominate AI.

Running a trained AI model to generate outputs — what your local GPU does when you chat with an LLM.

AMD's open-source answer to CUDA — enables AI inference on Radeon GPUs.

The open-source engine that made running 70B models on consumer hardware possible in 2023.