Running Ollama with Llama 3 on a machine with 32GB RAM and an RTX 3090. Response times are reasonable for most tasks. The setup took an afternoon. The model quality is genuinely impressive for a local setup. Privacy wins alone make it worth it for anything sensitive.
Comments