How to Install LLM: CLI, Local Models, and API Setup

"Installing an LLM" means different things depending on what you're trying to do. You might want a command-line tool to query models from your terminal, a local model running entirely on your machine, a Python library to build an app, or just API access without installing anything heavy at all. This guide walks through each path so you can pick the right one.

The short answer: if you want to experiment with models locally, install Ollama or llama.cpp. If you want a scriptable CLI, install Simon Willison's llm tool via pip. If you're building a product and don't want to manage GPUs or model weights, skip local installation entirely and call a hosted API instead. Below is how to do each.

Option 1: Install the llm command-line tool

If you searched "how to install llm" looking for the actual llm CLI (a popular open-source tool for talking to language models from the terminal), installation is straightforward with pip or pipx:

pip install llm

or, for an isolated environment:

pipx install llm

Once installed, you can run a prompt directly:

llm "Explain what a hash map is"

By default it uses OpenAI models and asks for an API key on first run:

llm keys set openai

You can add plugins for other providers, including local models:

llm install llm-gpt4all
llm install llm-ollama

This tool is useful for scripting, piping data through prompts, and quick terminal-based workflows, but it still depends on either a paid API key or a locally running model underneath.

Option 2: Install a local LLM with Ollama

Ollama is the fastest way to get a model running entirely on your own hardware, with no API key and no internet dependency after the initial download.

macOS / Linux:

curl -fsSL https://ollama.com/install.sh | sh

Windows: download the installer from ollama.com and run it.

Once installed, pull and run a model:

ollama pull llama3
ollama run llama3

This downloads several gigabytes of model weights, so make sure you have disk space and, ideally, a GPU with enough VRAM — 8GB is a reasonable minimum for smaller models. CPU-only inference works but is slow.

Ollama also exposes a local HTTP API on localhost:11434, so you can integrate it into scripts the same way you'd call any REST API:

curl http://localhost:11434/api/generate -d '{
  "model": "llama3",
  "prompt": "Write a haiku about compilers"
}'

Option 3: Install llama.cpp for lower-level control

If you want more control over quantization, memory usage, or you're running on constrained hardware, llama.cpp is the underlying engine many tools (including Ollama) build on:

git clone https://github.com/ggerganov/llama.cpp
cd llama.cpp
make

You then need a GGUF model file, which you can download from Hugging Face, and run it directly:

./main -m model.gguf -p "Summarize the following text:"

This route gives you the finest control but requires more manual setup — picking quantization levels, managing context length, and building from source on some platforms.

Option 4: Install a Python library to build with an LLM

If your goal is to write code that calls a model, you don't install "the LLM" itself — you install a client library and point it at an API:

pip install openai
# or
pip install anthropic

from anthropic import Anthropic

client = Anthropic(api_key="your-api-key")
response = client.messages.create(
    model="claude-3-5-sonnet-20241022",
    max_tokens=1024,
    messages=[{"role": "user", "content": "Hello"}]
)
print(response.content)

This is the standard approach for production apps: no model weights to manage, no GPU to provision, and the provider handles scaling.

Option 5: Skip installation entirely with a hosted API

If you already have access to Claude through a subscription and want to call it programmatically without installing local infrastructure or managing separate provider billing, SubToAPI turns that access into a standard HTTPS API. You get an application key (sub_live_...), and can call it from any language that can make an HTTP request — no local model install, no GPU:

curl https://api.subtoapi.app/v1/messages \
  -H "Authorization: Bearer $SUBTOAPI_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "claude-3-5-sonnet-20241022",
    "max_tokens": 1024,
    "messages": [{"role": "user", "content": "Hello"}]
  }'

It supports streaming, tool use, and usage tracking out of the box, which matters if you're building something beyond a one-off script. Setup takes a few minutes — see the quickstart and pricing if you want to compare against running your own infrastructure.

Which option should you pick?

• Just experimenting locally, privacy matters, or no internet reliance wanted: Ollama or llama.cpp.
• Scripting from the terminal, quick one-off prompts: the llm CLI tool.
• Building an application: a client library (openai, anthropic) plus an API key, or a service like SubToAPI if you already have Claude access and want it exposed as an API with keys and usage metadata.
• Fine-grained control over quantization and memory: llama.cpp directly.

There's no single "correct" installation — it depends on whether you need the model running on your own hardware or just need reliable programmatic access to one that already exists.

questions

Do I need a GPU to install and run an LLM locally? No, but it helps significantly. Small quantized models can run on CPU, though response times will be much slower than with a GPU that has enough VRAM to hold the model.

What's the difference between installing an LLM and installing an LLM CLI tool? Installing an LLM (like via Ollama) downloads actual model weights to run inference locally. A CLI tool like llm is just a client — it still needs either a local model or an API key to function.

Is there a way to use an LLM without installing anything? Yes — call a hosted API directly with a key, either from a provider or a service like SubToAPI if you already have Claude access and want it as a standard HTTPS API without managing your own infrastructure.