Ollama Stack

Implante uma stack de LLM local para fluxos de trabalho offline e focados em privacidade.

Quando Usar Esta Skill

Use esta skill quando:

• Configurar inferência de LLM privada/local para desenvolvimento
• Construir ambientes de IA isolados (air-gapped)
• Executar modelos em hardware pessoal (Mac, Linux, Windows com GPU)
• Criar endpoints de inferência compartilhados pela equipe
• Fazer prototipagem antes de se comprometer com APIs de LLM na nuvem

Pré-requisitos

• 8 GB+ de RAM (16 GB+ recomendado para modelos 7B+)
• Para aceleração por GPU: GPU NVIDIA com 6 GB+ de VRAM, ou Mac com Apple Silicon
• Docker (para implantação em contêineres)
• 20 GB+ de disco para armazenamento de modelos

Início Rápido

# Install Ollama
curl -fsSL https://ollama.com/install.sh -o /tmp/install-ollama.sh && sh /tmp/install-ollama.sh && rm /tmp/install-ollama.sh

# Start the server
ollama serve

# Pull and run a model
ollama pull llama3.1:8b
ollama run llama3.1:8b "Explain Kubernetes pods in one paragraph"

# List available models
ollama list

# Pull specific quantization
ollama pull llama3.1:8b-instruct-q4_K_M

Guia de Seleção de Modelos

Docker Compose — Stack Completa

# docker-compose.yml
services:
  ollama:
    image: ollama/ollama:latest
    container_name: ollama
    restart: unless-stopped
    ports:
      - "11434:11434"
    volumes:
      - ollama_data:/root/.ollama
    environment:
      - OLLAMA_HOST=0.0.0.0
      - OLLAMA_NUM_PARALLEL=4
      - OLLAMA_MAX_LOADED_MODELS=2
      - OLLAMA_FLASH_ATTENTION=1
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: all
              capabilities: [gpu]
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:11434/api/tags"]
      interval: 30s
      timeout: 10s
      retries: 3

  open-webui:
    image: ghcr.io/open-webui/open-webui:main
    container_name: open-webui
    restart: unless-stopped
    ports:
      - "3000:8080"
    volumes:
      - webui_data:/app/backend/data
    environment:
      - OLLAMA_BASE_URL=http://ollama:11434
      - WEBUI_AUTH=true
      - WEBUI_SECRET_KEY=${WEBUI_SECRET_KEY:-change-me-in-production}
      - DEFAULT_MODELS=llama3.1:8b
    depends_on:
      ollama:
        condition: service_healthy

  litellm:
    image: ghcr.io/berriai/litellm:main-latest
    container_name: litellm
    restart: unless-stopped
    ports:
      - "4000:4000"
    volumes:
      - ./litellm-config.yaml:/app/config.yaml
    command: ["--config", "/app/config.yaml"]
    depends_on:
      ollama:
        condition: service_healthy

volumes:
  ollama_data:
  webui_data:

Configuração do LiteLLM Proxy

# litellm-config.yaml
model_list:
  - model_name: llama3
    litellm_params:
      model: ollama/llama3.1:8b
      api_base: http://ollama:11434
  - model_name: codellama
    litellm_params:
      model: ollama/codellama:13b
      api_base: http://ollama:11434
  - model_name: embeddings
    litellm_params:
      model: ollama/nomic-embed-text
      api_base: http://ollama:11434

general_settings:
  master_key: sk-local-dev-key
  max_budget: 0  # unlimited for local

Uso da API

Ollama expõe uma API compatível com OpenAI:

# Chat completion
curl http://localhost:11434/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "llama3.1:8b",
    "messages": [{"role": "user", "content": "Hello"}],
    "stream": false
  }'

# Embeddings
curl http://localhost:11434/v1/embeddings \
  -H "Content-Type: application/json" \
  -d '{
    "model": "nomic-embed-text",
    "input": "The quick brown fox"
  }'

# List models
curl http://localhost:11434/api/tags

Cliente Python

# pip install ollama
import ollama

# Chat
response = ollama.chat(
    model="llama3.1:8b",
    messages=[{"role": "user", "content": "Explique Docker em 3 frases"}],
)
print(response["message"]["content"])

# Streaming
for chunk in ollama.chat(
    model="llama3.1:8b",

…