Modal

Overview

Modal is a cloud platform for running Python code serverlessly, with a focus on AI/ML workloads. Key capabilities:

• GPU compute on demand (including H100, H200, B200 and B300)
• Serverless functions with autoscaling from zero to thousands of containers
• Custom container images built entirely in Python code
• Persistent storage via Volumes for model weights and datasets
• Web endpoints for serving models and APIs
• Scheduled jobs via cron or fixed intervals
• Container reuse and warm pools to reduce cold starts (model loading still takes time)

Modal Apps and Images are defined in Python; existing Dockerfiles are also supported.

When to Use This Skill

Use this skill when:

• Deploy or serve AI/ML models in the cloud
• Run GPU-accelerated computations (training, inference, fine-tuning)
• Create serverless web APIs or endpoints
• Scale batch processing jobs in parallel
• Schedule recurring tasks (data pipelines, retraining, scraping)
• Need persistent cloud storage for model weights or datasets
• Want to run code in custom container environments
• Build job queues or async task processing systems

Installation and Authentication

Install

uv pip install "modal==1.6.0"

Reviewed against SDK 1.6.0 and current release notes. Local SDK construction and selected handlers were tested; remote builds, deployments, GPU inference and cloud limits were not executed. Snippets using model/data placeholders or third-party workloads are illustrative and require their stated dependencies.

Authenticate

Prefer existing credentials before creating new ones. Only the two Modal-specific variables below are relevant — do not read, load, or expose any other environment variables or .env file contents:

1. Reuse an existing Modal profile, or check whether both MODAL_TOKEN_ID and MODAL_TOKEN_SECRET are set; report presence only.
2. If not, look up only those two keys in a local .env file (ignore all other entries) and load them if appropriate for the workflow.
3. Only fall back to interactive modal setup if no usable profile or token pair exists. The SDK supports profiles in .modal.toml; environment tokens are optional when a profile authenticates.

modal setup

This opens a browser for authentication. For CI/CD or headless environments, use environment variables:

export MODAL_TOKEN_ID=<your-token-id>
export MODAL_TOKEN_SECRET=<your-token-secret>

If neither an existing profile nor a token pair is available, create credentials at https://modal.com/settings

Check current pricing and workspace quotas before sizing a run. GPU use requires a payment method even when credits remain.

Reference: See references/getting-started.md for detailed setup and first app walkthrough.

Core Concepts

App and Functions

A Modal App groups related functions. Functions decorated with @app.function() run remotely in the cloud:

import modal

app = modal.App("my-app")

@app.function()
def square(x):
    return x ** 2

@app.local_entrypoint()
def main():
    # .remote() runs in the cloud
    print(square.remote(42))

Run with modal run script.py. Deploy with modal deploy script.py.

Reference: See references/functions.md for lifecycle hooks, classes, .map(), .spawn(), and more.

Container Images

Modal builds container images from Python code. The recommended package installer is uv:

image = (
    modal.Image.debian_slim(python_version="3.11")
    .uv_pip_install("torch==2.12.0", "transformers==5.9.0", "accelerate==1.13.0")
    .apt_install("git")
)

@app.function(image=image, gpu="L40S")
def inference(prompt):
    from transformers import pipeline
    pipe = pipeline("text-generation", model="openai-community/gpt2", device=0)
    return pipe(prompt)

Key image methods:

• .uv_pip_install() — Install Python packages with uv (recommended)
• .pip_install() — Install with pip (fallback)
• .apt_install() — Install system packages
• .run_commands() — Run shell commands during build
• .run_function() — Run Python during build (e.g., download model weights)
• .add_local_python_source() — Add local modules
• .env() — Set environment variables

Reference: See references/images.md for Dockerfiles, micromamba, caching, GPU build steps.

GPU Compute

Request GPUs via the gpu parameter:

…