Object Storage

Configure and manage object storage solutions including AWS S3, MinIO (self-hosted), and compatible providers. Covers CLI operations, bucket policies, lifecycle rules, versioning, encryption, and the MinIO client (mc).

When to Use

• Storing application assets, backups, logs, or media files
• Setting up an S3-compatible object store on-premises with MinIO
• Configuring lifecycle rules to transition or expire objects automatically
• Implementing access control with bucket policies and IAM
• Syncing data between local filesystems and object storage
• Serving static content from S3 or MinIO

Prerequisites

• AWS CLI v2 installed and configured (aws configure) for S3 operations
• Docker installed for MinIO self-hosted setup
• MinIO client (mc) installed for MinIO management
• IAM credentials with appropriate S3 permissions
• Network access to the object storage endpoint

AWS S3 CLI Operations

Bucket Management

# Create a new bucket
aws s3 mb s3://my-app-assets-prod

# Create a bucket in a specific region
aws s3 mb s3://my-app-assets-eu --region eu-west-1

# List all buckets
aws s3 ls

# List objects in a bucket (with sizes)
aws s3 ls s3://my-app-assets-prod --recursive --human-readable --summarize

# Delete an empty bucket
aws s3 rb s3://my-old-bucket

# Delete a bucket and ALL its contents (destructive)
aws s3 rb s3://my-old-bucket --force

Upload and Download

# Upload a single file
aws s3 cp ./report.pdf s3://my-app-assets-prod/reports/

# Upload with a specific storage class
aws s3 cp ./archive.tar.gz s3://my-app-assets-prod/archives/ --storage-class GLACIER

# Upload with server-side encryption (AES-256)
aws s3 cp ./sensitive.dat s3://my-app-assets-prod/data/ --sse AES256

# Download a file
aws s3 cp s3://my-app-assets-prod/reports/report.pdf ./downloads/

# Sync a local directory to S3 (upload only changed files)
aws s3 sync ./build/ s3://my-app-assets-prod/static/ --delete

# Sync from S3 to local
aws s3 sync s3://my-app-assets-prod/static/ ./local-copy/

# Sync with exclusion patterns
aws s3 sync ./logs/ s3://my-app-logs/ --exclude "*.tmp" --exclude ".git/*"

# Copy between buckets
aws s3 sync s3://source-bucket/ s3://destination-bucket/ --source-region us-east-1 --region eu-west-1

# Generate a pre-signed URL (temporary access, 1 hour)
aws s3 presign s3://my-app-assets-prod/reports/report.pdf --expires-in 3600

# Recursive delete of a prefix
aws s3 rm s3://my-app-assets-prod/old-data/ --recursive

Versioning

# Enable versioning on a bucket
aws s3api put-bucket-versioning \
  --bucket my-app-assets-prod \
  --versioning-configuration Status=Enabled

# Check versioning status
aws s3api get-bucket-versioning --bucket my-app-assets-prod

# List object versions
aws s3api list-object-versions --bucket my-app-assets-prod --prefix reports/

# Restore a previous version (copy old version to current)
aws s3api copy-object \
  --bucket my-app-assets-prod \
  --copy-source "my-app-assets-prod/reports/report.pdf?versionId=abc123" \
  --key reports/report.pdf

# Delete a specific version permanently
aws s3api delete-object \
  --bucket my-app-assets-prod \
  --key reports/old-report.pdf \
  --version-id abc123

Bucket Policies

# Apply a bucket policy from a JSON file
aws s3api put-bucket-policy --bucket my-app-assets-prod --policy file://policy.json

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "PublicReadForStaticSite",
      "Effect": "Allow",
      "Principal": "*",
      "Action": "s3:GetObject",
      "Resource": "arn:aws:s3:::my-app-assets-prod/static/*"
    },
    {
      "Sid": "DenyUnencryptedUploads",
      "Effect": "Deny",
      "Principal": "*",
      "Action": "s3:PutObject",
      "Resource": "arn:aws:s3:::my-app-assets-prod/*",
      "Condition": {
        "StringNotEquals": {
          "s3:x-amz-server-side-encryption": "AES256"
        }
      }
    },
    {
      "Sid": "RestrictToVPC",
      "Effect": "Deny",
      "Principal": "*",
      "Action": "s3:*",
      "Resource": [
        "arn:aws:s3:::my-app-assets-prod",
        "arn:aws:s3:::my-app-assets-prod/*"
      ],
      "Condition": {
        "StringNotEquals": {
          "aws:sourceVpce": "vpce-abc123"
        }
      }
    }
  ]
}

Lifecycle Rules

# Apply lifecycle configuration
aws s3api put-bucket-lifecycle-configuration \
  --bucket my-app-assets-prod \

…