Performance Tuning

Optimize Linux system performance through kernel parameter tuning, I/O scheduler selection, memory management, CPU governor configuration, and benchmarking. Covers methodology, real sysctl settings, and tool-based validation.

When to Use

• Server experiencing high latency, throughput bottlenecks, or resource exhaustion
• Preparing infrastructure for high-traffic events or load tests
• Tuning a database server, web server, or application host for production
• Diagnosing whether a bottleneck is CPU, memory, disk I/O, or network
• Establishing baseline performance metrics before and after changes
• Configuring kernel parameters for containers, VMs, or bare-metal hosts

Prerequisites

• Root or sudo access on the target system
• sysstat package installed (provides sar, iostat, mpstat)
• linux-tools or perf package for CPU profiling
• Benchmarking tools: fio (disk), sysbench (CPU/memory), iperf3 (network)
• Baseline metrics collected before making any changes

Performance Analysis Methodology

Always follow this order:

1. Collect baseline -- measure current performance with tools
2. Identify bottleneck -- determine if CPU, memory, I/O, or network
3. Change one parameter -- apply a single tuning change
4. Measure impact -- re-run the same benchmark
5. Document -- record the change and its effect
6. Iterate or revert -- keep the change if beneficial, revert if not

System Monitoring Tools

# CPU and process monitoring
top                              # Interactive process viewer
htop                             # Enhanced interactive viewer
mpstat -P ALL 2                  # Per-CPU utilization every 2 seconds
pidstat -u 2                     # Per-process CPU usage

# Memory monitoring
free -h                          # Memory summary
vmstat 2                         # Virtual memory stats every 2 seconds
# Columns: r=runnable, b=blocked, si/so=swap in/out, bi/bo=block I/O

# Disk I/O monitoring
iostat -xz 2                     # Extended disk stats every 2 seconds
# Key columns: %util, await (latency), r/s, w/s
iotop -oP                        # Show processes doing I/O

# Network monitoring
sar -n DEV 2                     # Network interface stats
ss -s                            # Socket summary
nstat                            # Network counters

# CPU profiling (requires perf)
perf top                         # Real-time function-level CPU profiling
perf stat -a sleep 10            # System-wide counters for 10 seconds
perf record -g -a sleep 30       # Record 30 seconds of call stacks
perf report                      # Analyze recorded data

# One-liner: check all major resources
echo "=== CPU ===" && mpstat 1 1 && echo "=== MEM ===" && free -h && echo "=== DISK ===" && iostat -x 1 1 && echo "=== NET ===" && ss -s

Sysctl Kernel Parameter Tuning

Network Tuning

# /etc/sysctl.d/60-network-performance.conf

# Increase the maximum socket receive/send buffer sizes
net.core.rmem_max = 134217728
net.core.wmem_max = 134217728
net.core.rmem_default = 1048576
net.core.wmem_default = 1048576

# TCP buffer auto-tuning (min, default, max in bytes)
net.ipv4.tcp_rmem = 4096 1048576 134217728
net.ipv4.tcp_wmem = 4096 1048576 134217728

# Increase connection backlog for high-traffic servers
net.core.somaxconn = 65535
net.ipv4.tcp_max_syn_backlog = 65535
net.core.netdev_max_backlog = 65535

# Enable TCP fast open (client and server)
net.ipv4.tcp_fastopen = 3

# Reuse TIME_WAIT sockets for new connections
net.ipv4.tcp_tw_reuse = 1

# Increase the range of ephemeral ports
net.ipv4.ip_local_port_range = 1024 65535

# TCP keepalive tuning (detect dead connections faster)
net.ipv4.tcp_keepalive_time = 120
net.ipv4.tcp_keepalive_intvl = 30
net.ipv4.tcp_keepalive_probes = 3

# Disable slow start after idle (keeps congestion window open)
net.ipv4.tcp_slow_start_after_idle = 0

# Enable BBR congestion control (requires kernel 4.9+)
net.core.default_qdisc = fq
net.ipv4.tcp_congestion_control = bbr

Memory Tuning

# /etc/sysctl.d/60-memory-performance.conf

# Reduce swappiness (0-100, lower = less swap usage)
# 10 for general servers, 1 for database servers
vm.swappiness = 10

# Dirty page ratios (controls when dirty data is flushed to disk)
# Lower values = more frequent, smaller writes (better for SSDs)
vm.dirty_ratio = 20
vm.dirty_background_ratio = 5

# For large-memory systems writing to fast storage

…