volt/docs/troubleshooting.md

Troubleshooting

Common issues and solutions for the Volt Platform.

Quick Diagnostics

Run these first to understand the state of your system:

# Platform health check
volt system health

# Platform info
volt system info

# What's running?
volt ps --all

# Daemon status
volt daemon status

# Network status
volt net status

---

Container Issues

Container Won't Start

Symptom: volt container start <name> fails or returns an error.

Check the logs first:

volt container logs <name>
volt logs <name>

Common causes:

1. Image not found

   Error: image "ubuntu:24.04" not found
   

   Pull the image first:

   sudo volt image pull ubuntu:24.04
   volt image list
   

2. Name conflict

   Error: container "web" already exists
   

   Delete the existing container or use a different name:

   volt container delete web
   

3. systemd-nspawn not installed

   Error: systemd-nspawn not found
   

   Install the systemd-container package:

   # Debian/Ubuntu
   sudo apt install systemd-container
   
   # Fedora/Rocky
   sudo dnf install systemd-container
   

4. Rootfs directory missing or corrupt

   ls -la /var/lib/volt/containers/<name>/rootfs/
   

   If empty or missing, recreate the container:

   volt container delete <name>
   volt container create --name <name> --image <image> --start
   

5. Resource limits too restrictive Try creating without limits, then add them:

   volt container create --name test --image ubuntu:24.04 --start
   volt container update test --memory 512M
   

Container Starts But Process Exits Immediately

Check the main process:

volt container logs <name>
volt container inspect <name>

Common cause: the container has no init process or the specified command doesn't exist in the image.

# Try interactive shell to debug
volt container shell <name>

Can't Exec Into Container

Symptom: volt container exec fails.

1. Container not running:

   volt ps --all | grep <name>
   volt container start <name>
   

2. Shell not available in image: The default shell (/bin/sh) might not exist in minimal images. Check:

   volt container exec <name> -- /bin/bash
   volt container exec <name> -- /bin/busybox sh
   

Container Resource Limits Not Working

Verify cgroup v2 is enabled:

mount | grep cgroup2
# Should show: cgroup2 on /sys/fs/cgroup type cgroup2

Check the cgroup settings:

volt container inspect <name> -o json | grep -i memory
cat /sys/fs/cgroup/system.slice/volt-container@<name>.service/memory.max

---

VM Issues

VM Won't Start

Check prerequisites:

# KVM available?
ls -la /dev/kvm

# QEMU installed?
which qemu-system-x86_64

# Kernel modules loaded?
lsmod | grep kvm

If /dev/kvm doesn't exist:

# Load KVM modules
sudo modprobe kvm
sudo modprobe kvm_intel  # or kvm_amd

# Check BIOS: virtualization must be enabled (VT-x / AMD-V)
dmesg | grep -i kvm

If permission denied on /dev/kvm:

# Add user to kvm group
sudo usermod -aG kvm $USER
# Log out and back in

# Or check group ownership
ls -la /dev/kvm
# Should be: crw-rw---- 1 root kvm

VM Starts But No SSH Access

1. VM might still be booting. Wait 30-60 seconds for first boot.

2. Check VM has an IP:

   volt vm list -o wide
   

3. SSH might not be installed/running in the VM:

   volt vm exec <name> -- systemctl status sshd
   

4. Network connectivity:

   # From host, ping the VM's IP
   ping <vm-ip>
   

VM Performance Issues

Apply a tuning profile:

volt tune profile apply <vm-name> --profile database

Or tune individually:

# Pin CPUs
volt tune cpu pin <vm-name> --cpus 4,5,6,7

# Enable hugepages
volt tune memory hugepages --enable --size 2M --count 4096

# Set I/O scheduler
volt tune io scheduler /dev/sda --scheduler none

---

Service Issues

Service Won't Start

# Check status
volt service status <name>

# View logs
volt service logs <name>

# View the unit file for issues
volt service show <name>

Common causes:

1. ExecStart path doesn't exist:

   which <binary-path>
   

2. User/group doesn't exist:

   id <service-user>
   # Create if missing
   sudo useradd -r -s /bin/false <service-user>
   

3. Working directory doesn't exist:

   ls -la <workdir-path>
   sudo mkdir -p <workdir-path>
   

4. Port already in use:

   ss -tlnp | grep <port>
   

Service Keeps Restarting

Check the restart loop:

volt service status <name>
volt service logs <name> --tail 50

If the service fails immediately on start, systemd may hit the start rate limit. Check:

# View full systemd status
systemctl status <name>.service

Temporarily adjust restart behavior:

volt service edit <name> --inline "RestartSec=10"

Can't Delete a Service

# If it says "refusing to delete system unit"
# Volt protects system services. Only user-created services can be deleted.

# If stuck, manually:
volt service stop <name>
volt service disable <name>
volt service delete <name>

---

Networking Issues

No Network Connectivity from Container

1. Check bridge exists:

   volt net bridge list
   

   If volt0 is missing:

   sudo volt net bridge create volt0 --subnet 10.0.0.0/24
   

2. Check IP forwarding:

   volt tune sysctl get net.ipv4.ip_forward
   # Should be 1. If not:
   sudo volt tune sysctl set net.ipv4.ip_forward 1 --persist
   

3. Check NAT/masquerade rules:

   sudo nft list ruleset | grep masquerade
   

4. Check container has an IP:

   volt container inspect <name>
   

Workloads Can't Resolve Names

1. Check internal DNS:

   volt net dns list
   

2. Flush DNS cache:

   volt net dns flush
   

3. Check upstream DNS in config:

   volt config get network.dns.upstream
   

Port Forward Not Working

1. Verify the forward exists:

   volt net port list
   

2. Check the target is running and listening:

   volt ps | grep <target>
   volt container exec <target> -- ss -tlnp
   

3. Check firewall rules:

   volt net firewall list
   

4. Check for host-level firewall conflicts:

   sudo nft list ruleset
   sudo iptables -L -n  # if iptables is also in use
   

Firewall Rule Not Taking Effect

1. List current rules:

   volt net firewall list
   

2. Rule ordering matters. More specific rules should come first. If a broad deny rule precedes your accept rule, traffic will be blocked.

3. Flush and recreate if confused:

   volt net firewall flush
   # Re-add rules in the correct order
   

---

Daemon Issues

Daemon Not Running

volt daemon status
# If not running:
sudo volt daemon start

Check systemd:

systemctl status volt.service
journalctl -u volt.service --no-pager -n 50

Daemon Won't Start

1. Socket in use:

   ls -la /var/run/volt/volt.sock
   # Remove stale socket
   sudo rm /var/run/volt/volt.sock
   sudo volt daemon start
   

2. Config file invalid:

   volt config validate
   

3. Missing directories:

   sudo mkdir -p /var/lib/volt /var/run/volt /var/log/volt /var/cache/volt /etc/volt
   

4. PID file stale:

   cat /var/run/volt/volt.pid
   # Check if that PID exists
   ps -p $(cat /var/run/volt/volt.pid)
   # If no process, remove it
   sudo rm /var/run/volt/volt.pid
   sudo volt daemon start
   

Commands Timeout

# Increase timeout
volt --timeout 120 <command>

# Or check if daemon is overloaded
volt daemon status
volt top

---

Permission Issues

"Permission denied" Errors

Most state-changing operations require root or volt group membership:

# Add user to volt group
sudo usermod -aG volt $USER
# Log out and back in for group change to take effect

# Or use sudo
sudo volt container create --name web --image ubuntu:24.04 --start

Read-Only Operations Work, Write Operations Fail

This is expected for non-root, non-volt-group users. These commands always work:

volt ps                # Read-only
volt top               # Read-only
volt logs <name>       # Read-only
volt service list      # Read-only
volt config show       # Read-only

These require privileges:

volt container create  # Needs root/volt group
volt service create    # Needs root
volt net firewall add  # Needs root
volt tune sysctl set   # Needs root

---

Storage Issues

Disk Space Full

# Check disk usage
volt system info

# Clean up unused images
volt image list
volt image delete <unused-image>

# Clean CAS garbage
volt cas gc --dry-run
volt cas gc

# Clear cache (safe to delete)
sudo rm -rf /var/cache/volt/*

# Check container sizes
du -sh /var/lib/volt/containers/*/

CAS Integrity Errors

# Verify CAS store
volt cas verify

# If corrupted objects are found, re-pull affected images
volt image delete <affected-image>
volt image pull <image>

Volume Won't Attach

1. Volume exists?

   volt volume list
   

2. Already attached?

   volt volume inspect <name>
   

3. Target workload running? Volumes can typically only be attached to running workloads.

---

Compose Issues

volt compose up Fails

1. Validate the compose file:

   volt compose config
   

2. Missing images:

   volt compose pull
   

3. Dependency issues: Check that depends_on targets exist in the file and their conditions can be met.

4. Network conflicts: If subnets overlap with existing networks:

   volt net list
   

Environment Variables Not Resolving

# Check .env file exists in same directory as compose file
cat .env

# Variables must be set in the host environment or .env file
export DB_PASSWORD=mysecret
volt compose up

Undefined variables with no default cause an error. Use default syntax:

environment:
  DB_PASSWORD: "${DB_PASSWORD:-defaultpass}"

---

Exit Codes

Use exit codes in scripts for error handling:

Code	Meaning	Action
0	Success	Continue
2	Bad arguments	Fix command syntax
3	Not found	Resource doesn't exist
4	Already exists	Resource name taken
5	Permission denied	Use sudo or join volt group
6	Daemon down	sudo volt daemon start
7	Timeout	Retry with --timeout
9	Conflict	Resource in wrong state

volt container start web
case $? in
  0) echo "Started" ;;
  3) echo "Container not found" ;;
  5) echo "Permission denied — try sudo" ;;
  6) echo "Daemon not running — sudo volt daemon start" ;;
  9) echo "Already running" ;;
  *) echo "Error: $?" ;;
esac

---

Collecting Debug Info

When reporting issues, gather:

# Version
volt --version

# System info
volt system info -o json

# Health check
volt system health

# Daemon logs
journalctl -u volt.service --no-pager -n 100

# Run the failing command with debug
volt --debug <failing-command>

# Audit log
tail -50 /var/log/volt/audit.log

Factory Reset

If all else fails, reset Volt to defaults. This is destructive — it stops all workloads and removes all configuration.

sudo volt system reset --confirm

After reset, reinitialize:

sudo volt daemon start
volt system health