Ctrl K

GPU Monitoring

GPU utilization, memory, and process monitoring with nvidia-smi for one-shot or refreshed views, and nvtop for an interactive interface.

GPU monitoring from the terminal. nvidia-smi gives a snapshot or a refreshed view via watch. nvtop is an interactive monitor that handles multiple GPUs and shows utilization graphs over time.

One-shot nvidia-smi

Print a single snapshot of GPU utilization, memory used and free, temperature, and running processes.

nvidia-smi

Live refresh with watch

Run nvidia-smi on a fixed interval and refresh the display in place. Useful for watching utilization and memory during training runs.

watch -n 1 nvidia-smi

Adjust the refresh interval as needed:

watch -n 2 nvidia-smi    # refresh every 2 seconds
watch -n 0.5 nvidia-smi  # refresh every 500ms

Exit with Ctrl+c.

Interactive monitor with nvtop

nvtop is a interactive monitor for GPUs. It shows per-GPU utilization graphs over time, memory usage, temperature, and the processes using each GPU. Better than watch + nvidia-smi when monitoring multiple GPUs or watching utilization patterns over time.

Install on Ubuntu and Debian:

sudo apt install nvtop

Install on Arch:

sudo pacman -S nvtop

Run:

nvtop

Exit with q.