# Spinlocks vs Mutexes: When Burning CPU Is the Right Call

> One lock burns CPU while it waits; the other pays a syscall to sleep. The right choice depends on hold time versus context-switch cost — and the best answer, futexes and adaptive locks, is 'both, in that order'.

URL: https://harshith.in/blog/spinlocks-vs-mutexes/
Author: Harshith Sunku (Staff Engineer) — https://harshith.in/about/
Tags: concurrency, performance, linux

Two threads want the same lock. Thread B finds it taken. Now what? Everything
about lock design flows from the two possible answers: **keep asking** (spin)
or **go to sleep and get woken** (block). One wastes CPU to save latency; the
other pays latency to save CPU. Picking wrong in the kernel means wasted cores;
picking wrong in userspace means your latency-optimized spinlock performs
*worse* than the mutex you replaced — for reasons that only make sense once you
see what the scheduler does to spinners.

## What Spinning Really Does to the Machine

A naive spinlock hammers an atomic exchange in a loop. The problem isn't just
burned cycles — it's the **cache coherence traffic**. An atomic RMW needs the
line in exclusive state, so N spinners bounce the lock's cache line between
cores continuously, and that traffic slows down *the lock holder too* — the
one thread you want running fast. Hence the canonical shape:

```c
while (atomic_exchange(&lk, 1)) {          /* one RMW attempt */
    while (atomic_load(&lk)) cpu_relax();  /* then spin read-only on our copy */
}
```

Test-and-test-and-set: spinners wait on a *shared* cached copy (quiet), storm
the line only when it's freed. `cpu_relax()` is the `pause` instruction —
politeness to the pipeline and to SMT siblings. Past a dozen cores even this
storms too hard, which is why serious kernels moved to ticket locks (FIFO,
bounded unfairness) and then MCS/qspinlocks, where each waiter spins on its
*own* cache line and the queue hands the lock off point-to-point. Lock design
is cache-line choreography wearing a concurrency costume.

## The Decision Arithmetic

Blocking has a fixed toll: parking a thread and waking it later costs a couple
of syscalls and a context switch — call it 1–10 µs end to end. Spinning costs
(waiters × hold-time) of CPU. So:

- **Hold time ≪ context-switch cost** (nanoseconds: update a counter, unlink a
  node) → spinning wins; sleeping would cost 100× the wait.
- **Hold time ≥ switch cost** (I/O, allocation, anything unbounded) → blocking
  wins; spinning burns a core-second per second per waiter.

The kernel uses spinlocks precisely where code *cannot* sleep (interrupt
context) and holds are provably tiny; mutexes elsewhere. So far, symmetric. The
asymmetry — the trap — is userspace.

## Why Userspace Spinning Backfires

A kernel spinlock holder has preemption disabled: it *will* finish promptly.
Your userspace thread enjoys no such promise. The scheduler can preempt the
lock **holder** mid-critical-section — and now every spinner is burning its
full timeslice waiting for a thread that isn't running. Worse, the spinners
look like busy, CPU-hungry threads, so they *outcompete the holder for CPU*.
Add priority differences and you get priority inversion with a blowtorch.
Malte Skarupke's benchmarks of this ("measuring mutexes, spinlocks and how bad
the Linux scheduler really is", 2020) plus Linus Torvalds' memorable response
made the canonical point: in userspace, pure spinlocks are almost always
wrong, and your benchmark won't show it until the machine is loaded.

## The Grown-Up Answer: Futex and Friends

Linux mutexes aren't "syscall every time" — that's a 2003 problem. A glibc
mutex is a **futex**: an atomic word in userspace, and the kernel is involved
*only on contention*:

```text
  lock:   CAS 0→1 succeeds?           done — zero syscalls
          contended? set to 2, futex_wait(&word, 2)   → sleep
  unlock: set 0; was it 2? futex_wake(&word, 1)
```

Uncontended lock/unlock is a few nanoseconds of pure userspace atomics — the
fast path costs what a spinlock costs. And the real cure for the
holder-preempted problem is **adaptive locking**: spin briefly *while the
holder is running on another CPU*, then block
(`PTHREAD_MUTEX_ADAPTIVE_NP`, kernel rt-mutex optimistic spinning, most
language runtimes' locks). You get spin latency when holds are short and
blocking's citizenship when they aren't — the dichotomy dissolved into a
policy knob.

Practical defaults follow: use the platform mutex (it's already adaptive-ish
and futex-fast); reach for explicit spinning only for provably-tiny critical
sections on dedicated/pinned threads (DPDK-style polling designs, where the
core is spent anyway); and if a lock is hot enough that any of this matters,
the winning move is usually to shard or remove the shared state rather than
tune the lock guarding it.

## Takeaways

- Spin when expected wait < context-switch cost (~µs); block when holds are
  long or unbounded. Kernel interrupt context spins because it must.
- Real spinlocks are cache-line engineering: TTAS, ticket, MCS — each step
  reduces coherence storms.
- Userspace spinning breaks when the scheduler preempts the holder; spinners
  then starve the one thread that matters.
- Futexes make uncontended mutexes nearly free; adaptive mutexes spin-then-
  block, taking both wins.
- A truly hot lock is a design smell — shard the state before micro-optimizing
  the lock.