Async Rust & Tokio Ecosystem

Async Rust is a scaling technique for concurrent I/O, not a default programming mode. Use it when you have hundreds+ of concurrent connections where thread-per-connection becomes impractical.

The Mental Model

async fn → compiler → state machine (enum with one variant per .await point)
         → runtime polls it → Pending? park. I/O ready? waker.wake() → re-poll
         → 1000s of tasks on a handful of OS threads (~few hundred bytes/task vs 2MB/thread)

Core trait:

pub trait Future {
    type Output;
    fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Self::Output>;
}

Futures are lazy — they do nothing until polled. Pin prevents the state machine from being moved in memory (protecting self-referential internal pointers). The runtime (Tokio) manages polling, parking, and waking.

The Tokio Stack

┌─────────────────────────────────────────┐
│  Application (async/await)              │
├──────────────┬──────────────────────────┤
│ axum (HTTP)  │  tonic (gRPC)            │
├──────────────┴──────────────────────────┤
│ tower (Service trait, middleware layers) │
├─────────────────────────────────────────┤
│ hyper (HTTP/1.1, HTTP/2)                │
├─────────────────────────────────────────┤
│ tokio (runtime, I/O, sync, timers)      │
├────────────┬───────────┬────────────────┤
│ bytes      │ futures   │ tracing        │
│ (zero-copy)│(combinat.)│(observability) │
└────────────┴───────────┴────────────────┘

Quick Decision Trees

Concurrency primitive:

• Need all results, no orphans? → tokio::join! / try_join!
• Need true parallelism across threads? → tokio::spawn
• Need first-to-complete or shutdown signal? → tokio::select!
• Dynamic set of tasks with abort-on-drop? → JoinSet
• Graceful drain without collecting results? → TaskTracker

Shared state:

• Lock NOT held across .await, low contention? → std::sync::Mutex
• Lock held across .await (last resort)? → tokio::sync::Mutex
• Complex state needing async operations? → Actor pattern (mpsc + dedicated task)

Channel type:

• Work distribution (N:1)? → mpsc
• Request/response (1:1)? → oneshot
• Event fan-out (N:M)? → broadcast
• Config/state snapshot (latest value)? → watch

Before Going Async, Ask:

1. Do I actually have concurrent I/O? (If not → blocking I/O)
2. CPU-bound or I/O-bound? (CPU → threads/rayon)
3. Will I exceed ~500-1000 concurrent connections? (If not → threads are simpler)
4. Can the team afford the complexity tax? (Worse errors, harder debugging, Pin/Send/Sync)
5. What happens when a future is cancelled at each .await? (If unsure → you have bugs)

Reference Files

Detailed guidance organized by topic:

References: mental-model
References: pattern-catalog
References: crate-composition
References: footguns-and-pitfalls
References: cancellation-safety
References: performance
References: error-handling
References: graceful-shutdown
References: testing-patterns
References: tower-middleware
References: resources