Introduction

Deloxide is a runtime deadlock detection and diagnosis toolkit for Rust, with a secondary C interface. It turns a hanging tracked workload into a concrete thread-and-lock cycle, then gives you tools to reproduce, understand, and prevent the same failure.

The default detector follows waits between threads using Deloxide’s Mutex, RwLock, and Condvar. When the current waits form a cycle, the callback receives the participating thread IDs and the lock each thread is trying to acquire.

ThreadId(2) waits for LockId(7), owned by ThreadId(3)
ThreadId(3) waits for LockId(4), owned by ThreadId(2)

That is a WaitForGraph report: an active cycle observed among tracked synchronization. Deloxide also provides:

• an optional lock-order graph that finds risky acquisition patterns before they become an active deadlock;
• random and component-based stress modes that make rare schedules easier to reproduce;
• custom callbacks that run application-defined incident handling;
• asynchronous event logging and an interactive visualization; and
• C bindings for the same tracked primitives.

Typical workflow

Deloxide is designed to remain useful through the whole investigation:

1. Replace the locks around the suspicious path.
2. Reproduce the hang and receive an active cycle.
3. Add visualization when the IDs alone are not enough.
4. Use lock-order analysis to find the inversion earlier.
5. Use stress modes when the schedule rarely manifests.
6. Keep the default detector in production when the measured cost fits the application.

The Optimistic Fast Path keeps eligible uncontended Mutex and exclusive RwLock operations away from global graph work. The broader evaluation, methodology, and comparisons are described in the Deloxide preprint and the performance chapter.

Custom callbacks

The callback is part of the default detector and does not require logging. Your application can persist the report, export telemetry, notify an incident system, capture additional diagnostics, or signal a supervisor. Keep the callback bounded and hand slow work to an application-owned queue.

The lifecycle and callbacks chapter explains initialization, panic containment, queue handoff, and shutdown behavior with complete examples.

What Deloxide can observe

Deloxide sees synchronization performed through its wrappers. It cannot build a complete cycle through raw locks, channels, I/O, another process, or a remote service. That is why incremental adoption should cover every lock on the suspected cycle, not only the line where the final thread happened to block.

This manual complements docs.rs. It explains the workflow, feature choices, evidence, examples, C integration, and production trade-offs. Use the API documentation for exact signatures and trait details.

Continue with Installation, then run Your first diagnosis.

Installation

Deloxide’s default build includes active deadlock detection:

[dependencies]
deloxide = "1.1"

Then initialize it once, before creating the locks and threads you want to observe:

#![allow(unused)]
fn main() {
extern crate deloxide;
use deloxide::Deloxide;

Deloxide::new()
    .callback(|report| eprintln!("{report:#?}"))
    .start()
    .expect("start Deloxide");
}

Replace the synchronization on the suspicious path with Deloxide’s Mutex, RwLock, and Condvar. Their guards behave like familiar Rust lock guards. Using deloxide::thread also records thread lifecycle information.

Optional Cargo features add deeper investigation tools:

[dependencies]
deloxide = { version = "1.1", features = [
  "logging-and-visualization",
  "lock-order-graph",
  "stress-test",
] }

Start with the default build. Add logging when you need a timeline, lock-order analysis when you want to find risky acquisition patterns, and stress testing when a bug rarely reproduces. Choosing a mode explains the difference.

Rust is the primary interface. C projects can build Deloxide as a library and use include/deloxide.h; see C integration.

For exact methods and types, use the Rust API documentation.

Your first diagnosis

This example deliberately creates the classic two-lock cycle.

1. Start the detector

#![allow(unused)]
fn main() {
extern crate deloxide;
use deloxide::{DeadlockInfo, Deloxide};

Deloxide::new()
    .callback(|report: DeadlockInfo| {
        eprintln!("source: {:?}", report.source);
        eprintln!("threads: {:?}", report.thread_cycle);
        eprintln!("waited locks: {:?}", report.thread_waiting_for_locks);
    })
    .start()
    .expect("start Deloxide");
}

Initialize Deloxide before creating the tracked locks and worker threads.

2. Create opposite lock order

#![allow(unused)]
fn main() {
extern crate deloxide;
use deloxide::{Mutex, thread};
use std::sync::{Arc, Barrier};
let left = Arc::new(Mutex::new(()));
let right = Arc::new(Mutex::new(()));
let barrier = Arc::new(Barrier::new(2));
let left_a = Arc::clone(&left);
let right_a = Arc::clone(&right);
let barrier_a = Arc::clone(&barrier);
let left_b = Arc::clone(&left);
let right_b = Arc::clone(&right);
let barrier_b = Arc::clone(&barrier);
let first = thread::spawn(move || {
    let _left = left_a.lock();
    barrier_a.wait();
    let _right = right_a.lock();
});

let second = thread::spawn(move || {
    let _right = right_b.lock();
    barrier_b.wait();
    let _left = left_b.lock();
});
let _ = (first, second);
}

The barrier makes both threads keep their first lock before requesting the second. Deloxide sees the active cycle and calls the callback.

3. Read the result

source: WaitForGraph
threads: [ThreadId(2), ThreadId(3)]
waited locks: [(ThreadId(2), LockId(2)), (ThreadId(3), LockId(1))]

Fix the program by choosing one lock order and using it on every path.

Run the complete example:

cargo run --example diagnose_deadlock

Source: examples/diagnose_deadlock.rs. Continue with Reading a report for self-deadlocks, RwLock, condition variables, and missing evidence.

Choosing a mode

Start with the default detector. Add one optional feature only when it answers a question you actually have.

Active versus potential

WaitForGraph means Deloxide validated a current cycle among tracked waits and owners.

LockOrderViolation means the program previously acquired locks in conflicting orders. It is useful early warning, but it does not mean threads are blocked right now.

A practical progression

1. Reproduce with the default detector.
2. Add logging if the callback IDs are not enough to find the path.
3. Add lock-order analysis in development or CI to catch inversions earlier.
4. Add stress mode only when the failure rarely manifests.
5. Benchmark the exact feature combination before a broad rollout.

Optional features add graph work, event queueing, file I/O, or intentional delays. They are investigation tools, not a reason to enable everything at once.

Adopt the Tracked Primitives

Deloxide sees synchronization only when the code uses its tracked wrappers. Start by replacing the locks on the path you are investigating, initialize the detector before that path runs, and expand from there. The wrappers keep the familiar guard-based style while reporting supported lock activity to the detector.

#![allow(unused)]
fn main() {
extern crate deloxide;
use deloxide::{Condvar, Deloxide, Mutex, RwLock};

Deloxide::new()
    .callback(|report| eprintln!("deadlock report: {report:?}"))
    .start()
    .expect("detector initialization");

let counter = Mutex::new(0);
*counter.lock() += 1;

let settings = RwLock::new(String::from("ready"));
assert_eq!(settings.read().as_str(), "ready");
settings.write().push_str(" for work");

let ready = Condvar::new();
let _ = ready;
}

For complete item documentation, see Mutex, RwLock, Condvar, and Deloxide::start.

Replace imports, not the locking model

For selected code, replace either family of imports with Deloxide’s types:

// Before: use std::sync::{Condvar, Mutex, RwLock};
// Before: use parking_lot::{Condvar, Mutex, RwLock};
use deloxide::{Condvar, Mutex, RwLock};

This is an import diff rather than a Rust example. The runnable forms are the repository’s basic_mutex, rwlock, and condvar examples.

Mutex::lock, RwLock::read, and RwLock::write return guards directly. A guard dereferences to its protected value, and dropping it always releases the physical lock. The wrapper also reports the release globally when active tracking or logging requires it; uncontended fast paths avoid unnecessary global detector work. Keep the usual narrow scopes and explicit drop(guard) where the release point matters. The wrappers use parking_lot internally; they do not expose std::sync poisoning or LockResult/PoisonError. In particular, remove .unwrap() or poisoned-lock recovery that existed only to handle the standard library result:

#![allow(unused)]
fn main() {
extern crate deloxide;
use deloxide::Mutex;

let jobs = Mutex::new(Vec::<String>::new());
jobs.lock().push("index".to_owned()); // No LockResult to unwrap.
assert_eq!(jobs.lock().len(), 1);
}

That is a semantic migration, not merely a type alias: code that relies on poisoning as an application health signal needs its own explicit failure state.

Use each wrapper with its matching guard

Mutex is for exclusive access. RwLock has distinct read and write guards: multiple reads may coexist, while a write is exclusive. A Deloxide Condvar waits with a mutable Deloxide MutexGuard; do not mix it with a std::sync::Mutex or parking_lot::Mutex guard. Its wait methods release the associated mutex while waiting and reacquire it before returning.

#![allow(unused)]
fn main() {
extern crate deloxide;
use deloxide::{Condvar, Mutex};
use std::sync::Arc;

let state = Arc::new((Mutex::new(false), Condvar::new()));
let (lock, wake) = &*state;
let mut started = lock.lock();
while !*started {
    // `wait` returns with `started` holding the same tracked mutex again.
    wake.wait(&mut started);
}
}

Use the normal predicate loop: wakeups are not a reason to assume the predicate is true. The wrapper also offers Condvar::wait_timeout, wait_while, and wait_timeout_while. Unlike std::sync::Condvar, these methods mutate the supplied guard in place: wait_timeout and wait_timeout_while return bool (true means the timeout elapsed), rather than returning a guard/result pair.

There is no read-to-write upgrade method. Release a read guard before taking a write guard; attempting a blocking write while retaining a read guard can self-deadlock.

#![allow(unused)]
fn main() {
extern crate deloxide;
use deloxide::RwLock;

let cache = RwLock::new(vec![1, 2]);
{
    let read = cache.read();
    assert_eq!(read.len(), 2);
} // The read guard is gone before the write attempt.
cache.write().push(3);
}

Nonblocking probes

Mutex::try_lock, RwLock::try_read, and RwLock::try_write are nonblocking. They return Option<Guard>: Some owns the acquired guard and None means that attempt could not acquire the lock immediately. They do not return a TryLockError and must not be treated as an eventual wait.

#![allow(unused)]
fn main() {
extern crate deloxide;
use deloxide::{Mutex, RwLock};

let mutex = Mutex::new(1);
if let Some(mut value) = mutex.try_lock() {
    *value += 1;
}

let config = RwLock::new(10);
let snapshot = config.try_read().map(|value| *value);
if let Some(mut value) = config.try_write() {
    *value += 1;
}
assert!(snapshot.is_some());
}

Roll out without overstating coverage

Begin with the locks shared by the suspected operations, convert every endpoint of that dependency, and reproduce the scenario. Then migrate adjacent lock families and worker entry points. A useful rollout order is:

1. Initialize Deloxide before the instrumented workload.
2. Convert the locks and condition variables in one coherent operation.
3. Convert the threads that create that operation’s workers to deloxide::thread.
4. Exercise the path and use reports to guide the next boundary.

An untracked boundary is a visibility boundary. If a participant holds a standard/parking_lot lock, waits on a different synchronization primitive, or uses a condition variable paired with an untracked mutex, Deloxide cannot form all of that dependency’s edges. A report is evidence about the tracked primitives, not proof that the rest of the process is free of deadlocks. Keep the original primitives where migration is not yet safe, but document the gap and avoid interpreting the mixed deployment as complete coverage.

For active findings, distinguish WaitForGraph (a current, validated cycle) from LockOrderViolation (a potential historical order cycle). Continue with lifecycle and callback guidance before enabling the detector in a long-running process.

Manage Lifecycle and Callbacks

Initialize Deloxide once, at the process boundary, before instrumented threads begin work and before the locks whose behavior you want to diagnose are created. Configuration is process-wide; it is not a per-request, per-test-case, or per-worker service.

#![allow(unused)]
fn main() {
extern crate deloxide;
use deloxide::{DeadlockSource, Deloxide};

Deloxide::new()
    .callback(|report| match report.source {
        DeadlockSource::WaitForGraph => {
            eprintln!("active tracked deadlock: {:?}", report.thread_cycle);
        }
        DeadlockSource::LockOrderViolation => {
            eprintln!("potential lock-order cycle: {:?}", report.lock_order_cycle);
        }
    })
    .start()
    .expect("Deloxide must start before the workload");

// Construct tracked locks and start worker/request processing after this point.
}

WaitForGraph means an active, validated cycle among the detector’s currently tracked waits and incompatible owners. LockOrderViolation, available with the optional order-graph feature, is a potential ordering risk rather than an active deadlock. See Reading a Deadlock Report for the triage workflow and the exact Deloxide and DeadlockSource APIs.

One detector, partial repeated-start behavior

Deloxide keeps a global detector for the process. Calling Deloxide::start does not create an isolated detector or reject a call merely because one start already completed. A later call still runs initialization and, assuming any requested logger can be constructed, normally returns Ok(()). Its effects are deliberately not an all-or-nothing reconfiguration:

• The callback uses a OnceLock; the first callback successfully installed in the process handles later reports. A later builder’s callback is not installed.
• With logging-and-visualization, the global logger has its own OnceLock. The first logger successfully installed receives later events. An earlier no_logging() start leaves that slot empty, so a later start can install the first logger. Once installed, a later log path does not replace it, although that later start() still attempts to create its configured logger before the one-time install and can return an I/O error.
• With lock-order-graph, a later start with checking enabled creates or replaces the detector’s order graph with a new graph. A later start with checking disabled does not remove an order graph that already exists.
• With stress-test, every start overwrites the process-wide stress mode and stress configuration, including overwriting them with the builder defaults.

Existing ownership, wait, and other detector state is not cleared as one coherent reset while those feature-specific fields change. Repeated starts are therefore partial, unsupported reconfiguration, not a reliable reset or runtime toggle. Initialize once before instrumented work and use a separate process when a clean configuration or detector state is required. There is deliberately no public shutdown, reset, or reconfigure API in this guidance.

This matters in tests. Put cases requiring different Deloxide configurations in separate test processes (for example, separate integration-test binaries or separate cargo test --test name invocations), rather than parallel tests in one process. A test that installs the default callback can affect every later test in that binary.

The default callback panics with the report, but callback execution is isolated; choose an explicit callback for application policy instead of assuming that default behavior terminates the process.

Use tracked thread entry points

deloxide::thread re-exports common std::thread items such as JoinHandle, current, sleep, park, and yield_now. It provides tracked versions of:

• thread::spawn,
• thread::Builder with spawn, and
• Builder::spawn_scoped.

These helpers register thread creation and exit. On creation they retain the parent’s Deloxide thread ID; with logging enabled, that parent/child information is emitted with the thread-spawn event so a log can relate a worker to its creator.

#![allow(unused)]
fn main() {
extern crate deloxide;
use deloxide::thread;

let named = thread::Builder::new()
    .name("indexer".to_owned())
    .spawn(|| 42)
    .expect("worker creation");
assert_eq!(named.join().expect("worker result"), 42);

let value = 0;
thread::scope(|scope| {
    // `scope.spawn` is std's scoped spawn; use the tracked Builder helper here.
    thread::Builder::new()
        .spawn_scoped(scope, || assert_eq!(value, 0))
        .expect("scoped worker creation")
        .join()
        .expect("scoped worker result");
});
}

thread::scope provides the standard scoped-thread boundary, but its Scope is the standard library type. Therefore scope.spawn(...) is an ordinary scoped spawn; use thread::Builder::spawn_scoped(scope, ...) when creation/exit tracking matters.

Ordinary std::thread::spawn does not make the detector blind to every operation inside that thread: a standard thread that locks a Deloxide Mutex, RwLock, or uses a compatible Deloxide Condvar still runs the wrapper code, so supported lock waits and acquisitions can be observed. What it lacks is the tracked thread’s spawn/exit registration and parent/child log relationship. Ordinary threads also do not make std::sync or parking_lot locks observable; migrate those lock instances separately.

Keep callbacks an alert handoff

Deloxide queues callbacks to one background dispatcher thread rather than running them on the thread that detected the finding. A panic from one callback invocation is caught, reported to stderr, and does not stop that dispatcher from handling a later report. That isolation is useful, but it is not permission to do recovery work in the callback: callbacks are serialized, and a callback can still block on an application lock, do slow I/O, or delay every subsequent report.

Keep the handler bounded and nonblocking. Copy or move the DeadlockInfo into a bounded queue with try_send, count overload, and let a separate supervisor persist, page, or capture diagnostics.

#![allow(unused)]
fn main() {
extern crate deloxide;
use deloxide::{DeadlockInfo, Deloxide};
use std::sync::{
    Arc,
    atomic::{AtomicU64, Ordering},
    mpsc,
};

let (reports_tx, reports_rx) = mpsc::sync_channel::<DeadlockInfo>(64);
let dropped = Arc::new(AtomicU64::new(0));
let callback_dropped = Arc::clone(&dropped);

Deloxide::new()
    .callback(move |report| {
        if reports_tx.try_send(report).is_err() {
            callback_dropped.fetch_add(1, Ordering::Relaxed);
        }
    })
    .start()
    .expect("detector initialization");

let _supervisor_inputs = (reports_rx, dropped);
}

The process may exit, abort, or be terminated before a queued callback, a log write, or the supervising task completes. Do not make process termination from a callback your evidence-preservation strategy; persist or export what you need on the normal incident path, and test that path independently.

Select Features and Configuration

Deloxide’s features are Cargo compile-time choices, not runtime switches. Start with the default active wait-for detector, then add only the evidence or test behavior needed for the environment. The public base API is always Deloxide, Mutex, RwLock, Condvar, DeadlockInfo, and thread.

Use one dependency line in the application’s Cargo.toml; the cells above are complete alternatives, not lines to combine.

Builder defaults follow compiled features

Deloxide::new always supplies a callback that panics with the report unless you replace it with callback. When logging-and-visualization is compiled, it enables logging by default with the path deloxide.log; change that path with with_log, or disable logging for the initial configuration with no_logging. When lock-order-graph is compiled, lock-order checking is enabled by default; make the initial policy explicit with with_lock_order_checking, or use no_lock_order_checking for a controlled baseline. Stress compilation alone does not add delays: select random or component stress with its corresponding builder method.

#![allow(unused)]
fn main() {
extern crate deloxide;
#[cfg(feature = "logging-and-visualization")]
{
    use deloxide::Deloxide;

    Deloxide::new()
        .with_log("logs/deloxide_{timestamp}.log")
        .callback(|report| eprintln!("{report:?}"))
        .start()
        .expect("logging detector initialization");
}
#[cfg(not(feature = "logging-and-visualization"))]
{
    let _ = "this configuration needs the logging-and-visualization feature";
}
}

#![allow(unused)]
fn main() {
extern crate deloxide;
#[cfg(feature = "lock-order-graph")]
{
    use deloxide::{DeadlockSource, Deloxide};

    Deloxide::new()
        .with_lock_order_checking()
        .callback(|report| match report.source {
            DeadlockSource::WaitForGraph => eprintln!("active cycle"),
            DeadlockSource::LockOrderViolation => eprintln!("potential order cycle"),
        })
        .start()
        .expect("order-checking detector initialization");
}
#[cfg(not(feature = "lock-order-graph"))]
{
    let _ = "this configuration needs the lock-order-graph feature";
}
}

Both builder calls are feature-gated at compile time. Do not put them behind only a runtime if: a binary built without the feature has no such methods.

Choose the evidence level deliberately

The lock-order-graph feature can report DeadlockSource::LockOrderViolation when observed acquisitions close a historical order cycle. It is useful early in development, but it is not evidence that threads are blocked now. The base wait-for detector’s DeadlockSource::WaitForGraph is the active, validated-cycle finding. Keep those response paths distinct even when all features are compiled.

Stress mode changes scheduling to make a suspected bug easier to reproduce; it does not turn a potential order warning into a confirmed deadlock. Logging adds history for the supported events it receives, not a complete trace of every thread and primitive in the process. See Choosing a Mode for operational trade-offs, Finding Inconsistent Lock Order for potential findings, and Stress Test a Suspected Race for test-only stress workflows.

Cargo features are fixed when the application is built. Choose one runtime builder configuration and start it before instrumented work. Repeated start() calls are accepted, but their effects are asymmetric: the first successfully installed callback and global logger win; an enabled lock-order graph is created or replaced, while a later disabled setting does not remove an existing graph; and stress mode/configuration is overwritten on each start. Existing ownership and wait state is not reset coherently with those changes. Repeated starts are therefore partial, unsupported reconfiguration, not a reliable reset or toggle. Use separate processes for clean configurations; see Manage Lifecycle and Callbacks for the exact behavior.

Reading a Deadlock Report

The callback receives a DeadlockInfo value. Start with source: it decides whether the report describes a blocked cycle now or an ordering risk observed earlier.

• DeadlockSource::WaitForGraph is an active, validated wait-for cycle. Treat it as an incident: the listed threads are blocked on incompatible owners in the detector’s current snapshot.
• DeadlockSource::LockOrderViolation is a potential lock-order violation. It says executions have established a circular ordering rule, not that those threads are currently stuck. Reproduce and decide whether the paths can overlap before escalating it as an outage.

Field-by-field procedure

The ordered cycle is deliberately directional. Given thread_cycle: [101, 202] and thread_waiting_for_locks: [(101, 17), (202, 42)], investigate: thread 101 is waiting for 17, held incompatibly by 202; thread 202 is waiting for 42, held incompatibly by 101. The same list is not a claim that the IDs are sorted, or that every holder of a shared lock must be in the cycle.

Example: active wait-for report

source: WaitForGraph
thread_cycle: [101, 202]
thread_waiting_for_locks: [(101, 17), (202, 42)]
lock_order_cycle: None
timestamp: "2026-07-29T09:41:12.807Z"
verification_request: None

This is an active deadlock report. Capture the two thread stacks, then search the lock construction sites for IDs 17 and 42 in the optional event log. The likely shape is 101 holding 42 while requesting 17, and 202 holding 17 while requesting 42. Fix the overlapping critical sections or establish one acquisition order; see two-Mutex inversion.

Example: potential lock-order report

source: LockOrderViolation
thread_cycle: [202]
thread_waiting_for_locks: [(202, 17)]
lock_order_cycle: Some([17, 42, 17])
timestamp: "2026-07-29T09:48:33.042Z"
verification_request: None

This is not proof that 202 is blocked. It means prior execution recorded 17 before 42, while this acquisition, holding 42 and requesting 17, closes the historical cycle. Inspect both paths, determine whether they run concurrently and share the same lock instances, then use the lock-order workflow. If the overlap is real, use stress testing to seek an active WaitForGraph report; absence of one in a finite run does not make the ordering safe.

Keep the callback small

Register a handler with Deloxide::callback, but treat it as an alert handoff, not a recovery transaction. Deloxide dispatches callbacks away from the detecting thread, yet a callback that waits on application locks, performs slow network I/O, or invokes a complex shutdown path can still delay later notifications or compound an incident. Move DeadlockInfo into a bounded incident queue with a nonblocking send, record overload without taking an application lock, and return.

#![allow(unused)]
fn main() {
extern crate deloxide;
use deloxide::{DeadlockInfo, Deloxide};
use std::sync::{
    Arc,
    atomic::{AtomicU64, Ordering},
    mpsc,
};

let (reports_tx, reports_rx) = mpsc::sync_channel::<DeadlockInfo>(64);
let dropped_reports = Arc::new(AtomicU64::new(0));
let callback_drops = Arc::clone(&dropped_reports);
Deloxide::new()
    .callback(move |info| {
        if reports_tx.try_send(info).is_err() {
            // Counts a full queue or disconnected receiver without blocking.
            callback_drops.fetch_add(1, Ordering::Relaxed);
        }
    })
    .start()
    .expect("detector initialization");

// A separate supervisory task can correlate, persist, or page on reports_rx.
let _ = (reports_rx, dropped_reports);
}

try_send deliberately applies no backpressure to the callback. If the queue is full or its receiver has gone away, this example drops that report and increments a counter; production alerting should monitor the counter and size the queue for its incident burst budget. The supervisor may perform slower persistence and network work, provided it does not need an implicated application lock.

If logging is enabled, Deloxide::with_log and showcase_this can provide the event history. The structured callback payload remains the authoritative alert input; the visualization is supporting evidence.

Logging and visualization

An active callback tells you which threads and locks form a cycle. Logging adds the execution path that led there: thread lifecycle, lock attempts, acquisitions, releases, condition-variable events, and the final finding.

Enable logging

[dependencies]
deloxide = { version = "1.1", features = ["logging-and-visualization"] }

Choose a log path when Deloxide starts:

#![allow(unused)]
fn main() {
extern crate deloxide;
use deloxide::Deloxide;

Deloxide::new()
    .with_log("logs/deloxide_{timestamp}.log")
    .callback(|report| eprintln!("{report:#?}"))
    .start()
    .expect("start Deloxide");
}

Without with_log, a logging-enabled build uses deloxide.log. The logger creates missing parent directories and truncates an existing selected file. For production captures, add a PID, UUID, or another collision-proof component; the built-in timestamp has one-second precision.

Open the viewer

Use showcase for a retained file:

#![allow(unused)]
fn main() {
extern crate deloxide;
use deloxide::showcase;

showcase("logs/deloxide_20260729_120000.log")
    .expect("open visualization");
}

Use showcase_this() when the current process owns the active logger. It flushes pending records before opening the current file.

cargo run --features logging-and-visualization --bin deloxide -- \
  logs/deloxide_20260729_120000.log

The timeline shows the tracked events leading to the report. The graph lets you follow each waiting thread to the lock and incompatible owner that complete the cycle. Use the shared IDs to correlate the callback, log, and application telemetry.

Operational notes

The viewer compresses and encodes the log into a URL parameter, then opens https://deloxide.vercel.app/. Review the log for sensitive identifiers before opening it outside an approved environment.

The ordinary-event logger queue is currently unbounded. During a long capture, monitor memory, file growth, storage retention, and writer progress. Keep the structured callback as the primary alert; visualization is supporting evidence.

Browser launch and file handling can fail. Do not open the viewer inside the deadlock callback. Hand the report to an incident worker, retain the log, and open it from a controlled path.

Finding Inconsistent Lock Order

Lock-order checking is a development aid for discovering an ordering rule that could deadlock under a different schedule. It must never be presented as an active wait-for graph result: DeadlockSource::LockOrderViolation is potential, whereas DeadlockSource::WaitForGraph is an active, validated cycle.

Enable and control it

The builder methods exist only with the lock-order-graph Cargo feature:

[dependencies]
deloxide = { version = "1.1.0", features = ["lock-order-graph"] }

When that feature is compiled, Deloxide::new enables order checking by default. Make the choice visible in development or CI configuration with with_lock_order_checking, and turn it off explicitly for a controlled comparison with no_lock_order_checking.

#![allow(unused)]
fn main() {
extern crate deloxide;
use deloxide::{DeadlockSource, Deloxide};

Deloxide::new()
    .with_lock_order_checking()
    .callback(|info| match info.source {
        DeadlockSource::WaitForGraph => {
            // Active, validated deadlock: preserve evidence and page/escalate.
        }
        DeadlockSource::LockOrderViolation => {
            // Potential ordering cycle: send to code-review/triage workflow.
        }
    })
    .start()
    .expect("detector initialization");

// For a baseline run with the feature compiled:
// Deloxide::new().no_lock_order_checking().start()?;
}

Without the Cargo feature, neither control is available and no historical order graph is maintained. The normal detector can still emit active WaitForGraph reports.

How an edge becomes a finding

Whenever a thread holds A and then acquires B, the order graph records the directed relationship A -> B. It does not need to observe both locks blocked at once.

earlier path:  hold A, then acquire B    records A ──► B
later path:    hold B, then acquire A    would add B ──► A

result:        A ──► B ──► A             potential lock-order cycle

The report contains source: LockOrderViolation, a one-thread contextual thread_cycle, the requested (thread, lock) pair, and lock_order_cycle: Some(...). The historical graph deliberately survives the individual critical sections that created its edges. Consequently it may expose a dangerous inversion that never manifested during a run, but it cannot prove concurrent blockage. An active wait-for graph instead contains current thread-to-incompatible-owner dependencies and is independently validated before producing WaitForGraph.

Development and CI workflow

1. Enable lock-order-graph in a development/CI feature set and install a callback that records the full DeadlockInfo payload.
2. Exercise distinct entry points, error/rollback paths, and shutdown paths. These are the places most likely to acquire the same resources in a different order.
3. Group potential findings by the normalized lock_order_cycle, then identify the acquisition sites for every edge. Treat numeric IDs as run-local evidence; use logs and symbols to name the resources.
4. Decide whether the paths can hold the same lock instances concurrently. If they cannot, document why and keep a regression test; if they can, impose a consistent acquisition order or remove the nested hold.
5. Run the focused scenario repeatedly, optionally with stress mode, then run the normal test suite. A later active WaitForGraph report raises the issue from a potential warning to an incident-quality reproduction.

Keep a baseline run with no_lock_order_checking() when measuring the checking cost or isolating a report. Do not use that baseline to dismiss an already observed cycle.

Triage rules

Potential findings are most useful before release because they turn scheduling-dependent defects into reviewable lock-order evidence. They complement, and never replace, the current-state wait-for detector.

Reproducing Timing-Sensitive Deadlocks

The stress-test Cargo feature adds controlled scheduling disturbance around tracked lock operations. It is a test tool, not a production default: delays and yields change latency, throughput, and timing, and a run that does not manifest a bug is not a proof of safety. Deloxide’s normal default build has no optional stress feature enabled.

How stress testing exposes the schedule

Without scheduling disturbance, one thread may acquire A and then B before the other thread reaches the reverse B-then-A path. Both operations complete, so the latent lock-order problem remains hidden.

Component stress can inject a delay while the first thread holds A and is about to acquire B. That gives the second thread time to acquire B and request A. The first thread then requests B, completing the circular wait:

The delay does not create an impossible dependency. It widens a valid scheduling window that the application can already reach. Once the circular wait exists, Deloxide’s active detector reports the participating threads and locks through the configured callback.

[dependencies]
deloxide = { version = "1.1.0", features = ["stress-test"] }

Choose a stress mode

Random mode samples a delay before a lock acquisition when the thread already holds a tracked lock:

#![allow(unused)]
fn main() {
extern crate deloxide;
use deloxide::Deloxide;

Deloxide::new()
    .with_random_stress()
    .start()
    .expect("detector initialization");
}

Component mode learns held-lock/acquisition relationships and preferentially delays paths in the same component or reverse order:

#![allow(unused)]
fn main() {
extern crate deloxide;
use deloxide::Deloxide;

Deloxide::new()
    .with_component_stress()
    .start()
    .expect("detector initialization");
}

Both with_random_stress and with_component_stress select the default StressConfig if no configuration has been supplied. Set the mode as well as the configuration: with_stress_config alone supplies parameters but does not enable a stress mode.

Configure the disturbance

#![allow(unused)]
fn main() {
extern crate deloxide;
use deloxide::{Deloxide, StressConfig};

Deloxide::new()
    .with_random_stress()
    .with_stress_config(StressConfig {
        preemption_probability: 0.7,
        min_delay_us: 200,
        max_delay_us: 1_500,
        preempt_after_release: true,
    })
    .start()
    .expect("detector initialization");
}

preemption_probability is the chance (0.0 through 1.0) that random mode chooses a pre-acquisition delay. min_delay_us and max_delay_us bound that sampled delay in microseconds. preempt_after_release yields after tracked lock release; it is a scheduler yield, not an additional timed sleep.

Start low to keep ordinary tests quick, then increase only a focused reproduction:

#![allow(unused)]
fn main() {
extern crate deloxide;
use deloxide::{Deloxide, StressConfig};

let gentle = Deloxide::new()
    .with_random_stress()
    .with_stress_config(StressConfig::gentle());

let aggressive = Deloxide::new()
    .with_component_stress()
    .with_stress_config(StressConfig::aggressive());

let _ = (gentle, aggressive); // call start() in the isolated test process
}

StressConfig::gentle uses lower probability and shorter delays; StressConfig::aggressive increases both. Keep min_delay_us <= max_delay_us and record the exact four fields beside each failure.

A reliable reproduction loop

1. Make the competing threads reach the intended acquisition point with a barrier or channels. Do not use a sleep to create the bug; use a sleep only inside the configured stress disturbance.
2. Run the scenario in a separate test process (or otherwise disposable process). Deloxide initialization is process-wide, and an intentional deadlock must not strand the main test runner.
3. Give the parent a hard timeout. On timeout, collect thread stacks, the callback payload, and the event log before terminating the child.
4. Have the parent classify every launched attempt before calculating a manifestation rate. Count an active WaitForGraph callback received before the deadline as a detection. Count a child that exits without that callback, and a timeout with no such callback, as no active detection. The rate is active detections / all attempts classified as detection or no detection; exclude only harness infrastructure failures, and report those failures separately with the launched-attempt count. A potential LockOrderViolation callback is not an active detection.
5. Save the test-case seed, scheduler/input seed, attempt number, platform, feature set, and complete StressConfig. Deloxide currently draws stress randomness from its runtime RNG and exposes no seed-setting API, so record and control the surrounding harness seed when replayability matters.

The revalidated results chart is useful for comparing configurations, but it is not a promise that a particular machine or run will find the same defect:

Use it to choose a budget for reproduction, then validate a fix with deterministic synchronization and normal tests. Stress may turn a potential LockOrderViolation into an active WaitForGraph report, but failure to do so only means the tested schedules did not manifest the cycle.

Troubleshooting by Symptom

Start by deciding whether the expected outcome is an active WaitForGraph callback or a potential LockOrderViolation. A hang with neither can be contention, a missed notification, I/O, starvation, or an untracked synchronization path.

When escalating an issue, attach source, the full DeadlockInfo payload, the test command and feature set, relevant thread stacks, and the exact log file if enabled. That evidence lets another engineer distinguish an active cycle from a potential ordering pattern without reproducing the entire production workload first.

C guide

Rust is Deloxide’s primary interface, but C applications can use the same detector and tracked synchronization through include/deloxide.h.

Build and link

Build the library and C API:

cargo build --release --features c-api

Add other features when needed:

cargo build --release --features \
  c-api,logging-and-visualization,lock-order-graph,stress-test

Include include/deloxide.h and link the produced static or dynamic deloxide library. Exact filenames and platform libraries depend on the target. The repository’s c_examples/basic_mutex.c is the smallest buildable example.

Initialization and callback

Initialize once before creating tracked objects:

#include "deloxide.h"
#include <stdio.h>

static void on_deadlock(const char *json) {
    fprintf(stderr, "Deloxide report: %s\n", json);
}

int main(void) {
    int rc = deloxide_init(NULL, on_deadlock);
    if (rc != 0) {
        fprintf(stderr, "deloxide_init failed: %d\n", rc);
        return 1;
    }

    /* create locks and threads */
    return 0;
}

The callback receives a borrowed NUL-terminated JSON string. Copy it if another thread must retain it; do not free it or keep the pointer after the callback returns. Keep callback work bounded.

Initialization returns 0 on success and 1 if it has already run. Invalid log paths and logger failures use negative codes. Passing a non-null log path without the logging feature returns -3; the public header currently omits that code.

Mutex

void *mutex = deloxide_create_mutex();
if (mutex == NULL) return 1;

if (deloxide_lock_mutex(mutex) != 0) return 1;
/* protected work */
if (deloxide_unlock_mutex(mutex) != 0) return 1;

deloxide_destroy_mutex(mutex);

LOCK_MUTEX(mutex) and UNLOCK_MUTEX(mutex) provide checked convenience macros that terminate on failure. Destroy a mutex only after every thread has stopped using it.

RwLock

void *state = deloxide_create_rwlock();
if (state == NULL) return 1;

if (deloxide_rw_lock_read(state) != 0) return 1;
/* read shared state */
if (deloxide_rw_unlock_read(state) != 0) return 1;

if (deloxide_rw_lock_write(state) != 0) return 1;
/* update shared state */
if (deloxide_rw_unlock_write(state) != 0) return 1;

deloxide_destroy_rwlock(state);

The RWLOCK_READ, RWUNLOCK_READ, RWLOCK_WRITE, and RWUNLOCK_WRITE macros are the shorter checked form. A thread may hold read guards for different RwLocks, but it must release each matching guard correctly.

Condition variables

A Deloxide condition variable waits with a Deloxide mutex:

void *mutex = deloxide_create_mutex();
void *ready = deloxide_create_condvar();

if (deloxide_lock_mutex(mutex) != 0) return 1;
while (!predicate_is_ready()) {
    int rc = deloxide_condvar_wait(ready, mutex);
    if (rc != 0) return 1;
}
if (deloxide_unlock_mutex(mutex) != 0) return 1;

deloxide_destroy_condvar(ready);
deloxide_destroy_mutex(mutex);

deloxide_condvar_wait_timeout returns 1 when the timeout expires and 0 when notified. Negative values indicate invalid handles, a mutex not held by the caller, or another wait failure. Notify with deloxide_condvar_notify_one or deloxide_condvar_notify_all.

Tracked threads

Any native thread using a Deloxide lock contributes synchronization events. Register lifecycle events when logs should also show the thread relationship:

uintptr_t tid = deloxide_get_thread_id();
deloxide_register_thread_spawn(tid, parent_tid);

/* thread work */

deloxide_register_thread_exit(tid);

On POSIX, DEFINE_TRACKED_THREAD(worker) and CREATE_TRACKED_THREAD(thread, worker, arg) wrap this protocol around pthread_create. Those macros are not available on Windows; call the manual registration functions from the Windows thread entry point.

Logging, visualization, and stress

With logging-and-visualization, pass a log path to deloxide_init, flush it with deloxide_flush_logs, and open it with deloxide_showcase or deloxide_showcase_current.

With stress-test, C can enable random scheduling delays with deloxide_enable_random_stress, enable targeted component delays with deloxide_enable_component_stress, and return to normal scheduling with deloxide_disable_stress.

The C header is the exact API reference. This chapter focuses on correct lifecycle and common usage rather than duplicating every status-code comment.

Operate Deloxide in Production

Deloxide is most useful when its configuration, its coverage, and its incident path are decided before an outage. Begin with the default active wait-for detector, then add optional evidence features only after measuring their cost in the workload that will run them. An active WaitForGraph report is evidence of a validated cycle among supported, tracked synchronization; a LockOrderViolation is a potential historical ordering risk and needs a different response. See Reading a Deadlock Report for the triage distinction.

Roll out in stages

1. Inventory the coverage boundary. List every process, thread entry point, long-lived worker, and lock instance that matters to the service. Migrate those lock instances to Deloxide Mutex, RwLock, and Condvar wrappers. Prefer deloxide::thread::spawn or deloxide::thread::Builder where parent/child and exit events are useful. Record raw std::sync, raw parking_lot, third-party, and operating-system synchronization as explicit gaps; a cycle that crosses one is not fully visible to the detector.
2. Canary the default build first. Initialize one process-wide detector before creating tracked locks or accepting work. Install an explicit bounded callback, observe startup errors and report volume, and verify that normal lock behavior and service latency stay within the application’s budget.
3. Benchmark representative work before broad rollout. Include production contention shapes, RwLock reader/writer ratios, request concurrency, CPU limits, and the exact Cargo feature set. The focused figures in Measure Performance are a starting point, not capacity planning for another workload or machine.
4. Add optional features one at a time. Enable logging for a controlled incident capture, lock-order analysis in development or CI, and stress modes for focused reproductions. Measure each addition separately and again in the combined configuration if it will be deployed together.

Deloxide::start() configures global process state, not a worker-local detector. The first installed callback and logger persist; later starts are partial reconfiguration rather than a reliable reset. Use separate processes to compare clean configurations or isolate test cases. The lifecycle chapter explains the feature-specific repeated-start behavior in detail: Manage Lifecycle and Callbacks.

Make the callback an alert handoff

Callbacks run on a single dispatcher outside the detector mutex. They are a poor place to acquire an application lock, synchronously page an external service, open a browser, or attempt an involved shutdown: any of those can delay later reports or entangle an already blocked application. Transfer the report to a bounded application-owned queue, count failed handoffs, and let a supervisor perform slower enrichment, persistence, tracing, or paging.

#![allow(unused)]
fn main() {
extern crate deloxide;
use deloxide::{DeadlockInfo, Deloxide};
use std::sync::{
    Arc,
    atomic::{AtomicU64, Ordering},
    mpsc,
};

let (reports_tx, reports_rx) = mpsc::sync_channel::<DeadlockInfo>(64);
let dropped_reports = Arc::new(AtomicU64::new(0));
let callback_drops = Arc::clone(&dropped_reports);

Deloxide::new()
    .callback(move |report| {
        if reports_tx.try_send(report).is_err() {
            callback_drops.fetch_add(1, Ordering::Relaxed);
        }
    })
    .start()
    .expect("initialize before instrumented work");

// A separate task owns reports_rx and may persist or page without an implicated lock.
let _incident_inputs = (reports_rx, dropped_reports);
}

This code deliberately drops a report when the queue is full or disconnected. Choose queue capacity, overload telemetry, and supervisor durability for the incident burst the application expects. A process can still abort or be killed before the callback, logger, or supervisor finishes; test evidence preservation on the normal incident path instead of treating a callback as a transaction.

Capture, protect, and retain logs

The logging-and-visualization feature adds an asynchronous event logger. With that feature compiled, Deloxide::new().start() selects deloxide.log unless the builder uses with_log; the writer creates parent directories and truncates an existing selected file. Give each process or incident a unique, application-owned path that includes a PID, UUID, or another collision-proof component. The built-in {timestamp} substitution has one-second precision, so logs/deloxide_{timestamp}.log reduces collisions but is not sufficient to make concurrent process paths unique. Apply permissions, encryption, storage limits, retention, and access review as you would to any incident record. Do not let multiple processes reuse one filename.

The current optional logger has an unbounded ordinary-event queue. The correctness hardening report records that choosing a bounded drop, block, or coalescing policy still needs a separate saturation design and test. Accordingly, treat logging as incident evidence with memory and I/O capacity to monitor, rather than as a lossless audit trail with a fixed resource bound.

When an incident requires the timeline, flush and open the process’s current log from a controlled supervisor path with showcase_this, or use showcase for a retained file. The viewer encodes the local log in a URL and opens https://deloxide.vercel.app/ in the default browser. Review the log for identifiers or sensitive context before sending it to that browser destination; this workflow does not provide a local-only guarantee. Keep the structured callback report as the primary alert input and the visualization as supporting reconstruction evidence. Logging and Visualization describes the file format and failure cases.

Roll back by feature and preserve the evidence boundary

Cargo features are compile-time choices, so a rollback normally means deploying a binary built without the optional feature, not flipping a runtime switch in a running process.

Do not use a second start() call as an in-process rollback. It does not clear existing ownership and wait state coherently, cannot replace the first callback or logger, and has feature-specific side effects.

Deployment checklist

• The process initializes exactly one intended Deloxide configuration before instrumented work begins.
• The coverage inventory names tracked lock instances, thread entry points, and every known untracked synchronization boundary.
• The callback only performs bounded, nonblocking handoff; its queue loss or disconnection metric is monitored.
• The service has separate response playbooks for active WaitForGraph and potential LockOrderViolation reports.
• A representative benchmark covers the deployed features, contention pattern, resource limits, and roll-back threshold.
• Logging paths are unique per process/incident, writable, access-controlled, retained for an agreed period, and sized with the unbounded queue caveat in mind.
• The visualization export/privacy review is part of the incident procedure.
• Lock-order and stress features are enabled only in the environments where their historical-analysis or schedule-perturbation costs are intended.
• A fresh process/binary is available for each rollback configuration; no response depends on repeated start() calls.

Performance and benchmarks

Deloxide was evaluated with lock microbenchmarks, heavily contended workloads, correctness controls, stress-driven manifestation tests, and a shared-state raytracer. The complete study is described in the preprint “Deloxide: Low-Overhead Real-time Deadlock Detection and Visualization Framework for Rust”.

The full cross-tool benchmark suite has not been rerun for 1.1. A focused 1.1 microbenchmark checks whether the correctness fixes changed the default fast path. It is a no-regression check, not a replacement for the full evaluation.

Abbreviations:

• STD: std::sync;
• PL+DD: parking_lot with deadlock_detection;
• ND: no_deadlocks;
• DX: Deloxide default;
• DX (LOG): Deloxide logging and visualization; and
• DX (COMP): Deloxide component-based stress mode.

What was tested

The evaluation separates five questions:

1. Primitive cost: how much latency does tracking add to an isolated lock operation?
2. Contended throughput: how do the implementations behave from 4 to 64 competing threads?
3. Correctness: do deterministic cycles produce reports, and do nine complex safe patterns remain free of active WFG reports?
4. Manifestation: how often does each scheduling strategy make a timing-sensitive deadlock occur?
5. Application impact: what happens in a raytracer with 8 workers and 129,600 critical sections per 1080p frame?

These measurements should not be collapsed into one number. Stress mode is supposed to slow and perturb a test. The production-oriented comparison is the default fast path.

Lock microbenchmarks

Default Deloxide stays close to the primitive baselines in the focused Mutex, write-lock, and Condvar cases. RwLock reads cost more because Deloxide must count live readers for later writer dependencies. Logging adds event construction and queueing. Component stress adds intentional delay and should never be interpreted as production overhead.

Contended workloads

The macrobenchmarks measure complete workloads rather than one lock operation. The producer-consumer test is write-heavy; the concurrent-read test exercises shared RwLock access.

The default path follows PL+DD closely at low contention. It costs more in the write-heavy 64x64 case and is faster in the read-heavy 16-thread and 64-thread cases. These results describe the tested workloads, not a universal ranking.

Raytracing workload

The application benchmark uses a shared, tile-based framebuffer. At 1920×1080, workers perform 129,600 lock acquisitions per frame while tracing a scene with a maximum recursion depth of 50.

In the full evaluation, Deloxide completed the 1080p workload 9% faster than PL+DD. Detection does not inherently make an application faster; the result reflects the complete lock implementation and the workload’s contention shape.

Stress-testing manifestation

Each method ran the same deliberately deadlocking scenarios repeatedly without barriers. This measures whether the schedule forms the deadlock, not whether a detector can recognize a cycle that already exists.

The component strategy’s purpose is to make latent schedules appear during testing. These percentages are manifestation rates for the evaluated scenarios, not a claim that every real-world deadlock will reproduce.

Correctness and safe-pattern controls

With barriers enabled, Deloxide, PL+DD, and ND detected every deterministic ground-truth cycle in the evaluated suite. A separate set of nine deadlock-free programs checked whether the runtime WFG confused safe synchronization with an active cycle.

The active WFG produced zero reports across these nine safe patterns. The predictive lock-order graph flagged two patterns as potential risks, which is expected because it analyzes acquisition history rather than current waits. These are empirical results for the tested patterns, not a proof about every possible program.

Five representative safe-pattern workloads were also timed:

Focused 1.1 no-regression check

The current 1.1-focused Criterion run used 30 samples, a one-second warmup, and a two-second measurement window on an Apple M1 Pro:

The Mutex result is faster than the earlier Deloxide microbenchmark and the same-harness PL+DD point. The run is too short and narrow to support a general performance claim. Its purpose is to show that the latest correctness fixes did not introduce material default fast-path overhead.

Reproducing the evidence

The repository’s evaluation record contains the current commands, toolchain, commits, raw CSVs, and paired-seed controls.

Benchmark on the hardware, feature set, contention topology, and workload you plan to ship. Microbenchmarks establish mechanism cost; only the application can establish production impact.

Why Deloxide

The Rust ecosystem offers several approaches to concurrency safety, each with different trade-offs. Deloxide is built to bridge the gap between lightweight but passive monitoring and heavyweight synchronous debugging.

The landscape

Static analysis checks code before it runs. It can find useful ordering problems early, but complex path and concurrency assumptions can produce noisy results. It also cannot reconstruct the runtime schedule that produced an incident.

Passive dynamic detection, such as a periodic parking_lot deadlock check, keeps normal lock operations fast. Because observation happens later and the detector does not perturb scheduling, timing-sensitive bugs may fail to manifest or may only be reported at the next polling interval.

Synchronous graph analysis, represented in the evaluation by no_deadlocks, updates a global model around lock operations and finds cycles immediately. The full evaluation shows why that approach is normally treated as a debugging configuration rather than an always-on production path.

Deloxide combines synchronous active detection with an Optimistic Fast Path. Eligible uncontended Mutex and exclusive RwLock operations avoid global graph work, while contended operations publish the evidence needed for a current wait-for cycle. Optional features add predictive lock-order analysis, schedule stress, logging, and visualization only when the investigation needs them.

Feature matrix

STD = std::sync, PL+DD = parking_lot with deadlock_detection, ND = no_deadlocks, DX = Deloxide. Ratios and observed false-positive results are from the full evaluation.

What Deloxide adds

Deloxide covers the full lifecycle of a concurrency defect:

• Development: lock-order analysis finds dangerous inversions before they block a run.
• Testing: random and component-based stress modes make rare schedules substantially easier to manifest.
• Diagnosis: active WFG reports identify the participating threads and waited locks immediately.
• Response: custom callbacks can record evidence, send alerts, export telemetry, or notify an application supervisor.
• Investigation: structured logs become an interactive execution timeline and dependency graph.
• Production: the Optimistic Fast Path keeps the default detector close to primitive-baseline cost in the evaluated workloads.
• Integration: Rust applications get guard-based wrappers and C applications use the same detector through the shipped header.

That combination is Deloxide’s selling point. It is not only another lock implementation and not only a post-hoc deadlock check; it is one toolkit for finding, reproducing, explaining, and monitoring the bug.

The detailed methodology and results are in Performance and benchmarks and the Deloxide preprint.