{"docs":[{"id":154,"title":"Debugging Device Drivers: Tools and Techniques","heading":"Debugging Device Drivers: Tools and Techniques","subheading":null,"category":"Device Drivers","body":{"root":{"type":"root","children":[{"type":"heading","tag":"h3","version":1,"children":[{"type":"text","text":"Matching tool to symptom","version":1}]},{"type":"paragraph","version":1,"children":[{"type":"text","text":"For crashes, collect a dump and read the stack and state in WinDbg on Windows, or with kdump and the crash utility on Linux. For memory corruption, enable KASAN on a Linux debug kernel or the special pool option of Driver Verifier on Windows. For hangs and races, turn on lockdep on Linux or the Verifier deadlock checks on Windows, and trace with ftrace or Event Tracing for Windows.","version":1}]},{"type":"paragraph","version":1,"children":[{"type":"text","text":"For timing and protocol problems, put a logic analyser or protocol analyser on the bus and line its capture up with a software trace. On macOS, DriverKit drivers are user-space processes, so unified logging and crash reports cover much of the ground.","version":1}]}],"direction":"ltr","format":"","indent":0,"version":1}},"tags":"Device Drivers, Debugging, Linux, Windows","cardCta":{"label":null,"link":null},"pageCta":{"label":"Talk to AAVICO engineering","link":"/contact"},"excerpt":"Driver bugs hide in timing, concurrency and hardware state. Using the right tool for each kind of failure turns days of guessing into a short investigation.","featuredImage":null,"type":"Blog","readTime":"3 min read","publishedAt":"2026-11-15T19:09:00.000Z","heroImage":null,"introText":"Application bugs usually reproduce under a debugger. Driver bugs often do not: they depend on interrupt timing, on what the hardware did a microsecond earlier, or on two threads reaching a lock in a particular order, and the evidence may vanish with the crash. Each operating system offers tools built for this, and they work best alongside instruments that watch the bus.","metrics":[{"id":"debugging-device-drivers-tools-and-techniques-metric-1","value":"ftrace","label":"The Linux kernel tracer for function calls, events and latency, with little overhead when disabled.","icon":null},{"id":"debugging-device-drivers-tools-and-techniques-metric-2","value":"KASAN","label":"The Linux kernel address sanitiser, which finds out-of-bounds and use-after-free accesses.","icon":null},{"id":"debugging-device-drivers-tools-and-techniques-metric-3","value":"Verifier","label":"Windows Driver Verifier stresses a driver and stops at the first rule it breaks.","icon":null},{"id":"debugging-device-drivers-tools-and-techniques-metric-4","value":"WinDbg","label":"The Microsoft debugger for live kernel sessions and crash dump analysis.","icon":null}],"gallery":[],"challenge":{"root":{"type":"root","children":[{"type":"paragraph","version":1,"children":[{"type":"text","text":"The first instinct is to add print statements. They help, but printing from an interrupt handler changes timing and can hide the race being chased, and a message still buffered at the moment of a crash may never be seen.","version":1}]},{"type":"paragraph","version":1,"children":[{"type":"text","text":"The second instinct is to blame the hardware, or the software, without evidence either way. Without a record of what the driver did and what the bus carried, that argument can run for days.","version":1}]}],"direction":"ltr","format":"","indent":0,"version":1}},"blockquote":{"quote":"Capture what the driver did and what the hardware saw at the same moment; most driver arguments end there.","author":"","role":"AAVICO Engineering"},"supportImage1":null,"deliverables":{"root":{"type":"root","children":[{"type":"paragraph","version":1,"children":[{"type":"text","text":"Build a debug configuration early: on Linux a kernel with KASAN, lockdep and dynamic debug enabled; on Windows a test machine with kernel debugging and Driver Verifier turned on.","version":1}]},{"type":"paragraph","version":1,"children":[{"type":"text","text":"Reproduce the failure reliably, then narrow it down with tracing rather than prints, keeping the capture running until the fault occurs.","version":1}]},{"type":"paragraph","version":1,"children":[{"type":"text","text":"Once fixed, add a stress or fault-injection test that would have caught it.","version":1}]}],"direction":"ltr","format":"","indent":0,"version":1}},"results":[{"id":"debugging-device-drivers-tools-and-techniques-result-1","title":"Faster Diagnosis","description":"Evidence from traces and dumps replaces guesswork about the cause."},{"id":"debugging-device-drivers-tools-and-techniques-result-2","title":"Fewer Field Crashes","description":"Memory and locking checkers find bugs that ordinary testing misses."},{"id":"debugging-device-drivers-tools-and-techniques-result-3","title":"Clear Ownership","description":"Bus captures settle whether a fault lies in hardware or software."},{"id":"debugging-device-drivers-tools-and-techniques-result-4","title":"Lasting Quality","description":"Each fixed bug leaves a test behind that stops it returning."}],"lookingAhead":{"root":{"type":"root","children":[{"type":"paragraph","version":1,"children":[{"type":"text","text":"Design debug access into the hardware: a serial console, test points on the buses and, on development boards, a JTAG or SWD header.","version":1}]},{"type":"paragraph","version":1,"children":[{"type":"text","text":"Keep crash dump collection enabled on test fleets. A dump from the first occurrence is worth more than a description of the tenth.","version":1}]}],"direction":"ltr","format":"","indent":0,"version":1}},"supportImage2":null,"testimonial":{"quote":null,"author":null,"role":null,"photo":null},"challengesSolved":[{"id":"debugging-device-drivers-tools-and-techniques-challenge-1","title":"Races hidden by print statements","description":null,"icon":null},{"id":"debugging-device-drivers-tools-and-techniques-challenge-2","title":"Crashes with no dump to analyse","description":null,"icon":null},{"id":"debugging-device-drivers-tools-and-techniques-challenge-3","title":"Memory corruption found only in the field","description":null,"icon":null},{"id":"debugging-device-drivers-tools-and-techniques-challenge-4","title":"Hardware and software teams blaming each other","description":null,"icon":null}],"relatedPosts":[],"contactUs":{"show":true},"updatedAt":"2026-11-15T17:32:32.682Z","createdAt":"2026-11-15T17:32:32.682Z"}],"hasNextPage":false,"hasPrevPage":false,"limit":10,"nextPage":null,"page":1,"pagingCounter":1,"prevPage":null,"totalDocs":1,"totalPages":1}