SlideShare a Scribd company logo

Fast and Precise Symbolic Analysis of Concurrency Bugs in Device Drivers

Pantazis Deligiannis, profile picture
Pantazis Deligiannis

This document describes an approach for detecting concurrency bugs in device drivers through symbolic analysis. It first transforms a concurrent driver into a sequentialized approximation. It then performs symbolic lockset analysis to identify all potential data races. Finally, it uses a precise but bounded model checker to explore traces and find real bugs. The approach aims to efficiently and precisely detect concurrency issues like data races in complex driver code.

Software
1 of 62
Downloaded 14 times
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
Fast and Precise Symbolic Analysis of Concurrency Bugs in Device Drivers Pantazis Deligiannis Alastair Donaldson Zvonimir Rakamarić We acknowledge funding by a gift from Intel Corporation and from NSF CCF 1346756
The relative error-rate in device driver code is up to 10 times higher than the rest of the Linux kernel [1] [1] Chou et al. An empirical study of operating systems errors. SOSP 2001
85% of the system crashes in Windows XP are caused by faulty device drivers [2] [2] Swift et al. Improving the reliability of commodity operating systems. SOSP 2003
19% of the bugs in Linux device drivers are due to concurrency issues, such as data races [3] [3] Ryzhyk et al. Dingo: Taming device drivers. EuroSys 2009
Data races, can lead to subtle, hard to debug errors in complex programs such as device drivers How to efficiently detect them?
Overview of our approach Concurrent Driver
Overview of our approach Concurrent Driver Sequential Driver (sound over-approximation) transform
Overview of our approach Concurrent Driver Sequential Driver (sound over-approximation) Symbolic Lockset Analyser (identify all potential races) transform
Overview of our approach Concurrent Driver Sequential Driver (sound over-approximation) Symbolic Lockset Analyser (identify all potential races) Symbolic Verifier (precise but bounded exploration) We developed in this work … exploit race info to speedup trace transform
Concurrent Driver EP1 EP2 EPN… For each driver …
EP Pair Concurrent Driver EP1 EP2 EPN… Consider every pair of entry points that can execute concurrently EP1 EP2 EP Pair EP1 EP3 EP Pair EPM EPN … For each driver …
EP Pair EP1 EP2 For each pair …
EP Pair EP1 EP2 Sequential Pair call EntryPoint1 call EntryPoint2 Call 2 entry points in sequence (order, i.e. schedule, does not matter due to lockset analysis) Sequentialize For each pair …
Sequential Pair call EntryPoint1 call EntryPoint2 For each pair …
Sequential Pair call EntryPoint1 call EntryPoint2 Soundly model the effects of any other concurrently running EPs by over-approximating read accesses to shared memory For each pair …
Sequential Pair call EntryPoint1 call EntryPoint2 Instrument entry points with additional state to record locksets for symbolic lockset analysis Soundly model the effects of any other concurrently running EPs by over-approximating read accesses to shared memory For each pair …
Sequential Pair call EntryPoint1 call EntryPoint2 Instrument entry points with additional state to record locksets for symbolic lockset analysis Soundly model the effects of any other concurrently running EPs by over-approximating read accesses to shared memory Attempt to verify race-freedom by asserting that all accesses to each shared memory location are consistently protected by at least one common lock For each pair …
EP Pair Good reason: we can symbolically analyse each pair of entry points in parallel (but we leave this for future work) EP1 EP2 EP Pair EP1 EP3 EP Pair EPM EPN … Why pairwise analysis?
Lightweight data race detection method: - proposed in Eraser [4], a dynamic race detector Lockset analysis [4] Savage et al. Eraser: A Dynamic Data Race Detector for Multithreaded Programs. TOCS 1997
Lightweight data race detection method: - proposed in Eraser [4], a dynamic race detector - key idea: track sets of locks that consistently protect each memory location S during program execution Lockset analysis [4] Savage et al. Eraser: A Dynamic Data Race Detector for Multithreaded Programs. TOCS 1997
Lightweight data race detection method: - proposed in Eraser [4], a dynamic race detector - key idea: track sets of locks that consistently protect each memory location S during program execution - if the lockset of S ever becomes empty, report a potential race on S Lockset analysis [4] Savage et al. Eraser: A Dynamic Data Race Detector for Multithreaded Programs. TOCS 1997
Lightweight data race detection method: - proposed in Eraser [4], a dynamic race detector - key idea: track sets of locks that consistently protect each memory location S during program execution - if the lockset of S ever becomes empty, report a potential race on S - this is because an empty lockset suggests that S may be accessed simultaneously by two or more threads Lockset analysis [4] Savage et al. Eraser: A Dynamic Data Race Detector for Multithreaded Programs. TOCS 1997
Program CLST1 CLST2 LSA Initial { } { } { M, N } T1 lock (M); lock (N); write (A); unlock (N); write (A); unlock (M); T2 lock (N); write (A); unlock (N);
Program CLST1 CLST2 LSA Initial { } { } { M, N } T1 lock (M); lock (N); write (A); unlock (N); write (A); unlock (M); T2 lock (N); write (A); unlock (N); { M } { M, N }
Program CLST1 CLST2 LSA Initial { } { } { M, N } T1 lock (M); lock (N); write (A); unlock (N); write (A); unlock (M); T2 lock (N); write (A); unlock (N); { M } { M, N } { M, N } { M, N }
Program CLST1 CLST2 LSA Initial { } { } { M, N } T1 lock (M); lock (N); write (A); unlock (N); write (A); unlock (M); T2 lock (N); write (A); unlock (N); { M } { M, N } { M, N } { M, N } { M, N } { M, N } compute lockset intersection at access point
Program CLST1 CLST2 LSA Initial { } { } { M, N } T1 lock (M); lock (N); write (A); unlock (N); write (A); unlock (M); T2 lock (N); write (A); unlock (N); { M } { M, N } { M, N } { M, N } { M, N } { M, N } { M } { M, N } compute lockset intersection at access point
Program CLST1 CLST2 LSA Initial { } { } { M, N } T1 lock (M); lock (N); write (A); unlock (N); write (A); unlock (M); T2 lock (N); write (A); unlock (N); { M } { M, N } { M, N } { M, N } { M, N } { M, N } { M } { M, N } { M } { M } compute lockset intersection at access point
Program CLST1 CLST2 LSA Initial { } { } { M, N } T1 lock (M); lock (N); write (A); unlock (N); write (A); unlock (M); T2 lock (N); write (A); unlock (N); { M } { M, N } { M, N } { M, N } { M, N } { M, N } { M } { M, N } { M } { M } { } { M } compute lockset intersection at access point
Program CLST1 CLST2 LSA Initial { } { } { M, N } T1 lock (M); lock (N); write (A); unlock (N); write (A); unlock (M); T2 lock (N); write (A); unlock (N); { M } { M, N } { M, N } { M, N } { M, N } { M, N } { M } { M, N } { M } { M } { } { M } { N } { M } compute lockset intersection at access point
Program CLST1 CLST2 LSA Initial { } { } { M, N } T1 lock (M); lock (N); write (A); unlock (N); write (A); unlock (M); T2 lock (N); write (A); unlock (N); { M } { M, N } { M, N } { M, N } { M, N } { M, N } { M } { M, N } { M } { M } { } { M } { N } { M } { N } { } compute lockset intersection at access point compute lockset intersection at access point
Program CLST1 CLST2 LSA Initial { } { } { M, N } T1 lock (M); lock (N); write (A); unlock (N); write (A); unlock (M); T2 lock (N); write (A); unlock (N); { M } { M, N } { M, N } { M, N } { M, N } { M, N } { M } { M, N } { M } { M } { } { M } { N } { M } { N } { } compute lockset intersection at access point warning: access to A may not be consistently protected
Program CLST1 CLST2 LSA Initial { } { } { M, N } T1 lock (M); lock (N); write (A); unlock (N); write (A); unlock (M); T2 lock (N); write (A); unlock (N); { M } { M, N } { M, N } { M, N } { M, N } { M, N } { M } { M, N } { M } { M } { } { M } { N } { M } { N } { } { } { } compute lockset intersection at access point warning: access to A may not be consistently protected
Advantages of lockset analysis:
Advantages of lockset analysis: - easy to implement, lightweight, has the potential to scale well (in contrast with happens-before based analyses)
Limitations of lockset analysis: Advantages of lockset analysis: - easy to implement, lightweight, has the potential to scale well (in contrast with happens-before based analyses)
Limitations of lockset analysis: - imprecision (a violation of locking discipline is not always a race) Advantages of lockset analysis: - easy to implement, lightweight, has the potential to scale well (in contrast with happens-before based analyses)
Limitations of lockset analysis: - imprecision (a violation of locking discipline is not always a race) - code coverage in dynamic tools is limited by execution paths that are explored Advantages of lockset analysis: - easy to implement, lightweight, has the potential to scale well (in contrast with happens-before based analyses)
Limitations of lockset analysis: - imprecision (a violation of locking discipline is not always a race) - code coverage in dynamic tools is limited by execution paths that are explored - to counter the latter, we apply lockset analysis in a symbolic context by analysing source code Advantages of lockset analysis: - easy to implement, lightweight, has the potential to scale well (in contrast with happens-before based analyses)
We use Read and Write Sets during lockset analysis to keep track of shared memory location accesses — not in previous example for simplicity Some other (driver-specific) optimisations read our paper!
Boogie IVL code, instrumented with yields Data Race Reports No Errors (Under Given Bounds) WHOOP Error Traces Z3 Chauffeur SMACK Linux driver source code in C Boogie IVL code llvm-IR Linux Environmental Model Instrumentation Sequentialization Invariant Generation Boogie Verification Engine CORRAL A. Translation Phase B. Symbolic Lockset Analysis Phase C. Bug-Finding Phase Clang / LLVM entry point information Toolchain
W Inst Sequ Invari Ve Translation Phase Chauffeur SMACK Linux driver source code in C Boogie IVL code llvm-IR Linux Environmental Model Clang / LLVM entry point information
W Inst Sequ Invari Ve Translation Phase Chauffeur SMACK Linux driver source code in C Boogie IVL code llvm-IR Linux Environmental Model Clang / LLVM entry point information
W Inst Sequ Invari Ve Translation Phase Chauffeur SMACK Linux driver source code in C Boogie IVL code llvm-IR Linux Environmental Model Clang / LLVM entry point information Debugging information is preserved so that we can map bugs back to C source code
Symbolic Lockset Analysis Phase ACK Boogie IVL code entry point information Data Race Reports WHOOP Z3 Instrumentation Sequentialization Invariant Generation Boogie Verification Engine Boogie IVL code, instrumented with yields No Errors (Under Given Bounds) Error Tr CORRAL
Symbolic Lockset Analysis Phase ACK Boogie IVL code entry point information Data Race Reports WHOOP Z3 Instrumentation Sequentialization Invariant Generation Boogie Verification Engine Boogie IVL code, instrumented with yields No Errors (Under Given Bounds) Error Tr CORRAL
Challenges and Solutions Aliasing is a big challenge — we exploit LLVM alias analyses and SMACK heap modelling Scalability is a big issue — we generate and infer invariants and perform summarisation for modular verification (each procedure verified in isolation) read our paper!
- imprecision as it inherits the limitations of lockset analysis Limitations
- imprecision as it inherits the limitations of lockset analysis - sound over-approximation of shared state and lock-free data structures can lead to false alarms Limitations
- imprecision as it inherits the limitations of lockset analysis - sound over-approximation of shared state and lock-free data structures can lead to false alarms - do not currently handle interrupt handlers in a special way, we just assume they execute concurrently at all times (which however is sound) Limitations
To improve Whoop’s precision, we use Corral [5], an industrial strength bug-finder for device drivers from Microsoft Research that is currently used by Microsoft in the backend of the Static Driver Verifier (SDV) Improving precision [5] A. Lal, S. Qadeer, and S. K. Lahiri. A solver for reachability modulo theories. CAV 2012
The sequentialised program is bounded by a number of context-switches per thread [6] A. Lal and T. W. Reps. Reducing concurrent analysis under a context bound to sequential analysis. CAV 2008 Corral uses the Lal-Reps sequentialisation [6] to transform a concurrent program into a non-deterministic sequential program Bounded Bug-Finding
Bounded Bug-Finding Corral can then explore the sequential program to detect data races, encoded as race-checking assertions If a race is found, a trace (mapped on the original C code) is reported to the user
Bug-Finding Phase ie de int ion Data Race Reports WHOOP Z3 Instrumentation Sequentialization Invariant Generation Boogie Verification Engine Boogie IVL code, instrumented with yields No Errors (Under Given Bounds) Error Traces CORRAL
State-space explosion Corral inserts a context switch before each shared memory access and at each synchronisation point (e.g. lock/unlock)
Problem: this can be too coarse and state- space can explode in large drivers! State-space explosion Corral inserts a context switch before each shared memory access and at each synchronisation point (e.g. lock/unlock)
Sound partial-order reduction If our symbolic lockset analysis finds that a given shared memory access is not racy, then we do not instrument a context switch before this memory access
This tames the sequentialisation and can greatly speedup Corral as we have found in our experiments Sound partial-order reduction If our symbolic lockset analysis finds that a given shared memory access is not racy, then we do not instrument a context switch before this memory access
We applied our technique on 16 drivers from the Linux 4.0 kernel: - block, char, ethernet, nfc, usb and watchdog (250 — 7300 LoC) - exploiting our symbolic lockset analysis we can significantly accelerate Corral by 1.5-20x ! - in 1 driver there was slowdown, because we did not manage to verify race freedom - detected potential races (filled bug report, but have not confirmed yet) Evaluation
Whoop Corral Whoop + Corral Benchmarks Time (sec) # Context Switches CSB = 9 Time (sec) # Context Switches CSB = 9 Time (sec) generic_nvram 2.3 92 197.5 29 49.3 pc9736x_gpio 4.1 691 27337.6 167 1358.9 intel_scu_wd 2.4 314 1571.7 196 689.3 fs3270 3.2 321 T.O. (10 hours) 211 8883.0 intel_nfcsim 3.6 732 1539.9 601 278.0 sx8 2.8 227 21.4 217 24.3 Read the paper for more results!
Applied our symbolic lockset analysis on >1000 concurrency benchmarks from SVCOMP 2016: - Early results: reducing Corral runtime by ~75% - No bugs missed, gives more confidence in our partial-order reduction! We want to extend our Linux model so that we can apply our technique on more drivers! We want to experiment with applying our technique on other Linux modules (e.g. filesystems) New / Future work
Thanks! http://www.doc.ic.ac.uk/~pd1113/ p.deligiannis@imperial.ac.uk https://github.com/pdeligia/whoop https://github.com/pdeligia/lockpwn

More Related Content

PDF
SCP
YongraeJo
 
PDF
Vft
YongraeJo
 
PDF
Omni ledger
YongraeJo
 
PDF
FastBFT
YongraeJo
 
PDF
ITFT_Semaphores and bounded buffer
Sneh Prabha
 
PDF
Ekiden
YongraeJo
 
PDF
Post Quantum Cryptography - Emerging Frontiers
Gokul Alex
 
PPT
DES Block Cipher Hao Qi
Information Security Awareness Group
 
SCP
YongraeJo
 
Vft
YongraeJo
 
Omni ledger
YongraeJo
 
FastBFT
YongraeJo
 
ITFT_Semaphores and bounded buffer
Sneh Prabha
 
Ekiden
YongraeJo
 
Post Quantum Cryptography - Emerging Frontiers
Gokul Alex
 
DES Block Cipher Hao Qi
Information Security Awareness Group
 

What's hot (20)

PPT
Stream ciphers presentation
degarden
 
PDF
An effective RC4 Stream Cipher
Tharindu Weerasinghe
 
PPT
RC4&RC5
guestff64339
 
PPT
OSCh7
Joe Christensen
 
PPT
Os3
gopal10scs185
 
PDF
Defeating RSA Multiply-Always and Message Blinding Countermeasures
Riscure
 
PPTX
Key reinstallation attacks forcing nonce reuse in wpa2
Università Degli Studi Di Salerno
 
PDF
Next generation block ciphers
Roman Oliynykov
 
PPTX
Mutual Exclusion
David Evans
 
PPTX
Information and data security pseudorandom number generation and stream cipher
Mazin Alwaaly
 
PDF
Practical Differential Fault Attack on AES
Riscure
 
PPT
Lecture 5
Chamila Fernando
 
PPT
Chord Algorithm
Sijia Lyu
 
PDF
2. Stream Ciphers
Sam Bowne
 
PPTX
Operating system 24 mutex locks and semaphores
Vaibhav Khanna
 
PDF
Implementing STM in Java
Misha Kozik
 
PDF
St hack2015 dynamic_behavior_analysis_using_binary_instrumentation_jonathan_s...
Jonathan Salwan
 
PDF
CRYPTOGRAPHY AND NETWORK SECURITY
Kathirvel Ayyaswamy
 
PPT
Ch06
Joe Christensen
 
PPT
Lec11 semaphores
anandammca
 
Stream ciphers presentation
degarden
 
An effective RC4 Stream Cipher
Tharindu Weerasinghe
 
RC4&RC5
guestff64339
 
OSCh7
Joe Christensen
 
Os3
gopal10scs185
 
Defeating RSA Multiply-Always and Message Blinding Countermeasures
Riscure
 
Key reinstallation attacks forcing nonce reuse in wpa2
Università Degli Studi Di Salerno
 
Next generation block ciphers
Roman Oliynykov
 
Mutual Exclusion
David Evans
 
Information and data security pseudorandom number generation and stream cipher
Mazin Alwaaly
 
Practical Differential Fault Attack on AES
Riscure
 
Lecture 5
Chamila Fernando
 
Chord Algorithm
Sijia Lyu
 
2. Stream Ciphers
Sam Bowne
 
Operating system 24 mutex locks and semaphores
Vaibhav Khanna
 
Implementing STM in Java
Misha Kozik
 
St hack2015 dynamic_behavior_analysis_using_binary_instrumentation_jonathan_s...
Jonathan Salwan
 
CRYPTOGRAPHY AND NETWORK SECURITY
Kathirvel Ayyaswamy
 
Ch06
Joe Christensen
 
Lec11 semaphores
anandammca
 
Ad

Viewers also liked (13)

PPT
The goMo™ Story
Kall Ramanathan
 
PPTX
Types of PANTOGRAPH
gauravjagdale
 
PPTX
Location based reminder
junnubabu
 
PPTX
Abes engineering college ghaziabad
Palash Awasthi
 
DOCX
Location sharing and automatic message sender Android Application
Kavita Sharma
 
PPTX
Training report of Electric Loco Shed
Aarti Ojha
 
PPT
Dlw varanasi training ppt
Praveen Tiwari
 
DOC
Android Report
Ganesh Waghmare
 
PPTX
Android ppt with example of budget manager
Nalini Mehta
 
PPTX
NORTHERN RAILWAY EMU CAR SHED INDUSTRIAL TRAINING PRESENTATION
INDUSTRIAL ENGINEERING
 
PDF
Final Year Project Report on Self Tacit Zone (Location Based Android App)
Parthik Poshiya
 
PPT
Use Case Diagram
Ashesh R
 
PPTX
Railway loco ppt
Anand Dwivedi
 
The goMo™ Story
Kall Ramanathan
 
Types of PANTOGRAPH
gauravjagdale
 
Location based reminder
junnubabu
 
Abes engineering college ghaziabad
Palash Awasthi
 
Location sharing and automatic message sender Android Application
Kavita Sharma
 
Training report of Electric Loco Shed
Aarti Ojha
 
Dlw varanasi training ppt
Praveen Tiwari
 
Android Report
Ganesh Waghmare
 
Android ppt with example of budget manager
Nalini Mehta
 
NORTHERN RAILWAY EMU CAR SHED INDUSTRIAL TRAINING PRESENTATION
INDUSTRIAL ENGINEERING
 
Final Year Project Report on Self Tacit Zone (Location Based Android App)
Parthik Poshiya
 
Use Case Diagram
Ashesh R
 
Railway loco ppt
Anand Dwivedi
 
Ad

Similar to Fast and Precise Symbolic Analysis of Concurrency Bugs in Device Drivers (20)

PDF
Man in the middle attacks
BurtPepper
 
DOCX
Contents namp
shwetha mk
 
DOCX
Contents namp
shwetha mk
 
PPT
Iss lecture 2
Ali Habeeb
 
PDF
Fast and Precise Symbolic Analysis of Concurrency Bugs in Device Drivers
Pantazis Deligiannis
 
PDF
Dsp file
Rakesh Thakur
 
PPT
Georgy Nosenko - An introduction to the use SMT solvers for software security
DefconRussia
 
PPT
1 Cryptography Introduction_shared.ppt
ssuser0cd7c9
 
PDF
IRJET- Securing Cloud Data Under Key Exposure
IRJET Journal
 
PDF
IRJET- Data Analysis for Braking System in Time Domain for Fault Diagnosis
IRJET Journal
 
PPT
Rsa
ismaelhaider
 
PDF
A REVIEW ON NMAP AND ITS FEATURES
IRJET Journal
 
PDF
Cyber-Security Enhancements of Networked Control Systems Using Homomorphic En...
Kiminao Kogiso
 
PPTX
Lattice based Merkle for post-quantum epoch
DefCamp
 
PPTX
Unit 1.pptx
DeepakYadav656387
 
PDF
STA STATIC TIMING ANALYSIS AND ITS USE HOW IT WORKS
akshayaigali
 
PDF
A Typed Assembly Language for Non-interference.pdf
Yasmine Anino
 
PDF
Specifying and Implementing SNOW3G with Cryptol
Ulisses Costa
 
PDF
Cs8792 cns - Public key cryptosystem (Unit III)
ArthyR3
 
PDF
Cryptosystem An Implementation of RSA Using Verilog
ijcncs
 
Man in the middle attacks
BurtPepper
 
Contents namp
shwetha mk
 
Contents namp
shwetha mk
 
Iss lecture 2
Ali Habeeb
 
Fast and Precise Symbolic Analysis of Concurrency Bugs in Device Drivers
Pantazis Deligiannis
 
Dsp file
Rakesh Thakur
 
Georgy Nosenko - An introduction to the use SMT solvers for software security
DefconRussia
 
1 Cryptography Introduction_shared.ppt
ssuser0cd7c9
 
IRJET- Securing Cloud Data Under Key Exposure
IRJET Journal
 
IRJET- Data Analysis for Braking System in Time Domain for Fault Diagnosis
IRJET Journal
 
Rsa
ismaelhaider
 
A REVIEW ON NMAP AND ITS FEATURES
IRJET Journal
 
Cyber-Security Enhancements of Networked Control Systems Using Homomorphic En...
Kiminao Kogiso
 
Lattice based Merkle for post-quantum epoch
DefCamp
 
Unit 1.pptx
DeepakYadav656387
 
STA STATIC TIMING ANALYSIS AND ITS USE HOW IT WORKS
akshayaigali
 
A Typed Assembly Language for Non-interference.pdf
Yasmine Anino
 
Specifying and Implementing SNOW3G with Cryptol
Ulisses Costa
 
Cs8792 cns - Public key cryptosystem (Unit III)
ArthyR3
 
Cryptosystem An Implementation of RSA Using Verilog
ijcncs
 

Recently uploaded (20)

PDF
How to Choose the Right IT Partner for Your Business in Malaysia
Asian Business Services & Technologies
 
PDF
Nekopoi APK 2025 free lastest update
umarali35kp
 
PDF
Addressing The Cult of Project Management Tools-Why Disconnected Work is Hold...
OnePlan Solutions
 
PDF
Understanding Forklifts - TECH EHS Solution
TECH EHS Solution
 
PDF
Design an Analysis of Algorithms I-SECS-1021-03
DilanEscobar1
 
PDF
PTS Company Brochure 2025 (1).pdf.......
pts464036
 
PPTX
Odoo POS Development Services by CandidRoot Solutions
CandidRoot Solutions Private Limited
 
PPTX
L1 - Introduction to python Backend.pptx
MoizAhmed480282
 
PDF
wealthsignaloriginal-com-DS-text-... (1).pdf
skmiani786
 
PDF
iTop VPN Free 5.6.0.5262 Crack latest version 2025
itskinga12
 
PPTX
Operating system designcfffgfgggggggvggggggggg
rmskumara09
 
PDF
SAP S4 Hana Brochure 3 (PTS SYSTEMS AND SOLUTIONS)
pts464036
 
PPTX
Log360_SIEM_Solutions Overview PPT_Feb 2020.pptx
nonameist3434
 
PDF
Claude Code: Everyone is a 10x Developer - A Comprehensive AI-Powered CLI Tool
William Chong
 
PDF
medical staffing services at VALiNTRY
Tiyan Grayson
 
PDF
Digital Strategies for Manufacturing Companies
Virendra Rai, PMP
 
PPTX
CHAPTER 2 - PM Management and IT Context
KevinPutra14
 
PDF
Upgrade and Innovation Strategies for SAP ERP Customers
Virendra Rai, PMP
 
PPTX
Agentic AI : A Practical Guide. Undersating, Implementing and Scaling Autono...
IT IDOL Technologies
 
PDF
Internet Downloader Manager (IDM) Crack 6.42 Build 42 Updates Latest 2025
itskinga12
 
How to Choose the Right IT Partner for Your Business in Malaysia
Asian Business Services & Technologies
 
Nekopoi APK 2025 free lastest update
umarali35kp
 
Addressing The Cult of Project Management Tools-Why Disconnected Work is Hold...
OnePlan Solutions
 
Understanding Forklifts - TECH EHS Solution
TECH EHS Solution
 
Design an Analysis of Algorithms I-SECS-1021-03
DilanEscobar1
 
PTS Company Brochure 2025 (1).pdf.......
pts464036
 
Odoo POS Development Services by CandidRoot Solutions
CandidRoot Solutions Private Limited
 
L1 - Introduction to python Backend.pptx
MoizAhmed480282
 
wealthsignaloriginal-com-DS-text-... (1).pdf
skmiani786
 
iTop VPN Free 5.6.0.5262 Crack latest version 2025
itskinga12
 
Operating system designcfffgfgggggggvggggggggg
rmskumara09
 
SAP S4 Hana Brochure 3 (PTS SYSTEMS AND SOLUTIONS)
pts464036
 
Log360_SIEM_Solutions Overview PPT_Feb 2020.pptx
nonameist3434
 
Claude Code: Everyone is a 10x Developer - A Comprehensive AI-Powered CLI Tool
William Chong
 
medical staffing services at VALiNTRY
Tiyan Grayson
 
Digital Strategies for Manufacturing Companies
Virendra Rai, PMP
 
CHAPTER 2 - PM Management and IT Context
KevinPutra14
 
Upgrade and Innovation Strategies for SAP ERP Customers
Virendra Rai, PMP
 
Agentic AI : A Practical Guide. Undersating, Implementing and Scaling Autono...
IT IDOL Technologies
 
Internet Downloader Manager (IDM) Crack 6.42 Build 42 Updates Latest 2025
itskinga12
 

Fast and Precise Symbolic Analysis of Concurrency Bugs in Device Drivers

  • 1. Fast and Precise Symbolic Analysis of Concurrency Bugs in Device Drivers Pantazis Deligiannis Alastair Donaldson Zvonimir Rakamarić We acknowledge funding by a gift from Intel Corporation and from NSF CCF 1346756
  • 2. The relative error-rate in device driver code is up to 10 times higher than the rest of the Linux kernel [1] [1] Chou et al. An empirical study of operating systems errors. SOSP 2001
  • 3. 85% of the system crashes in Windows XP are caused by faulty device drivers [2] [2] Swift et al. Improving the reliability of commodity operating systems. SOSP 2003
  • 4. 19% of the bugs in Linux device drivers are due to concurrency issues, such as data races [3] [3] Ryzhyk et al. Dingo: Taming device drivers. EuroSys 2009
  • 5. Data races, can lead to subtle, hard to debug errors in complex programs such as device drivers How to efficiently detect them?
  • 6. Overview of our approach Concurrent Driver
  • 7. Overview of our approach Concurrent Driver Sequential Driver (sound over-approximation) transform
  • 8. Overview of our approach Concurrent Driver Sequential Driver (sound over-approximation) Symbolic Lockset Analyser (identify all potential races) transform
  • 9. Overview of our approach Concurrent Driver Sequential Driver (sound over-approximation) Symbolic Lockset Analyser (identify all potential races) Symbolic Verifier (precise but bounded exploration) We developed in this work … exploit race info to speedup trace transform
  • 10. Concurrent Driver EP1 EP2 EPN… For each driver …
  • 11. EP Pair Concurrent Driver EP1 EP2 EPN… Consider every pair of entry points that can execute concurrently EP1 EP2 EP Pair EP1 EP3 EP Pair EPM EPN … For each driver …
  • 12. EP Pair EP1 EP2 For each pair …
  • 13. EP Pair EP1 EP2 Sequential Pair call EntryPoint1 call EntryPoint2 Call 2 entry points in sequence (order, i.e. schedule, does not matter due to lockset analysis) Sequentialize For each pair …
  • 14. Sequential Pair call EntryPoint1 call EntryPoint2 For each pair …
  • 15. Sequential Pair call EntryPoint1 call EntryPoint2 Soundly model the effects of any other concurrently running EPs by over-approximating read accesses to shared memory For each pair …
  • 16. Sequential Pair call EntryPoint1 call EntryPoint2 Instrument entry points with additional state to record locksets for symbolic lockset analysis Soundly model the effects of any other concurrently running EPs by over-approximating read accesses to shared memory For each pair …
  • 17. Sequential Pair call EntryPoint1 call EntryPoint2 Instrument entry points with additional state to record locksets for symbolic lockset analysis Soundly model the effects of any other concurrently running EPs by over-approximating read accesses to shared memory Attempt to verify race-freedom by asserting that all accesses to each shared memory location are consistently protected by at least one common lock For each pair …
  • 18. EP Pair Good reason: we can symbolically analyse each pair of entry points in parallel (but we leave this for future work) EP1 EP2 EP Pair EP1 EP3 EP Pair EPM EPN … Why pairwise analysis?
  • 19. Lightweight data race detection method: - proposed in Eraser [4], a dynamic race detector Lockset analysis [4] Savage et al. Eraser: A Dynamic Data Race Detector for Multithreaded Programs. TOCS 1997
  • 20. Lightweight data race detection method: - proposed in Eraser [4], a dynamic race detector - key idea: track sets of locks that consistently protect each memory location S during program execution Lockset analysis [4] Savage et al. Eraser: A Dynamic Data Race Detector for Multithreaded Programs. TOCS 1997
  • 21. Lightweight data race detection method: - proposed in Eraser [4], a dynamic race detector - key idea: track sets of locks that consistently protect each memory location S during program execution - if the lockset of S ever becomes empty, report a potential race on S Lockset analysis [4] Savage et al. Eraser: A Dynamic Data Race Detector for Multithreaded Programs. TOCS 1997
  • 22. Lightweight data race detection method: - proposed in Eraser [4], a dynamic race detector - key idea: track sets of locks that consistently protect each memory location S during program execution - if the lockset of S ever becomes empty, report a potential race on S - this is because an empty lockset suggests that S may be accessed simultaneously by two or more threads Lockset analysis [4] Savage et al. Eraser: A Dynamic Data Race Detector for Multithreaded Programs. TOCS 1997
  • 23. Program CLST1 CLST2 LSA Initial { } { } { M, N } T1 lock (M); lock (N); write (A); unlock (N); write (A); unlock (M); T2 lock (N); write (A); unlock (N);
  • 24. Program CLST1 CLST2 LSA Initial { } { } { M, N } T1 lock (M); lock (N); write (A); unlock (N); write (A); unlock (M); T2 lock (N); write (A); unlock (N); { M } { M, N }
  • 25. Program CLST1 CLST2 LSA Initial { } { } { M, N } T1 lock (M); lock (N); write (A); unlock (N); write (A); unlock (M); T2 lock (N); write (A); unlock (N); { M } { M, N } { M, N } { M, N }
  • 26. Program CLST1 CLST2 LSA Initial { } { } { M, N } T1 lock (M); lock (N); write (A); unlock (N); write (A); unlock (M); T2 lock (N); write (A); unlock (N); { M } { M, N } { M, N } { M, N } { M, N } { M, N } compute lockset intersection at access point
  • 27. Program CLST1 CLST2 LSA Initial { } { } { M, N } T1 lock (M); lock (N); write (A); unlock (N); write (A); unlock (M); T2 lock (N); write (A); unlock (N); { M } { M, N } { M, N } { M, N } { M, N } { M, N } { M } { M, N } compute lockset intersection at access point
  • 28. Program CLST1 CLST2 LSA Initial { } { } { M, N } T1 lock (M); lock (N); write (A); unlock (N); write (A); unlock (M); T2 lock (N); write (A); unlock (N); { M } { M, N } { M, N } { M, N } { M, N } { M, N } { M } { M, N } { M } { M } compute lockset intersection at access point
  • 29. Program CLST1 CLST2 LSA Initial { } { } { M, N } T1 lock (M); lock (N); write (A); unlock (N); write (A); unlock (M); T2 lock (N); write (A); unlock (N); { M } { M, N } { M, N } { M, N } { M, N } { M, N } { M } { M, N } { M } { M } { } { M } compute lockset intersection at access point
  • 30. Program CLST1 CLST2 LSA Initial { } { } { M, N } T1 lock (M); lock (N); write (A); unlock (N); write (A); unlock (M); T2 lock (N); write (A); unlock (N); { M } { M, N } { M, N } { M, N } { M, N } { M, N } { M } { M, N } { M } { M } { } { M } { N } { M } compute lockset intersection at access point
  • 31. Program CLST1 CLST2 LSA Initial { } { } { M, N } T1 lock (M); lock (N); write (A); unlock (N); write (A); unlock (M); T2 lock (N); write (A); unlock (N); { M } { M, N } { M, N } { M, N } { M, N } { M, N } { M } { M, N } { M } { M } { } { M } { N } { M } { N } { } compute lockset intersection at access point compute lockset intersection at access point
  • 32. Program CLST1 CLST2 LSA Initial { } { } { M, N } T1 lock (M); lock (N); write (A); unlock (N); write (A); unlock (M); T2 lock (N); write (A); unlock (N); { M } { M, N } { M, N } { M, N } { M, N } { M, N } { M } { M, N } { M } { M } { } { M } { N } { M } { N } { } compute lockset intersection at access point warning: access to A may not be consistently protected
  • 33. Program CLST1 CLST2 LSA Initial { } { } { M, N } T1 lock (M); lock (N); write (A); unlock (N); write (A); unlock (M); T2 lock (N); write (A); unlock (N); { M } { M, N } { M, N } { M, N } { M, N } { M, N } { M } { M, N } { M } { M } { } { M } { N } { M } { N } { } { } { } compute lockset intersection at access point warning: access to A may not be consistently protected
  • 35. Advantages of lockset analysis: - easy to implement, lightweight, has the potential to scale well (in contrast with happens-before based analyses)
  • 36. Limitations of lockset analysis: Advantages of lockset analysis: - easy to implement, lightweight, has the potential to scale well (in contrast with happens-before based analyses)
  • 37. Limitations of lockset analysis: - imprecision (a violation of locking discipline is not always a race) Advantages of lockset analysis: - easy to implement, lightweight, has the potential to scale well (in contrast with happens-before based analyses)
  • 38. Limitations of lockset analysis: - imprecision (a violation of locking discipline is not always a race) - code coverage in dynamic tools is limited by execution paths that are explored Advantages of lockset analysis: - easy to implement, lightweight, has the potential to scale well (in contrast with happens-before based analyses)
  • 39. Limitations of lockset analysis: - imprecision (a violation of locking discipline is not always a race) - code coverage in dynamic tools is limited by execution paths that are explored - to counter the latter, we apply lockset analysis in a symbolic context by analysing source code Advantages of lockset analysis: - easy to implement, lightweight, has the potential to scale well (in contrast with happens-before based analyses)
  • 40. We use Read and Write Sets during lockset analysis to keep track of shared memory location accesses — not in previous example for simplicity Some other (driver-specific) optimisations read our paper!
  • 41. Boogie IVL code, instrumented with yields Data Race Reports No Errors (Under Given Bounds) WHOOP Error Traces Z3 Chauffeur SMACK Linux driver source code in C Boogie IVL code llvm-IR Linux Environmental Model Instrumentation Sequentialization Invariant Generation Boogie Verification Engine CORRAL A. Translation Phase B. Symbolic Lockset Analysis Phase C. Bug-Finding Phase Clang / LLVM entry point information Toolchain
  • 42. W Inst Sequ Invari Ve Translation Phase Chauffeur SMACK Linux driver source code in C Boogie IVL code llvm-IR Linux Environmental Model Clang / LLVM entry point information
  • 43. W Inst Sequ Invari Ve Translation Phase Chauffeur SMACK Linux driver source code in C Boogie IVL code llvm-IR Linux Environmental Model Clang / LLVM entry point information
  • 44. W Inst Sequ Invari Ve Translation Phase Chauffeur SMACK Linux driver source code in C Boogie IVL code llvm-IR Linux Environmental Model Clang / LLVM entry point information Debugging information is preserved so that we can map bugs back to C source code
  • 45. Symbolic Lockset Analysis Phase ACK Boogie IVL code entry point information Data Race Reports WHOOP Z3 Instrumentation Sequentialization Invariant Generation Boogie Verification Engine Boogie IVL code, instrumented with yields No Errors (Under Given Bounds) Error Tr CORRAL
  • 46. Symbolic Lockset Analysis Phase ACK Boogie IVL code entry point information Data Race Reports WHOOP Z3 Instrumentation Sequentialization Invariant Generation Boogie Verification Engine Boogie IVL code, instrumented with yields No Errors (Under Given Bounds) Error Tr CORRAL
  • 47. Challenges and Solutions Aliasing is a big challenge — we exploit LLVM alias analyses and SMACK heap modelling Scalability is a big issue — we generate and infer invariants and perform summarisation for modular verification (each procedure verified in isolation) read our paper!
  • 48. - imprecision as it inherits the limitations of lockset analysis Limitations
  • 49. - imprecision as it inherits the limitations of lockset analysis - sound over-approximation of shared state and lock-free data structures can lead to false alarms Limitations
  • 50. - imprecision as it inherits the limitations of lockset analysis - sound over-approximation of shared state and lock-free data structures can lead to false alarms - do not currently handle interrupt handlers in a special way, we just assume they execute concurrently at all times (which however is sound) Limitations
  • 51. To improve Whoop’s precision, we use Corral [5], an industrial strength bug-finder for device drivers from Microsoft Research that is currently used by Microsoft in the backend of the Static Driver Verifier (SDV) Improving precision [5] A. Lal, S. Qadeer, and S. K. Lahiri. A solver for reachability modulo theories. CAV 2012
  • 52. The sequentialised program is bounded by a number of context-switches per thread [6] A. Lal and T. W. Reps. Reducing concurrent analysis under a context bound to sequential analysis. CAV 2008 Corral uses the Lal-Reps sequentialisation [6] to transform a concurrent program into a non-deterministic sequential program Bounded Bug-Finding
  • 53. Bounded Bug-Finding Corral can then explore the sequential program to detect data races, encoded as race-checking assertions If a race is found, a trace (mapped on the original C code) is reported to the user
  • 54. Bug-Finding Phase ie de int ion Data Race Reports WHOOP Z3 Instrumentation Sequentialization Invariant Generation Boogie Verification Engine Boogie IVL code, instrumented with yields No Errors (Under Given Bounds) Error Traces CORRAL
  • 55. State-space explosion Corral inserts a context switch before each shared memory access and at each synchronisation point (e.g. lock/unlock)
  • 56. Problem: this can be too coarse and state- space can explode in large drivers! State-space explosion Corral inserts a context switch before each shared memory access and at each synchronisation point (e.g. lock/unlock)
  • 57. Sound partial-order reduction If our symbolic lockset analysis finds that a given shared memory access is not racy, then we do not instrument a context switch before this memory access
  • 58. This tames the sequentialisation and can greatly speedup Corral as we have found in our experiments Sound partial-order reduction If our symbolic lockset analysis finds that a given shared memory access is not racy, then we do not instrument a context switch before this memory access
  • 59. We applied our technique on 16 drivers from the Linux 4.0 kernel: - block, char, ethernet, nfc, usb and watchdog (250 — 7300 LoC) - exploiting our symbolic lockset analysis we can significantly accelerate Corral by 1.5-20x ! - in 1 driver there was slowdown, because we did not manage to verify race freedom - detected potential races (filled bug report, but have not confirmed yet) Evaluation
  • 60. Whoop Corral Whoop + Corral Benchmarks Time (sec) # Context Switches CSB = 9 Time (sec) # Context Switches CSB = 9 Time (sec) generic_nvram 2.3 92 197.5 29 49.3 pc9736x_gpio 4.1 691 27337.6 167 1358.9 intel_scu_wd 2.4 314 1571.7 196 689.3 fs3270 3.2 321 T.O. (10 hours) 211 8883.0 intel_nfcsim 3.6 732 1539.9 601 278.0 sx8 2.8 227 21.4 217 24.3 Read the paper for more results!
  • 61. Applied our symbolic lockset analysis on >1000 concurrency benchmarks from SVCOMP 2016: - Early results: reducing Corral runtime by ~75% - No bugs missed, gives more confidence in our partial-order reduction! We want to extend our Linux model so that we can apply our technique on more drivers! We want to experiment with applying our technique on other Linux modules (e.g. filesystems) New / Future work