Static partitioning virtualization on RISC-V
Download as PPTX, PDF0 likes683 views
This document summarizes a presentation on static partitioning virtualization for RISC-V. It discusses the motivation for embedded virtualization, an overview of static partitioning hypervisors like Jailhouse and Xen, and the Bao hypervisor. It then provides an overview of the RISC-V hypervisor specification and extensions, including implemented features. It evaluates the performance overhead and interrupt latency of a prototype RISC-V hypervisor implementation with and without interference mitigations like cache partitioning.
Static partitioning virtualization on RISC-V
- 1. Static Partitioning Virtualization on RISC-V José Martins and Sandro Pinto RISC-V Summit @ Virtual December 8th, 2020
- 2. Agenda Introduction Motivation and Background 01 RISC-V Virtualization RISC-V virtualization in a nutshell 02 RISC-V H-Extension Implementation & enhancements 03 Evaluation Tests, experiments, and results 04 Conclusion Final thoughts and Takeaways 05
- 4. Mixed-Criticality 4 ESRGv3 RISC-V Summit 20 • High Complexity • Multiple Subsystems • Heterogeneous Software Stacks (RTOS, GPOS) • Different Criticality Levels • Size, Weight, Power and Cost (SWaP-C) Automotive Industrial Automation
- 5. Embedded Virtualization 5 ESRGv3 ESRGv3 RISC-V Summit 20 Consolidation Size Weight Power Cost Low Engineering Cost Full-Virtualization allows direct porting of guest OSs Fault Containment Sandboxed Enviroments Performance Low Virtualization Overhead Close to Native Performance Security Small TCB Side-channel TEE support Open Design Real-time Low latencies Determinism/Predictability Freedom-from-interference
- 6. Traditional Hypervisors 6 ESRGv3 ▪ Not designed for embedded/MCS: ▪ Although retrofitted with success ▪ High-overhead IO: ▪ Emulated ▪ Para-virtualization/Backend-drivers ▪ Large Code Base : ▪ Hosted hypervisor ▪ Privileged VMs run large monolithic OSs (typically Linux) part of TCB. ESRGv3 RISC-V Summit 20
- 7. Static Partitioning Virtualization 7 ESRGv3 Jailhouse Static Partitioning: Thin configuration/partition layer 1:1 vCPU-to-pCPU mapping Static memory assignment Device Pass-through Hardware interrupts Jailhouse: Needs “root cell” to boot and manage VMs Large boot time Xen Dom0-less*: DomUs may boot w/o Dom0 Direct Device assignment Dom0-less ESRGv3 RISC-V Summit 20 * https://www.youtube.com/watch?v=OrtV6gyHW74
- 8. Bao Overview 8 ESRGv3 01 02 03 Type-1 / Bare-metal Static Partitioning: 1:1 vCPU-to-pCPU mapping Static memory assignment Device Pass-through Hardware-assisted: 2nd-stage translation Interrupt virtualization support IOMMU Dependencies: No external libraries / privileged VMs Small TCB (~7K SLoC) Real-time & Security: Predictability / Freedom-from-interference Side-channels / TEE support ESRGv3 RISC-V Summit 20 José Martins, Adriano Tavares, Marco Solieri, Marko Bertogna, and Sandro Pinto. "Bao: A Lightweight Static Partitioning Hypervisor for Modern Multi-Core Embedded Systems". In NG-RES 2020 https://drops.dagstuhl.de/opus/volltexte/2020/11779/
- 9. Bao Hypervisor Support 9 ESRGv3 01 03 Architectures: Armv8-A RISC-V (v0.6.1 Hyp Spec.) Platforms: Zynq US+ (ZCUx and Ultra96) HiSilicon96 (Hikey96) NXP i.MX8 Nvidia Tegra TX2 QEMU Rocket @ ZCU Firmware: Arm Trusted Firmware (PSCI) on Arm Supervisor Binary Interface (SBI) on RISC-V U-boot Guests: Bare-metal Linux / Android RTOSs (FreeRTOS, Erika) ESRGv3 RISC-V Summit 20
- 10. RISC-V Virtualization RISC-V virtualization in a nutshell
- 11. Hypervisor Extension 11 ESRGv3 Designed for type-1 and type-2 hypervisors Additional execution modes HS-mode (hypervisor-extended supervisor) VS-mode & VU-mode Additional CSR registers: HS-mode CSRs for hypervisor capabilities (e.g., hstatus) HS-mode CSRs for accessing Guest/VM state (e.g., vsstatus) Two-Stage MMU: Stage 1: VS-mode page table (GVA -> GPA) Stage 2: HS-mode guest page table (GPA -> HPA) ESRGv3 RISC-V Summit 20 https://github.com/riscv/riscv-isa-manual/releases/latest
- 12. Hypervisor Extension 12 ESRGv3 ESRGv3 RISC-V Summit 20 Hypervisor Firmware (SBI) Decreasing Privilege level M Guest OS Guest User Space Host User Space Virtualised Environment Non-virtualised Environment HS VS Adapted from: Alistair Francis (WD), “Developing the RISC-V Hypervisor Extensions in QEMU”, Embedded Linux Conference Europe, 2019 VU M HS U
- 13. Spec Status 13 ESRGv3 ESRGv3 RISC-V Summit 20 Currently version 0.6.1 Feedback from open source projects Contributions from organizations and individuals H-Extension spec close to freeze state Hypervisor “group” (join RISC-V Hypervisor sync-up calls!) Function completeness (KVM & Xvisor) RTL implementations (we have contributed with one ) KVM and Xvisor running on FPGA implementation (we have ran Xvisor on an FPGA ) Open source projects support: QEMU (v0.6.1) KVM and Xvisor (v0.6.1)
- 14. RISC-V Hypervisors 14 ESRGv3 KVM: Type-2 hypervisor, Linux-based architecture, open-source Enterprise virtualization setups (datacenters and private clouds) https://github.com/kvm-riscv/linux Xvisor: Type-1 hypervisor, monolithic architecture, open-source Embedded systems with soft real-time requirements https://github.com/avpatel/xvisor-next Bao: Type-1 hypervisor, static partitioning architecture, open-source Mixed-criticality systems with strong real-time and security requirements https://github.com/bao-project/bao-hypervisor ESRGv3 RISC-V Summit 20
- 15. RISC-V H-Extension Implementation & enhancements
- 16. Spec Checklist 16 ESRGv3 01 02 03 H-Extension, Version 0.6.1 RV64 and sv39 only Implemented: h- and vs- csrs and respective functionality m- registers extended accordingly hypervisor load/store instructions new virtual instruction exception hfence instructions (limited, flushes complete TLB) guest external interrupts (w/ PLIC virtualization extension) Not Implemented: ASID/VMID support Transformed Instruction or Pseudoinstruction for htinst/mtinst ESRGv3 RISC-V Summit 20
- 17. H-Extension Implementation 17 ESRGv3 01 02 03 Available at: https://github.com/josecm/rocket-chip/tree/hyp Synced with chipyard’s master branch commit of rocket-chip (@1872f5d, 6 months ago) ~1100 sLOC modifications to rocket-chip. Mainly on: CSR PTW TLB ESRGv3 RISC-V Summit 20
- 18. H-Extension Implementation 18 ESRGv3 01 02 03 Next Steps: Refactoring/Optimizations/Clean-up ASID/VMID support Increase hfence granularity Improve 2nd-stage translation data structures (e.g. dedicated guest physical address TLB) Software: Ad hoc testing Bao Hypervisor Xvisor ESRGv3 RISC-V Summit 20
- 19. PLIC H-Extension 19 ESRGv3 01 02 03 Main requirements: minimal design direct injection of physical interrupts for the active guest no traps on claim/complete mix of physical and virtual interrupts Extra GEILEN VS-contexts per physical hart Hypervisor must trap-and-emulate VS-context priority and enable registers. Hypervisor gives guests direct access to VS- context’s claim/complete registers. Virtual interrupts are injected by writing to virtual interrupt injection registers (VIIR) which can be grouped. ESRGv3 RISC-V Summit 20 02 PLIC RISC-V Hart 0 RISC-V Hart 1 RISC-V Hart n M-mode External Interrupt S-mode External Interrupt VS-mode External Interrupt 0 VS-mode GEILEN-1 * The PLIC itself might be deprecated as a new standard interrupt controller is in the workings. https://github.com/josecm/riscv-plic-spec/tree/virt
- 20. Cache and Bandwidth Partitioning 20 ESRGv3 Software-based techniques: Increases TCB High Overheads Coarse-grained E.g.: page coloring, PMU event based throttling Cache Partitioning: Per bus-master way-locking Lightweight modification to eviction circuit Memory bandwidth throttling: Bandwidth regulation unit (BRU) Per partition memory bandwidth budgets ESRGv3 RISC-V Summit 20 Adapted from: "BRU: Bandwidth Regulation Unit for Real- Time Multicore Processors,". F. Farshchi, Q. Huang and H. Yun. RTAS 2020
- 21. Evaluation Tests, experiments, and results
- 22. Experimental Setup 22 ESRGv3 Hardware ▪ Hexa-core rocket ▪ 3.2GHz ▪ 16 KB iL1$ and dL1$ ▪ 8-way, 512 KB unified L2$ w/ way-locking ▪ Plic Virt extensions ▪ BRU System Configurations ▪ Bare-metal / no hypervisor (bare) ▪ Single guest (solo) ▪ Interference (interf) ▪ Partitioning (solo+part) ▪ Partitioning w/ Interference (interf+part) Metrics ▪ Performance Overhead ▪ Interrupt Latency RISC-V Summit 20 Software Stack ▪ OpenSBI ▪ Bao Hypervisor ▪ Linux OS (VM) ▪ Baremetal App (VMs) Memory Resource Partitioning ▪ 4 L2$ ways ▪ 1800 MiB/s
- 23. Performance Overhead 23 ESRGv3 ESRGv3 RISC-V Summit 20 ▪ MiBench Benchmark Suite ▪ Automotive subset ▪ Relative to bare ▪ Bare absolute values ▪ Higher is worse ▪ Negligible variance Hosted execution brings non-negligible overheads (2 to 6%) . We believe this is mainly due to high cost of tick timer handling and non-optimized two-stage translation Memory subsystem interference has performance impacts from 5% to 70% Cache and memory bandwidth partition mitigations significantly reduce interference (1 to 23%) but not completely
- 24. Interrupt Latency 24 ESRGv3 ESRGv3 RISC-V Summit 20 ▪ Custom benchmark ▪ Auto-Reload timer at 100 Hz ▪ Absolute time in ns ▪ L1 Instruction cache invalidated at each tick Trap-and-emulate approach introduces an extra 650 ns (≈800%) in guest average interrupt latency Interference increases latency but can be somewhat attenuated by memory resource partitioning Direct interrupt injection keeps native latency and is not very susceptible to the effects of interference
- 25. Conclusion Final thoughts and Takeaways
- 26. We have presented the first public implementation valuation of the hypervisor extensions in a RISC-V
- 27. 27 ESRGv3 No "real" silicon with H-extension yet probably not for the foreseeable future Static Partitioning “Virtualization” M-mode = hypervisor PMP configuration per hart PLIC emulation IOPMP Bao porting WiP version @ PolarFire Icicle (Renode) Virtualization for RISC-V “as of today” Extended platform support ESRGv3 RISC-V Summit 20 But… do we really need it ?!
- 28. TAKEAWAYS 28 • Current state of the RISC-V hypervisor extension specification • First implementation of the hypervisor extensions in a RISC-V core • Static partitioning virtualization and what hardware supports it needs The spec is close to freeze but we encourage the community to contribute 1 There is a need for additional implementations in other RISC-V cores 2 There is a need to define standard extensions that specify memory resource partition interfaces and IOMMUs 3 ESRGv3 RISC-V Summit
- 29. THANK YOU! [email protected] LinkedIn - https://www.linkedin.com/in/josecmar/ Twitter - https://twitter.com/josecarmartins Github - https://github.com/josecm [email protected] LinkedIn - https://www.linkedin.com/in/sandro2pinto Twitter - https://twitter.com/sandro2pinto Github - https://github.com/sandro2pinto/