Expectations for optical network from the viewpoint of system software research
4 likes1,820 views
The document discusses the challenges and opportunities of integrated photonics in future datacenters, emphasizing the importance of optical networks for achieving high bandwidth and energy efficiency. It outlines trends in datacenter architecture, including disaggregation and the concept of 'datacenter in a box', while proposing new software architectures to manage data processing efficiently. Additionally, it highlights various initiatives and research directions, including AIST's impulse project and the necessity of a dedicated datacenter operating system.
![Direct Memory Copy over DWDM
• Assume processor-‐‑‒
memory embedded
package with WDM
interconnect.
• To fully utilize the huge
I/O bandwidth realized
by DWDM.
• Multiple memory blocks
can be sent/received
simultaneously using
multiple wavelengths.
• Memory-‐‑‒centric network
is a similar idea [PACT13]
31
Processor
Memory block
Processor
cores
Memory Bank
WDM
Interconnect
Memory block
Single package compute node
Cache
/
MMU
From
Wavelength bank](https://image.slidesharecdn.com/acsi2015-150128050955-conversion-gate02/85/Expectations-for-optical-network-from-the-viewpoint-of-system-software-research-31-320.jpg)
![40
Optical Interconnect:
As the bandwidth demand for traditionally electrical wireline interconnects has accelerated, optics has become an increa
alternative for interconnects within computing systems. Optical communication offers clear benefits for high-speed an
interconnects. Relative to electrical interconnects, optics provides lower channel loss. Circuit design and packaging techn
traditionally been used for electrical wireline are being adapted to enable integrated optical with extremely low power
resulted in rapid progress in optical ICs for Ethernet, backplane and chip-to-chip optical communication. ISSCC 201
dimensional (12×5) optical array achieving an aggregate data-rate of 600Gb/s [8.2]. Pre-emphasis using group-delay
the useful date rate of a 25Gb/s VCSEL to 40Gb/s [8.9]. Additional examples of low-power-linear and non-linear e
electronic dispersion compensation in multi-mode and long-haul cables [8.1, 8.3].
Concluding Remarks:
Continuing to aggressively scale I/O bandwidth is both essential for the industry and extremely challenging. Innovatio
higher performance and lower power will continue to be made in order to sustain this trend. Advances in circuit architectu
topologies, and transistor scaling are together changing how I/O will be done over the next decade. The most exciting and
of these emerging technologies for wireline I/O will be highlighted at ISSCC 2014.
Per-pin data-rate vs. year for a variety of common I/O standards.
2000 2002 2004 2006 2008 2010 2012 2014 2016
1
10
50
Year
Data Rate [Gbps]
HyperTranport
QPI
PCIe
S‐ATA
SAS
OIF/CEI
PON
Fibre Channel
DDR
GDDR
DRAM Data bandwidth trends.
Non-Volatile Memories (NVMs):
the past decade, significant investment has been put into emerging memories to find an alternative to floating-gate based non-volatile
emory. The emerging NVMs, such as phase-change memory (PRAM), ferroelectric RAM (FeRAM), magnetic spin-torque-transfer (STT-
RAM), and Resistive memory (ReRAM), are showing potential to achieve high cycling capability and lower power per bit in read/write
perations. Some commercial applications, such as cellular phones, have recently started to use PRAM, demonstrating that reliability
nd cost competitiveness in emerging memories is becoming a reality. Fast write speed and low read-access time are the potential
enefits of these emerging memories. At ISSCC 2014, a high-density ReRAM with a buried WL access device is introduced to improve
e write performance and area. The next Figure highlights how MLC NAND Flash write throughput continues to improve. However,
hile the Figure following shows no increase in NAND Flash density over the past year, recent devices are built with finer dimensions
more sophisticated 3-dimensional vertical bit cells.
Per-‐‑‒pin data rate of common I/O standards
High Bandwidth Memory
ISSCC2014 Trends
ISSCC2014 Trends
Data source:
http://cpudb.stanford.edu/
2x/1.5yrs
Processor scaling trends](https://image.slidesharecdn.com/acsi2015-150128050955-conversion-gate02/85/Expectations-for-optical-network-from-the-viewpoint-of-system-software-research-40-320.jpg)
Expectations for optical network from the viewpoint of system software research
- 1. Ryousei Takano National Institute of Advanced Industrial Science and Technology (AIST) Special session on challenges and opportunities of integrated photonics in future datacenters ACSI 2015@Tsukuba, 27 Jan. 2015 Expectations for optical network from the viewpoint of system software research
- 2. Outline • Trends in datacenter research and development • AIST IMPULSE Project • Workload analysis • Proposed architecture: Dataflow-‐‑‒centric computing 2
- 3. Introduction • BigData is a killer app in datacenters, it requires a clean slate architecture like “disaggregation”, “datacenter in a box”. • Optical network is key to making them. • Optical path network (all optical path between end-‐‑‒to-‐‑‒end) in a datacenter – Pros: huge bandwidth, energy efficiency – Cons: path switching latency, utilization • To take advantage of optical path network, a new datacenter OS is essential. – Key idea: control/data plane separation 3
- 4. Optical Network in DCs • Similar concept (“disaggregation” or “datacenter in a box”) projects are launched recently. – Open Compute Project (Facebook) – Rack Scale Computing (Intel) – Extremely Shrinking Computing (IBM) – The Machine (HP) – FireBox (UCB) – CTR Consortium (MIT) • Optical network, including photonic-‐‑‒electronic convergence and short (<1km) reach inter-‐‑‒ connection, is key to drive innovation in future datacenters. 4
- 5. 5 Architecture Service Cluster Back-End Cluster Front-End Cluster Web 250 racks Ads 30 racks Cache (~144TB) Search Photos Msg Others UDB ADS-DB TaoLeader Multifeed 9 racks Other small services “Flash at Facebook”, Flash Summit 2013 Standard Systems I Web III Database IV Hadoop V Photos VI Feed CPU High 2&x&E5*2670 High 2&x&E5*2660 High 2&x&E5*2660 Low High 2&x&E5*2660 Memory Low High 144GB Medium 64GB Low High 144GB Disk Low High&IOPS 3.2&TB&Flash High 15&x&4TB&SATA High 15&x&4TB&SATA Medium Services Web,&Chat Database Hadoop (big&data) Photos,&Video MulPfeed, Search,&Ads Five Standard Servers
- 6. Open Compute Project 6 • OCP was founded to openly share designs of datacenter products by Facebook in April 2011. • Shift from commodity products to user-‐‑‒driven design to improve the energy efficiency of large scale datacenters – Industry Standard: 1.9 PUE – Open Compute Project: 1.07 PUE • Specifications: server, storage, rack, network switch, etc. • Products: Quanta Rackgo X, GIGABYTE DataCenter Solution Open Compute Rack v2
- 7. 7
- 8. HP “The Machine” 8 The Machine could be six %mes more powerful than an equivalent conven2onal design, while using just 1.25 percent of the energy and being around 1/100 the size. h:p://www.hpl.hp.com/research/systems-‐research/themachine/
- 9. Datacenter in a Box • The machine is six times faster with 1.25 percent of the energy comparing with K. 9 HPC Challengeʼ’s RandomAccess benchmark
- 10. The Machine: Architecture 10 Photonic Interconnect Compute Elements Memory Elements NV Memory Elements Storage Elements Architecture evolution/revolution “Computing Ensemble”: bigger than a server, smaller than a datacenter, built-in system software – Disaggregated pools of uncommitted compute, memory, and storage elements – Optical interconnects enable dynamic, on-demand composition – Ensemble OS software using virtualization for composition and management – Management and programming virtual appliances add value for IT and application developers On-demand composition Ensemble OS Management Ensemble Programming
- 11. Machine OS • Linux++: Linux-‐‑‒based OS for The Machine – A new concept of memory management – An emulator to make a conventional computer behave like The Machine – A developerʼ’s preview released in June 2015? • Carbon – HP will replace Linux++ with Carbon. 11
- 12. UC Berkeley 1 Terabit/sec optical fibers FireBox Overview! High Radix Switches SoC SoC SoC SoC SoC SoC SoCSoC SoC SoC SoC SoC SoC SoC SoC SoC Up to 1000 SoCs + High-BW Mem (100,000 core total) NVM NVM NVM NVM NVM NVM NVM NVMNVM NVM NVM NVM NVM NVM NVM NVM Up to 1000 NonVolatile Memory Modules (100PB total) InterXBox& Network& Many&Short&Paths& Thru&HighXRadix&Switches& FireBox Overview 12 A similar concept of The Machine
- 13. UC Berkeley Photonic-Switches- ! Monolithically&integrated&silicon&photonics&with&WaveXDivision& MulCplexing&(WDM)& - A&fiber&carries&32&wavelengths,&each&32Gb/s,&in&each&direcCon& - OffXchip&laser&opCcal&supply,&onXchip&modulators&and&detectors& ! MulCple&radixX1000&photonic&switch&chips&arranged&as&middle& stage&of&Clos&network&(first&and&last&Clos&stage&inside&sockets)& ! 2K&endpoints&can&be&configured&as&either&SoC&or&NVM&modules& ! In&Box,&all&paths&are&two&fiber&hops:& - ElectricalXphotonic&at&socket& - One&fiber&hop&socketXtoXswitch& - PhotonicXelectrical&at&switch& - Electrical&packet&rouCng&in&switch& - ElectricalXphotonic&at&socket& - One&fiber&hop&switchXtoXsocket& - PhotonicXelectrical&at&socket& 30 SoC& Switch& Switch& SoC& NVM& 13 Electrical packet switching
- 14. IMPULSE: Initiative for Most Power-efficient Ultra-Large-Scale data Exploration 2014 2020 2030 ・・・ ・・・ Op2cal Network 3D stacked package 2.5D stacked package Separated packages Future data center Logic I/O NVRAM Logic NVRAM I/O I/O Logic NVRAM High-Performance Logic Architecture Non-Volatile Memory Optical Network - Voltage-controlled, magnetic RAM mainly for cache and work memories - 3D build-up integration of the front-end circuits including high-mobility Ge-on- insulator FinFETs. / AIST-original TCAD - Silicon photonics cluster SW - Optical interconnect technologies - Future data center architecture design / Dataflow-centric warehouse- scale computing
- 15. 15 HPC Big data Architecture for concentrated data processing 3D installation Non-‐‑‒volatile memory Energy-‐‑‒ saving logic Optical path between chips Non-‐‑‒volatile memory Energy-‐‑‒ saving logic Storage class memory（non-‐‑‒volatile memory） HDD storage High performance server module Energy-‐‑‒saving high-‐‑‒speed network Energy-‐‑‒saving large-‐‑‒capacity storage Creating a rich and eco-‐‑‒friendly society Initiative for Most Power-‐‑‒efficient Ultra-‐‑‒Large-‐‑‒Scale data Exploration IMPULSE STrategic AIST integrated R&D (STAR) program ＊STAR program is AIST research that will produce a large outcome in the future. AIST’s IMPULSE Program
- 16. Voltage-controlled Nonvolatile Magnetic RAM Nonvolatile CPU Nonvolatile Cash Nonvolatile Display Power saved storage NAND Flash Voltage Controlled Spin RAM • voltage-induced magnetic anisotropy change • Less than 1/100 rewriting power • Resistance change by the Ge displacement • Loss by entropy: < 1/100 Voltage Controlled Topological RAM Memory keeping w/o power Insulation Layer Thin film Ferro- magnetics
- 17. Low Power High-performance Logic Wiring layer Front -end Front-end 3D integration ソース ドレイン ソース ドレイン nMOS pMOS 絶縁膜 Ge Ge Ge Ge Fin CMOS Tech. • Low-power/high-speed by Ge • Toward 0.4V - Ge Fin CMOS S D S D Insulation layer ● Dense integration w/o miniaturization ● Reduction of the wiring length for power saving ● Introduction of Ge and III-V channels by simple stacking process ● Innovative circuit by using Z direction
- 18. Simulations Anisotropic magnetic material First-principle sim. TCAD - Large-scale simulation for 3D structure, novel devices, and latest material Simulation for temperature distribution - Clarification for voltage-control - Parameter setting to the TCAD Phase-change
- 19. DEMUX Wavelength bank (Optical comb) ・ ・ ・ MOD. MUX Fiber Silicon Photonics Integration Datacenter server racks Silicon photonics cluster switches DWDM, multi-level modulation optical interconnects DSP Tx RxComb source Memory cube CPU /GPU 2.5D-CPU Card No of λs Order of mod. Bit rate 1 1 20 Gbps 4 8 640 Gbps 32 8 5.12 Tbps ●Large-scale silicon photonics based cluster switches ●DWDM, multi-level modulation, highly integrated “elastic” optical interconnects ●Ultra-low energy consumption network by making use of optical switches Ø Ultra-compact switches based on silicon photonics Ø 3D integration by amorphous silicon Ø A new server architecture Current state-of-the-art Tx 100Gbps → ～ 5.12Tbps Current electrical switches： ～130Tbps → ～500Pbps Optical Network Technology for Future Datacenters
- 20. Architecture for Big Data and Extreme-scale Computing Real-time Big data Optimal arrangement of the data flow Resource management / Monitoring Storage Server Module Data center OS Input Output Conv. Ana. Data flow Data flow centric warehouse scale compu%ng 1 - Single OS controls entire data center 2 - Split a data center OS into the data plane and the control plane to guarantee real-time data processing Connect to universal processor / hardware and storage by using optical network
- 21. Performance Estimation • Estimate the performance of typical both HPC and BigData workloads on a future datacenter system • SimGrid simulator – Simulator of large-‐‑‒scale distributed Systems, such as Grids, Clouds, HPC, and P2P. 21 http://simgrid.gforge.inria.fr mGrid Overview MSG Simple application- level simulator SimDag Framework for DAGs of parallel tasks applications on top of a virtual environment Library to run MPI SMPI virtual platform simulator SURF Contrib Grounding features (logging, etc.), data structures (lists, etc.) and portability XBT TRACETracingsimulation User Code Grid user APIs If your application is a DAG of (parallel) tasks ; use SimDag To study an existing MPI code ; use SMPI SimGrid is not a Simulator logs stats visu Availibility Changes Platform Topology Application Deployment Simulation Kernel Application Simulator OutcomesScenario Applicative Workload Parameters Input That’s a Generic Simulation Framework Da SimGrid Team SimGrid User 101 Introduction Installing MSG Java lua Ruby Trace Config xbt Performance CC 23/28
- 22. Workload 1: Simple Message Passing • Iteration of neighbor communication (bottom left) • Big impact of increasing link bandwidth if an application is network intensive. 22 Compute power (FLO) Relative execution time Data size (byte) #node: 10000 Link bandwidth: 0.1, 1, 10Tbps Link latency 100ns CPU power: 10TFLOPS Data size：1012〜1024B … … 1/100
- 23. Workload 2: HPC Application • NAS Parallel Benchmark (256 procs, class C) – Low latency is more important than huge bandwidth. – The problem size is too small to utilize huge bandwidth. 23 CPU power：1TFLOPS Link bandwidth：1Tbps CPU power：1TFLOPS Link latency：0.1us Effect of reducing the link latency Effect of increasing the link bandwidth Relative execution time Relative execution time
- 24. Workload 3: MapReduce 24 • KDD Cup 2012, Track 2: predict the click-‐‑‒through rate of ads (using Hadoop and Hivemall) • Machine learning is CPU intensive. • The effect of huge bandwidth is limited, because... – The concurrency of the used model is not enough. – Hadoop is optimized to make jobs run faster on the current I/O devices. Disk I/O bandwidth (Mbps) Execution time (second) Execution time (second) Relative CPU power (base 172GFLOPS) CPU power: 17.2TFLOPS Disk bandwidth: 200Mbps Network bandwidth: 10Gbps
- 25. Memory Optical I/O Logic Board Chassis ・・・ 10TB 1TB/s 100GB/s 1TB/s 100 TFLOPS ・・・ Data Data Movement Problems 25 Memory Electrical I/O Logic Board Chassis ・・・ Current* Future 128GB 14.9GB/s (DDR3) 7GB/s (IB FDR) 224*2 GFLOPS 1TB/s ・・・ 8GB/s (PCIe G3) * AIST Super Green Cloud (ASGC) Data Bottle neck Bottle neck
- 26. Parallelization inside Package Memory Logic 10TB 1TB/s 100 TFLOPS • Need efficient parallelized structure • On-‐‑‒chip network • Interconnection
- 27. Bottlenecks in MapReduce 27 Map Map Map Map Reduce Reduce Reduce Reduce 1. Disk Throughput 2. Network Congestion 3. Serialized Reduce Shuffle
- 28. In-‐‑‒storage, network processing 28 Mappers Shuffle and Reduce Hierarchical and partial reduce processing in each network node to avoid network congestion and serialized reduce. Compute modules are attached in storage to maximize the read throughput from storage.
- 29. In-‐‑‒storage, network processing 29 Mappers Shuffle and Reduce Processing Component Optical Interconnect NVRAM
- 30. 30 PU MEM PU MEM PU MEM PU MEM PU MEM PU MEM PU MEM PU MEM Direct optical I/O connection to non-‐‑‒volatile memory modules distributed on a chip Communication over DWDM In-‐‑‒storage, network processing: hardware design
- 31. Direct Memory Copy over DWDM • Assume processor-‐‑‒ memory embedded package with WDM interconnect. • To fully utilize the huge I/O bandwidth realized by DWDM. • Multiple memory blocks can be sent/received simultaneously using multiple wavelengths. • Memory-‐‑‒centric network is a similar idea [PACT13] 31 Processor Memory block Processor cores Memory Bank WDM Interconnect Memory block Single package compute node Cache / MMU From Wavelength bank
- 32. 32 2014 2020 2030 ・・・ ・・・ Optical Network 3D stacked package2.5D stacked packageSeparated packages Future data center Logic I/O NVRAM Logic NVRAM I/O I/O Logic NVRAM Goal: 100x energy efficiency of data processing Our Vision of Future Datacenter
- 33. DPF DPF DPF DPF Data Flow Application Datacenter OS Data flow planning Slice DPCs Dataflow Processing System 33 DPF: Data Processing Function DPC: Data Processing Component Optical Network Storage Server modules Co-‐‑‒allocate DPCs and network path between them. Resource monitoring
- 34. IMPULSE Datacenter OS • A single OS for datacenter-‐‑‒wide optimization of the energy efficiency and the performance • Separation of data plane and control plane: - Data plane is an application specific library OS. - Control plane manages resources (server, network, etc). Datacenter OS App. Data-‐‑‒ plane App. Data-‐‑‒ plane App. Data-‐‑‒ plane … deploy/launch/destroy/ monitor Control-‐‑‒plane Control-‐‑‒plane 34
- 35. IMPULSE Datacenter OS Data plane • Application specific library OS - E.g., machine learning, data store, etc. • Mitigate the OS overhead to fully utilize high performance devices. App. Data-‐‑‒ plane App. Data-‐‑‒ plane App. Data-‐‑‒ plane Control-‐‑‒ plane CPU CPU GPU Mem I/O Mem I/O Mem I/O Control plane • Resource management • Logical and secure resource partitioning for data planes • Running on the firmware 35
- 36. Related Work • Datacenter-‐‑‒wide resource management – OpenStack, Apache CloudStack, Kubernetes – Hadoop YARN, Apache Mesos • Dataflow processing engine – Google Cloud Dataflow – Lambda architecture • Control and data planes separated design in OS – Arrakis (U. Washington) – IX (Stanford) 36
- 37. Summary • New visions of future datacenters: “disaggregation” and “datacenter in a box” • Optical network is key to making them. • Hardware and software co-‐‑‒design is critical. • Optical path network encourages C/D separation in a datacenter OS. – Control plane manages resources and establishes a path between data processing components. – Data plane fully utilizes the huge bandwidth. 37
- 38. Join us! Thanks for your attention 38
- 39. Reference • Rack scale architecture for Cloud, IDC2013 – https://intel.activeevents.com/sf13/connect/fileDownload/session/ 6DE5FDFBF0D0854E73D2A3908D58E1E2/SF13_̲CLDS001_̲100.pdf • Intel rack scale architecture overview, Interop2013 – http://presentations.interop.com/events/las-‐‑‒vegas/2013/free-‐‑‒ sessions-‐‑‒-‐‑‒-‐‑‒keynote-‐‑‒presentations/download/463 • New technologies that disrupt our complete ecosystem and their limits in the race to Zettascale, HPC2014 – http://www.hpcc.unical.it/hpc2014/pdfs/demichel.pdf • HPが「Tech Power Club」で⾒見見せた“未来のサーバー技術”, ASCII.jp – http://ascii.jp/elem/000/000/915/915508/ 39
- 40. 40 Optical Interconnect: As the bandwidth demand for traditionally electrical wireline interconnects has accelerated, optics has become an increa alternative for interconnects within computing systems. Optical communication offers clear benefits for high-speed an interconnects. Relative to electrical interconnects, optics provides lower channel loss. Circuit design and packaging techn traditionally been used for electrical wireline are being adapted to enable integrated optical with extremely low power resulted in rapid progress in optical ICs for Ethernet, backplane and chip-to-chip optical communication. ISSCC 201 dimensional (12×5) optical array achieving an aggregate data-rate of 600Gb/s [8.2]. Pre-emphasis using group-delay the useful date rate of a 25Gb/s VCSEL to 40Gb/s [8.9]. Additional examples of low-power-linear and non-linear e electronic dispersion compensation in multi-mode and long-haul cables [8.1, 8.3]. Concluding Remarks: Continuing to aggressively scale I/O bandwidth is both essential for the industry and extremely challenging. Innovatio higher performance and lower power will continue to be made in order to sustain this trend. Advances in circuit architectu topologies, and transistor scaling are together changing how I/O will be done over the next decade. The most exciting and of these emerging technologies for wireline I/O will be highlighted at ISSCC 2014. Per-pin data-rate vs. year for a variety of common I/O standards. 2000 2002 2004 2006 2008 2010 2012 2014 2016 1 10 50 Year Data Rate [Gbps] HyperTranport QPI PCIe S‐ATA SAS OIF/CEI PON Fibre Channel DDR GDDR DRAM Data bandwidth trends. Non-Volatile Memories (NVMs): the past decade, significant investment has been put into emerging memories to find an alternative to floating-gate based non-volatile emory. The emerging NVMs, such as phase-change memory (PRAM), ferroelectric RAM (FeRAM), magnetic spin-torque-transfer (STT- RAM), and Resistive memory (ReRAM), are showing potential to achieve high cycling capability and lower power per bit in read/write perations. Some commercial applications, such as cellular phones, have recently started to use PRAM, demonstrating that reliability nd cost competitiveness in emerging memories is becoming a reality. Fast write speed and low read-access time are the potential enefits of these emerging memories. At ISSCC 2014, a high-density ReRAM with a buried WL access device is introduced to improve e write performance and area. The next Figure highlights how MLC NAND Flash write throughput continues to improve. However, hile the Figure following shows no increase in NAND Flash density over the past year, recent devices are built with finer dimensions more sophisticated 3-dimensional vertical bit cells. Per-‐‑‒pin data rate of common I/O standards High Bandwidth Memory ISSCC2014 Trends ISSCC2014 Trends Data source: http://cpudb.stanford.edu/ 2x/1.5yrs Processor scaling trends
- 41. 41 http://www.linuxfoundation.org/collaborate/workgroups/networking/kernel_̲flow Linux kernel is highly complex 64byte TCP Echo A.Belay “IX: A Protected Dataplane Opera3ng System for High Throughput and Low Latency”, OSDI2014 Hardware Linux Latency 10μs 480μs Request per second 8.8 M 1 M Datacenter OS is not the federation of a general purpose OS like Linux.