Apache Geode (incubating) Introduction with Docker
Download as PPTX, PDF0 likes865 views
This document provides an agenda for a hands-on introduction and hackathon kickoff for Apache Geode. The agenda includes details about the hackathon, an introduction to Apache Geode including its history, key features, and roadmap. It also covers hands-on labs for building, running, and clustering Geode as well as creating a first application. The document concludes with information on how to contribute to the Geode project.
Apache Geode (incubating) Introduction with Docker
- 1. Hands-on Introduction & Hackathon Kickoff Ashvin Agrawal William Markito @william_markito@aasoj Powered by Pivotal Open Source Hub (POSH) (incubating)
- 2. • Hackathon Details • Apache Geode Introduction • History • Key features and components • Roadmap • Hands-on lab • Build & run • Starting a cluster • Using docker for clustering • Your first app • Q&A 2 Agenda
- 3. Hackathon details Powered by Pivotal Open Source Hub (POSH) http://ambitious-apps.challengepost.com/
- 5. A distributed, memory-based data management platform for data oriented apps that need: • high performance, scalability, resiliency and continuous availability • fast access to critical data set • location aware distributed data processing • event driven data architecture 5 Introduction
- 6. 6 One size fits all ? Cost of sorting is nlog(n)
- 7. 7 One size fits all ? Cost of sorting is nlog(n) • Data quality and quantity differences • Eventual consistency • Response time expectation • Scalability challenges: disk, memory, network and external systems
- 8. • 1000+ systems in production (real customers) • Cutting edge use cases 8 Incubating… but rock solid
- 9. • 17 billion records in memory • GE Power & Water's Remote Monitoring & Diagnostics Center • 3 TB operational data in-memory, 400 TB archived • China Railways • 4.6 Million transactions a day / 40K transactions a second • China Railways 9 Incubating… but rock solid
- 10. • Performance optimized persistence • Configurable consistency • Elastic capacity • Latency minimizing distribution • Heterogenous deployment Designed for High Performance 10 +/- L2 ~10 ns, memory ~100 ns, network <1ms, disk ~10ms
- 11. • Cache • Region • Member • Client Cache • Functions • Listeners 11 Concepts
- 12. • Cache • In-memory storage and management for your data • Configurable through XML, Spring, Java API or CLI • Collection of Region 12 Concepts
- 13. • Region • Distributed java.util.Map on steroids (Key/Value) • Consistent API regardless of where or how data is stored • Observable (reactive) • Highly available, redundant on cache Member (s). 13 Concepts
- 14. • Region • Local, Replicated or Partitioned • In-memory or persistent • Redundant • LRU • Overflow 14 Concepts LOCAL LOCAL_HEAP_LRU LOCAL_OVERFLOW LOCAL_PERSISTENT LOCAL_PERSISTENT_OVERFLOW PARTITION PARTITION_HEAP_LRU PARTITION_OVERFLOW PARTITION_PERSISTENT PARTITION_PERSISTENT_OVERFLOW PARTITION_PROXY PARTITION_PROXY_REDUNDANT PARTITION_REDUNDANT PARTITION_REDUNDANT_HEAP_LRU PARTITION_REDUNDANT_OVERFLOW PARTITION_REDUNDANT_PERSISTENT PARTITION_REDUNDANT_PERSISTENT_OVERFLOW REPLICATE REPLICATE_HEAP_LRU REPLICATE_OVERFLOW REPLICATE_PERSISTENT REPLICATE_PERSISTENT_OVERFLOW REPLICATE_PROXY
- 15. • Persistent Regions • Durability • WAL for efficient writing • Consistent recovery • Compaction 15 Concepts Server 1 Server N
- 16. • Member • A process that has a connection to the system • A process that has created a cache • Embeddable within your application 16 Concepts Client Locator Server
- 17. • Client cache • A process connected to the Geode server(s) • Can have a local copy of the data • Can be notified about events on the servers 17 Concepts
- 18. • Functions • Used for distributed concurrent processing (Map/Reduce, stored procedure) • Highly available • Data oriented • Member oriented 18 Concepts
- 20. • Listeners • CacheWriter / CacheListener • AsyncEventListener (queue / batch) • Parallel or Serial • Conflation 20 Concepts
- 21. Hands on
- 22. • Clone & Build 22 Hands-on: Build & run git clone https://github.com/apache/incubator-geode cd incubator-geode ./gradlew build -Dskip.tests=true • Start a server cd gemfire-assembly/build/install/apache-geode ./bin/gfsh gfsh> start locator --name=locator gfsh> start server --name=server gfsh> create region --name=myRegion --type=REPLICATE
- 24. • Containers • FreeBSD Jails (2000) • Solaris Zones (2004) • Docker (2013) • Operating system level virtualization • Isolated user space instances 24 * https://linuxcontainers.org/ Hands-on: Docker
- 25. 25 Container vs VM pervisor abstracts the entire device, containers just abstract the operating
- 26. Hands-on: Docker & Compose 26 • Single instance docker run -it apachegeode/geode:nightly gfsh • Cluster docker-compose up • Scale docker-compose scale server=3
- 27. Hands-on: Application 27 • Teeny URL • Fast response time • Statistics • Hits • User agent ? • IPs ? • URL will last for 5 minutes • Distribute data & load • Highly scalable createURL getURL stats
- 28. • HDFS Persistence • Off-heap memory storage • Lucene Search • Spark Integration • Cloud Foundry service 28 Roadmap
- 29. • Code • New features • Bug fixes • Writing tests • Documentation • Wiki • Web site • User guide 29 How to Contribute • Community • Join the mailing list • Ask or answer • Join our HipChat • Become a speaker • Finding bugs • Testing an RC/Beta
- 30. • JIRA https://issues.apache.org/jira/browse/GEODE • Wiki cwiki.apache.org/confluence/display/GEODE • GitHub https://github.com/apache/incubator-geode • Mailing lists mail-archives.apache.org/mod_mbox/incubator-geode-dev/ 30 Links