Reactive Databases for Big Data applications
0 likes592 views
This document discusses reactive databases for big data applications. It proposes an architecture with front-end and back-end components to support reactive programming and queries over NoSQL databases. The front-end would use a reactive language to declaratively define queries that observe changes. The back-end would integrate a rules engine with the NoSQL store to evaluate queries incrementally based on data changes and trigger notifications. Future work is outlined to optimize queries, integrate the system with Apache Spark, and implement more reactive language constructs.
Reactive Databases for Big Data applications
- 1. Humberto Rodríguez Avila, Wolfgang de Meuter, Elisa González Boix Reactive Databases for BigData Applications Graph-TA 4th Edition Barcelona, 2016
- 2. Context 1REACTIVE DATABASES FOR BIGDATA APPLICATIONS/HUMBERTO RODRIGUEZ AVILA
- 3. Reactive Programming 2 Server-Side REACTIVE DATABASES FOR BIGDATA APPLICATIONS/HUMBERTO RODRIGUEZ AVILA Behaviours 8 5 7 15 +2 *2 + Dependency graphExpression x =#(a+2) y =#(b*2) z =#x#+#y a x y z Sub<expressions 4 b NO application in Database Contexts
- 4. Databases 3REACTIVE DATABASES FOR BIGDATA APPLICATIONS/HUMBERTO RODRIGUEZ AVILA PubNub Pusher
- 5. Databases 4REACTIVE DATABASES FOR BIGDATA APPLICATIONS/HUMBERTO RODRIGUEZ AVILA r.table(“users”).filter( r.row(“age”).lt(18) .changes().run(conn, function(err, cursor) { // update application layer }) • Collaborative web and mobile apps • Streaming analytics apps • Multiplayer games • Realtime marketplaces • Connected devices Limited reactive capabilities Notifications through callbacks Changefeeds example NO shares intermediate results
- 6. 5REACTIVE DATABASES FOR BIGDATA APPLICATIONS/HUMBERTO RODRIGUEZ AVILA Reactive Programming Databases Reactive BigData applications +
- 7. Architecture overview 6REACTIVE DATABASES FOR BIGDATA APPLICATIONS/HUMBERTO RODRIGUEZ AVILA Front-end Component Back-end Component Reactive Language Language Integrated Query Graph Dependency Manager Reactive Database NoSQL Store Rules Engine Notification Manager BigData Analysis
- 8. Front-end component 8REACTIVE DATABASES FOR BIGDATA APPLICATIONS/HUMBERTO RODRIGUEZ AVILA var dbBooks = from b in R.Books where b.tag = “database” select b; var reactDbBooks = from b in R.Books where b.tag = “database” select b observe as “database books”; DataSource Declarative syntax 1 2 Reactive Query 3 var response = R.unregisterQuery(“database books”); true | Error Reactive Language Language Integrated Query Graph Dependency Manager
- 9. Back-end component 7REACTIVE DATABASES FOR BIGDATA APPLICATIONS/HUMBERTO RODRIGUEZ AVILA Rules Engine Working Memory RuleSet NoSQL In-Memory Database Persistent - Rete-based - Node sharing - Accumulate functions NoSQL Store 1 2 Notification Manager Socket-based 3 triggers Graph Dependency Manager Reactive Language BigData Analysis 4
- 10. Back-end component 7REACTIVE DATABASES FOR BIGDATA APPLICATIONS/HUMBERTO RODRIGUEZ AVILA var reactDbBooks = from c in R.Customer join!a!in R.Account on c.id == a.id_costumer where c.age < 20 and a.balance <= 100 select c observe as “customers C”; 1 2 Top node Customer Account age < 20 balance <= 100 age < 20 & balance <= 100 Object-type node Alpha node Join node Terminal node
- 11. Future Work 7REACTIVE DATABASES FOR BIGDATA APPLICATIONS/HUMBERTO RODRIGUEZ AVILA - Implement of garbage collector in the rule engine - Apply query optimisations techniques to the Rete-Graph - Integration of our Riak extension with Apache Spark - Implementation of reactive constructs for Apache Spark - Finish the implementation of our reactive-query language Garbage collector Apache Spark … reactive constr… Query optimi…
- 12. Contact Humberto Rodríguez Avila [email protected] Prof. Dr. Elisa Gonzalez Boix [email protected] Prof. Dr. Wolfgang De Meuter [email protected] 14 http://soft.vub.ac.be http://vub.ac.be
- 13. Humberto Rodríguez Avila, Wolfgang de Meuter, Elisa González Boix Reactive Databases for BigData Applications Graph-TA 4th Edition Barcelona, 2016