Zeppelin meetup 2016 madrid
2 likes191 views
The document discusses advanced features of Apache Zeppelin, including new visualization extensions, multi-user capabilities, and enhanced security through user permissions and authentication. It outlines the pros and cons of shared, isolated, and scoped multi-tenancy models for resource management. Key features mentioned include pluggable visualizations and analytics, as well as the integration with Apache Shiro for enterprise authentication.
Zeppelin meetup 2016 madrid
- 1. Advanced features of Apache Zeppelin http://zeppelin.apache.org
- 2. Jongyoul Lee PMC of Apache Zeppelin from Sep. 2015. Software Development Engineer at NFLabs
- 3. Advanced? • lium • A new extension for visualization • Multi-users features • Users & Permissions • Per user/Per note & Shared/Scoped/Isolated • Futures • Impersonation & Personalized mode • Scalability & Reliability He 2
- 4. liumHe 2
- 5. Zeppelin Visualizations : 6 Built-in visualizations comes with pivot Table Bar Pie Area Line Scatter Free to draw any customized visualizations inside of notebook …
- 6. He liumHe 2 Interpreter Notebook Storage Spark Flink Geode JDBC … FileSystem AmazonS3 Git … Application Visualizations Map WordCloud … Resource Pool SparkContext Flink Environment JDBC connection … Analytics … … User object Extend pluggable visualization to pluggable analytics application Working in progress to make visualization pluggable
- 8. • Company complains • Why security works … • Why authentication works … • Why Zeppelin stores my password as plain … • Why two user use same Spark … • Why I wait while other run somethings & Enterprise
- 9. Auhentication : Integrated with Apache Shiro Contributions - PAM - ActiveDirectory - Jdbc - Jndi - Ldap - Properties Zeppelin
- 10. Notebook Authorization : Owners, Writers, Readers per Note Zeppelin
- 11. Multi-tenancy Per user/Per note & Shared/Scoped/Isolated
- 12. SHARED ISOLATED SCOPED PROCESS 1 N 1 THREADS 1 1 N Multi-tenancyZeppelin
- 13. ZeppelinServer SparkInterpreter Run P1 on NoteA Run SparkInterpreter for P1 User1 User2 Run P2 on NoteB Run SparkInterpreter for P2 SharedZeppelin
- 14. • Originally implemented • Pros • Simple structure • Predictable behavior • Cons • All resources shared • Interference among users SharedZeppelin
- 15. ZeppelinServer SparkInterpreter Run P1 on NoteA Run SparkInterpreter for P1 User1 User2 Run P2 on NoteB Run SparkInterpreter for P2 SparkInterpreter IsolatedZeppelin
- 16. • Pros • No pending • No resources shared • Cons • Lots of memory • Inefficiency of using memory • Limited by resources IsolatedZeppelin
- 17. ZeppelinServer JDBCInterpreter Run P2 on NoteA Run SparkInterpreter for P2 User1 User2 Run P3 on NoteB Run SparkInterpreter for P3 Scoped JDBCInstance User1 JDBCInstance User2 Zeppelin
- 18. • Pros • Less memory • Some resources Isolated • Cons • Some resources shared • Big single process ScopedZeppelin
- 19. SHARED ISOLATED SCOPED PROCESS 1 N 1 THREADS 1 1 N Multi-tenancyZeppelin
- 20. • ~ 0.7.0 • Impersonation of JDBC/Spark Interpreter • Personalized mode • 0.7.0 ~ • Scalability & Reliability • … & Futures