Oracle OpenWorld 2016 Review - Focus on Data, BigData, Streaming Data, Machine Learning & Database Development
Download as pptx, pdf1 like820 views
The Oracle OpenWorld 2016 highlighted advancements in database development, including the introduction of Oracle Database 12cR2, machine learning, and real-time analytics. Key discussions focused on the need for better data processing methods, cloud-first strategies, and the evolving role of citizen data scientists. The event emphasized the integration of various technologies and the adoption of open-source projects, reflecting current industry trends.
![4
Machine Learning
• Analyze Historical Data (input and result – training set) to discover
Patterns & Models
• Iteratively apply Models to [additional] Input (test set) and compare model
outcome with known actual result to improve the model
• Use Model to predict outcome for entirely new data](https://image.slidesharecdn.com/oow2016-review-blocke-dbdev-bigdata-13thoctober-distribution-161019084554/85/Oracle-OpenWorld-2016-Review-Focus-on-Data-BigData-Streaming-Data-Machine-Learning-Database-Development-4-320.jpg)
![22
Traditional approach
• All enterprise data is in the Oracle [relational] Database
– Except very unstructured documents - and sometimes even those](https://image.slidesharecdn.com/oow2016-review-blocke-dbdev-bigdata-13thoctober-distribution-161019084554/85/Oracle-OpenWorld-2016-Review-Focus-on-Data-BigData-Streaming-Data-Machine-Learning-Database-Development-22-320.jpg)
![25
Trends around data storage
and data processing
• Take processing to data [to reduce data movement]
– Exadata SmartScan in Storage Cells (SQL & R processing)
– Hadoop MapReduce/Spark
– Coherence Processors
– Streaming Analytics
– Microservices, stand alone data domains
• Distributed data partitions – for scalability and parallelization [and fault
tolerance when also replicated]:
– Shards (Oracle Database 12cR2) and Partitioned External Tables
– TimesTen Velocity Scale – distributed In-Memory OLTP
– Hadoop HDFS, Apache Kafka
• New paradigms regarding transactional data
– CQRS (for example Oracle Database In Memory (read) / In Flash/On disk
(read/write), Write behind cache)
– Event Sourcing, Transaction Log](https://image.slidesharecdn.com/oow2016-review-blocke-dbdev-bigdata-13thoctober-distribution-161019084554/85/Oracle-OpenWorld-2016-Review-Focus-on-Data-BigData-Streaming-Data-Machine-Learning-Database-Development-25-320.jpg)
![26
Oracle Database
• How much of your data
– Arrives through (business) transactions that require true ACID?
– Is involved in current business operations?
– Will ever be updated [again]?
– Plays a direct role in integrity [of other records]?
– Is actively accessed [on a regular basis] ?
– Really has to be in the OLTP engine?
• How much of the data currently in your OLTP engine could be off-loaded
– If that data remains accessible through SQL (even from within the OLTP engine,
without altering existing queries) with reasonable response times
• What if such off-loading
– Improves performance of the OLTP engine for transactions
– Shortens batch jobs [by engaging distributed, scale out processing options]
– Opens up possibilities for advanced analytics
– Potentially lowers the cost [licenses & specialized hardware] for the OLTP engine
– Introduces some change and complexity](https://image.slidesharecdn.com/oow2016-review-blocke-dbdev-bigdata-13thoctober-distribution-161019084554/85/Oracle-OpenWorld-2016-Review-Focus-on-Data-BigData-Streaming-Data-Machine-Learning-Database-Development-26-320.jpg)
![33
New in PL/SQL in 12cR2
• Deprecated procedures and functions
• Accessible by at procedure or function level
• JSON support: generation of JSON documents using PL/SQL API and
Oracle supplied Object Types (somewhat akin to XMLType)
– JSON SQL functions available in PL/SQL expressions
• Supplied package dbms_plsql_code_coverage to identify code units not
touched in specific
[test] scenarios
• PL/Scope enhancements –
more fine grained reporting
• Edition Based Redefinition
does ‘garbage collection’ –
editioned objects no
longer in use
are cleaned up](https://image.slidesharecdn.com/oow2016-review-blocke-dbdev-bigdata-13thoctober-distribution-161019084554/85/Oracle-OpenWorld-2016-Review-Focus-on-Data-BigData-Streaming-Data-Machine-Learning-Database-Development-33-320.jpg)
![36
Summary of
Oracle OpenWorld 2016
• (5 days filled to brim with 1800+ sessions, 12 keynotes, 150+ demo
booths, hundreds of vendors and quite a few rumors & hallway tales)
• Infrastructure [as a Service]
– Generation 2 Data Center
– Network & IOPS (storage, NVMe, Flash)
– Exadata SL
• Abdication of the single, central, enterprise Oracle RDBMS – and the
limelight for data
– PDBs
– Sharding
– Hadoop & Spark (& SQL & R)
– Machine Learning
• Adoption of open source projects, industry trends & community darlings
– Node.js, Docker, Microservices, Git(Hub), Python, Slack, …](https://image.slidesharecdn.com/oow2016-review-blocke-dbdev-bigdata-13thoctober-distribution-161019084554/85/Oracle-OpenWorld-2016-Review-Focus-on-Data-BigData-Streaming-Data-Machine-Learning-Database-Development-36-320.jpg)
![37
Summary of
Oracle OpenWorld 2016 (2)
• Cloud [First] strategy
– Migrate & Extend i/o [bidirectional] Lift & Shift
– Cloud@Customer
– Status and future of On Premises software (and yet Engineered Systems)
– Ops in Oracle Data Centers
– Subscription Models, Suites (i/o a la carte)
– How fast can Oracle move [without spreading itself too thin]?
• SaaS and [Unlimited] Applications
– SaaS portfolio quite extensive
– UX is important asset of the SaaS applications
– Real cloud elements are improving (APIs, extensibility)
– Traditional Apps are still evolving [as promised] – and seem to benefit from SaaS and
technological advances across the board
• Oracle Public Cloud consistency, architecture and the Dogfood Doctrine
– Fabric and foundational components
– Designated capabilities and mutual integration](https://image.slidesharecdn.com/oow2016-review-blocke-dbdev-bigdata-13thoctober-distribution-161019084554/85/Oracle-OpenWorld-2016-Review-Focus-on-Data-BigData-Streaming-Data-Machine-Learning-Database-Development-37-320.jpg)
Oracle OpenWorld 2016 Review - Focus on Data, BigData, Streaming Data, Machine Learning & Database Development
- 1. The AMIS Team Oracle OpenWorld 2016, Nieuwegein, 13th October 2016 Oracle OpenWorld 2016 Review Data - Database Development, BigData, BI
- 2. 2 Data • ACID is expensive • OLTP is a niche • Moving data takes long – bring processing to data • SQL is omnipresent – expose all data in an SQL friendly way – Including NoSQL and data on Hadoop • Data from the past should be able to help us predict the future – Bring on machine learning (aka AI aka predictive analytics) – Aided by the citizen data scientist in (Big) Data Discovery • Fast Data (big data at high velocity) should be handled in real time – Enter: Streaming Analytics & Apache Kafka • Oh and ehm …. a next major release of Oracle Database is available – 12cR2 – only on the cloud for now – Highlights: Sharding, More PDB (“virtual database”), Approximate Query Processing, Leverage In-Memory even more, JSON document generation and faster JSON processing, Analytic Views
- 3. 3 Learning • How do we learn? – Try something (else) => get feedback => learn • Eventually: – We get it (understanding) so we can predict the outcome of a certain action in a new situation – Or we have experienced enough situations to predict the outcome in most situations with high confidence • Through interpolation, extrapolation, etc. – We remain clueless
- 4. 4 Machine Learning • Analyze Historical Data (input and result – training set) to discover Patterns & Models • Iteratively apply Models to [additional] Input (test set) and compare model outcome with known actual result to improve the model • Use Model to predict outcome for entirely new data
- 5. 5 Data Discovery A B C D E F G 1104534 ZTR 0.1 anijs 2 36 T 631148 ESE 132 rivier 0 21 S -3 WGN 71 appel 0 1 - 1262300 ZTR 56 zes 2 41 T 315529 HVN 1290 hamer 0 11 - 788914 ASM 676 zwaluw 0 26 T 157762 HVN 9482 wie 0 6 - 946681 DHG 42 rond 1 31 T -31539 WGN 2423 bruin 0 0 - 47338 HVN 54 hamer 0 16 P
- 6. 6 Scatter Plot Attribute F (Y-axis)vs Attribute A 0 5 10 15 20 25 30 35 40 45 -500000 0 500000 10000001500000 Y-Values Y-Values
- 7. 7 Scatter Plot Attribute F (Y-axis)vs Attribute A 0 5 10 15 20 25 30 35 40 45 1960 1980 2000 2020 Age of Lucas Jellema vs Year Y-Values
- 8. 8 Data Discovery Time City - - #Kids Age Level of Education 1104534 ZTR 0.1 anijs 2 36 T 631148 ESE 132 rivier 0 21 S -3 WGN 71 appel 0 1 - 1262300 ZTR 56 zes 2 41 T 315529 HVN 1290 hamer 0 11 - 788914 ASM 676 zwaluw 0 26 T 157762 HVN 9482 wie 0 6 - 946681 DHG 42 rond 1 31 T -31539 WGN 2423 bruin 0 0 - 47338 HVN 54 hamer 0 16 P
- 9. 9 Machine Learning, Data Mining & Predictive Analytics
- 10. 10 Recent developments • More compute capacity, smarter algorithms and better analytical tooling – Evolving Machine Learning – Smart text and speech analysis (NLP, ESA) – Real time predictions become a reality – Streaming (event) Analytics – Visualization – Citizen Data Scientist – SQL against Big Data • More data available & accessible (IoT, Social, Media, IT operations, business processes,…) • Better/larger/cheaper/faster data storage capabilities
- 11. 11 Many cloud services around Big Data & Analytics
- 12. 12 Big Data Integration Reference Architecture IngestPrepare Transform, Merge, Enrich Save GovernGovern Serve Analyze & Act Present, Leverage & ‘Action’ Extract Explore Purge
- 14. 14 Mapping Oracle portfolio to Reference Architecture Big Data Discovery Data Visualization BI CS IT Analytics Security Analytics Log Analytics
- 15. 15 Tip: OEMM - Oracle Enterprise Metadata Management
- 16. 16 Citizen Data Scientist • Data Visualization CS • Big Data Preparation CS • Big Data Discovery CS • Machine Learning CS
- 17. 17 Oracle Machine Learning Cloud Service
- 18. 18
- 19. 19
- 22. 22 Traditional approach • All enterprise data is in the Oracle [relational] Database – Except very unstructured documents - and sometimes even those
- 23. 23 Center of the Data Universe is shifting
- 24. 24 Variety in data – Data Tiering • How long relevant (hot vs cold vs dead)? • How fine grained and how accurate? • What is it used for? – By whom, where, in what way, using which tools • What format is it in/should it be in? • How to be processed? • What confidentiality & integrity is required? • How much of it?
- 25. 25 Trends around data storage and data processing • Take processing to data [to reduce data movement] – Exadata SmartScan in Storage Cells (SQL & R processing) – Hadoop MapReduce/Spark – Coherence Processors – Streaming Analytics – Microservices, stand alone data domains • Distributed data partitions – for scalability and parallelization [and fault tolerance when also replicated]: – Shards (Oracle Database 12cR2) and Partitioned External Tables – TimesTen Velocity Scale – distributed In-Memory OLTP – Hadoop HDFS, Apache Kafka • New paradigms regarding transactional data – CQRS (for example Oracle Database In Memory (read) / In Flash/On disk (read/write), Write behind cache) – Event Sourcing, Transaction Log
- 26. 26 Oracle Database • How much of your data – Arrives through (business) transactions that require true ACID? – Is involved in current business operations? – Will ever be updated [again]? – Plays a direct role in integrity [of other records]? – Is actively accessed [on a regular basis] ? – Really has to be in the OLTP engine? • How much of the data currently in your OLTP engine could be off-loaded – If that data remains accessible through SQL (even from within the OLTP engine, without altering existing queries) with reasonable response times • What if such off-loading – Improves performance of the OLTP engine for transactions – Shortens batch jobs [by engaging distributed, scale out processing options] – Opens up possibilities for advanced analytics – Potentially lowers the cost [licenses & specialized hardware] for the OLTP engine – Introduces some change and complexity
- 27. 27 Oracle Big Data SQL • Big Data SQL: A ‘franchised query engine,’ enables scalable, integrated access in situ to the entire Big Data Management System (BDMS) – Meta data, Query execution, Workload Management, Data Optimization – Primary role for Oracle Database – foundation for BDMS See Statement of Direction: http://www.oracle.com/technetwork/database/bigdata-appliance/overview/sod-bdms-2015-04-final-2516729.pdf
- 29. 29 Oracle Forms • Release 12.2.1.1 is available – 12.2.2 (or 12.2.1.2) is planned for late 2016 – Support for Forms 12c: Premier Oct 2020, Extended 2023 – (and moving) • Forms usually runs in browser – using the Java JRE plugin for Applets – Modern browsers have stopped or will stop supporting the Java plugin – Forms will either have to run on outdated browsers (IE, old versions of Firefox or Chrome) or run outside the browser – The main changes around Forms are around running Forms outside the browser – as standalone Java Web Start (jnlp) application – Also: Forms Helper – script for customizing post-install environment (simplified WLST) • On Reports: – Reports 12c exists – it is the terminal release – From now on, reporting should be done using BI Publisher – BI Publisher has become part of the Developer Suite and will be included in the WebLogic Suite
- 30. 30 Forms in the Cloud
- 31. 31 APEX – 5.1 • Interactive Grid – A new rich, client-side region type that allows editing multiple rows of data in a dynamic, JSON-enabled grid, and supports multiple grids on a single page. • Master / Detail / Detail – Provide a wizard interface to define declarative master/detail/detail regions. • New Charting Engine – Include a new JavaScript (Oracle JET) based charting engine developed by Oracle which produces responsive and accessible HTML5 charts. – AnyChart is on the way out • Ability to have multiple tabs open to the same APEX application and isolate session state • Improved Wizards - fewer steps and more attributes set by default. • Declarative RTL Support –declarative methods to control user interface direction- support for both Left-to-Right and Right-To-Left languages. • Packaged Applications – Improved framework and enhancements to the packaged applications. • Status: EA 2 is available (hosted) as of September 2016 – APEX 5.1 Production – early 2017?
- 32. 32 New in SQL in 12cR2 • Listagg improvements • Error handling for CAST function & new Validate_Conversion function • Materialized Views – Real Time Materialized Views (stale plus logs) – Statement Level Refresh • AL32UTF8 As the Default Database Character Set • New capabilities for generating JSON documents directly from SQL queries, improved JSON support in In Memory processing • Beyond 12cR2 • Approximate Query Processing (using HyperLogLog) • Analytical Views • Band Join- better performance for non-equijoins • Temporary, cached in memory tables for duration of cursor • Partitioned External Tables
- 33. 33 New in PL/SQL in 12cR2 • Deprecated procedures and functions • Accessible by at procedure or function level • JSON support: generation of JSON documents using PL/SQL API and Oracle supplied Object Types (somewhat akin to XMLType) – JSON SQL functions available in PL/SQL expressions • Supplied package dbms_plsql_code_coverage to identify code units not touched in specific [test] scenarios • PL/Scope enhancements – more fine grained reporting • Edition Based Redefinition does ‘garbage collection’ – editioned objects no longer in use are cleaned up
- 34. 34 Other Database Development News • JS Stored Procedures • SQL Developer GUI Debugger – One session can have another start debugging – At breakpoint: execute SQL to inspect run context – including PL/SQL state • SQLcl • ORDS – Oracle REST Data Services
- 35. 35
- 36. 36 Summary of Oracle OpenWorld 2016 • (5 days filled to brim with 1800+ sessions, 12 keynotes, 150+ demo booths, hundreds of vendors and quite a few rumors & hallway tales) • Infrastructure [as a Service] – Generation 2 Data Center – Network & IOPS (storage, NVMe, Flash) – Exadata SL • Abdication of the single, central, enterprise Oracle RDBMS – and the limelight for data – PDBs – Sharding – Hadoop & Spark (& SQL & R) – Machine Learning • Adoption of open source projects, industry trends & community darlings – Node.js, Docker, Microservices, Git(Hub), Python, Slack, …
- 37. 37 Summary of Oracle OpenWorld 2016 (2) • Cloud [First] strategy – Migrate & Extend i/o [bidirectional] Lift & Shift – Cloud@Customer – Status and future of On Premises software (and yet Engineered Systems) – Ops in Oracle Data Centers – Subscription Models, Suites (i/o a la carte) – How fast can Oracle move [without spreading itself too thin]? • SaaS and [Unlimited] Applications – SaaS portfolio quite extensive – UX is important asset of the SaaS applications – Real cloud elements are improving (APIs, extensibility) – Traditional Apps are still evolving [as promised] – and seem to benefit from SaaS and technological advances across the board • Oracle Public Cloud consistency, architecture and the Dogfood Doctrine – Fabric and foundational components – Designated capabilities and mutual integration
- 38. 38 Oracle OpenWorld 2016 Tag Cloud
Editor's Notes
- #21: Example of a notebook: https://github.com/rhiever/Data-Analysis-and-Machine-Learning-Projects/blob/master/example-data-science-notebook/Example%20Machine%20Learning%20Notebook.ipynb
- #32: Oracle Application Express 5.1 introduces a new component, the interactive grid, that combines the best of both interactive reports and tabular forms while lifting restrictions such as allowing for only a single tabular form region on a page. The interactive grid is a rich, client-side region type that allows editing multiple rows of data in a dynamic, JavaScript Object Notation-enabled grid that supports multiple grids on a single page. I build highly interactive master detail pages so loved by Oracle Forms developers. Oracle Application Express 5.1 is a huge leap forward in end user productivity. In this session see how the interactive grid brings dynamic, rich-client reporting and multirow editing capabilities to your Oracle Application Express applications and how the new, powerful charting engine (based on Oracle JavaScript Extension Toolkit ) enables users to visualize and interact with their data like never before. The session also takes an in-depth look at numerous other enhancements, such as the new live template options, support for right-to-left languages, and the future of the page designer with the integrated component view. the new data visualization capabilities available with Oracle Application Express 5.1, based on the Oracle JavaScript Extension Toolkit and other components. Oracle JavaScript Extension Toolkit is a JavaScript charting solution that is highly customizable, accessible, interactive, and incorporates automatic responsive design support. See how easily you can integrate these great looking visualizations to tell a story in your application.
- #33: See for example thread on AskTom on SQL Assertions: https://asktom.oracle.com/pls/apex/f?p=100:11:0::::P11_QUESTION_ID:698031000346429496
- #35: 12cR2: http://docs.oracle.com/cloud/latest/exadataexpress-cloud/CSDBP/GUID-F1E74056-FB12-414A-9AAA-579E9540801E.htm#CSDBP-GUID-40C9753E-34CC-403C-8840-581875FC34E1
- #39: https://tagul.com/wuxmhhmc65n4/oow2016-word-cloud