Talend Open Studio Data Integration
12 likes13,918 views
Talend provides data integration and management solutions. It focuses on combining data from different sources into a unified view for users. Talend offers an open source tool called Talend Open Studio that allows users to visually design procedures to extract, transform, and load data between various databases and file types. It also offers features for data quality, storage optimization, master data management, and reporting.
Talend Open Studio Data Integration
- 1. Talend Data Integration and Management
- 2. Data Integration Data Integration involves combining data residing in differente sources and providing the user with a unified view of the data Data Management combines different disciplines to manage data as a valuable resource www.robertomarchetto.com
- 3. Talend ● Talend is a company focused on Data Integration and Data Management solutions ● Talend is a „Cool Vendor“ for Gartner (2010) ● Present in more than 12 locations around the World ● Fast growing company www.robertomarchetto.com
- 4. Talend Open Studio www.robertomarchetto.com
- 5. Talend Open Studio ● Open Source, professional tool ● Draw procedures linking components, each component performs an operation ● DB vendor-specific optimized components ● Produces fully editable Java (or Perl) code ● Deployment with small and fast compiled Java or as Web Service ● Eclipse based IDE, excellent flexibility ● BI Platform indipendent, DB Vendor indipendent www.robertomarchetto.com
- 6. Automatic code generation, diffent deployment www.robertomarchetto.com
- 7. Extracion Transformation Loading ● ETL is a common process in Data Integration ● Extract, reading data from different datasources (database, flat files, spreadsheet files, web services, etc) ● Transfom, converting data in a form so that it can be placed in another container (database, web services, files, etc). Cleaning, computations and verifications are also performed ● Load, write the data in the target format www.robertomarchetto.com
- 8. Tutorial, Source data www.robertomarchetto.com
- 9. Tutorial, Destination data (Datawarehouse) www.robertomarchetto.com
- 10. Tutorial, Metadata ● Talend requires a preliminary definition of the metadata ● Often a strong metadata definition means, as in programming languages, fast, robust and maintenable applications ● ..demo.. www.robertomarchetto.com
- 11. Tutorial, Talend jobs basics ● Place components on the designer ● Link components to build a transformation ● Main type of link: Rows flow ● Schema metadata is propagated and must be coherent ● ..demo.. www.robertomarchetto.com
- 12. Tutorial, users_dimension www.robertomarchetto.com
- 13. Test the job www.robertomarchetto.com
- 14. Tutorial, accounts_dimension www.robertomarchetto.com
- 15. Tutorial, dates_dimension www.robertomarchetto.com
- 16. Tutorial, write a Java library www.robertomarchetto.com
- 17. Tutorial, opportunities_fact www.robertomarchetto.com
- 18. Tutorial, define a root job www.robertomarchetto.com
- 19. Deploy and run www.robertomarchetto.com
- 20. Extensibility, comunity plugins ● Many official components ● Components for every task released by the comunity ● Geospatial components, log analysis, Google analytics, data encryption, etc www.robertomarchetto.com
- 21. Scheduler www.robertomarchetto.com
- 22. And now.. reports, dashboards, OLAP, Geoanalysis, KPIs.. www.robertomarchetto.com
- 23. Do you trust your data? www.robertomarchetto.com
- 24. What about data quality? ● Customer A is present 5 times with different names ● Null values can vary statistical indexes like mean calculation ● Duplicated records ● Blank values ● Some records can contain errors (es -1 field values) ● Some records can be garbage www.robertomarchetto.com
- 25. Talend Open Profiler www.robertomarchetto.com
- 26. What abount data storage size? ● Some fields can be oversized for the data they contain ● Sometimes fields are related and can be calculated ● Some keys or values are never used ● When data grow garbage grow ● Data storage is not free (disks, electricity, backups, DB licenses) www.robertomarchetto.com
- 27. Data is „the black gold“ that can produce knowledge ● Data is a resource, you can extract knowledge ● A lot of Data produces concise informations ● Data storage is not free and a lot of data can make system not fast ● Data cleansing is a central process in statistical analysis and Data Mining www.robertomarchetto.com
- 28. Talend Master Data Management www.robertomarchetto.com