An overview of data and web-application development with Python
0 likes55 views
This document provides an overview of Python for data and web application development. It discusses that Python is a widely used general purpose programming language. It then covers common Python applications like web development, data science, and machine learning. It also discusses key Python libraries like Pandas and Numpy for data analysis. Important Python web frameworks like Django are explained. Finally, it briefly discusses data engineering and tools used for tasks like ETL, data warehousing, and analytics.
An overview of data and web-application development with Python
- 2. TechVariable 2 An overview of data and web-application development with PYTHON Sivaranjan Goswami Team Lead, Data Tech
- 3. TechVariable 3 CONTENTS ● Python – what and why? ● Common applications of Python ● Is Python Slow? ● Pandas and Numpy ● What makes Python a great choice for web back-end? ● Web Frameworks in Python – Django ● Web Scrapping ● Data Science and Data Engineering
- 4. TechVariable 4 Python ● Python is a general purpose programming language released by Guido van Rossum, Dutch programmer in 1991. ● Major Timeline: – First release: 0.9.0 in 1991 – Released Python 2.0 in 2000 – Released Python 3.0 in 2008 – Discontinued Python 2 (2.7.18) in 2020 – Latest version: 3.11 (2022-10-24) Most commonly used versions nowadays: Python 3.7, Python 3.8
- 5. TechVariable 5 Some Characteristics of Python: ● Python works on different platforms (Windows, Mac, Linux, Raspberry Pi, etc). ● Python has a simple syntax similar to the English language. ● Python has syntax that allows developers to write programs with fewer lines than some other programming languages. ● Python runs on an interpreter system, meaning that code can be executed as soon as it is written. This means that prototyping can be very quick. ● Dynamically typed language ● Object oriented programming language – everything in Python is an object ● Supports functional programming.
- 6. TechVariable 6 Common applications of Python ● Web applications ● Workflow ● Interact with database ● Implement complex mathematical expressions for data processing ● Image processing and computer vision ● Machine Learning ● Data Science and Engineering
- 8. TechVariable 8 Is Python Slow? ● It depends on how you use Python ● Loops are slow in any interpreted programming language ● Use in-built functions and libraries as much as possible
- 9. TechVariable 9 Pandas and Numpy ● Numpy is a fundamental library in Python for performing scientific computing. ● Numpy provides high-performance multidimensional arrays and tools to deal with them. ● Pandas is built on the numpy library and written in languages like Python, Cython, and C. ● Pandas provide high performance, fast, easy-to-use data structures, and data analysis tools for manipulating numeric data and time series.
- 10. TechVariable 10 What makes Python a great choice for web back-end? ● Modern web applications are much more than just record keeping and CRUD operations. ● There are dashboards that need to aggregate data dynamically to support multiple charts and graphs. ● There are web applications that use image processing, computer vision, machine learning etc. ● Python comes with one of the most popular web development framework – Django.
- 11. TechVariable 11 Python web Framework - Django ● The web framework for perfectionists with deadlines. ● Django is a high-level Python web framework that enables rapid development of secure and maintainable websites. – Complete – Versatile – Secure – Scalable – Maintainable – Portable
- 12. TechVariable 12 Django MVC ● Model: Django ORM that allows you to perform complex database operations with Python Code. Django models have in-built support for filter, sort, pagination etc. ● View: Python class or functions that implements the business logic. ● Template: HTML templates where contents can be manipulated dynamically on server side using Python-Like code.
- 14. TechVariable 14 Django REST Framework ● A Django project except the Templates ● The view returns JSON response that any front-end framework can work with. ● Ideal for projects where front-end and back-end are completely decoupled. ● Enables front-end developers to use libraries or frameworks of their own choice such as React, Angular, React Native, Flutter etc.
- 15. TechVariable 15 Popular Websites powered by Django: ● Google Search ● Youtube ● Instagram ● Spotify ● Dropbox ● NASA ● The Washington Post
- 16. TechVariable 16 Web Scrapping ● Web scraping is the process of collecting and parsing raw data from the Web, and the Python community has come up with some pretty powerful web scraping tools. ● Web scrapping is a popular application of Python. ● Python comes with a variety of tools to scrap websites and extract data form the websites. ● Web scrapping is useful for collecting data for training machine learning models. ● Web scrapping may be used for generating leads for sales and marketing. ● However, scrapping is usually a complex task and every website may require a different approach for scrapping.
- 17. TechVariable 17 Data Engineering ● Acquire datasets that align with business needs ● Develop algorithms to transform data into useful, actionable information ● Build, test, and maintain database pipeline architectures ● Collaborate with management to understand company objectives ● Create new data validation methods and data analysis tools ● Ensure compliance with data governance and security policies
- 18. TechVariable 18 Tools used for Data Engineering ● Data Warehouse – Snowflake, AWS Redshift ● Data Stage – AWS S3 ● Data Transformation – Platforms: AWS EC2, AWS Glue, AWS Lambda, DBT – Tools: Pandas, PySpark, SQL ● Data Analytics and Visualization – Tableau, Sisense