Using python and docker for data science
3 likes2,256 views
The document provides a guide to using Docker for data science, highlighting its advantages such as portability, project isolation, and fast local file access. It includes instructions for setting up and managing Docker containers, creating data-only containers, and building custom Docker images using the IPython SciPy server. Additional resources and links to Docker documentation and related profiles are also provided.
Using python and docker for data science
- 1. USING DOCKER FOR DATA SCIENCE
- 2. WHY DOCKER Portable environment Isolated between projects Stateless Fast local file access Hetrogenous
- 3. GET DOCKER https://docs.docker.com/installation/ boot2docker .dmg or .exe apt-get install docker.io ...
- 4. RUN SCIPYSERVER $ docker run -d -e "PASSWORD=YourPassword?" ipython/scipyserver $ docker run -d -e "PASSWORD=YourPassword?" --name dev_notebook -p 443:8888 ipython/scipyserver https://localhost:443 https://{boot2docker ip}:443
- 5. CREATE DATA-ONLY CONTAINERS $ docker run -d -v ~/notebooks:/notebooks --name notebooks_container ubuntu echo notebooks $ docker run -d -v ~/data:/data --name data_container ubuntu echo data
- 6. MOUNT DATA-ONLY CONTAINERS $ docker stop dev_notebook $ docker rm dev_notebook $ docker run -d -e "PASSWORD=YourPassword?" --name dev_notebook -p 443:8888 --volumes-from data_container --volumes-from notebooks_container ipython/scipyserver
- 7. CREATE A DOCKERFILE FROM ipython/scipyserver MAINTAINER Calvin Giles <[email protected]> COPY requirements.txt /requirements.txt RUN pip2 install -r /requirements.txt RUN pip3 install -r /requirements.txt $ docker build -t calvingiles/ds-notebook . $ docker run -d -e "PASSWORD=YourPassword?" --name dev_notebook -p 443:8888 --volumes-from data_container --volumes-from notebooks_container calvingiles/ds-notebook
- 8. NEXT TIME Connecting to local database containers Tweaking the boot2docker vm memory from 2GB to 8 (or more...) Linking to private git repositories Automated builds with github linking
- 9. MORE? Docker User Guide: http://docs.docker.com/userguide/ Docker Reference: http://docs.docker.com/reference/commandline/cli/ ipython docker images: https://registry.hub.docker.com/repos/ipython/ my docker image: https://github.com/calvingiles/ds-notebook https://registry.hub.docker.com/u/calvingiles/ds-notebook/
- 10. ABOUT ME Calvin Giles Data Scientist at Adthena PyData Meetup Organiser untangleconsulting.io [email protected] @calvingiles on twitter, github, docker hub (and many more)