Elasticsearch python
- 1. ElasticSearch + python Getting started with ElasticSearch Valeria Chemtai Software Developer, Andela @valeriachemtai
- 2. Search… What is search? What is the main objective of search? How search works Technologies involved - web crawlers, inverted index, scoring, search
- 3. Inverted Index Some image from stackoverflow
- 4. Expectations: 1. Understand Elasticsearch and its’ basic concepts 2. Install and setup a node server 3. Index, update and delete documents 4. Incorporate Elasticsearch into a simple python application.
- 5. Prerequisites 1. Command line knowledge 2. Familiarity with RESTful APIs
- 7. Elasticsearch is a highly scalable open-source full- text search and analytics engine. It can be considered as a nosql distributed full text database. Elasticsearch is built on top of Apache Lucene which is free (open sourced)
- 8. Other Technologies similar to Elasticsearch 1. Apache Solr 2. Nutch 3. CrateDB - open source SQL distributed DB
- 9. ● Speed - Operates near real time ● Easy to use with REST API calls ● Scalable ● Robust search - Flexible Query DSL ● Offers statistical analysis tool. ● Many extensions - cloud services, client libraries in many languages Why Use Elasticsearch
- 10. 1. Very poor documentation 2. Not good for use with relational data 3. Only supports JSON data Limitations of Elasticsearch
- 11. Where can I use Elasticsearch Elasticsearch is mainly used for non-relational data: ● Blogs ● Data analytics ● Documents with schemaless structure
- 12. 1. Document - Information to be indexed in JSON form. 2. Type - Logical grouping of documents 3. Index - Collection of types of documents with some similarities. 4. Node - Single server that takes part in indexing 5. Shards - Multiple elements within a node 6. Replicas - Copies of shards 7. Cluster - A collection of nodes Basic Concepts
- 13. SQL Database vs Elasticsearch
- 16. 1. Install Java version 7 and above from: https://www.java.com/en/ 2. Install Elasticsearch version 5 and above: https://www.elastic.co/downloads/elasticsearch 3. Then configure system variables. Installation and Setup
- 17. 1. Check cluster health 2. Create an index 3. Add documents to an index 4. Retrieve documents 5. Update documents 6. Delete documents and an entire index Exploring Elasticsearch with curl
- 19. Create a virtual environment Install elasticsearch in the environment: ● $ pip install elasticsearch Invoke python shell: ● $ python Exploring Elasticsearch with Python
- 20. 1. Create an index 2. Add documents to an index 3. Retrieve documents 4. Update documents 5. Delete documents and an entire index Exploring Elasticsearch with python
- 22. Code Implementation Wrapping it all up into a simple command line app
- 24. More on Elasticsearch Getting started with ElasticSearch-Python :: Part One Getting started with ElasticSearch-Python :: Part Two Elastic Website
Editor's Notes
- #3: Find most relevant documents with our search terms. Search has to know of the documents existence, index the document, know the relevancy of the document, present/retrieve searched document by level of relevancy. Data is tokenized to individual terms extracted (from text to words) and stored in a data structure called the inverted index.
- #4: Inverted index is the heart of every search engine
- #8: Nosql - document based. Has no tables and columns, no schemas or structured query language for searching. Apache Lucene - High performance indexing and search library with full text search engine
- #10: Easy to scale horizontally
- #13: Index is like a database Clustering is what makes elasticsearch easy to scale horizontally
- #14: SQL table with schema Documents in JSON format
- #16: Scaling elasticsearch