Introduction to Elasticsearch
2 likes7,835 views
Elasticsearch is a distributed, RESTful search and analytics engine that allows for fast searching, filtering, and analysis of large volumes of data. It is document-based and stores structured and unstructured data in JSON documents within configurable indices. Documents can be queried using a simple query string syntax or more complex queries using the domain-specific query language. Elasticsearch also supports analytics through aggregations that can perform metrics and bucketing operations on document fields.
Introduction to Elasticsearch
- 1. Introduction to Elasticsearch Jason Austin - @jason_austin
- 2. The Problem • You are building a website to find beers • You have a huge database of beers and breweries to sift through • You want simple keyword-based searching • You also want structured searching, like finding all beers > 7% ABV • You want to run some analytics on what beers are in your dataset
- 3. Enter Elasticsearch • Lucene based • Distributed • Fast • RESTful interface • Document-Based with JSON
- 5. Install Elasticsearch • Download from http://elasticsearch.org • Requires Java to run
- 6. Run Elasticsearch • From the install directory: ./bin/elasticsearch -d! ! http://localhost:9200/!
- 7. Communicating • Elasticsearch listens to RESTful HTTP requests • GET, POST, PUT, DELETE • CURL works just fine
- 9. ES Structure Elasticsearch Indices Types Documents Fields Elasticsearch phpbeer beer Pliny the Elder ABV, Name, Desc
- 10. Create an Index curl -XPOST 'http://localhost:9200/phpbeer'
- 11. What to Search? • Define the types of things to search • Beer • Brewery - Maybe later
- 12. Define a Beer • Name • Style • ABV • Brewery ‣ Name ‣ City
- 13. Beer JSON { ! "name": "Pliny the Elder", ! "style": "Imperial India Pale Ale", ! "abv": 7.0, ! "brewery": { ! ! "name": "Russian River Brewing Co.", ! ! "city": "Santa Rosa", "state": "California" ! } }
- 14. SavingThe Beer curl -XPOST 'http://localhost:9200/ phpbeer/beer/1' -d '{ ! "name": "Pliny the Elder", ! "style": "Imperial India Pale Ale", ! "abv": 7.0, ! "brewery": { ! ! "name": "Russian River Brewing Co.", ! ! "city": "Santa Rosa", "state": "California" ! } }'
- 15. Getting a beer curl -XGET 'http://localhost:9200/phpbeer/ beer/1?pretty'
- 16. Updating a Beer curl -XPOST 'http://localhost:9200/ phpbeer/beer/1' -d '{ ! "name": "Pliny the Elder", ! "style": "Imperial India Pale Ale", ! "abv": 8.0, ! "brewery": { ! ! "name": "Russian River Brewing Co.", ! ! "city": "Santa Rosa", "state": "California" ! } }'
- 17. POSTvs PUT • POST • No ID - Creates new doc, assigns ID • With ID - Updates or creates new doc • PUT • No ID - Error • With ID - Updates doc
- 18. Delete a Beer curl -XDELETE 'http://localhost:9200/ phpbeer/beer/1'
- 19. Finally! Searching! curl -XGET 'http://localhost:9200/_search? pretty&q=pliny'
- 20. Specific Field Searching curl -XGET 'http://localhost:9200/_search? pretty&q=style:pliny'! curl -XGET 'http://localhost:9200/_search? pretty&q=style:imperial'
- 21. Alternate Approach • Search using DSL (Domain Specific Language) • JSON in request body
- 22. DSL Searching curl -XGET 'http://localhost:9200/_search? pretty' -d '{ "query" : { "match" : { "style" : "imperial" } } }'
- 23. DSL = Query + Filter • Query - “How well does the document match” • Filter - Yes or No question on the field
- 24. Query DSL • match • Used to query across all fields for a string • match_phrase • Used to query an exact phrase • match_all • Matches all documents • multi_match • Runs the same match query on multiple fields
- 25. Filter DSL • term • Exact match on a field • range • Match numbers over a specified range • exists / missing • Match based on the existence of a value for a field
- 26. More Complex Search • Find beer whose styles include “Pale Ale” that are less than 7% ABV
- 27. Match + Range curl -XGET 'http://localhost:9200/_search?pretty' -d '{ "query" : { "match" : { "style" : "pale ale" } }, "filter" : { "range" : { "abv" : { "lt" : 7 } } } }'
- 28. Embedded Field Search curl -XGET 'http://localhost:9200/_search?pretty' -d '{ "query" : { "match" : { "brewery.state" : "California" } } }'
- 30. Highlighting Search Results curl -XGET 'http://localhost:9200/_search?pretty' -d '{ "query" : { "match" : { "style" : "pale ale" } }, "highlight": { "fields" : { "style" : {} } } }'
- 31. Aggregations • Collect analytics on your documents • 2 main types • Bucketing • Produce a set of buckets with documents in them • Metric • Compute metrics over a set of documents
- 33. Metric Aggregations • How many beers exist of each style? • What is the average ABV of beers for each style? • How many beers exist that are brewed in California?
- 34. What is the average ABV of beers for each style? curl -XGET 'http://localhost:9200/_search?pretty' -d '{ "aggs" : { "all_beers" : { "terms" : { "field" : "style" }, "aggs" : { "avg_abv" : { "avg" : { "field" : "abv" } } } } } }'
- 35. Mappings • Define how ES searches • Completely optional • Must re-index after defining mapping
- 36. Create Index with Mapping curl -XPOST localhost:9200/phpbeer -d '{ "mappings" : { "beer" : { "_source" : { "enabled" : true }, "properties" : { "style" : { "type" : "string", "index" : "not_analyzed" } } } } }' curl -XDELETE localhost:9200/phpbeer
- 37. What is the average ABV of beers for each style? curl -XGET 'http://localhost:9200/_search?pretty' -d '{ "aggs" : { "all_beers" : { "terms" : { "field" : "style" }, "aggs" : { "avg_abv" : { "avg" : { "field" : "abv" } } } } } }'
- 38. Non-Analyzed Fields curl -XGET 'http://localhost:9200/_search? pretty&q=style:imperial'! curl -XGET 'http://localhost:9200/_search? pretty&q=style:hefeweizen'
- 39. Flexibility • Mixing aggregations, filters and queries all together • What beers have the word “night” in the name that are between 4 and 6 % ABV, broken down by style.
- 40. Elasticsearch and PHP • Elasticsearch PHP Lib https://github.com/elasticsearch/elasticsearch-php • Elastica http://elastica.io/
- 41. Other Awesome ES Features • Search analyzers • Geo-based searching • Elasticsearch Plugins • kopf - http://localhost:9200/_plugin/kopf
- 42. Questions? • @jason_austin • http://www.pintlabs.com • https://joind.in/10821