Chapter 2. Getting Started with ES-Hadoop

Hadoop provides you with a batch-oriented distributed storage and a computing engine. Elasticsearch is a full-text search engine with rich aggregation capabilities. Getting the data from Hadoop to Elasticsearch can open doors to run some data discovery tools to find out interesting patterns and perform full-text search or geospatial analytics. ES-Hadoop is a library that bridges Hadoop with Elasticsearch. The goal of this book is to get you up-and-running with ES-Hadoop and enable you to solve real-world analytics problems.

Our goal in this chapter is to develop MapReduce jobs to write/read the data to/from Elasticsearch. You probably already know how to write basic MapReduce jobs using Hadoop that writes its output to HDFS. ES-Hadoop is a connector library that provides a dedicated InputFormat and OutputFormat that you can use to read/write data from/to Elasticsearch in Hadoop jobs. To take the first step in this direction, we will start with how to set up Hadoop, Elasticsearch, and the related toolsets, which you will use throughout the rest of the book.

We encourage you to try the examples in the book to speed up the learning process.

We will cover the following topics in this chapter: