OpenSearch: Up and Running in 10 minutes — On local or Amazon EKS

Say Hello to OpenSearch Release Candidate 1

Step by Step guide to run OpenSearch “Release Candidate 1' on local and on Amazon Elastic Kubernetes Service (EKS)

OpenSearch is a community-driven, open source search and analytics suite derived from Apache 2.0 licensed Elasticsearch 7.10.2 & Kibana 7.10.2. It consists of a search engine daemon, OpenSearch, and a visualization and user interface, OpenSearch Dashboards. OpenSearch enables people to easily ingest, secure, search, aggregate, view, and analyze data. These capabilities are popular for use cases such as application search, log analytics, and more.

https://opensearch.org/

Now let’s get started and set up OpenSearch on your local machine.

OpenSearch team publishes all docker containers on docker hub, that makes it super simple to run it anywhere.

Install Docker from“here” and Verify the installation using

Now OpenSearch has two main components. First opensearch itself the engine and second open search dashboards. Just like Elasticsearch and Kibana. To run both together and have them talk to each other we will use docker compose.

To created docker compose file follow this

Create docker compose file, below file will start 1 opensearch container and 1 opensearch-dashboard container

Lets fire it up

Thats it.. if you started web server like million times like me look for this in rolling logs

Lets look at the Dashboards

On your browser open

voilà

Thats it. You have OpenSearch Running on your local machine with OpenSearch Dashboard

Lets Add some Data: Click on Add data.

Lets add Sample Web Logs to simulate log aggregation use case

View Data

For details logs ust like in Kibana, click on discover

And there you go all your logs are here (from Sample Data)

Lets get this going on EKS

To create EKS Cluster I am using eksctl from friends at weaveworks https://eksctl.io/

Download eksctl from

Create Cluster

Once EKS cluster is created, create namespace

Deploy opensearch and opensearch dashboard to EKS

Deploy on Kubernetes

Check status of deployment.

Now lets get the endpoint for Dashboard

Open your browser and go to external IP url:

voilà

Code used in this blog is available at https://github.com/amulyas/opensearch

More Configuration options and details for OpenSearch at https://docs-beta.opensearch.org/

More information on creating EKS Cluster : https://www.eksworkshop.com/

More Information on eksctl: https://eksctl.io/

OpenSearch: https://aws.amazon.com/blogs/opensource/introducing-opensearch/

Cloud, Containers, SaaS, PaaS, Automation, Infrastructure, Dad, Husband, Sillicon Valley