Embedding ElasticSearch In a Spring Application

This blog post will show how to have ElasticSearch embedded in a Spring Application.

Motivation

Why Embed ElasticSearch in an application? Simple. To reduce the infrastructure overhead an application needs in other to run, making it more portable.

It is also for the same reason you will want to embed, say a web server, within your application.

It is especially appropriate to have ElasticSearch embedded when the use case does not require the distributed features of ElasticSearch and all that is needed is just a single node.

The Approach

There are two things we are going to use to accomplish the objective of  embedding ElasticSearch into a Spring Application: The ElasticSearch Node and Spring’s FactoryBean interface.

ElasticSearch Node
An ElasticSearch Node is the smallest unit which you can interact with in ElasticSearch. It is the gateway to the receiving, analyzing and storing of the data that is to be retrieved later via searching. When you have ElasticSearch running as a stand alone application, you have a node. (by the way, the combination of one or more nodes, configured with the same cluster name, gives you a cluster).

It is also possible to have this ElasticSearch node created and have it running in the computer’s memory, but within an application, without the need to kickstart it as a standalone process.

FactoryBean Interface
Since Spring, is a framework that is built around the idea of  Inversion of Control, and at it's core, is an Inversion of Control container (often referred to as Context), then having an embedded ElasticSearch in a Spring application is effectively saying we want to have the ElasticSearch Node as a bean that is managed by Spring in its inversion of control container.

For us to achieve this, we use Spring’s FactoryBean interface which, as explained in the post: Difference between BeanFactory and FactoryBean in Spring Framework, allows us to have an object that created by a Factory class to be managed by Spring Framework as a bean.

So in summary, taking the above two facts together, our approach to having the embedded ElasticSearch in a Spring Application will be to have a factory class which can create an Instance of an ElasticSearch node and have the factory class implement FactoryBean interface which then makes it possible to have the ElasticSearch node created managed as a Bean by Spring, so it can be used by other components within the Spring application.

Let see how to get this done.

The Steps

The remainder of the post shows how:

Add ElasticSearch as a Dependency
Even though we are not going to be running ElasticSearch as a standalone application, we still need to have it available one way or the other. This we do by having it as a dependency in our application.

If you working with Maven the following dependency declaration will do exactly that:

<dependency>
   <groupId>org.elasticsearch</groupId>
   <artifactId>elasticsearch</artifactId>
   <version>${elastic.search.version}</version>
</dependency>

If using gradle, then this is what will be needed:

dependencies {
compile "org.elasticsearch:elasticsearch:$elastic.search.version}"
}

Create A FactoryBean For ElasticSearch Node
The code snippet below shows how we can have a FactoryBean that creates an ElasticSearch Node.

import org.elasticsearch.common.settings.ImmutableSettings;
import org.elasticsearch.common.settings.Settings;
import org.elasticsearch.node.Node;
import org.elasticsearch.node.NodeBuilder;
import org.springframework.beans.factory.FactoryBean;
import org.springframework.stereotype.Component;

/**
 * Factory bean that creates an embedded ElasticSearch node
 *
 */
@Component
public class ElasticSearchNodeFactoryBean implements FactoryBean<Node> {

    private Node node;

    @Override
    public Node getObject() throws Exception {
        return getNode();
    }

    @Override
    public Class getObjectType() {
        return Node.class;
    }

    @Override
    public boolean isSingleton() {
        return true;
    }

    private Node getNode() {

        ImmutableSettings.Builder settingsBuilder =
                ImmutableSettings.settingsBuilder();

        settingsBuilder.put("node.name", ElasticSearchConfig.NODE_NAME);
        settingsBuilder.put("path.data", ElasticSearchConfig.DATA_PATH);
        settingsBuilder.put("http.enabled", false);

        Settings settings = settingsBuilder.build();

        node = NodeBuilder.nodeBuilder()
                          .settings(settings)
                          .clusterName(ElasticSearchConfig.CLUSTER_NAME)
                          .data(true).local(true).node();
        return node;
    }
}

The above code implements the FactoryBean interface and it is parameterized with Elasticsearch Node. You can check Customizing instantiation logic with a FactoryBean section in the Spring reference guide for more details on the specifics of the interface.

The most important part for us, in the above code snippet is the private method: getNode() which is where we create an instance of the ElasticSearch node we are interested in.

The most important part is the .local(true)settings of the NodeBuilder.nodeBuilder()as it is the setting that ensures our Node runs embedded. It indicates that our node should live within the Jvm processes it is created in. We also have .data(true) which indicate that our node should be able to hold data, irrespective of the fact it's runs within the JVM or not.

In ImmutableSettings.settingsBuilder() we used the "path.data" to specify the location where our embedded ElasticSearch should store its data, plus we set http.enabled to be false: to disable the ability for our node to respond to external query requests via http; which makes sense since our ElasticSearch node is going to be embedded.

Using the Embedded ElasticSearch Node
With the FactoryBean for creating a bean of ElasticSearch node in place, we can then inject it into other beans and put it to use. For example, as done in the following code snippet:

@Service
public class IndexService {

// Our embedded ElasticSearch Node
private Node node;

// Injecting the embedded ElasticSearch Node
@Autowired
IndexService(Node node) {
this.node = node;
}

public SearchRespons getAll() {

// we retrieve the client from the node
Client client = this.node.client();

// we execute our search query using the client
return client.prepareSearch("index_name")
                   .setQuery(QueryBuilders.matchAllQuery())
                   .execute()
                   .actionGet(); 
}

}

We have the ElasticSearch node injected by Spring into our IndexService and in the getAll method we retrieve the client from the node, with which we perform a query against the ElasticSearch Data.

And this way, we have an embedded ElasticSearch, which we can easily make use of, within a Spring Application.

A popular alternative of achieving the same within a Spring application is to use Spring Data ElasticSearch, a community driven project within the Spring Data project, but this involves having to embrace the way Spring Data abstracts persistence sources: something that may be desirable (as it provides helpers and a consistent data access idiom across the Spring Data Family of projects) , or not (as it introduces another layer between your application and ElasticSearch)...If all that is needed is to have an embedded ElasticSearch and still keep the approach of interacting with ElasticSearch directly, then the FactoryBean method is the way to go.