Searching and indexing

When creating a clean storage that is based on key-values, we may find that we have lost some of the extra searching capabilities that traditional databases offer. Mainly, we now can't find records within a dataset without knowing the primary key to the entry. However, fear not. Hazelcast provides similar capabilities that allow you to search its maps using predefined indexes. These can be either ordered (ascending) or unordered, depending on our particular data needs. However, keep in mind that indexing doesn't come for free. The internal lookup table that is used to quickly search the reads is maintained as we make changes to the map. This will add latency to every write operation in the namespace.

So firstly, let's create a new plain old Java object (POJO) to represent a city, as follows:

import java.io.Serializable;

public class City implements Serializable, Comparable<City> {
  private String name;
  private String country;
  private int population;

  public City(String name, String country, int population) {
    this.name = name;
    this.country = country;
    this.population = population;
  }

  public String getName() {
    return name;
  }

  public void setName(String name) {
    this.name = name;
  }

  public String getCountry() {
    return country;
  }

  public void setCountry(String country) {
    this.country = country;
  }

  public int getPopulation() {
    return population;
  }

  public void setPopulation(int population) {
    this.population = population;
  }

  @Override
  public int compareTo(City o) {
    return getPopulation() - o.getPopulation();
  }

  @Override
  public boolean equals(Object o) {
    if (this == o) return true;
    if (o == null || getClass() != o.getClass()) return false;
    City other = (City) o;
    if (!this.country.equals(other.country)) return false;
    if (!this.name.equals(other.name)) return false;

    return true;
  }

  @Override
  public int hashCode() {
    int result = name.hashCode();
    result = 31 * result + country.hashCode();
    return result;
  }

  @Override
  public String toString() {
    return String.format(
      "City{name='%s', country='%s', population=%d}",
      name, country, population);
  }
}

As you can see, we created the City class to implement Serializable so that it can be correctly persisted within Hazelcast. We have also implemented the equals() and hashCode() methods so that the required behavior is ensured. Additionally, a toString() method has been added for convenience during the process of debugging.

Using this, we can update the previous map to use the new City, POJO. One major difference between the previous and the current example is that in this example, in order to access the additional indexing functionality, we have to use the specific Hazelcast IMap interface rather than the standard Java Map object that we used before.

Note

Note that because we introduced a new data class, we will need to shut down any Hazelcast instances that may still be running from our earlier examples.

In order to search the map, we need to provide a Predicate object to define our filter. One such implementation of this is that we can use SqlPredicate, which provides us with the ability to use an SQL-like syntax to describe the filter, in the following way:

IMap<String, City> capitals = hz.getMap("capitals");
capitals.addIndex("name", false);
capitals.addIndex("population", true);

capitals.put("GB",
  new City("London", "GB", 8174100));

capitals.put("FR",
  new City("Paris", "FR", 2268265));

capitals.put("US",
  new City("Washington DC", "US", 601723));

capitals.put("AU",
  new City("Canberra", "AU", 354644));

Collection<City> possibleLondons = capitals.values(
  new SqlPredicate("name = 'London'")););

System.err.println(possibleLondons);


Collection<City> largeCities = capitals.values(
  new SqlPredicate("population > 1000000"));

System.err.println(largeCities);

The supported syntax is a limited subset of SQL, but it should feel familiar:

• AND/OR: This is used to combine multiple expressions
• =, !=, <, <=, >, >=: These are used for expression comparison
• LIKE: This is used for simple string pattern matching expressions
• IN: This is used to provide a defined list of the sought values
• BETWEEN: This is used to provide a range of the sought numeric values
• NOT: This can be used as a prefix to negate the expression

The preceding functions are used in the following code:

country = 'GB' AND population BETWEEN 10000 AND 100000

country NOT IN ('GB', 'FR')

name LIKE 'L%'

If you prefer constructing a query more programmatically, you can either use a JPA-like criteria API provided by PredicateBuilder, or manually use various helper methods in Predicates. We can use the following alternative code in place of the previous SQL based predicates:

EntryObject c = new PredicateBuilder().getEntryObject();
Predicate londonPredicate = c.get("name").equal("London");

Collection<City> possibleLondons = capitals.values(londonPredicate);

System.err.println(possibleLondons);

Predicate largeCityPredicate = Predicates.greaterThan("population", 1000000);

Collection<City> largeCities = capitals.values(largeCityPredicate);

System.err.println(largeCities);

Depending on how big the dataset gets, querying for all the records in one go might become unacceptably large and slow to return. In order to provide for consistent query times, we can split the queries into smaller chunks by using the process of pagination. In order to do this, we can use the PagingPredicate class to wrap the underlying search predicate. So, we can extend the previous code in the following way so that it returns each city that was found, one at a time (in this case, a page size of 1):

PagingPredicate pagingPredicate =
  new PagingPredicate(largeCityPredicate, 1);

System.err.println(capitals.values(pagingPredicate));
pagingPredicate.nextPage();
System.err.println(capitals.values(pagingPredicate));

A caveat that you should remember when using PagingPredicate is that it requires a comparator so that it can determine the order in which it can return the results—either by having the object's class implement Comparable (like we have done), or by providing Comparator to the wrapping pager object.