de.picturesafe.search:picturesafe-search

= picturesafe-search

== An Elasticsearch service wrapper

picturesafe-search is a Java service wrapper for the search engine Elasticsearch.

It allows the fast, flexible and easy integration of Elasticsearch functions into new or existing Java applications.

The following features are included:

• Functions for creating and maintaining Elasticsearch indices ** Create field definitions ** Create and delete Elasticsearch indices ** Add and remove documents
• Query API for creating simple and complex (nested) search queries. ** Fulltext queries ** Field queries ** Complex queries
• Filter builder API for easy implementation of customized search filters
• Aggregation builder API for easy implementation of customized aggregation builders (facets)

== Make Elasticsearch queries easy

Elasticsearch is a very powerful search engine, but has a high complexity and requires a long learning curve.

The following example compares the execution of the logical search expression fulltext contains "(test && title)" and count >= 102 sort by id between the Elasticsearch REST API, the Elasticsearch Java High Level REST Client and the picturesafe-search Java service wrapper:

.Elasticsearch REST API [source]

POST /picturesafe-search-sample/_search

[source,json]

{ "query": { "bool": { "must": [{ "bool": { "filter": [{ "query_string": { "query": "(test && title)", "fields": ["fulltext^1.0"] } }] } }], "filter": [{ "range": { "count": { "from": 102, "to": null } } }] } }, "sort": [{ "id.keyword": { "order": "desc", "missing": "_last" } }] }

.Elasticsearch Java High Level REST Client [source,java]

SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder() .query(QueryBuilders.boolQuery() .must(QueryBuilders.queryStringQuery("test && title") .field("fulltext") .defaultOperator(Operator.AND)) .filter(QueryBuilders.rangeQuery("count").gte(102))) .sort(SortBuilders .fieldSort("id.keyword") .missing("_last") .order(SortOrder.DESC));

SearchRequest searchRequest = new SearchRequest("picturesafe-search-sample"); searchRequest.source(searchSourceBuilder); SearchResponse searchResponse = new RestClientSearchAction() .action(restHighLevelClient, searchRequest);

.picturesafe-search Java service wrapper [source,java]

SearchResult searchResult = elasticsearchService.search("picturesafe-search-sample", OperationExpression.and( new FulltextExpression("test title"), new ValueExpression("count", ValueExpression.Comparison.GE, 102)), SearchParameter.builder().sortOptions(SortOption.desc("id")).build());

As you can see, picturesafe-search focuses more on WHAT to search for than on HOW to search it and abstracts some complexity.

== Getting started

Only a few steps are necessary to get started with a standard configuration. A complete code example can be found https://github.com/picturesafe/picturesafe-search-samples[here]. If you want to create a Spring Boot application, you can alternatively use the https://github.com/picturesafe/picturesafe-search-starter[picturesafe-search Starter].

For more information, please see following https://picturesafe-search.io/docs/[documentation].

=== Start Elasticsearch

picturesafe-search requires a running Elasticsearch server from version 7.x.

==== Local installation of Elasticsearch

For a new application an Elasticsearch server must be installed first:

• https://www.elastic.co/downloads/elasticsearch[Download] and unpack the Elasticsearch official distribution.
• Run bin/elasticsearch on Linux or macOS. Run bin\elasticsearch.bat on Windows.

Some features of picturesafe-search (for example sorting documents in a language-specific word order) require the https://www.elastic.co/guide/en/elasticsearch/plugins/current/analysis-icu.html[ICU Analysis Plugin] for Elasticsearch:

• Run bin/elasticsearch-plugin install analysis-icu on Linux or macOS. Run bin\elasticsearch-plugin install analysis-icu on Windows.

==== Run Elasticsearch in a Docker container

As an alternative to installing Elasticsearch you can run it in a Docker container. To do this you can use the provided Docker Compose file.

• Install https://docs.docker.com/install/[Docker] and https://docs.docker.com/compose/install/[Docker Compose].
• Clone the picturesafe-search GitHub repository.
• Run docker-compose -f src/test/docker/docker-compose.yml up -d from the project directory to start Elasticsearch.
• To stop Elasticsearch run docker-compose -f src/test/docker/docker-compose.yml stop from the project directory.

=== Include java library

Add the current version of the picturesafe-search library to your project.

.Maven dependency [source,xml]

de.picturesafe.search picturesafe-search 3.5.0 ----

=== Configuration

==== Configuration bean

Implement a configuration class that imports the DefaultElasticConfiguration.class. This configuration can be extended later.

The following example defines three fields for the Elasticsearch index:

• Field 'id' (Elasticsearch type integer, sortable)
• Field 'fulltext' (Elasticsearch type text)
• Field 'title' (Elasticsearch type text, within fulltext, aggregatable, sortable)

.Spring configuration [source,java]

@Configuration @ComponentScan(basePackages = {"de.picturesafe.search.elasticsearch"}) @Import({DefaultElasticConfiguration.class}) public class Config {

@Bean
List<FieldConfiguration> fieldConfigurations() {
    return Arrays.asList(
        FieldConfiguration.ID_FIELD,
        FieldConfiguration.FULLTEXT_FIELD,
        StandardFieldConfiguration.builder(
                "title", ElasticsearchType.TEXT).copyToFulltext(true).sortable(true).build(),
        StandardFieldConfiguration.builder(
                "count", ElasticsearchType.INTEGER).sortable(true).build()
    );
}

}

==== Configuration properties

Add a file elasticsearch.properties to the classpath of your application and define the following key:

.Property file [source]

elasticsearch.index.alias=picturesafe-search-sample

This configuration can be extended later, see src/main/resources/elasticsearch.template.properties.

==== Service implementation

Inject the SingleIndexElasticsearchService and implement an expression-based search:

• Create an Elasticsearch index with alias
• Add some documents to the index
• Create an OperationExpression with two terms
• Run the search query
• Delete the Elasticsearch index

If you want to implement searches for more than one index, please use ElasticsearchService instead of SingleIndexElasticsearchService.

.Spring service implementation [source,java]

@Component @ComponentScan public class GettingStarted {

private static final Logger LOGGER = LoggerFactory.getLogger(GettingStarted.class);

@Autowired
private SingleIndexElasticsearchService singleIndexElasticsearchService;

public static void main(String[] args) {
    try (AnnotationConfigApplicationContext ctx
            = new AnnotationConfigApplicationContext(GettingStarted.class)) {
        final GettingStarted gettingStarted = ctx.getBean(GettingStarted.class);
        gettingStarted.run();
    }
}

private void run() {
    try {
        singleIndexElasticsearchService.createIndexWithAlias();

        singleIndexElasticsearchService
                .addToIndex(DataChangeProcessingMode.BLOCKING, Arrays.asList(
                DocumentBuilder.id(1).put("title", "This is a test title")
                    .put("count", 101).build(),
                DocumentBuilder.id(2).put("title", "This is another test title")
                    .put("count", 102).build(),
                DocumentBuilder.id(3).put("title", "This is one more test title")
                    .put("count", 103).build()
        ));

        final Expression expression = OperationExpression.and(
                new FulltextExpression("test title"),
                new ValueExpression("count", ValueExpression.Comparison.GE, 102));

        final SearchResult searchResult = singleIndexElasticsearchService
            .search(expression, SearchParameter.DEFAULT);

        LOGGER.info(searchResult.toString());
    } finally {
        singleIndexElasticsearchService.deleteIndexWithAlias();
    }
}

}

With implementations of the picturesafe-search Expression-Interface complex terms of different search conditions can be easily defined.

Here are some examples:

.Simple fulltext search [source,java]

Expression expression = new FulltextExpression("test title");

.Simple field search [source,java]

Expression expression = new ValueExpression("title", "test");

.Simple field search with comparison operator [source,java]

Expression expression = new ValueExpression("count", ValueExpression.Comparison.GE, 102);

.Search with two terms [source,java]

Expression expression = OperationExpression.and( new FulltextExpression("test title"), new ValueExpression("count", ValueExpression.Comparison.GE, 102));

In addition there are further expressions like InExpression, MustNotExpression, RangeValueExpression, DayExpression, https://picturesafe-search.io/docs/reference/expressions/[more]

== Building picturesafe-search

If you want to build picturesafe-search yourself there are two prerequisites:

=== JDK

You need to have installed a Java Development Kit. The picturesafe-search project is currently developed using Java 8, but has also been tested on Java 11.

Note when using Java 11: + There is a JavaDoc related https://bugs.openjdk.java.net/browse/JDK-8212233[bug] which has not been fixed in Adopt or Corretto OpenJDK at the moment. If you are using OpenJDK 11 and you are facing a build error like

Failed to execute goal org.apache.maven.plugins:maven-javadoc-plugin:3.2.0:jar (attach-javadocs) on project picturesafe-search: MavenReportException: Error while generating Javadoc:

[ERROR] Exit code: 1 - javadoc: error - The code being documented uses packages in the unnamed module, but the packages defined in https://docs.oracle.com/en/java/javase/11/docs/api/ are in named modules.

, please skip generating JavaDoc until the fix has become part of the OpenJDK build you are using.

.Skipping the JavaDoc generation:

mvn -Dmaven.javadoc.skip=true install

Alternatively you could use the OpenJDK 11 reference build provided by https://jdk.java.net/java-se-ri/11[Oracle], which has the fix included.

Side note on java modules: + We are not able to provide a module-info.java at the moment, because we are using the Elasticsearch high level rest client which has the monolithic elasticsearch.jar as dependency. The elasticsearch.jar has no module-info and it makes auto module detection impossible because of its internal structure. Please see this https://github.com/elastic/elasticsearch/issues/38299[issue] for details.

=== Apache Maven

You also need to have installed https://maven.apache.org/[Apache Maven] version 3.6.

=== Build

Change to the project directory and run the following command in your shell:

[source,bash]

mvn install