Support for Elasticsearch

[ZenitK]

Hello Contributors

I wanted to ask about your opinion about the following options for supporting Elasticsearch:

1. Extend this project to support ES
2. Make a fork of this project for ES
3. Make a completely separate project (possibly with a different stack)

What would make the most sense in terms of maintainability and effort needed?

[pbartusch]

hey @ZenitK ,

thanks a lot for bringing this up. there exists a discussion in the community (incl @renekrie) to provide ES support in SMUI. conceptually , this should be possible without forking / separating the SMUI development branch here.

as of now , there does not exist a concrete project / product demand for SMUI @ ES (different than for querqy itself - which already supports ES).
where do you see the need?

cheers,
Paul

[renekrie]

Hi @ZenitK,

Thank you for bringing this up. @pbartusch and I have had this discussion for a while and I think the overall opinion is your solution 1 - 'Extend this project to support ES'

The are only two places that need to be adjusted:

1. 'raw queries' - that is queries in the native syntax of the search engine (which are used for field specific rules)
2. rule deployment.

The rule deployment is more or less about changing a batch script (though we have ideas to change the Querqy rule deployment in Solr to make it more similar to the ES version = using an HTTP request to push the rules - I'm currently working on it)

'raw queries' might need a bit more work. There is an idea to use 'rule templates' - basically a piece of 'text' like in:

def my_business_rule(term):
  UP(100): * category:$term
  UP(100): * date:[NOW-10DAYS TO *]

= boost the term in the category field and boost everything which has date within the last 10 days.

ES would use a different syntax for the native UP queries but in SMUI we would only have to expose 'my_business_rule' and give the user the chance to apply it to 'term'. This could also be a way to generalise the field-specific raw queries like the are used for field queries in SMUI at the moment. On the other hand, we might not want to wait for this feature in Querqy and just find a model in SMUI to maintain 'raw queries' for ES.

[ZenitK]

That was fast, thank you both!

@pbartusch we're starting an initiative in the coming weeks to support search management with querqy and we are using Elasticsearch. We'll definitely need a UI, so we're weighing out different options about tackling it.

Based on @renekrie's response it seems worthwhile for us to dig deeper this option instead of building our own basic UI.

I like the idea of being able to write the rules in a search engine agnostic way, but I guess that might limit the set of possible queries to the defined generic queries, which might not be that bad actually.

[renekrie]

I'd definitely recommend trying to use SMUI over writing your own tool. SMUI feature development has recently gained some momentum due to several companies contributing to the development. It would be hard to keep abreast and more expensive if you developed this as a single company.

[epugh]

Had a nice chat with a client, and SMUI + Querqy on ES would be great for them.

[pbartusch]

exciting , well DM you @epugh

[goodniceweb]

Hi @pbartusch

Thank you for taking care of this! How is it coming?

[pbartusch]

Hey @goodniceweb , the support for ES is not fully implemented yet. But if you would like to try it out , there is may be a way to use it with some limitations with a custom deployment mechanism.

[dobestler]

@goodniceweb There is a way SMUI can be used with Elasticsearch. SMUI just needs to be configured as mentioned here: #56 (comment)

Then add a custom deployment script to upload the rules to Querqy@ES

toggle.rule-deployment.custom-script=true 
toggle.rule-deployment.custom-script-SMUI2SOLR-SH_PATH="/path/to/script.sh"

and you should be ready to go.

[luisherrera13]

@goodniceweb There is a way SMUI can be used with Elasticsearch. SMUI just needs to be configured as mentioned here: #56 (comment)

Then add a custom deployment script to upload the rules to Querqy@ES

toggle.rule-deployment.custom-script=true 
toggle.rule-deployment.custom-script-SMUI2SOLR-SH_PATH="/path/to/script.sh"

and you should be ready to go.

Hello i'm trying to "connect" SMUI to remote Elastic Search instance do you think this aproach would apply? if yes then how would be an example of script.sh and the path it should have or how to do this?

[renekrie]

Hi @luisherrera13,

Have you noticed Chorus for Elasticsearch https://github.com/querqy/chorus-elasticsearch-edition? - It has an example integration of SMUI with ES (example scripts: https://github.com/querqy/chorus-elasticsearch-edition/tree/main/smui/conf)