4

I've seen on other questions that Gremlin & Neptune do not support full text search natively.

How can I provide this feature as part of my web-site?

Ideally it would be not require running up more infrastructure/software that I have to look after.

Thinking that some options are using an external search service like Solr or ElasticSearch. What about another AWS service? Cloudsearch?

thanks

James Render
  • 1,490
  • 1
  • 14
  • 26
  • Just saw this blog post on neptune streams. Could that be used as a way to feed a search engine? https://aws.amazon.com/blogs/database/capture-graph-changes-using-neptune-streams/ – James Render Nov 21 '19 at 14:24

2 Answers2

7

Your question is very timely. Integration between Amazon Neptune and ElasticSearch was just launched [1]. As you add data to a graph, Neptune will automatically keep an ElasticSearch index up to date using the Streams feature. This is similar in approach to what you were considering building but with the added advantage that you can access the index directly from your graph queries rather than needing to write wrapper code that calls the index and then calls Neptune. You can use the ElasticSearch index in your Gremlin and/or SPARQL queries by simply including some "magic" strings in the query that will tell Neptune to use the ElasticSearch index rather than its own internal indices. You have control over things like which ElasticSearch query API a given Gremlin/SPARQL query should use. Instructions on how to setup the environment, which does not take long are included at the same link [1]. Hopefully this will help with your use case. As a side note, another benefit of the feature is that, when working with Gremlin, you do not need a specially modified client library to take advantage of these new capabilities.

[1] https://docs.aws.amazon.com/neptune/latest/userguide/full-text-search.html

Kelvin Lawrence
  • 14,674
  • 2
  • 16
  • 38
0

CloudSearch is good choice but note that "looking after" it will be inevitable. You will at least need an error monitoring/logging mechanism that enables you to see what queries have failed and to track down why. Maybe diacritics handling was not ok, for instance. However, do note that some code for wiring Neptune with CloudSearch will be needed, I do not know of any out of the box method to transfer certain data from Neptune as index to CS. At least a Lambda-function. Lambda functions are worth considering.

Eszter
  • 331
  • 1
  • 3
  • 19
  • agree with the observability requirements. The Neptune streams looks like an interesting way to keep a search engine fresh with changes. We have an Elasticsearch instance running so will try integrating the two. – James Render Nov 22 '19 at 14:01