2

I am trying to wrap my head around Bleve and I understand everything that is going on in the tutorials, videos and documentation. I however get very confused when I am using it on BoltDB and don't know how to start.

Say I have an existing BoltDB database called data.db populated with values of struct type Person

type Person struct {
   ID int          `json:"id"`             
   Name string     `json:"name"` 
   Age int         `json:"age"`
   Sex string      `json:"sex"`
}

How do I index this data so that I can do a search? How do I handle the indexing of data that will be stored in the database in the future?

Any help will be highly appreciated.

themihai
  • 7,903
  • 11
  • 39
  • 61
Mardwan
  • 139
  • 3
  • 14
  • Bleve appears to use BoltDB to store its own indexes (rather than building them from an existing Bolt database): https://godoc.org/github.com/blevesearch/bleve/index/store/boltdb – elithrar Jan 24 '16 at 18:44
  • Further: what are you building with BoltDB? It's an excellent KV store for things like auth sessions, tokens, etc but is not suited to ad-hoc queries or full text search. – elithrar Jan 24 '16 at 18:46

2 Answers2

8

Bleve uses BoltDB as one of several backend stores and is separate from where you store your application data. To index your data in Bleve, simply add your Index:

index.Index(person.ID, person)

That index exists separately from your application data (whether it's in Bolt, Postgres, etc).

To retrieve your data, you'll need to construct a search request using bleve.NewSearchRequest(), then call Index.Search(). This will return a SearchResult which includes a Hits field where you can retrieve the ID for your object. You can use this to look up the object in your application data store.

Disclaimer: I am the author of BoltDB.

Ben Johnson
  • 375
  • 2
  • 7
  • Thank you for your reply. I do understand, from the documentation, how data is being indexed and how searches are being done. I however still do not understand how the actual data from BoltDB is being employed here. – Mardwan Jan 26 '16 at 00:22
  • 1
    Bleve uses a BoltDB store for its own internal format of the indexed data. It's better to think of Bleve as a black box. Your BoltDB store doesn't interact with it except through the Bleve API. – Ben Johnson Jan 26 '16 at 12:58
2

How you index your data depends on how you want to query for it.

If you want to query by any arbitrary fields, like {Age:15, Name:"Bob"} then BoltDB isn't an awesome fit for your problem.

BoltDB is simply a key value store with fast access to sequential keys and efficient prefix seeking. It's not really a replacement for general use databases.

You likely want something more like a document store (ie: MongoDB) or RDBMS (ie: PostgreSQL).

If you just wanted something that uses simple files and is embedded, you could also use SQlite with the Go module

If you want to search by only a single field, like ID or Name, then use that as the key.

If lookup speed doesn't matter at all, I guess you can use Bolt to just iterate over the entire db, parse the json and check the fields. But that's probably the worst approach you could take.

David Budworth
  • 11,248
  • 1
  • 36
  • 45
  • I find Bolt works great as an application data store when paired with a search index. For many applications, data retrieval occurs either through primary key or full text search. Key/value stores perform great at primary key search and your FTE index works great for full text search. – Ben Johnson Jan 24 '16 at 20:55
  • Hi. Thank you for you reply. It however doesn't answer my question :( – Mardwan Jan 26 '16 at 00:28
  • Yep, both of y'all are right. I didn't really dig enough in to the topic. – David Budworth Jan 26 '16 at 07:02