4

I've just discovered this promising storage engine, which is Xodus.

Which are the use cases where Virtual File System mode should be preferred over a traditional File System?

I've also seen the ExodusDirectory lucene implementation, how can it be better than the reference FSDirectory?

riccardo.tasso
  • 978
  • 2
  • 10
  • 28

1 Answers1

6

Xodus VFS is a simple transactional file system implemented over Xodus Environment API. Traditional file system is not transactional, it doesn't allow to work with snapshots of data with consistency requirements defined by an application.

Xodus VFS was implemented directly for storing Lucene indices used by text search in JetBrains YouTrack. Xodus is a primary storage engine in YouTrack. Apart from snapshot isolation, it has configurable shared cache consuming constant resources (RAM, file handles, etc.) regardless of the number of opened environments (database instances) in single JVM. In YouTrack, Lucene is configured to consume as little resources as possible delegating all caching and performance tuning to Xodus. Xodus has out-of-the-box stream ciphering, so if XodusDirectory is used one can easily encrypt Lucene indices to satisfy GDPR or any other similar regulation. Finally, Xodus has probably more facilities to recover (compared to default Lucene backend) after a system/hardware failure without index corruption and the necessity to rebuild it, as its log-structured architecture allows to rollback to a recent valid snapshot. Probably - since there are no strong evidences of that.

Vyacheslav Lukianov
  • 1,913
  • 8
  • 12
  • `Lucene is configured to consume as little resources as possible delegating all caching and performance tuning to Xodus.` could you be able to share how it is done? Or otherwise any pointers on where to look at. Or, even better, any best practices and lessons learned from running youtrack regarding `ExodusDirectory` would be great. – Vin Jan 01 '21 at 11:04