3

I am playing around with structure from motion (Sfm) and 3D model reconstruction for the first time. The end result is to be able to generate a (fairly accurate) 3D model from a series of images.

Being a JVM developer, my natural inclination was to look for an available Java lib. I found BoofCV which seems to be an active project with decent features & capabilities. But I took a half-day and followed their Sfm examples and wasn't real happy with the results. If you're interested in reproducing them I made a little example project that you can clone and build yourself. But the TL;DR of that is that the code was supposed to render a 3D model of this chair:

enter image description here

...but instead gave me something that only very vaguely resembles a chair:

enter image description here

Very awesome that it can even do this (kudos to the BoofCV team!!!), but still not what I was expecting (then again, being brand new to CV, perhaps this result is completely expected and represents best-in-show Sfm capabilities, I dunno!). I compare these results to the Python Photogrammetry Toolbox ("PyPT") library results of a 3D generated skull, and, well, BoofCV just leaves me wanting a little more...

That PyPT example clearly demonstrates awesomeness, and apparently Google uses PMVS and CMVS for its 3D mapping capabilities. So there are clearly other open source libs out there that can get the job done...and well.

So I ask:

  1. Is the example code in my project (linked above) just not fully leveraging BoofCV's Sfm/reconstruction capabilities?; or
  2. Does BoofCV just not work as well as these other libs? If so, does anybody know offhand what could be contributed to BoofCV (feature/capability-wise) to get it up to par with these other libs?; or
  3. Am I just interpreting these results incorrectly, and perhaps the chair results above are just as accurate as what these other libraries will give me, but I'm just not looking at it the right way?!?
smeeb
  • 27,777
  • 57
  • 250
  • 447
  • Update: it looks like PyPT is just a GUI/usability/convenience wrapper *over* Bundler/PMVS/CMVS. I might try to get those three tools working together and then see if I can whip up some JNI to call them from inside a JVM process. – smeeb Aug 25 '16 at 14:50
  • Or, contrarily, to use Thrift to communicate between a JVM process and PyPT (if possible, probably not, I dunno). – smeeb Aug 25 '16 at 14:51
  • Turns out SFM software just produces **sparse** point clouds, and so the result I posted above is totally expected. The idea is that you take this sparse point cloud (that the SFM component outputted) and use it as input to a Multi-View Stereo (MVS) software, forming a tool pipeline. The MVS software then converts this into a **dense** point cloud which accurately represents the image. Problem is, BoofCV doesn't do MVS (that I can tell). So I found a modern, open source SFM/MVS pipeline called [OpenMVS](https://github.com/cdcseacave/openMVS) and I'll try that out next. – smeeb Aug 31 '16 at 11:08
  • 3
    thank you so much for taking your time to post this. I'm now in your exact same shoes from 3 years ago. How did you manage to solve this? Do you recall? I'm going to try OpenMVS but anything you can offer would be excellent. – Mathomatic Jun 16 '19 at 16:12

0 Answers0