31

I used 'Anglican' which is based on Clojure, and I think that is not good for me. Bad documents and a too small community to find help. Also, I still can't get familiar with the Scheme-based languages. So I want to change the language to something based on Python.

Maybe Pyro or PyMC could be the case, but I totally have no idea about both of those.

  • What are the difference between the two frameworks?
  • Can they be used for the same problems?
  • Are there examples, where one shines in comparison?
sophros
  • 14,672
  • 11
  • 46
  • 75
MinKi Jo
  • 457
  • 1
  • 5
  • 10
  • 6
    I'd vote to keep open: There is nothing on Pyro [AI] so far on SO. It remains an opinion-based question but difference about Pyro and Pymc would be very valuable to have as an answer. – Unapiedra Jan 28 '18 at 11:08
  • In Terms of community and documentation it might help to state that as of today, there are 414 questions on stackoverflow regarding pymc and only 139 for pyro. – mrk Aug 01 '18 at 07:57

2 Answers2

53

(Updated for 2022)

Pyro is built on PyTorch. It has full MCMC, HMC and NUTS support. It has excellent documentation and few if any drawbacks that I'm aware of.

PyMC was built on Theano which is now a largely dead framework, but has been revived by a project called Aesara. PyMC3 is now simply called PyMC, and it still exists and is actively maintained. Its reliance on an obscure tensor library besides PyTorch/Tensorflow likely make it less appealing for widescale adoption--but as I note below, probabilistic programming is not really a widescale thing so this matters much, much less in the context of this question than it would for a deep learning framework.

There still is something called Tensorflow Probability, with the same great documentation we've all come to expect from Tensorflow (yes that's a joke). My personal opinion as a nerd on the internet is that Tensorflow is a beast of a library that was built predicated on the very Googley assumption that it would be both possible and cost-effective to employ multiple full teams to support this code in production, which isn't realistic for most organizations let alone individual researchers.

That said, they're all pretty much the same thing, so try them all, try whatever the guy next to you uses, or just flip a coin. The best library is generally the one you actually use to make working code, not the one that someone on StackOverflow says is the best. As for which one is more popular, probabilistic programming itself is very specialized so you're not going to find a lot of support with anything.

stuart
  • 1,005
  • 1
  • 10
  • 18
  • 1
    I also think this page is still valuable two years later since it was the first google result. Firstly, OpenAI has recently officially adopted PyTorch for all their work, which I think will also push PyRO forward even faster in popular usage. One thing that PyMC3 had and so too will PyMC4 is their super useful forum (https://discourse.pymc.io/) which is very active and responsive. Regard tensorflow probability, it contains all the tools needed to do probabilistic programming, but requires a lot more manual work. – Josh Albert Mar 04 '20 at 12:34
  • 3
    Good disclaimer about Tensorflow there :). I was furiously typing my disagreement about "nice Tensorflow documention" already but stop. – Phúc Lê Jun 16 '20 at 16:52
  • I'm currently using BUGS or Stan, and I wish these Python-based PPLs were a bit more intuitive, but I guess it is what it is. Seems like Pyro might have a bit easier learning curvce. – Evan Zamir Oct 06 '20 at 17:47
  • Same Evan. PyMc3 looked familiar, and the 'hello world' OLS was straightforward. However, the multiprocessing support seems hopelessly broken. – Nathan D Oct 12 '20 at 19:47
  • 7
    Update as of 12/15/2020, PyMC4 has been discontinued. Instead, the PyMC team has taken over maintaining Theano and will continue to develop PyMC3 on a new tailored Theano build. See here for PyMC roadmap: https://github.com/pymc-devs/pymc3/wiki/Timeline – ZAR Jan 03 '21 at 18:21
  • 6
    The latest edit makes it sounds like PYMC in general is dead but that is not the case. As per @ZAR PYMC4 is no longer being pursed but PYMC3 (and a new Theano) are both actively supported and developed. – JJR4 Mar 02 '21 at 19:09
  • 1
    Seconding @JJR4 , PyMC3 has become PyMC and Theano has a been revived as Aesara by the developers of PyMC. So PyMC is still under active development and it's backend is not "completely dead". As the answer stands, it is misleading – Richard Border Mar 14 '22 at 22:29
  • 1
    updated my answer to reflect what JJR4 and Richard Border mentioned – stuart Mar 15 '22 at 23:21
24

From here

Pyro is a deep probabilistic programming language that focuses on variational inference, supports composable inference algorithms. Pyro aims to be more dynamic (by using PyTorch) and universal (allowing recursion).

Pyro embraces deep neural nets and currently focuses on variational inference. Pyro doesn't do Markov chain Monte Carlo (unlike PyMC and Edward) yet.

Pyro is built on pytorch whereas PyMC3 on theano. So you get PyTorch’s dynamic programming and it was recently announced that Theano will not be maintained after an year. However, I found that PyMC has excellent documentation and wonderful resources. Another alternative is Edward built on top of Tensorflow which is more mature and feature rich than pyro atm. Authors of Edward claim it's faster than PyMC3.
I guess the decision boils down to the features, documentation and programming style you are looking for.

Littleone
  • 641
  • 6
  • 14
  • 7
    Does this answer need to be updated now since Pyro now appears to do MCMC sampling? – SARose Jul 03 '18 at 19:44
  • @SARose yes, but it should also be emphasized that Pyro is only in beta and its HMC/NUTS support is considered experimental. – merv Oct 13 '18 at 19:38
  • 3
    PyMC4 will be built on Tensorflow, replacing Theano. – Hatshepsut Feb 24 '19 at 10:00
  • Now NumPyro supports a number of inference algorithms, with a particular focus on MCMC algorithms like Hamiltonian Monte Carlo, including an implementation of the No U-Turn Sampler. Additional MCMC algorithms include MixedHMC (which can accommodate discrete latent variables) as well as HMCECS. https://github.com/pyro-ppl/numpyro – skan Oct 07 '22 at 17:45