Packages for Variational Bayes inference? Turing?

Specific Domains Probabilistic programming
1

What is the most versatile and/or fasest Julia package for doing “Variational Bayes inference”?
I’ve only found two packages: VarBayes.jl and TopicModelsVB.jl, the former seems deprecated.

How can I do Variational Bayes inference with Turing.jl? it’s “universal”, though maybe only useful for MCMC. I can’t find it on its documentation.

Stan can do it (some methods) but I prefer to use a 100% Julia solution.

1 Like
2

Did you find what you were looking for?

3

I’ve also been looking for this and have not found it. As far as I can tell there is no robust package for variational bayesian inference in pure Julia.

4

I asked at Turing.jl and they ( xukai92) replied:

VI is not-supported at the moment. But this is something we will look at in the future.

I guess this we will need to wait for a long time or a new package.

5

At the moment, your best bet is via Stan. I am not sure if Stan.jl can do VI, but it’s mostly an interface thing so it should be easy to incorporate.

6

My understanding is that this can be done in Stan but it’s not a fully developed feature of the language. So it’s possible using the tools but still an experimental application.

1 Like
7

PyMC3 has good VI support:
https://docs.pymc.io/notebooks/variational_api_quickstart.html

If you want to do more elaborate things, Pyro focuses on stochastic VI:
http://pyro.ai/examples/

I’m hoping to get this kind of thing going in Soss.jl, but that will be summer at the earliest.

4 Likes
8

I looked into this a little more and Stan has experimental implementations of both fullrank and meanfield advi accessible through CmdStan.jl.

1 Like
9

While it doesn’t appear to officially released yet, you can try Gen.jl < GitHub - probcomp/Gen.jl: A general-purpose probabilistic programming system with programmable inference > Looks like it’s aiming to be a pretty full-featured and sophisticated probabilistic programming language.

1 Like
11

How do the packages Gen.jl and Turing.jl compare?

12

Here is something related to this discussion: https://github.com/StanJulia/StanVariational.jl

13

FYI: Variational Inference is now supported in Turing.jl. At the moment you will need to use the master branch, but this will change soon.

4 Likes
14

You can find an example of the current ADVI implementation in Turing here: https://github.com/TuringLang/Turing.jl/blob/master/test/variational/advi.jl

Note that there are some open PRs (https://github.com/TuringLang/Turing.jl/pull/903, https://github.com/TuringLang/Turing.jl/pull/902) which will extend/improve the current state but it is already possible to use ADVI for most Turing models.

3 Likes