Are there ways to break symmetries in Turing models?

[Noricc]

Hello,

I am trying to make a gaussian mixture model. In the Turing tutorial, the proposed model is symmetric: You can swap the label of the clusters and still get the same likelihood. The tutorial states:

You’ll note here that it appears the location means are switching between chains. We will address this in future tutorials.

I didn't find any other tutorial that mentioned handling the symmetry. I tried to read the linked blog post about the same topic, but it was a bit steep for my background, so I am not really sure how it helps.

This PyMC3 example seems to do it by setting the likelihood to zero when a certain condition is not satisfied, but I couldn't find a way to do the same thing with Turing. Is it possible to get a distribution of ordered vectors (vectors where x0 < x1, x1 < x2, etc.), or is there a way to reject a sample if a condition is not met? If not, is there a way to break this symmetry in the model?

[devmotion]

I didn't find any other tutorial that mentioned handling the symmetry. I tried to read the linked blog post about the same topic, but it was a bit steep for my background, so I am not really sure how it helps.

This PyMC3 example seems to do it by setting the likelihood to zero when a certain condition is not satisfied, but I couldn't find a way to do the same thing with Turing. Is it possible to get a distribution of ordered vectors (vectors where x0 < x1, x1 < x2, etc.), or is there a way to reject a sample if a condition is not met?

The first link suggests to either specify non-exchangeable priors or to break the symmetry of exchangeable priors by forcing a specific ordering of the parameters of the components. In the first case one tries to "force" the different components to be distinct and to adhere to certain prior beliefs that can be mapped to specific components by, e.g., specifying a prior with positive mean and one with negative mean for two components of a Gaussian mixture model. However, as the presented example shows, the components might still collapse if enough data is provided that outweighs the prior information. The second alternative is to break the symmetry be specifying a prior that is zero if the condition alpha_1 <= alpha_2 <= ... <= alpha_n is not satisfied, where alpha_i are the parameters of the mixture components.

In Turing you can reject samples by defining a model in the following way:

@model function model()
    # some sampling steps before...

    # check the condition
    if any(alpha[i] > alpha[i+1] for i in 1:(length(alpha) - 1))
        # update the joint log probability of the samples
        # we set it to -Inf such that the samples are rejected
        Turing.acclogp!(_varinfo, -Inf)

        # it's not necessary to perform any further computations, so we can just exit here
        return
    end

    # some computations afterwards if the condition is met...
end

I hope that helps.

[Noricc]

Hello again,

Yeah, I thought about using priors to nudge Turing in the right direction, so to speak. I didn't know it was also possible to set the log probability to a constant, thank you for your help!. What does _varinfo mean, though?

[devmotion]

_varinfo is an internal variable that Turing uses to keep track of samples and their corresponding distributions and the accumulated log probability when evaluating the model.

[Noricc]

Ok, thanks. Closing the issue!