How to implement the product of multiple likelihood functions

Questions
1

In my problem, I have M observations y_i (y_i is a vector of N elements), and a set of parameters \theta_i (a vector of 3 elements) for each observation. There is a deterministic model that link the parameter \theta_i to the i-th observation y_i. My objective is to estimate the parameters \theta_i for every observations.

My model is built as follow:

  • Each set of observation is independent and have an independent likelihood function p(y_i | \theta_i).
  • For each observation i, the parameters \theta_i has a multivariate Gaussian prior with parameter \mu and \Sigma_\mu. \mu and \Sigma_\mu are the same for each observation i
  • \mu and \Sigma_\mu follow some prior p(\mu, \Sigma_\mu)

Hence, the posterior for my model is:
p(\theta_{1:M}, \mu, \Sigma_\mu | y_{1:M}) \propto p(\mu, \Sigma_\mu) \prod_{i=1}^Mp(y_i|\theta_i)p(\theta_i|\mu, \Sigma_\mu)

The likelihood p(y_i | \theta_i) for each observation is ad-hoc, so it is implemented through its logp with DensityDist function.

What do you think would be the best strategy to implement the product of the likelihoods functions:

  1. Implement it with one DensityDist and one big logp function, inside which I will do the sum of \log p(y_i|\theta_i) for i=1..M

or

  1. Should I define a generic function for the logp of each observation i and DensityDist, and then do the sum with the Potential function?

The second solution raises an other question: how to do the sum M likelihood function within the Potential function?

Thanks!

2

There is a third solution: Do like in 2. but don’t sum the in the potential as PyMC3 already does that for you if you define multiple observed variables.

1 Like
3

I think the line model.py:713 is what you’re looking for. It shows how the Model class multiplies all of the terms in your posterior equation. Those include

  • the likelihoods (observed_RVs \subset basic_RVs)
  • the series of terms from however you’ve factored your joint prior (free_RVs \subset basic_RVs)
  • any potentials
4

Thanks @twiecki and @tfg. I will try what you propose and let you know if I managed to make it work!
Best regards

5

@tboutelier So can you tell us which approach you ended up using? It will benefit others reading this question.