Multinomial probabilities are not rescaled as tensors

[Armavica]

Description of your problem

Hi, it looks like the Multinomial distribution does not rescale the probabilities (as indicated by the documentation) when they are themselves random variables.

import pymc as pm
import aesara.tensor as at
with pm.Model() as model:
    a = pm.Exponential("a", 1)
    b = pm.Exponential("b", 1)
    x = pm.Multinomial("x", n=10, p=at.stack([a, b])) # Does not work
    # x = pm.Multinomial("x", n=10, p=at.stack([a, b]) / (a+b)) # Works fine
    # x = pm.Multinomial("x", n=10, p=[1, 1]) # Works fine
    pm.sample()

Please provide the full traceback.

Complete error traceback

File ~/.miniconda3/envs/nathan_sequencing/lib/python3.10/site-packages/pymc/sampling.py:558, in sample(draws, step, init, n_init, initvals, trace, chain_idx, chains, cores, tune, progressbar, model, random_seed, discard_tuned_samples, compute_convergence_checks, callback, jitter_max_retries, return_inferencedata, idata_kwargs, mp_ctx, **kwargs)
    556 # One final check that shapes and logps at the starting points are okay.
    557 for ip in initial_points:
--> 558     model.check_start_vals(ip)
    559     _check_start_shape(model, ip)
    561 sample_args = {
    562     "draws": draws,
    563     "step": step,
   (...)
    573     "discard_tuned_samples": discard_tuned_samples,
    574 }

File ~/.miniconda3/envs/nathan_sequencing/lib/python3.10/site-packages/pymc/model.py:1794, in Model.check_start_vals(self, start)
   1791 initial_eval = self.point_logps(point=elem)
   1793 if not all(np.isfinite(v) for v in initial_eval.values()):
-> 1794     raise SamplingError(
   1795         "Initial evaluation of model at starting point failed!\n"
   1796         f"Starting values:\n{elem}\n\n"
   1797         f"Initial evaluation results:\n{initial_eval}"
   1798     )

SamplingError: Initial evaluation of model at starting point failed!
Starting values:
{'a_log__': array(0.), 'b_log__': array(0.), 'x': array([ 0, 10])}

Initial evaluation results:
{'a': -1.0, 'b': -1.0, 'x': -inf}

Versions and main components

• PyMC/PyMC3 Version: 4.1.2
• Aesara/Theano Version: 2.7.5
• Python Version: 3.10.5
• Operating system: linux
• How did you install PyMC/PyMC3: conda

[ricardoV94]

Yes, that's by design. You can rescale them manually.

[Armavica]

Thank you for your answer, however the documentation notes

Elements are expected to sum to 1 along the last axis, and they will be automatically rescaled otherwise.

It doesn't seem clear to me that this only applies to a numerical vector of probabilites but not to random variables. Moreover, the error message isn't really helpful at directing to what is wrong, only mentioning that the initial log-likelihood evaluation failed.

Would you like me to add a line to the documentation of this function to help people understand that this is on purpose? Or try to provide a better error message?

[ricardoV94]

Thanks for your input.

About the error message, this touches on a more widespread question of how to help users understand when they have created invalid models (probabilities not summing to one, negative standard deviations, etc)

We could have a debug mode where the logp raises with the specific parameter error instead of switching to -inf, when given invalid values. The logp graph is actually built with such asserts initially (check pymc.aesaf.compile_pymc, where we include the rewrites that switch the behavior).

About the docs, we should remove that line about the automatic normalization (probably also still exists in the categorical). We only do automatic normalization when the inputs are not symbolic and always with a warning to move users away from it. Feel free to open a PR to change the docs!