TypeError: expected type_num 12 (NPY_FLOAT64) got 11

[mdekauwe]

When I attempt to apply pym3 to fit two parameters in my own model I run into the following error with expected types:

TypeError: expected type_num 12 (NPY_FLOAT64) got 11

It seems a bit like this issue, but this was quite old and was closed.

#994

I don't have an easily reproducible example because it relies on being able to run my model and all the relevant driving files. I'm happy to share these files but this wouldn't be trivial.

My code is: https://github.com/mdekauwe/CABLE_scripts/blob/master/run_pymc_wrapper.py

The full traceback:

Sampling chain 0, 0 divergences:   0%|                                                                                                                         | 0/510 [00:47<?, ?it/s]
Traceback (most recent call last):
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/theano/compile/function_module.py", line 903, in __call__
    self.fn() if output_subset is None else\
TypeError: expected type_num 12 (NPY_FLOAT64) got 11

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "./run_pymc_wrapper.py", line 148, in <module>
    trace = pm.sample(10, chains=1, step=pm.Slice())
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pymc3/sampling.py", line 489, in sample
    trace = _sample_many(**sample_args)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pymc3/sampling.py", line 539, in _sample_many
    **kwargs
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pymc3/sampling.py", line 605, in _sample
    for it, (strace, diverging) in enumerate(sampling):
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/tqdm/std.py", line 1104, in __iter__
    for obj in iterable:
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pymc3/sampling.py", line 707, in _iter_sample
    point = step.step(point)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pymc3/step_methods/compound.py", line 41, in step
    point = method.step(point)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pymc3/step_methods/arraystep.py", line 142, in step
    apoint = self.astep(bij.map(point), *inputs)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pymc3/step_methods/slicer.py", line 56, in astep
    y = logp(q) - nr.standard_exponential()
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pymc3/blocking.py", line 257, in __call__
    return self.fa(self.fb(x))
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pymc3/model.py", line 1269, in __call__
    return self.f(**state)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/theano/compile/function_module.py", line 917, in __call__
    storage_map=getattr(self.fn, 'storage_map', None))
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/theano/gof/link.py", line 325, in raise_with_op
    reraise(exc_type, exc_value, exc_trace)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/six.py", line 695, in reraise
    raise value.with_traceback(tb)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/theano/compile/function_module.py", line 903, in __call__
    self.fn() if output_subset is None else\
TypeError: expected type_num 12 (NPY_FLOAT64) got 11
Apply node that caused the error: Elemwise{Composite{Switch(i0, (i1 * ((-(i2 * sqr((i3 - i4)))) + i5)), i6)}}[(0, 4)](Elemwise{Composite{Cast{int8}(GT(i0, i1))}}.0, TensorConstant{(1,) of 0.5}, Elemwise{Composite{inv(sqr(i0))}}[(0, 0)].0, TensorConstant{[114.17002...58988953]}, FromFunctionOp{run_and_unpack_cable}.0, Elemwise{Composite{log((i0 * i1))}}.0, TensorConstant{(1,) of -inf})
Toposort index: 11
Inputs types: [TensorType(int8, (True,)), TensorType(float64, (True,)), TensorType(float64, (True,)), TensorType(float64, vector), TensorType(float64, vector), TensorType(float64, (True,)), TensorType(float32, (True,))]
Inputs shapes: [(1,), (1,), (1,), (1461,), (1461,), (1,), (1,)]
Inputs strides: [(1,), (8,), (8,), (8,), (4,), (8,), (4,)]
Inputs values: [array([1], dtype=int8), array([0.5]), array([0.01]), 'not shown', 'not shown', array([-6.44304725]), array([-inf], dtype=float32)]
Outputs clients: [[Sum{acc_dtype=float64}(Elemwise{Composite{Switch(i0, (i1 * ((-(i2 * sqr((i3 - i4)))) + i5)), i6)}}[(0, 4)].0)]]

HINT: Re-running with most Theano optimization disabled could give you a back-trace of when this node was created. This can be done with by setting the Theano flag 'optimizer=fast_compile'. If that does not work, Theano optimizations can be disabled with 'optimizer=None'.
HINT: Use the Theano flag 'exception_verbosity=high' for a debugprint and storage map footprint of this apply node.
HAL 9000 $ cp /Users/mdekauwe/research/CABLE_runs/CABLE_scripts/run_pymc_wrapper.py .
HAL 9000 $ ./run_pymc_wrapper.py


===========
[ 4. 65.]
===========


TumbaFluxnet
 THE NAME LIST IS cable_TumbaFluxnet.nml
 Use spatially-specific soil properties;          360         150
           0
           0
 When choosing spatially-specific soil properties,
 snow-free albedo is also overwritten by this data set.
 Total number of patches (countPatch):            1
 iveg           1           2
 patchfrac           1   1.00000000
  Could neither find restart file ./
  nor ./
  Pre-loaded default initialisations are used.
  Loaded some parameters from met input file: TumbaFluxnet_tmp.nc the rest are default values - check log file

 time-space-averaged energy & water balances
 Ebal_tot[Wm-2], Wbal_tot[mm per timestep]   5.28684723E-06  -1.07890363E-08
 time-space-averaged latent heat and net photosynthesis
 sum_fe[Wm-2], sum_fpn[umol/m2/s]   64.714239402571209       -6.3941715931624318E-005

 NB. Offline-serial runs spinup cycles:           1
 Internal check shows in this version new_sumbal != trunk sumbal
 Writing new_sumbal to the file:new_sumbal
 setting SPINON -> FALSE           0           1
 Finished.    30.8996754      seconds needed for year
 Finished.    30.9007416      seconds needed for        70128  hours
Only 10 samples in chain.
Sequential sampling (1 chains in 1 job)
CompoundStep
>Metropolis: [sigma]
>Metropolis: [vcmax]
>Metropolis: [g1]
Sampling chain 0, 0 divergences:   0%|                                                                                                                         | 0/510 [00:00<?, ?it/s]

===========
[ 4. 65.]
===========


TumbaFluxnet
 THE NAME LIST IS cable_TumbaFluxnet.nml
 Use spatially-specific soil properties;          360         150
           0
           0
 When choosing spatially-specific soil properties,
 snow-free albedo is also overwritten by this data set.
 Total number of patches (countPatch):            1
 iveg           1           2
 patchfrac           1   1.00000000
  Could neither find restart file ./
  nor ./
  Pre-loaded default initialisations are used.
  Loaded some parameters from met input file: TumbaFluxnet_tmp.nc the rest are default values - check log file

 time-space-averaged energy & water balances
 Ebal_tot[Wm-2], Wbal_tot[mm per timestep]   5.28684723E-06  -1.07890363E-08
 time-space-averaged latent heat and net photosynthesis
 sum_fe[Wm-2], sum_fpn[umol/m2/s]   64.714239402571209       -6.3941715931624318E-005

 NB. Offline-serial runs spinup cycles:           1
 Internal check shows in this version new_sumbal != trunk sumbal
 Writing new_sumbal to the file:new_sumbal
 setting SPINON -> FALSE           0           1
 Finished.    31.5556889      seconds needed for year
 Finished.    31.5565720      seconds needed for        70128  hours
Sampling chain 0, 0 divergences:   0%|                                                                                                                         | 0/510 [00:53<?, ?it/s]
Traceback (most recent call last):
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/theano/compile/function_module.py", line 903, in __call__
    self.fn() if output_subset is None else\
TypeError: expected type_num 12 (NPY_FLOAT64) got 11

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "./run_pymc_wrapper.py", line 145, in <module>
    trace = pm.sample(10, chains=1, step=pm.Metropolis())
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pymc3/sampling.py", line 489, in sample
    trace = _sample_many(**sample_args)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pymc3/sampling.py", line 539, in _sample_many
    **kwargs
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pymc3/sampling.py", line 605, in _sample
    for it, (strace, diverging) in enumerate(sampling):
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/tqdm/std.py", line 1104, in __iter__
    for obj in iterable:
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pymc3/sampling.py", line 700, in _iter_sample
    point, stats = step.step(point)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pymc3/step_methods/compound.py", line 27, in step
    point, state = method.step(point)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pymc3/step_methods/arraystep.py", line 175, in step
    apoint, stats = self.astep(self.bij.map(point))
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pymc3/step_methods/metropolis.py", line 162, in astep
    accept = self.delta_logp(q, q0)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/theano/compile/function_module.py", line 917, in __call__
    storage_map=getattr(self.fn, 'storage_map', None))
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/theano/gof/link.py", line 325, in raise_with_op
    reraise(exc_type, exc_value, exc_trace)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/six.py", line 695, in reraise
    raise value.with_traceback(tb)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/theano/compile/function_module.py", line 903, in __call__
    self.fn() if output_subset is None else\
TypeError: expected type_num 12 (NPY_FLOAT64) got 11
Apply node that caused the error: Elemwise{Composite{sqr((i0 - i1))}}[(0, 1)](TensorConstant{[114.17002...58988953]}, FromFunctionOp{run_and_unpack_cable}.0)
Toposort index: 11
Inputs types: [TensorType(float64, vector), TensorType(float64, vector)]
Inputs shapes: [(1461,), (1461,)]
Inputs strides: [(8,), (4,)]
Inputs values: ['not shown', 'not shown']
Outputs clients: [[Elemwise{Composite{Switch(i0, (i1 * ((-(i2 * i3)) + i4)), i5)}}(Elemwise{Composite{Cast{int8}(GT(i0, i1))}}.0, TensorConstant{(1,) of 0.5}, Elemwise{Composite{inv(sqr(i0))}}[(0, 0)].0, Elemwise{Composite{sqr((i0 - i1))}}[(0, 1)].0, Elemwise{Composite{log((i0 * i1))}}.0, TensorConstant{(1,) of -inf}), Elemwise{Composite{Switch(i0, (i1 * ((-(i2 * i3)) + i4)), i5)}}[(0, 3)](Elemwise{Composite{Cast{int8}(GT(i0, i1))}}.0, TensorConstant{(1,) of 0.5}, Elemwise{Composite{inv(sqr(i0))}}[(0, 0)].0, Elemwise{Composite{sqr((i0 - i1))}}[(0, 1)].0, Elemwise{Composite{log((i0 * i1))}}.0, TensorConstant{(1,) of -inf})]]

HINT: Re-running with most Theano optimization disabled could give you a back-trace of when this node was created. This can be done with by setting the Theano flag 'optimizer=fast_compile'. If that does not work, Theano optimizations can be disabled with 'optimizer=None'.
HINT: Use the Theano flag 'exception_verbosity=high' for a debugprint and storage map footprint of this apply node.

Versions and main components

• PyMC3 Version: 3.8
• Theano Version: 1.0.4
• Python Version: 3.7
• Operating system: Mac OSX
• How did you install PyMC3: macports

Thanks,

Martin

[michaelosthege]

Hi Martin,

There's little reference on the numpy type nums, but a trace in this comment indicates that type_num 11 may correspond to float32.

Are you running with floatX=float64 ?

There are some -inf showing up in your trace, so make sure that your data has not only the right floatX type, but also that you're not inputting invalid values somewhere.

Here you put a Uniform(0, 20) prior on a standard deviation. That's guaranted to give you -inf in the limit and will most definitely cause problems in sampling:

https://github.com/mdekauwe/CABLE_scripts/blob/38344ff4b22030edb90c30450a9b2c6bd159ef04/run_pymc_wrapper.py#L130-L137

[mdekauwe]

Hi Michael,

Thanks a lot for those suggestions, I removed the sigma bit, good point.

I wasn't running with python floatX=float64, I assume you meant something like:

$ THEANO_FLAGS='floatX=float64' ./run_pymc_wrapper.py

I found that didn't fix things, but when I explicitly recast everything to float64 as you implied, it runs. So, that is good news.

Thanks again,

Martin

[ColCarroll]

Closing because it sounds fixed -- feel free to reopen!