Skip to content

COMPAT: df.to_dict('index') is not preserving dtypes #18580

Closed
@Anton-vBR

Description

@Anton-vBR

Code Sample

import pandas as pd
from pandas.core.common import standardize_mapping

df = pd.DataFrame(dict(ints=range(5),floats=[float(i) for i in range(5)]))

into_c = standardize_mapping(dict)

print('Using iterrows:')
print(into_c((k, v.to_dict(dict)) for k, v in df.iterrows()))

Problem description

The problem appears when trying to export dataframe to dict using 'index' as param and when that dataframe conatains both float and int. Expected behaviour would be to separate ints and floats (as shown below). iAs recommended by the docs, we could use itertuples instead. I have included a code-snippet that could work. (Disclaimer: Sorry, this might be a pull request but I thought it could fit here since I'm not feeling familiar with the whole process)

Expected Output

{0: {'ints': 0, 'floats': 0.0}, 1: {'ints': 1, 'floats': 1.0}, 2: {'ints': 2, 'floats': 2.0}, 3: {'ints': 3, 'floats': 3.0}, 4: {'ints': 4, 'floats': 4.0}}

print('\nUsing itertuples:')
print(into_c((t[0],dict(zip(df.columns,t[1:]))) for t in df.itertuples()))

Output of pd.show_versions()

INSTALLED VERSIONS

commit: None
python: 3.5.2.final.0
python-bits: 64
OS: Darwin
OS-release: 14.5.0
machine: x86_64
processor: i386
byteorder: little
LC_ALL: en_US.UTF-8
LANG: en_US.UTF-8
LOCALE: en_US.UTF-8

pandas: 0.21.0
pytest: 2.9.2
pip: 9.0.1
setuptools: 27.2.0
Cython: 0.24.1
numpy: 1.13.1
scipy: 0.18.1
pyarrow: None
xarray: None
IPython: 5.1.0
sphinx: 1.4.6
patsy: 0.4.1
dateutil: 2.5.3
pytz: 2016.6.1
blosc: None
bottleneck: 1.1.0
tables: 3.2.3.1
numexpr: 2.6.1
feather: None
matplotlib: 1.5.3
openpyxl: 2.3.2
xlrd: 1.0.0
xlwt: 1.1.2
xlsxwriter: 0.9.3
lxml: 3.6.4
bs4: 4.5.1
html5lib: None
sqlalchemy: 1.0.13
pymysql: None
psycopg2: None
jinja2: 2.8
s3fs: None
fastparquet: None
pandas_gbq: None
pandas_datareader: None

Metadata

Metadata

Assignees

No one assigned

    Labels

    Compatpandas objects compatability with Numpy or Python functionsDtype ConversionsUnexpected or buggy dtype conversionsReshapingConcat, Merge/Join, Stack/Unstack, Explode

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions