-
Notifications
You must be signed in to change notification settings - Fork 8
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Pandas/pyArrow/read_parquet error #17
Comments
Hi @sophiamyang, I am facing the same issue here. Cheers, |
Hi @sophiamyang, The error seems to be persistent. I now get the following:
Is there anything we can do? Cheers, |
Hi, Does anyone know of a workaround to this bug ? Or an alternative to condastats ? Cheers, |
Thanks for the issue. Could you open the issue in this repo instead? condastats depends on the data pipeline in this repo: https://github.com/ContinuumIO/anaconda-package-data cc @cappadona |
Hi @sophiamyang and thanks a lot for this package and this answer. But, as I only experiment this error when using condastats, I don't see how to reproducibly describe the bug in this repo ... |
Hey Pierre-Henri, I respond to your message posted in #18 here to keep the threads separated and focused. Quick inspection of my environment (see below) indicates that the main differences are in the Environment# Name Version Build Channel
_libgcc_mutex 0.1 conda_forge conda-forge
_openmp_mutex 4.5 2_gnu conda-forge
aiobotocore 2.5.0 pyhd8ed1ab_0 conda-forge
aiohttp 3.8.5 py311h459d7ec_0 conda-forge
aioitertools 0.11.0 pyhd8ed1ab_0 conda-forge
aiosignal 1.3.1 pyhd8ed1ab_0 conda-forge
alsa-lib 1.2.9 hd590300_0 conda-forge
anyio 3.7.1 pyhd8ed1ab_0 conda-forge
asttokens 2.2.1 pyhd8ed1ab_0 conda-forge
async-timeout 4.0.2 pyhd8ed1ab_0 conda-forge
attr 2.5.1 h166bdaf_1 conda-forge
attrs 23.1.0 pyh71513ae_1 conda-forge
aws-c-auth 0.7.0 hf8751d9_2 conda-forge
aws-c-cal 0.6.0 h93469e0_0 conda-forge
aws-c-common 0.8.23 hd590300_0 conda-forge
aws-c-compression 0.2.17 h862ab75_1 conda-forge
aws-c-event-stream 0.3.1 h9599702_1 conda-forge
aws-c-http 0.7.11 hbe98c3e_0 conda-forge
aws-c-io 0.13.28 h3870b5a_0 conda-forge
aws-c-mqtt 0.8.14 h2e270ba_2 conda-forge
aws-c-s3 0.3.13 heb0bb06_2 conda-forge
aws-c-sdkutils 0.1.11 h862ab75_1 conda-forge
aws-checksums 0.1.16 h862ab75_1 conda-forge
aws-crt-cpp 0.20.3 he9c0e7f_4 conda-forge
aws-sdk-cpp 1.10.57 hbc2ea52_17 conda-forge
backcall 0.2.0 pyh9f0ad1d_0 conda-forge
backports 1.0 pyhd8ed1ab_3 conda-forge
backports.functools_lru_cache 1.6.5 pyhd8ed1ab_0 conda-forge
bokeh 3.2.1 pyhd8ed1ab_0 conda-forge
botocore 1.29.76 pyhd8ed1ab_0 conda-forge
brotli 1.0.9 h166bdaf_9 conda-forge
brotli-bin 1.0.9 h166bdaf_9 conda-forge
brotlipy 0.7.0 py311hd4cff14_1005 conda-forge
bzip2 1.0.8 h7f98852_4 conda-forge
c-ares 1.19.1 hd590300_0 conda-forge
ca-certificates 2023.7.22 hbcca054_0 conda-forge
cairo 1.16.0 hbbf8b49_1016 conda-forge
certifi 2023.7.22 pyhd8ed1ab_0 conda-forge
cffi 1.15.1 py311h409f033_3 conda-forge
chardet 5.1.0 py311h38be061_0 conda-forge
charset-normalizer 3.2.0 pyhd8ed1ab_0 conda-forge
click 8.1.6 unix_pyh707e725_0 conda-forge
cloudpickle 2.2.1 pyhd8ed1ab_0 conda-forge
comm 0.1.4 pyhd8ed1ab_0 conda-forge
condastats 0.2.1 pyhd8ed1ab_0 conda-forge
contourpy 1.1.0 py311h9547e67_0 conda-forge
cryptography 41.0.2 py311h63ff55d_0 conda-forge
cycler 0.11.0 pyhd8ed1ab_0 conda-forge
cytoolz 0.12.2 py311h459d7ec_0 conda-forge
dask 2023.7.1 pyhd8ed1ab_0 conda-forge
dask-core 2023.7.1 pyhd8ed1ab_0 conda-forge
dataproperty 1.0.1 pyhd8ed1ab_0 conda-forge
dbus 1.13.6 h5008d03_3 conda-forge
debugpy 1.6.8 py311hb755f60_0 conda-forge
decorator 5.1.1 pyhd8ed1ab_0 conda-forge
distributed 2023.7.1 pyhd8ed1ab_0 conda-forge
dominate 2.8.0 pyhd8ed1ab_0 conda-forge
exceptiongroup 1.1.2 pyhd8ed1ab_0 conda-forge
executing 1.2.0 pyhd8ed1ab_0 conda-forge
expat 2.5.0 hcb278e6_1 conda-forge
font-ttf-dejavu-sans-mono 2.37 hab24e00_0 conda-forge
font-ttf-inconsolata 3.000 h77eed37_0 conda-forge
font-ttf-source-code-pro 2.038 h77eed37_0 conda-forge
font-ttf-ubuntu 0.83 hab24e00_0 conda-forge
fontconfig 2.14.2 h14ed4e7_0 conda-forge
fonts-conda-ecosystem 1 0 conda-forge
fonts-conda-forge 1 0 conda-forge
fonttools 4.41.1 py311h459d7ec_0 conda-forge
freetype 2.12.1 hca18f0e_1 conda-forge
frozenlist 1.4.0 py311h459d7ec_0 conda-forge
fsspec 2023.6.0 pyh1a96a4e_0 conda-forge
gettext 0.21.1 h27087fc_0 conda-forge
gflags 2.2.2 he1b5a44_1004 conda-forge
glib 2.76.4 hfc55251_0 conda-forge
glib-tools 2.76.4 hfc55251_0 conda-forge
glog 0.6.0 h6f12383_0 conda-forge
graphite2 1.3.13 h58526e2_1001 conda-forge
gst-plugins-base 1.22.5 hf7dbed1_0 conda-forge
gstreamer 1.22.5 h98fc4e7_0 conda-forge
h11 0.14.0 pyhd8ed1ab_0 conda-forge
h2 4.1.0 pyhd8ed1ab_0 conda-forge
harfbuzz 7.3.0 hdb3a94d_0 conda-forge
hpack 4.0.0 pyh9f0ad1d_0 conda-forge
httpcore 0.17.3 pyhd8ed1ab_0 conda-forge
httpx 0.24.1 pyhd8ed1ab_0 conda-forge
hyperframe 6.0.1 pyhd8ed1ab_0 conda-forge
icu 72.1 hcb278e6_0 conda-forge
idna 3.4 pyhd8ed1ab_0 conda-forge
importlib-metadata 6.8.0 pyha770c72_0 conda-forge
importlib_metadata 6.8.0 hd8ed1ab_0 conda-forge
ipykernel 6.25.0 pyh71e2992_0 conda-forge
ipython 8.14.0 pyh41d4057_0 conda-forge
jedi 0.19.0 pyhd8ed1ab_0 conda-forge
jinja2 3.1.2 pyhd8ed1ab_1 conda-forge
jmespath 1.0.1 pyhd8ed1ab_0 conda-forge
jupyter_client 8.3.0 pyhd8ed1ab_0 conda-forge
jupyter_core 5.3.1 py311h38be061_0 conda-forge
keyutils 1.6.1 h166bdaf_0 conda-forge
kiwisolver 1.4.4 py311h4dd048b_1 conda-forge
krb5 1.21.1 h659d440_0 conda-forge
lame 3.100 h166bdaf_1003 conda-forge
lcms2 2.15 haa2dc70_1 conda-forge
ld_impl_linux-64 2.40 h41732ed_0 conda-forge
lerc 4.0.0 h27087fc_0 conda-forge
libabseil 20230125.3 cxx17_h59595ed_0 conda-forge
libarrow 12.0.1 h657c46f_7_cpu conda-forge
libblas 3.9.0 17_linux64_openblas conda-forge
libbrotlicommon 1.0.9 h166bdaf_9 conda-forge
libbrotlidec 1.0.9 h166bdaf_9 conda-forge
libbrotlienc 1.0.9 h166bdaf_9 conda-forge
libcap 2.69 h0f662aa_0 conda-forge
libcblas 3.9.0 17_linux64_openblas conda-forge
libclang 15.0.7 default_h7634d5b_3 conda-forge
libclang13 15.0.7 default_h9986a30_3 conda-forge
libcrc32c 1.1.2 h9c3ff4c_0 conda-forge
libcups 2.3.3 h4637d8d_4 conda-forge
libcurl 8.2.1 hca28451_0 conda-forge
libdeflate 1.18 h0b41bf4_0 conda-forge
libedit 3.1.20191231 he28a2e2_2 conda-forge
libev 4.33 h516909a_1 conda-forge
libevent 2.1.12 hf998b51_1 conda-forge
libexpat 2.5.0 hcb278e6_1 conda-forge
libffi 3.4.2 h7f98852_5 conda-forge
libflac 1.4.3 h59595ed_0 conda-forge
libgcc-ng 13.1.0 he5830b7_0 conda-forge
libgcrypt 1.10.1 h166bdaf_0 conda-forge
libgfortran-ng 13.1.0 h69a702a_0 conda-forge
libgfortran5 13.1.0 h15d22d2_0 conda-forge
libglib 2.76.4 hebfc3b9_0 conda-forge
libgomp 13.1.0 he5830b7_0 conda-forge
libgoogle-cloud 2.12.0 h840a212_1 conda-forge
libgpg-error 1.47 h71f35ed_0 conda-forge
libgrpc 1.56.2 h3905398_0 conda-forge
libiconv 1.17 h166bdaf_0 conda-forge
libjpeg-turbo 2.1.5.1 h0b41bf4_0 conda-forge
liblapack 3.9.0 17_linux64_openblas conda-forge
libllvm15 15.0.7 h5cf9203_3 conda-forge
libnghttp2 1.52.0 h61bc06f_0 conda-forge
libnsl 2.0.0 h7f98852_0 conda-forge
libnuma 2.0.16 h0b41bf4_1 conda-forge
libogg 1.3.4 h7f98852_1 conda-forge
libopenblas 0.3.23 pthreads_h80387f5_0 conda-forge
libopus 1.3.1 h7f98852_1 conda-forge
libpng 1.6.39 h753d276_0 conda-forge
libpq 15.3 hfc447b1_2 conda-forge
libprotobuf 4.23.3 hd1fb520_0 conda-forge
libsndfile 1.2.0 hb75c966_0 conda-forge
libsodium 1.0.18 h36c2ea0_1 conda-forge
libsqlite 3.42.0 h2797004_0 conda-forge
libssh2 1.11.0 h0841786_0 conda-forge
libstdcxx-ng 13.1.0 hfd8a6a1_0 conda-forge
libsystemd0 254 h3516f8a_0 conda-forge
libthrift 0.18.1 h8fd135c_2 conda-forge
libtiff 4.5.1 h8b53f26_0 conda-forge
libutf8proc 2.8.0 h166bdaf_0 conda-forge
libuuid 2.38.1 h0b41bf4_0 conda-forge
libvorbis 1.3.7 h9c3ff4c_0 conda-forge
libwebp-base 1.3.1 hd590300_0 conda-forge
libxcb 1.15 h0b41bf4_0 conda-forge
libxkbcommon 1.5.0 h5d7e998_3 conda-forge
libxml2 2.11.4 h0d562d8_0 conda-forge
libzlib 1.2.13 hd590300_5 conda-forge
locket 1.0.0 pyhd8ed1ab_0 conda-forge
lz4 4.3.2 py311h9f220a4_0 conda-forge
lz4-c 1.9.4 hcb278e6_0 conda-forge
markupsafe 2.1.3 py311h459d7ec_0 conda-forge
matplotlib 3.7.2 py311h38be061_0 conda-forge
matplotlib-base 3.7.2 py311h54ef318_0 conda-forge
matplotlib-inline 0.1.6 pyhd8ed1ab_0 conda-forge
matplotx 0.3.10 pyhd8ed1ab_0 conda-forge
mbstrdecoder 1.1.3 pyhd8ed1ab_1 conda-forge
mpg123 1.31.3 hcb278e6_0 conda-forge
msgpack-python 1.0.5 py311ha3edf6b_0 conda-forge
multidict 6.0.4 py311h2582759_0 conda-forge
munkres 1.1.4 pyh9f0ad1d_0 conda-forge
mysql-common 8.0.33 hf1915f5_2 conda-forge
mysql-libs 8.0.33 hca2cd23_2 conda-forge
ncurses 6.4 hcb278e6_0 conda-forge
nest-asyncio 1.5.6 pyhd8ed1ab_0 conda-forge
networkx 3.1 pyhd8ed1ab_0 conda-forge
nspr 4.35 h27087fc_0 conda-forge
nss 3.89 he45b914_0 conda-forge
numpy 1.25.1 py311h64a7726_0 conda-forge
openjpeg 2.5.0 hfec8fc6_2 conda-forge
openssl 3.1.2 hd590300_0 conda-forge
orc 1.9.0 h385abfd_1 conda-forge
packaging 23.1 pyhd8ed1ab_0 conda-forge
pandas 1.5.3 py311h2872171_1 conda-forge
parso 0.8.3 pyhd8ed1ab_0 conda-forge
partd 1.4.0 pyhd8ed1ab_0 conda-forge
pathvalidate 3.1.0 pyhd8ed1ab_0 conda-forge
patsy 0.5.3 pyhd8ed1ab_0 conda-forge
pcre2 10.40 hc3806b6_0 conda-forge
pexpect 4.8.0 pyh1a96a4e_2 conda-forge
pickleshare 0.7.5 py_1003 conda-forge
pillow 10.0.0 py311h0b84326_0 conda-forge
pip 23.2.1 pyhd8ed1ab_0 conda-forge
pixman 0.40.0 h36c2ea0_0 conda-forge
platformdirs 3.10.0 pyhd8ed1ab_0 conda-forge
ply 3.11 py_1 conda-forge
pooch 1.7.0 pyha770c72_3 conda-forge
prettytable 3.8.0 pyhd8ed1ab_0 conda-forge
prompt-toolkit 3.0.39 pyha770c72_0 conda-forge
prompt_toolkit 3.0.39 hd8ed1ab_0 conda-forge
psutil 5.9.5 py311h2582759_0 conda-forge
pthread-stubs 0.4 h36c2ea0_1001 conda-forge
ptyprocess 0.7.0 pyhd3deb0d_0 conda-forge
pulseaudio-client 16.1 hb77b528_4 conda-forge
pure_eval 0.2.2 pyhd8ed1ab_0 conda-forge
pyarrow 12.0.1 py311h39c9aba_7_cpu conda-forge
pycparser 2.21 pyhd8ed1ab_0 conda-forge
pygments 2.15.1 pyhd8ed1ab_0 conda-forge
pyopenssl 23.2.0 pyhd8ed1ab_1 conda-forge
pyparsing 3.0.9 pyhd8ed1ab_0 conda-forge
pypistats 1.4.0 pyhd8ed1ab_0 conda-forge
pyqt 5.15.9 py311hf0fb5b6_4 conda-forge
pyqt5-sip 12.12.2 py311hb755f60_4 conda-forge
pysocks 1.7.1 pyha2e5f31_6 conda-forge
pytablewriter 1.0.0 pyhd8ed1ab_0 conda-forge
python 3.11.4 hab00c5b_0_cpython conda-forge
python-dateutil 2.8.2 pyhd8ed1ab_0 conda-forge
python-slugify 8.0.1 pyhd8ed1ab_0 conda-forge
python-snappy 0.6.1 py311h48ad69a_0 conda-forge
python-tzdata 2023.3 pyhd8ed1ab_0 conda-forge
python_abi 3.11 3_cp311 conda-forge
pytz 2023.3 pyhd8ed1ab_0 conda-forge
pyyaml 6.0 py311hd4cff14_5 conda-forge
pyzmq 25.1.0 py311h75c88c4_0 conda-forge
qt-main 5.15.8 h7fe3ca9_15 conda-forge
rdma-core 28.9 h59595ed_1 conda-forge
re2 2023.03.02 h8c504da_0 conda-forge
readline 8.2 h8228510_1 conda-forge
requests 2.31.0 pyhd8ed1ab_0 conda-forge
s2n 1.3.46 h06160fa_0 conda-forge
s3fs 2023.6.0 pyhd8ed1ab_0 conda-forge
scipy 1.11.1 py311h64a7726_0 conda-forge
seaborn 0.12.2 hd8ed1ab_0 conda-forge
seaborn-base 0.12.2 pyhd8ed1ab_0 conda-forge
setuptools 68.0.0 pyhd8ed1ab_0 conda-forge
sip 6.7.10 py311hb755f60_0 conda-forge
six 1.16.0 pyh6c4a22f_0 conda-forge
snappy 1.1.10 h9fff704_0 conda-forge
sniffio 1.3.0 pyhd8ed1ab_0 conda-forge
sortedcontainers 2.4.0 pyhd8ed1ab_0 conda-forge
stack_data 0.6.2 pyhd8ed1ab_0 conda-forge
statsmodels 0.14.0 py311h1f0f07a_1 conda-forge
tabledata 1.3.1 pyhd8ed1ab_0 conda-forge
tblib 1.7.0 pyhd8ed1ab_0 conda-forge
tcolorpy 0.1.3 pyhd8ed1ab_0 conda-forge
termcolor 2.3.0 pyhd8ed1ab_0 conda-forge
text-unidecode 1.3 py_0 conda-forge
tk 8.6.12 h27826a3_0 conda-forge
toml 0.10.2 pyhd8ed1ab_0 conda-forge
tomli 2.0.1 pyhd8ed1ab_0 conda-forge
toolz 0.12.0 pyhd8ed1ab_0 conda-forge
tornado 6.3.2 py311h459d7ec_0 conda-forge
traitlets 5.9.0 pyhd8ed1ab_0 conda-forge
typepy 1.3.1 pyhd8ed1ab_0 conda-forge
typing-extensions 4.7.1 hd8ed1ab_0 conda-forge
typing_extensions 4.7.1 pyha770c72_0 conda-forge
tzdata 2023c h71feb2d_0 conda-forge
ucx 1.14.1 hf587318_2 conda-forge
unidecode 1.3.6 pyhd8ed1ab_0 conda-forge
urllib3 1.26.15 pyhd8ed1ab_0 conda-forge
wcwidth 0.2.6 pyhd8ed1ab_0 conda-forge
wheel 0.41.0 pyhd8ed1ab_0 conda-forge
wrapt 1.15.0 py311h2582759_0 conda-forge
xcb-util 0.4.0 hd590300_1 conda-forge
xcb-util-image 0.4.0 h8ee46fc_1 conda-forge
xcb-util-keysyms 0.4.0 h8ee46fc_1 conda-forge
xcb-util-renderutil 0.3.9 hd590300_1 conda-forge
xcb-util-wm 0.4.1 h8ee46fc_1 conda-forge
xkeyboard-config 2.39 hd590300_0 conda-forge
xorg-kbproto 1.0.7 h7f98852_1002 conda-forge
xorg-libice 1.1.1 hd590300_0 conda-forge
xorg-libsm 1.2.4 h7391055_0 conda-forge
xorg-libx11 1.8.6 h8ee46fc_0 conda-forge
xorg-libxau 1.0.11 hd590300_0 conda-forge
xorg-libxdmcp 1.1.3 h7f98852_0 conda-forge
xorg-libxext 1.3.4 h0b41bf4_2 conda-forge
xorg-libxrender 0.9.11 hd590300_0 conda-forge
xorg-renderproto 0.11.1 h7f98852_1002 conda-forge
xorg-xextproto 7.3.0 h0b41bf4_1003 conda-forge
xorg-xf86vidmodeproto 2.3.1 h7f98852_1002 conda-forge
xorg-xproto 7.0.31 h7f98852_1007 conda-forge
xyzservices 2023.7.0 pyhd8ed1ab_0 conda-forge
xz 5.2.6 h166bdaf_0 conda-forge
yaml 0.2.5 h7f98852_2 conda-forge
yarl 1.9.2 py311h459d7ec_0 conda-forge
zeromq 4.3.4 h9c3ff4c_1 conda-forge
zict 3.0.0 pyhd8ed1ab_0 conda-forge
zipp 3.16.2 pyhd8ed1ab_0 conda-forge
zlib 1.2.13 hd590300_5 conda-forge
zstd 1.5.2 hfc55251_7 conda-forge
|
ok, that's it !! pandas<2.0.0 is required ! Thanks @nicrie |
Hi @sophiamyang and @nicrie, The requirement Any comments or ideas ? Thanks ! shell>> # python 3.12.5 with brand new virtual env
shell>> pip install condastat
shell>> condastat overall numpy --monthly
Traceback (most recent call last):
[...]
ValueError: An error occurred while calling the read_parquet method registered to the pandas backend.
shell>> pip install "pandas<2.0.0"
Installing collected packages: pandas
Attempting uninstall: pandas
Found existing installation: pandas 2.2.3
Uninstalling pandas-2.2.3:
Successfully uninstalled pandas-2.2.3
ERROR: pip dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
dask-expr 1.1.14 requires pandas>=2, but you have pandas 1.5.3 which is incompatible.
Successfully installed pandas-1.5.3
shell>> condastat overall numpy --monthly
Traceback (most recent call last):
[...]
ValueError: numpy.dtype size changed, may indicate binary incompatibility. Expected 96 from C header, got 88 from PyObject |
Description
Unable to use condastats.cli.overall (internal error on pandas->pyArrow)
The text was updated successfully, but these errors were encountered: