NEST compiles without Python bindings on Travis CI

[Helveg]

Hi, because of #1332 I've switched from Semaphore CI to Travis CI, where NEST does compile, but for some reason it doesn't add the Python bindings. The only relevant error in the build log I could find seems to be this:

-- Could NOT find GSL (missing: GSL_INCLUDE_DIR GSL_LIBRARY GSL_CBLAS_LIBRARY) (found version "")
-- Found PythonInterp: /home/travis/virtualenv/python3.7.1/bin/python3 (found suitable version "3.7.1", minimum required is "3") 
-- Could NOT find PythonLibs (missing: PYTHON_LIBRARIES PYTHON_INCLUDE_DIRS) (Required is exact version "3.7.1")

Could this cause CMake to decide to compile without Python support?

Built-in modules    : models;precise;topology
User modules        : None
Python bindings     : No
Use threading       : Yes (OpenMP: -fopenmp)
Use GSL             : No
Use Readline        : Yes (GNU Readline 6.3)
    Includes        : /usr/include
    Libraries       : /usr/lib/x86_64-linux-gnu/libreadline.so;/usr/lib/x86_64-linux-gnu/libncurses.so

How could this be resolved?

The compile commands are:

sudo apt-get install -y build-essential cmake cython libgsl-dev libltdl-dev libncurses-dev libreadline-dev python3-all-dev python3-numpy python3-scipy python3-matplotlib python3-nose openmpi-bin libopenmpi-dev
wget https://github.com/nest/nest-simulator/archive/v$NEST_VERSION.tar.gz -O nest-simulator-$NEST_VERSION.tar.gz
tar -xzf nest-simulator-$NEST_VERSION.tar.gz
mkdir nest-simulator-$NEST_VERSION-build
mkdir nest-install-$NEST_VERSION
cd nest-simulator-$NEST_VERSION-build

cmake -Dwith-python=3 -DCMAKE_INSTALL_PREFIX:PATH=/home/travis/nest-$NEST_VERSION /home/travis/nest-simulator-$NEST_VERSION
make
make install
make installcheck
source /home/travis/nest-$NEST_VERSION/bin/nest_vars.sh

And I've pasted the full log here

[Helveg]

I was excited to see the updated installation instructions, but the nest installation from the PPA also installs without Python bindings.

[Helveg]

Now that I managed to find the right Python executables/includes and libraries (using the 3.5 library because Travis doesn't really seem to have the right 3.7 library, it uses pyenv shims (?) ) it just throws a duplicate of #1332.

Perhaps this thread shines some light on the issue: mcfletch/pyopengl#11

[lekshmideepu]

We also had the same problem in finding the python libraries and includes in Travis workers and I set the path explicitly like here. As you said, you were able to find the libraries and headers, hope that your problem has been solved.

[Helveg]

Ok thanks @lekshmideepu, I was using a different include dir and library file. But so NEST is untested on Python 3.7? Because reverting to 3.6 on Travis fixed the PyThreadState errors on Travis, but not on Semaphore ... If the cmake version is recent enough the errors become exactly the same as in the external issue I linked.

[Helveg]

I'm not familiar with the issue or frameworks used to build NEST but the last post in pytries/DAWG#40 might also help solve the problem in the pynestkernel? There's many occurences of the mentioned except + statement there.

[Silmathoron]

Just to mention that I'm using NEST fine on Python 3.7/3.8. The except + statement means that Cython will intercept thrown errors, convert them to Python errors, then throw them to have a proper termination and not a segfault crash and burn, not sure it's related to anything here...

[Helveg]

Well have you looked at the errors in #1332? There seems to be an issue with the `PyThreadState` missing members `exc_*` which I also get here on Travis when I try to compile with 3.7. But the main original issue of this thread is solved by copying the cmake flags! Let's close this issue but we can keep going on #1332 because I haven't had any advice there :) I suppose it'll be a cmake/cython version thing but I'm on the latest versions.

…

On Thu, 12 Dec 2019, 13:14 Tanguy Fardet, ***@***.***> wrote: Just to mention that I'm using NEST fine on Python 3.7/3.8. The except + statement means that Cython will intercept thrown errors, convert them to Python errors, then throw them to have a proper termination and not a segfault crash and burn, not sure it's related to anything here... — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#1365?email_source=notifications&email_token=AG4VQS75TOGMQXUE27AOJRDQYITLDA5CNFSM4JZXK632YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEGWPFPA#issuecomment-564982460>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AG4VQS5FFOI5JRPQH6R7VD3QYITLDANCNFSM4JZXK63Q> .

[terhorstd]

reported as fixed by user.