Pinning cudf version to 0.10

[randerzander]

Due to the many existing pip packages installed by default in Colab, if you try to conda install -c rapidsai-nightly cudf conda will likely install an older version of cudf.

Explicitly passing cudf=0.10 fixes that and will result in installing the latest nightly for the rest of the RAPIDS packages as well.

[randerzander]

With the latest commits, users can now specify a rapids-nightly version argument in the "header" cell.

For instance, to install RAPIDS 0.9 nightlies:

!wget -nc https://github.com/rapidsai/notebooks-contrib/raw/master/utils/rapids-colab.sh
!bash rapids-colab.sh 0.9

import sys, os

sys.path.append('/usr/local/lib/python3.6/site-packages/')
os.environ['NUMBAPRO_NVVM'] = '/usr/local/cuda/nvvm/lib64/libnvvm.so'
os.environ['NUMBAPRO_LIBDEVICE'] = '/usr/local/cuda/nvvm/libdevice/'

By default (if no argument is given), the 0.10 nightly packages will be installed.

[gumdropsteve]

This resolves #149

Warnings

• have been getting these warnings for a while when installing RAPIDS in Colab
  • this pr didn't cause, but seems like a good time to mention

WARNING:
    You currently have a PYTHONPATH environment variable set. This may cause
    unexpected behavior when running the Python interpreter in Miniconda3.
    For best results, please verify that your PYTHONPATH only points to
    directories of packages that are compatible with the Python interpreter
    in Miniconda3: /usr/local
Installing RAPIDS packages
Please standby, this will take a few minutes...


==> WARNING: A newer version of conda exists. <==
  current version: 4.5.4
  latest version: 4.7.11

Please update conda by running

    $ conda update -n base conda

[randerzander]

This resolves #149

Warnings

• have been getting these warnings for a while when installing RAPIDS in Colab

  • this pr didn't cause, but seems like a good time to mention

WARNING:
    You currently have a PYTHONPATH environment variable set. This may cause
    unexpected behavior when running the Python interpreter in Miniconda3.
    For best results, please verify that your PYTHONPATH only points to
    directories of packages that are compatible with the Python interpreter
    in Miniconda3: /usr/local
Installing RAPIDS packages
Please standby, this will take a few minutes...


==> WARNING: A newer version of conda exists. <==
  current version: 4.5.4
  latest version: 4.7.11

Please update conda by running

    $ conda update -n base conda

Unfortunately this is an artifact of using the only version of miniconda (an outdated one) that works with the version of Python used by Colab.

I searched but couldn't find a way to disable the warning during install of miniconda itself. If you can figure out how to, please do let us know.

[gumdropsteve]

read_csv error

occurs when installation is done via proposed script

• code functions as expected when using old/og install script
  • old script pasted below, still in use on rapids-colab.ipynb
• was seeing same error before issue Colab Install Script syntax error [BUG] #149 arose
  • may have something to do with version?

input

# import train 2016  data
train2016 = cudf.read_csv('/content/train_2016_v2.csv',
                          parse_dates=["transactiondate"])
# peek display 2016 train
print(train2016.head())

output

---------------------------------------------------------------------------

TypeError                                 Traceback (most recent call last)

<ipython-input-6-ff87c45bf2d8> in <module>()
      1 train2016 = cudf.read_csv('/content/train_2016_v2.csv',
----> 2                           parse_dates=["transactiondate"])
      3 # peek display 2016 train
      4 print(train2016.head())

TypeError: read_csv() got an unexpected keyword argument 'parse_dates'

Old install script being used

# intall miniconda
!wget -c https://repo.continuum.io/miniconda/Miniconda3-4.5.4-Linux-x86_64.sh
!chmod +x Miniconda3-4.5.4-Linux-x86_64.sh
!bash ./Miniconda3-4.5.4-Linux-x86_64.sh -b -f -p /usr/local

# install RAPIDS packages
!conda install -q -y --prefix /usr/local -c conda-forge \
  -c rapidsai-nightly/label/cuda10.0 -c nvidia/label/cuda10.0 \
  cudf cuml

# set environment vars
import sys, os, shutil
sys.path.append('/usr/local/lib/python3.6/site-packages/')
os.environ['NUMBAPRO_NVVM'] = '/usr/local/cuda/nvvm/lib64/libnvvm.so'
os.environ['NUMBAPRO_LIBDEVICE'] = '/usr/local/cuda/nvvm/libdevice/'

# copy .so files to current working dir
for fn in ['libcudf.so', 'librmm.so']:
  shutil.copy('/usr/local/lib/'+fn, os.getcwd())

• old script cons
  • needs to be run twice to install RAPIDS packages
  • takes longer to run
    • each time, total time saved by new script is large

to reproduce

Notebook being run is WIP colab #148

• notebook with latest updates viewable in drive

install script being used

• tried with specifying RAPIDS 0.9 nightlies and without

!wget -nc https://github.com/rapidsai/notebooks-contrib/raw/master/utils/rapids-colab.sh
!bash rapids-colab.sh 0.9

import sys, os

sys.path.append('/usr/local/lib/python3.6/site-packages/')
os.environ['NUMBAPRO_NVVM'] = '/usr/local/cuda/nvvm/lib64/libnvvm.so'
os.environ['NUMBAPRO_LIBDEVICE'] = '/usr/local/cuda/nvvm/libdevice/'

!wget -nc https://github.com/rapidsai/notebooks-contrib/raw/master/utils/rapids-colab.sh
!bash rapids-colab.sh

import sys, os

sys.path.append('/usr/local/lib/python3.6/site-packages/')
os.environ['NUMBAPRO_NVVM'] = '/usr/local/cuda/nvvm/lib64/libnvvm.so'
os.environ['NUMBAPRO_LIBDEVICE'] = '/usr/local/cuda/nvvm/libdevice/'

[randerzander]

This PR is updating https://github.com/rapidsai/notebooks-contrib/raw/master/utils/rapids-colab.sh, so you'll not be able to test using that same URL until this PR merges.

The way I tested was with the URL for my fork: https://github.com/randerzander/notebooks-contrib/raw/master/utils/rapids-colab.sh

[gumdropsteve]

Awesome, was hoping this was the case. Thanks.