Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Prepare spark-tensorflow-distributor 0.1.0 release #163

Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open

Prepare spark-tensorflow-distributor 0.1.0 release #163

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
388be86
prepare 0.1.0 release
mengxr Jun 22, 2020
9199fe9
use Spark 3.0
mengxr Jun 22, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 1 addition & 4 deletions spark/spark-tensorflow-distributor/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,10 +15,7 @@ RUN apt-get update && \
update-alternatives --install /usr/local/bin/python python /usr/bin/python${PYTHON_INSTALL_VERSION} 1

# Install Spark and update env variables
# Modify this when the next Spark 3 release includes the allGather barrier mode API
RUN pip install \
https://ml-team-public-read.s3-us-west-2.amazonaws.com/pyspark-3.1.0.dev0-60dd1a690fed62b1d6442cdc8cf3f89ef4304d5a.tar.gz \
--force-reinstall --upgrade
RUN pip install pyspark==3.0.0

# Set SPARK_HOME so that tests can easily find the scripts in $SPARK_HOME/sbin
ENV SPARK_HOME /usr/local/lib/python${PYTHON_INSTALL_VERSION}/dist-packages/pyspark
Expand Down
2 changes: 1 addition & 1 deletion spark/spark-tensorflow-distributor/setup.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@

setuptools.setup(
name="spark_tensorflow_distributor",
version="0.0.3",
version="0.1.0",
author="sarthfrey",
author_email="[email protected]",
description="This package helps users do distributed training with TensorFlow on their Spark clusters.",
Expand Down