Commit 22b51d28 authored by Daniele Venzano's avatar Daniele Venzano
Browse files

Testing and fixing the Spark ZApp

parent 041ed5b3
Pipeline #10406 passed with stages
in 1 minute and 6 seconds
# Configure the necessary Spark environment
import os
import sys
# make sure pyspark tells workers to use python3 not 2 if both are installed
os.environ['PYSPARK_PYTHON'] = '/opt/conda/bin/python'
sys.path += ['/opt/spark/python', '/opt/spark/python/lib/py4j-0.10.7-src.zip']
if 'SPARK_MASTER' in os.environ:
import pyspark
......
#!/bin/bash
echo $PYTHONPATH
exec /opt/conda/bin/jupyter lab --no-browser --NotebookApp.token='' --allow-root --ip=0.0.0.0
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment