Spark in Docker with AWS credentials
Running spark in docker container Setting up spark is tricky. Therefore it is useful to try out things locally before deploying to the cluster. Docker is of a good help here. There is a great docker image to play with spark locally. gettyimages/docker-spark Examples Running SparkPi sample program (one of the examples from the docs of Spark): docker run --rm -it -p 4040:4040 gettyimages/spark bin/run-example SparkPi 10 Running a small example with Pyspark:...