This code sample shows how to use Spark 3.2.0 with Python 3.8 within a Docker container.

Running the code

To run this code, the submit.sh script will need to be executed in a bash shell. The machine from which you execute this code will need to have Spark 3.2.0 installed and configured correctly.

Extended documentation

More documentation can be found on the terrascope website: https://docs.terrascope.be/#/Developers/Hadoop/UsingDockerOnHadoop