The python version currently installed on the cluster is 3.11.2
Note that Python scripts must be run using the Slurm workload manager.
A lot of packages are already installed globally.
You can see the list of installed packages and their version with these commands.
pip3 list
pip3 freeze
pip3 list | grep package-name
pip3 freeze | grep package-name
In case you need a different version of a certain package or a package that isn’t installed the preferred method is using virtualenv.
virtualenv allows creating an isolated (or semi-isolated) python environment where you can install your own packages without requiring super-user privileges.
virtualenv can take a lot of disk space, so it should be created on a lab storage space and not on the home directory.
To create the virtual environment:
virtualenv /sci/labs/pi-user-name/your-user-name/my-python-venv
This command will create a folder with the name my-python-venv.
The Python virtual environment will be created in that folder.
We suggest you give that virtual environment a meaningful name so you would remember the purpose of creating it.
Please avoid copying the example name.
To install packages in the virtualenv you created you first need to activate it:
For csh shell (default shell in cluster):
source /sci/labs/pi-user-name/your-user-name/my-python-venv/bin/activate.csh
For sh, bash or zsh shells
. /sci/labs/pi-user-name/your-user-name/my-python-venv/bin/activate
Notice that there is a dot at the beginning of the command instead of the command “source” as for csh shell.
Now you can install packages inside the virtualenv with pip command
pip install package-name
To install a specific version of the package use:
pip install package-name==version
To uninstall a package:
pip uninstall package-name
Packages that come from a source with a setup.py can also be installed inside the virtualenv
python setup.py install
When the installation of packages is finished you need to exit the virtualenv with this command:
deactivate
To use the packages you installed in virtualenv you need to activate it the same way you activate it for installing packages.
For interactive work in the cluster with csh shell (default shell in cluster):
source /sci/labs/pi-user-name/your-user-name/my-python-venv/bin/activate.csh
For sbatch scripts sh, bash or zsh shells and scripts:
. /sci/labs/pi-user-name/your-user-name/my-python-venv/bin/activate
After activating, any Python related commands will work from the virtualenv.
In scripts use this line at the beginning so that your script will use the Python interpreter from the virtualenv :
#!/usr/bin/env python
You can verify that you are using the Python in the virtualenv with this command:
which python
Output should be
/sci/labs/pi-user-name/your-user-name/my-python-venv/bin/python
When you finish working with the virtualenv deactivate it:
deactivate
To make sure that your Python script will use the first Python interpreter in the PATH you need to use this line at the beginning of your script:
#!/usr/bin/env python
The Python interpreter will change when you load some software with the module command that has Python packages.
To check which Python interpreter you are using run this command:
which python