Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update existing Doc on how to use EESSI #317

Open
wants to merge 22 commits into
base: main
Choose a base branch
from
Open
Changes from all commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
48e46e0
Update existing Doc on how to use EESSI
Oct 8, 2024
4e72953
Update existing Doc on how to use EESSI
Oct 9, 2024
069216f
Update setting_up_environment.md
ocaisa Oct 16, 2024
49abc11
Update docs/using_eessi/setting_up_environment.md
TopRichard Oct 17, 2024
4dcbc25
Update docs/using_eessi/setting_up_environment.md
TopRichard Oct 17, 2024
0c0882d
Update docs/using_eessi/setting_up_environment.md
TopRichard Oct 17, 2024
56297c7
Update docs/using_eessi/setting_up_environment.md
TopRichard Oct 17, 2024
f5d6855
Update docs/using_eessi/setting_up_environment.md
TopRichard Oct 17, 2024
6bfca5d
Update docs/using_eessi/setting_up_environment.md
TopRichard Oct 17, 2024
0dd9a07
Update docs/using_eessi/setting_up_environment.md
TopRichard Oct 17, 2024
0b8bd58
Update docs/using_eessi/setting_up_environment.md
TopRichard Oct 17, 2024
d458ab9
Update docs/using_eessi/setting_up_environment.md
TopRichard Oct 17, 2024
5c251d5
Update docs/using_eessi/setting_up_environment.md
TopRichard Oct 17, 2024
76d87b1
Update docs/using_eessi/setting_up_environment.md
TopRichard Oct 18, 2024
0f8e9e7
Update docs/using_eessi/setting_up_environment.md
TopRichard Oct 18, 2024
1dff95b
Merge branch 'EESSI:main' into eessi-using-EESSI-as-module
TopRichard Oct 18, 2024
ee020c4
Update setting_up_environment.md
TopRichard Oct 18, 2024
bf98d89
Update setting_up_environment.md
TopRichard Oct 18, 2024
e5f4cad
Update setting_up_environment.md
TopRichard Oct 18, 2024
554590e
Update setting_up_environment.md
TopRichard Oct 18, 2024
5d9d30b
Update setting_up_environment.md
TopRichard Oct 18, 2024
b726a4f
Update setting_up_environment.md
TopRichard Oct 18, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
74 changes: 64 additions & 10 deletions docs/using_eessi/setting_up_environment.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,57 @@
# Setting up your environment

To set up the EESSI environment, simply run the command:
To set up the EESSI environment, you can either load an EESSI module with Lmod or source a bash initialisation script.

## Loading an EESSI module with Lmod

There are a few different scenarios where you may want to set up the EESSI environment by loading an EESSI module with Lmod:

!!! note "Why do we recommend to unset `MODULEPATH`?"

Unsetting MODULEPATH may not be possible at some sites, and some compromise or workaround may
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Unsetting MODULEPATH may not be possible at some sites, and some compromise or workaround may
Unsetting the `$MODULEPATH` environment variable, which tells Lmod in which directories environment module files are available, may not be possible at some sites, and some compromise or workaround may

be necessary. The underlying reason to suggest this is that EESSI and your system are most likely based on
two different operating system distributions - EESSI uses Gentoo Prefix, your system almost certainly uses some other
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wouldn't mention Gentoo Prefix here, keep it more general (compatibility layer + link to the corresponding page)

distribution. If you can find a way to ensure that the _software stacks_ from your site and EESSI do not mix
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

distribution -> Linux distribution

(in particular when someone is building new software!), then this should be good enough.

1. You are already using Lmod with version >= 8.6

In this case, we _recommend_ unsetting the `MODULEPATH`, because EESSI is not designed to mix modules coming from EESSI and from your system.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

    In this case, we _recommend_ unsetting `$MODULEPATH`, because EESSI is not designed to mix modules coming from EESSI and from your system.


``` { .bash .copy }
unset MODULEPATH
export MODULEPATH=/cvmfs/software.eessi.io/init/modules
Comment on lines +22 to +23
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you're doing a hard set, then there's no need to unset?

module load EESSI/2023.06
```

TopRichard marked this conversation as resolved.
Show resolved Hide resolved
:clap: Your environment is now set up, you are ready to start running software provided by EESSI!

2. You are using an Lmod with a version older than 8.6 or any other tool utilizing `MODULEPATH` (Tmod, etc.)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tmod is not the official name, that's sort of "slang", use "Tcl-based Environment Modules" + link that to https://modules.sourceforge.net/


You should unset MODULEPATH to prevent Lmod from attempting to build a cache for your module tree (as this can be very slow if you have
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
You should unset MODULEPATH to prevent Lmod from attempting to build a cache for your module tree (as this can be very slow if you have
You should unset `$MODULEPATH` to prevent Lmod from attempting to build a cache for your module tree (as this can be very slow if you have

a lot of modules). Again, unsetting the MODULEPATH should be considered as a good idea in general so you do not mix local and EESSI
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
a lot of modules). Again, unsetting the MODULEPATH should be considered as a good idea in general so you do not mix local and EESSI
a lot of modules). Again, unsetting the `$MODULEPATH` should be considered as a good idea in general so you do not mix local and EESSI

modules:

``` { .bash .copy }
unset MODULEPATH
source /cvmfs/software.eessi.io/versions/2023.06/init/lmod/bash
```

:clap: Your environment is now set up, you are ready to start running software provided by EESSI!

3. Should Lmod be unavailable and `MODULEPATH` not utilized, you can initialise EESSI via an Lmod module by directly sourcing the Lmod initialisation script (this script automatically loads the EESSI module):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To keep things simple, we should probably start with this and then have a collapsed section for "Other scenarios" that mentions the two other approaches (and the note/warning about unsetting the MODULEPATH). That way the two approaches are equivalent for typical use case.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

3.  Should Lmod be unavailable and `$MODULEPATH` not used, you can initialise EESSI via an Lmod module by directly sourcing the Lmod initialisation script (this script automatically loads the EESSI module):


``` { .bash .copy }
source /cvmfs/software.eessi.io/versions/2023.06/init/lmod/bash
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should make it clear that there are a range of options available here

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

or use $SHELL instead of harcoding bash?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I checked yesterday and $SHELL is not reliable for the supported shells, $0 is more reliable but not very visually helpful

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(or rather the even worse $(basename $0))

```

:clap: Your environment is now set up, you are ready to start running software provided by EESSI!

## Sourcing the EESSI `bash` initialisation script

You can initialise EESSI (in a non-reversible way) by running the command:
Copy link
Member

@ocaisa ocaisa Oct 18, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it is worth explicitly saying that this command only works with bash, and if you use another shell you should take the other approach.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • how to check (echo $SHELL)


!!! info "This is supported exclusively for bash shell users. If you're using a different shell, please use the alternative approach"

``` { .bash .copy }
source /cvmfs/software.eessi.io/versions/2023.06/init/bash
Expand All @@ -10,23 +61,26 @@ This may take a while as data is downloaded from a Stratum 1 server which is
part of the CernVM-FS infrastructure to distribute files. You should see the
following output:

``` { .yaml .no-copy }
``` { .bash .no-copy }
Found EESSI repo @ /cvmfs/software.eessi.io/versions/2023.06!
archdetect says x86_64/amd/zen2
archdetect says x86_64/amd/zen2 # (1)
archdetect could not detect any accelerators
Using x86_64/amd/zen2 as software subdirectory.
Using /cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/amd/zen2/modules/all as the directory to be added to MODULEPATH.
Found Lmod configuration file at /cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/amd/zen2/.lmod/lmodrc.lua
Found Lmod SitePackage.lua file at /cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/amd/zen2/.lmod/SitePackage.lua
Using /cvmfs/software.eessi.io/host_injections/2023.06/software/linux/x86_64/amd/zen2 as the site extension directory for installations.
Using /cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/amd/zen2/modules/all as the directory to be added to MODULEPATH.
Using /cvmfs/software.eessi.io/host_injections/2023.06/software/linux/x86_64/amd/zen2/modules/all as the site extension directory to be added to MODULEPATH.
Found libcurl CAs file at RHEL location, setting CURL_CA_BUNDLE
Initializing Lmod...
Prepending /cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/amd/zen2/modules/all to $MODULEPATH...
Prepending site path /cvmfs/software.eessi.io/host_injections/2023.06/software/linux/x86_64/amd/zen2/modules/all to $MODULEPATH...
Environment set up to use EESSI (2023.06), have fun!
{EESSI 2023.06} [user@system ~]$ # (2)!
{EESSI 2023.06} [user@system ~]$ # (2)!
```

1. What is reported here depends on the CPU architecture of the machine you are
running the `source` command.
2. This is the prompt indicating that you have access to the EESSI software
stack.
What is reported at `(1)` depends on the CPU architecture of the machine you are running the `source` command.
TopRichard marked this conversation as resolved.
Show resolved Hide resolved

The last line is the shell prompt.
At `(2)` is the prompt indicating that you have access to the EESSI software stack.

:clap: Your environment is now set up, you are ready to start running software provided by EESSI!