From bb5781e730c85a0bd40dca50564c540802f34e55 Mon Sep 17 00:00:00 2001 From: Bruno Carrez Date: Mon, 17 Jul 2023 03:17:02 -0400 Subject: [PATCH 1/2] add manual procedure for account matching --- docs/account_matching_manual_procedure.md | 109 ++++++++++++++++++++++ 1 file changed, 109 insertions(+) create mode 100644 docs/account_matching_manual_procedure.md diff --git a/docs/account_matching_manual_procedure.md b/docs/account_matching_manual_procedure.md new file mode 100644 index 00000000..7ca38acc --- /dev/null +++ b/docs/account_matching_manual_procedure.md @@ -0,0 +1,109 @@ +# Account Matching procedure + +--- +# In a nutshell + +The manual procedure steps are: + +**Once a month** (or each time we have new users files from DRAC): + +- Copy the two DRAC files (`users` and `roles`) in the `secrets/account_matching` folder +- Adjust the `account_matching` section of the config file to point to these two files +- run the `acquire users` command +- if any errors, adjust the exceptions in the `secrets/make_matches_config.json` file, and re-run the account matching script. + +--- +# In details + +This procedure consists in making matches between the DRAC user account and the Mila acocunts. + +- Mila accounts are gathered directly by connecting to Mila LDAP. +- On the other hand, DRAC accounts must be provided manually. + +After that, the `users` collection of mongoDB contains the aggregated users database. + +## Access rights + +The operator executing the account matching procedure must have wite access to the DRAC folder. + +Two possible scenarios : + +- have write access to the running SARC server (production server) +- use SARC from a local machine, with a SSH connection to the production server (see below). **This is the prefered method.** + +### Remote access to MongoDB (via SSH tunneling) + +#### SSH config +Refer to `remote_ssh_access.md` for ssh connection with port redirection, to connect to mongoDB form the local machine. + +#### SARC config file +To use the remote mongoDB connection, tunneled from localhost:27018, the `mongo` section in the config file like this: + +``` + "mongo": { + "connection_string": "127.0.0.1:27018", + "database_name": "sarc" + }, +``` + +## data source 1: Mila LDAP credentials + +The credentials for the Mila LDAP are in the `secrets/ldap` folder. + +They are refered to in the ldap section of the sarc config file : +``` + "ldap": { + "local_private_key_file": "secrets/ldap/Google_2026_01_26_66827.key", + "local_certificate_file": "secrets/ldap/Google_2026_01_26_66827.crt", + "ldap_service_uri": "ldaps://ldap.google.com", + "mongo_collection_name": "users" + }, + +``` + +## data source 2: DRAC account files + +Compute Canada must provide 2 CSV files: +- One "members" file +- One "roles" file + +### copy the files in the right directory + +The two file must be copied to the `secrets/account_matching/` folder of SARC, on the server or the local machine, depending on the scenario. + +### Configuration file + + + +## Exceptions handling + +The exception are manually handled in the `secrets/make_matches_config.json` file. + +``` +{ + "L_phantom_mila_emails_to_ignore": + [ + "ignoreme@mila.quebec", + "idontexistanymore@mila.quebec" + ], + "D_override_matches_mila_to_cc_account_username": + { + "john_doe@mila.quebec": "jdoe01", + "janedoe@mila.quebec": "unguessableusername" + } +} +``` +The `L_phantom_mila_emails_to_ignore` list contains the Mila emails present in the DRAC users listings that do not exist in the Mila LDAP. + +The `D_override_matches_mila_to_cc_account_username` dictionnary is used to bypass the automatic matching algorythm, when no link can be made with the name or the email address. + +The procedure is: +- run the matching script +- if there are mathcing errors, modify `make_matches_config.json` accordingly and re-run the matching script. + +## Run the matching script + +From the SARC folder: +``` +$ SARC_CONFIG= poetry run sarc acquire users +``` \ No newline at end of file From 5fb93fda67095af924132d24d0651008e69b16b0 Mon Sep 17 00:00:00 2001 From: Bruno Carrez Date: Mon, 17 Jul 2023 03:22:45 -0400 Subject: [PATCH 2/2] Remote mongo access (ssh port tunneling) --- docs/account_matching_manual_procedure.md | 2 +- docs/remote_mongo_access.md | 22 ++++++++++++++++++++++ 2 files changed, 23 insertions(+), 1 deletion(-) create mode 100644 docs/remote_mongo_access.md diff --git a/docs/account_matching_manual_procedure.md b/docs/account_matching_manual_procedure.md index 7ca38acc..a8b86757 100644 --- a/docs/account_matching_manual_procedure.md +++ b/docs/account_matching_manual_procedure.md @@ -34,7 +34,7 @@ Two possible scenarios : ### Remote access to MongoDB (via SSH tunneling) #### SSH config -Refer to `remote_ssh_access.md` for ssh connection with port redirection, to connect to mongoDB form the local machine. +Refer to `remote_mongo_access.md` for ssh connection with port redirection, to connect to mongoDB form the local machine. #### SARC config file To use the remote mongoDB connection, tunneled from localhost:27018, the `mongo` section in the config file like this: diff --git a/docs/remote_mongo_access.md b/docs/remote_mongo_access.md new file mode 100644 index 00000000..68c6457e --- /dev/null +++ b/docs/remote_mongo_access.md @@ -0,0 +1,22 @@ + +# MongoDB port tunneling + +With an SSH access to the production machine, you can easilly tunnel the mongoDB. Example in the `~/.ssh/config` file: + +``` +Host sarc + (...) + LocalForward 27018 127.0.0.1:27017 +``` + +# SARC config file + +Simply modify the config JSON file you use: + +``` + "mongo": { + "connection_string": "localhost:27018", + "database_name": "sarc" + }, + +``` \ No newline at end of file