remind2 (>= 1.137.4) to avoid reporting fail, IIASA tutorial update, bugfix fixOnRef

DESCRIPTION

@@ -50,15 +50,15 @@ Imports:
     nleqslv,
     optparse,
     piamenv (>= 0.4.0),
-    piamInterfaces (>= 0.17.0),
+    piamInterfaces (>= 0.17.4),
     plotly,
     purrr,
     quitte (>= 0.3128.4),
     R.utils,
     raster,
     readr,
     readxl,
-    remind2 (>= 1.135.15),
+    remind2 (>= 1.137.4),
     renv,
     reshape2,
     reticulate,

scripts/output/comparison/varListHtml.R

@@ -31,7 +31,7 @@ mifs <- c(
 
 details <- 
   readr::read_delim(
-    "https://raw.githubusercontent.com/pik-piam/piamInterfaces/master/inst/templates/mapping_template_AR6.csv",
+    "https://raw.githubusercontent.com/pik-piam/piamInterfaces/master/inst/mappings/mapping_AR6.csv",
     delim = ";",
     col_select = c(piam_variable, Definition),
     col_types = "cc"

scripts/output/single/fixOnRef.R

@@ -37,7 +37,7 @@ findRefMif <- function(outputdir, envi) {
   return(refmif)
 }
 
-fixMAGICC <- function(d, dref, startyear, scenario) {
+fixMAGICC <- function(d, dref, startyear, scen) {
   magiccgrep <- "^Forcing|^Temperature|^Concentration"
   message("Fixing MAGICC6 data before ", startyear)
   dnew <-
@@ -47,7 +47,7 @@ fixMAGICC <- function(d, dref, startyear, scenario) {
       filter(d, ! grepl(magiccgrep, .data$variable) |
              .data$period >= startyear)
     ) %>%
-    mutate(scenario = factor(scenario)) %>%
+    mutate(scenario = factor(scen)) %>%
     droplevels()
   return(dnew)
 }

scripts/start/prepare.R

@@ -335,7 +335,7 @@ prepare <- function() {
 
     writeLines(levs, "levs.gms")
 
-    # Replace fixing.gms with level values
+    # Replace fixings.gms with level values
     file.copy("levs.gms", "fixings.gms", overwrite = TRUE)
 
     fixings_manipulateThis <- c(fixings_manipulateThis, list(c(".L ", ".FX ")))

tutorials/13_Submit_to_IIASA_database.md

@@ -8,7 +8,7 @@ Many projects have to upload their scenario data to the database provided by [II
 
 At the beginning of the project, there should be a process to get access to the project Internal Scenario Explorer. In case of problems, contact [Daniel Huppmann](https://iiasa.ac.at/staff/daniel-huppmann).
 
-Model, scenarios and project variables should be registered in the IIASA database. The variable list can be based on the `AR6` template once generate for the IPCC Sixth Assessment Report, or the new template for the NAVIGATE project.
+Model, scenarios and project variables should be registered in the IIASA database. Often, the variable list is based on the `AR6` template once generated for the IPCC Sixth Assessment Report, or the new template for the NAVIGATE project.
 
 A template file containing the list of variables and associated units may be provided as yaml or xlsx file. It can be used to check the variable names and units of your submission. Save it in your REMIND repository, the suggested place is `./output/export/`.
 
@@ -23,19 +23,19 @@ You can generate the file to be uploaded by either calling [`piamInterfaces::gen
 - `iiasatemplate`: optional path to the xlsx or yaml file obtained in the project with the variables and units that are accepted in the database
 - `addToScen`: optional string added in front of all scenario names
 - `removeFromScen`: optional regular expression of parts to be deleted from the scenario names, such as "C_|_bIT|_bit|_bIt"
-- `mapping`: vector of templates from [this directory](https://github.com/pik-piam/piamInterfaces/tree/master/inst/templates) (such as `c("AR6", "AR6_NGFS")` or `c("NAVIGATE", "SHAPE")`) or a local file with identical structure.
+- `mapping`: vector of mappings from [this directory](https://github.com/pik-piam/piamInterfaces/tree/master/inst/mappings) (such as `c("AR6", "AR6_NGFS")` or `c("NAVIGATE", "SHAPE")`) or a local file with identical structure.
 
-Usually, you will find the result in in the `output` subdirectory, but you can adapt this, see  [the function documentation](https://github.com/pik-piam/piamInterfaces/blob/master/R/generateIIASASubmission.R).
+Usually, you will find the result in in the `output` subdirectory, but you can adapt this, see [the function documentation](https://github.com/pik-piam/piamInterfaces/blob/master/R/generateIIASASubmission.R).
 
 Starting from your REMIND directory, you can start this process by running `./output.R`, then selecting `export` and `xlsx_IIASA`. Then choose the directories of the runs you would like to use. This works also for coupled runs, as the `REMIND_generic_*.mif` contains the MAgPIE output since [October 4, 2022](https://github.com/remindmodel/remind/pull/992).
 
 The script requires the inputs as above, expect that it lets you select the `mifs` from a list, and provides additional options:
-- mapping: either the path to a mapping template or a vector of template names such as `c("NAVIGATE", "SHAPE")` referring to the last part of the file names in [this piamInterfaces directory](https://github.com/pik-piam/piamInterfaces/tree/master/inst/templates)
+- mapping: either the path to a mapping or a vector of mapping names such as `c("NAVIGATE", "SHAPE")` referring to the last part of the file names in [this piamInterfaces directory](https://github.com/pik-piam/piamInterfaces/tree/master/inst/mappings)
 - filename_prefix: optional prefix of the resulting outputFile, such as your project name
 
-You can specify the information above in two ways: Either edit [`xlsx_IIASA.R`](../scripts/output/export/xlsx_IIASA.R) and add a project in a similar way to `NGFS_v4` or `ENGAGE_4p5`. You can then start the scripts with:
+You can specify the information above in two ways: Either edit [`xlsx_IIASA.R`](../scripts/output/export/xlsx_IIASA.R) and add a project in a similar way to `NGFS` or `ENGAGE`. You can then start the scripts with:
 ```
-Rscript output.R comp=export output=xlsx_IIASA project=NGFS_v4
+Rscript output.R comp=export output=xlsx_IIASA project=NGFS
 ```
 You do not need to specify `comp` and `output` in the command line, you can just wait to be asked for it.
 An alternative is to specify everything individually as command-line arguments:
@@ -46,13 +46,15 @@ All the information printed to you during the run will also be present in the lo
 
 ## Step 3: check submission
 
-Check the logfile carefully for the variables that were omitted, failing summation checks etc. If you need information on a specific variable such as "Emi|CO2", you can run `piamInterfaces::variableInfo("Emi|CO2")` and it will provide a human-readable summary of the places this variable shows up in mapping templates and summation checks. Running `piamInterfaces::variableInfo("Emi|CO2", template = c("AR6", "mapping.csv"))` allows to compare the mapping with the mapping template with respect to this variable.
+Check the logfile carefully for the variables that were omitted, failing summation checks etc.
+If you need information on a specific variable such as "Emi|CO2", you can run `piamInterfaces::variableInfo("Emi|CO2")` and it will provide a human-readable summary of the places this variable shows up in mappings and summation checks.
+Running `piamInterfaces::variableInfo("Emi|CO2", mapping = c("AR6", "mapping.csv"))` allows to compare your local mapping with the AR6 mapping with respect to this variable.
 
 If you specify `iiasatemplate`, the scripts will delete all the variables not in the template. This can be the reason that summation checks fail, simply because some of the variables that were reported by REMIND were omitted.
 
 Additionally, unit mismatches can cause the script to fail. In the past, IIASA has sometimes changed unit names to correct spelling mistakes or harmonize them.
 If there were unit mismatches where the units are identical, just spelled differently, you can add them to the named vector `identicalUnits` in [`piamInterfaces::checkFixUnits`](https://github.com/pik-piam/piamInterfaces/blob/master/R/checkFixUnits.R).
-So if the project template expects `Mt/yr`, but our templates export it as `Mt/year`, add `"Mt/yr" = "Mt/year"` to the vector, and it will in the future not fail on this unit mismatch but correct it to what is required for the submission.
+So if the project template expects `Mt/yr`, but our mappings export it as `Mt/year`, add `c("Mt/yr", "Mt/year")` to the vector, and it will in the future not fail on this unit mismatch but correct it to what is required for the submission.
 Never use this mechanism if the units are not actually identical in their meaning.
 
 ## Step 4: upload file