MR #1154 broke REMIND EU21

[0UmfHxcvx5J7JoaOhFSs5mncnisTJJ6q]

Due to hard-coded H12 regions in ./modules/33_CDR/portfolio/input/p33_transport_costs.inc

$ sed -n 's/.*"\([A-Z]\{3\}\)".*/\1/p' ./modules/33_CDR/portfolio/input/p33_transport_costs.inc | uniq 
CAZ
CHA
EUR
IND
JPN
LAM
MEA
NEU
OAS
REF
SSA
USA

That file has to be generated dynamically from mrremind to provide the correct regional data.
REMIND EU21 is inoperable until either that is done or this change is reverted, as are the mandatory model tests.

---

Also, MRs #1154, #1477, #1478 cannot possibly have had valid passing model tests

• All automated model tests pass (FAIL 0 in the output of make test)

@orichters @robertpietzcker

[robertpietzcker]

Also, MRs #1154, #1477, #1478 cannot possibly have had valid passing model tests

    All automated model tests pass (FAIL 0 in the output of make test)

@orichters @robertpietzcker

hm, how can that be? I executed "make test", and I got "fail 0":

any ideas, Michaja?

[0UmfHxcvx5J7JoaOhFSs5mncnisTJJ6q]

any ideas, Michaja?

Sure.
The commit you tested

/p/projects/remind/users/robertp/savePR/2023-11-24_deltacapFixCCS/new_onlyNoCCSdeltacapFx (deltacapFixCCS|?? M)$ git log -n 1 --format="%h"
52e2532

is waaay down there in the commit history of your branch at the point of merging.

$ git log --graph --format="%h | %ch | %s" 52e2532^..96d5a0b
*   96d5a0b | Mon 13:57 | Merge pull request #1478 from robertpietzcker/deltacapFixCCS
|\  
| *   56242a0 | Mon 10:53 | Merge branch 'deltacapFixCCS' of https://github.com/robertpietzcker/remind into deltacapFixCCS
| |\  
| | *   173d040 | Mon 10:12 | Merge branch 'remindmodel:develop' into deltacapFixCCS
| | |\  
| |_|/  
|/| |   
* | | 323d6fc | Fri 17:05 +0000 | Release development version 3.2.1.dev412
* | |   494e2cf | Fri 18:04 | Merge pull request #1154 from katarkow/cdr-cleaning-session
|\ \ \  
| * | | 160616b | Wed Nov 22 11:21 | Merge remote-tracking branch 'upstream/develop' into cdr-cleaning-session
| * | | 69f734e | Wed Nov 22 11:21 | Update documentation in 33_CDR
| * | | a64b7c1 | Tue Sep 12 13:39 | Fix DAC fegas emissions
| * | | d932e1d | Thu Sep 7 16:53 | Merge remote-tracking branch 'upstream/develop' into cdr-cleaning-session
| * | | 9fc8a28 | Wed Jun 14 11:03 | Remove unnecessary input file generated by mrremind
| * | | 42ee456 | Wed Jun 14 10:20 | Changes after PR review
| * | | 17d8148 | Wed Jun 14 10:20 | Fix indentation
| * | | 7ac34f3 | Wed Jun 14 10:20 | Set CDR switches to default values
| * | | 966fdea | Wed Jun 14 10:20 | Add the input dir
| * | | 9a643a6 | Wed Jun 14 10:20 | CDR module cleanup - new realization 'portfolio'
| * | | ee5e3c6 | Wed Jun 14 10:20 | New rlf corresponding to climate zones
| * | | 3b48313 | Wed Jun 14 10:20 | Rename grindrock to weathering
|  / /  
* | | 9aef93f | Fri 15:56 +0000 | Release development version 3.2.1.dev398
* | | b98ce3f | Fri 16:55 | Merge pull request #1476 from orichters/fixtech
* | | 10ad8a1 | Fri 16:17 | fix Tech|*|Capital Costs|w/ Adj Costs on reference run
 / /  
* / 9f609b8 | Mon 10:53 | update CES parameter number
|/  
* ed9acc8 | Fri 20:46 | shift to 2030
* 52e2532 | Fri 15:12 | lower bound on deltcap except ccs

After testing, you did another commit (ed9acc8), and then merged develop into your branch 173d040 (lots of commits), then merged some other branch (56242a0) with a further commit (9f609b8), all of which are potential errors and invalidate your test.

[robertpietzcker]

ah, thanks for the explanation!

I thought I had executed git pull before doing make test, but apparently not

[orichters]

I apparently rebased after testing, which I shouldn't have done.

[0UmfHxcvx5J7JoaOhFSs5mncnisTJJ6q]

I thought I had executed git pull before doing make test, but apparently not

Even if you had, that would not have changed anything.
For one thing, git pull on your deltacapFixCCS branch would have only pulled changes from you upstream tracking branch (github.com:robertpietzcker/remind/deltacapFixCCS), which does not seem to have diverged at that point.
For another, had you merged origin/develop before testing, you would not have merged the changes by Katarzyna, as she merged those changes only three hours later.

Basically you should not change the state of your branch between testing and merging.

[robertpietzcker]

For another, had you merged origin/develop before testing, you would not have merged the changes by Katarzyna, as she merged those changes only three hours later.

Basically you should not change the state of your branch between testing and merging.

Your first point I get.

The second one I don't: My understanding of the work flow I should have followed would have been to update the deltacapFixCCS to remindmodel/remind/develop, then pull, make the test, make the PR, get approval, and then merge.
or am I wrong in this assumption?

now if Kasha had merged her changes after I made my PR but before I got an approval and merged my PR, I would have had to sync with the head in order to be able to merge, right? so in the correctest execution of the work flow, I would have also had to redo the tests...

[0UmfHxcvx5J7JoaOhFSs5mncnisTJJ6q]

The second one I don't: My understanding of the work flow I should have followed would have been to

1. update the deltacapFixCCS to remindmodel/remind/develop,
2. then pull,
3. make the test,
4. make the PR,
5. get approval,
6. and then merge.

or am I wrong in this assumption?

Point 1 gets you all the changes on remindmodel/remind/develop up to that point and is A-OK.
But point 2 would do nothing. Your local branch would be ahead of your remote, so there is nothing from the remote to merge into your local branch.
Points 3 through 6 are fine.
But you did point 5.a. "commit and merge a bunch of stuff that was not tested".

now if Kasha [sic!] had merged her changes after I made my PR but before I got an approval and merged my PR, I would have had to sync with the head in order to be able to merge, right? so in the correctest execution of the work flow, I would have also had to redo the tests...

Depends. Probably you could have merged without any conflicts. EU21 would still fail, but that way #1154 could be reverted. As it is now, there are three sources for Kasia's buggy commit: the merges #1154, #1477, and #1478. Reverting the first one in the row would be an unholy mess and not change the state of develop much.

[robertpietzcker]

Ok, so in principle it is better to merge directly - if possible - and NOT update to the head revision? good to know (maybe briefly mention at the next REMIND meeting for those people like me who only partially grok git

(and as explanation on point 2: I have the work step 2 as I usually do step 1 (the update to the head of remindmodel) in the web interface and then need step 2 to get the stuff to my cluster folder, but that is just my idiosyncratic work flow)

[katarkow]

Hi, sorry for the trouble! I never tested my changes with EU21. I suggest merging #1482 which would temporarily fix this issue while I work on mrremind to remove these hardcoded parameters. How does that sound?

[0UmfHxcvx5J7JoaOhFSs5mncnisTJJ6q]

Ok, so in principle it is better to merge directly - if possible - and NOT update to the head revision? good to know (maybe briefly mention at the next REMIND meeting for those people like me who only partially grok git

Not necessarily. That makes for a messy commit history. (I always rebase before merging.)
The point is to merge what has been tested, not something a dozen commits further along (only some of which one did oneself and has any chance of assessing what they do). And if need be, redo the tests. (They take only 20 minutes. Get a coffee. ;)

(and as explanation on point 2: I have the work step 2 as I usually do step 1 (the update to the head of remindmodel) in the web interface and then need step 2 to get the stuff to my cluster folder, but that is just my idiosyncratic work flow)

You do you.

[katarkow]

Merged the temporary fix, working on mrremind.