Conditional for NEON usermods is too broad

[wwieder]

This statement is still true in a BGC case because of the extra wildcard

https://github.com/ESCOMP/CTSM/blob/bb2a8d2c0c05ebb5417724f98fb0586dc7584ae6/cime_config/usermods_dirs/NEON/defaults/shell_commands#L35C1-L36C1

Something like the following would be more accurate

if [[ $compset =~ .*CLM[0-9]+%SP* ]]; then

[ekluzek]

The one problem there is that SP isn't always immediately after the % in the compset name. One way to do this would be to have a wildcard imbetween the % and "SP" that doesn't match underscores.

So maybe...

if [[ $compset =~ .*CLM[0-9]+%[^_]*SP.* ]]; then

[wwieder]

Yes! That suggestion works @ekluzek

[ekluzek]

It looks like my comment only applies to the FATES-SP compset which doesn't matter now, and the NWP compsets like: 2000_DATM%NLDAS2_CLM50%NWP-SP_SICE_SOCN_MOSART_SGLC_SWAV (I2000Ctsm50NwpSpNldas)

[ekluzek]

These regular expressions are notoriously tricky, but for longevity good to get right.

[wwieder]

I don't understand why, but @ekluzek's suggestion if [[ $compset =~ .*CLM[0-9]+%[^_]*SP.* ]]; then works as intended when creating a clone, but not for using run_neon.py for a new case and building it?

[briandobbins]

My guess (and I'll fix this momentarily so you can try again) is that you used a notebook doing 'run_neon', which runs the *central* copy of the source code -- I only updated the regex in the copies in everyone's home directory. If that's the case, give it two minutes and try again.

…

On Thu, Jun 22, 2023 at 6:23 PM will wieder ***@***.***> wrote: I don't understand why, but @ekluzek <https://github.com/ekluzek>'s suggestion if [[ $compset =~ .*CLM[0-9]+%[^_]*SP.* ]]; then works as intended when creating a clone, but not for using run_neon.py for a new case and building it? — Reply to this email directly, view it on GitHub <#2039 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACL2HPK4OVSF4MHGEFE5GQ3XMTOWXANCNFSM6AAAAAAZQXHYJE> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[briandobbins]

Updated. And, just to be more clear, a './run_neon.py' from your ~/CTSM directory should've worked fine, whereas 'qcmd -- run_neon ...' would've launched via the central directory, and not had that regex fix. (It does now.)

[wwieder]

One more issue that needs correction is for shell commands at SJER. Which should look like this example from TEAK.

Were there other manual changes you made for the tutorial @briandobbins and @TeaganKing ?
@ekluzek is this the right place to log these other NEON bugs that can be addressed in #2044?

[TeaganKing]

There was a CTSM shell command update that needs to be updated in the cloud instance specifically in order to switch off the mpi-serial option, but that change is not desired for the general CTSM use. Perhaps there would be some way to have an if statement for the case when we are on the cloud versus on Cheyenne, or we could continue updating the CTSM version used in the cloud.

Other than that, there were a few tutorial changes that were already pushed to the repository prior to Flux Course, and some manual changes to the scripts used for setting up the AWS instance (but not involving the CTSM nor tutorial repositories).

If we are making some additional updates, can we also make the run_neon.py update to ensure that we use is not instead of != and avoid printing additional warnings? This is also mentioned in this issue

[ekluzek]

@wwieder @TeaganKing yes let's collect all the little things you had to do for the tutorial here. So please add anything you haven't mentioned yet. I'll organize the things you've already mentioned and make sure they are coming in the new PR.

@briandobbins is there a way to detect if the cloud is being used?

[wwieder]

@ekluzek are there any upcoming PRs that we can include these fixes for NEON usermods?

[ekluzek]

I've got this fix started in #2044 and it's listed in upcoming tags as another set of simple fixes.

[wwieder]

@ekluzek we didn't get to discuss when your long list of simple fixes will be coming in the upcoming tag queue. Is this something that's basically ready to go? I'm asking because we need to update the NEON container, and it would be good have have this merged to main.

Thanks
Will

[ekluzek]

@wwieder the plan is to get back to that ASAP, which should be right away. When do you need to update the NEON container? I hope we can coordinate this and make it happen when you need it...

[wwieder]

Thanks, @ekluzek. We can't update the container until Brian is back, but after he returns I'm hoping to be able to do this quickly.