Interpreting patch calibration error rates

[tgestabrook]

Hello, I have a question about the patch calibration component of GRASS_FUTURES that I have not been able to find answers for in the documentation:

How should the area_error and compactness_error values be interpreted? For one of the cities I am modeling, area error is very high while compactness error is very low for all parameter combinations. It seems like that should help me diagnose some issue with my setup but I am unsure exactly what.

Any help is greatly appreciated.
calib2022_7.csv

[petrasovaa]

Your discount factor varies between 0.45-0.55, I would suggest to spread it out more. Compactness range seems to be less important than mean, so I typically vary mean and keep range static to keep the number of calibration combinations lower. For example, you could use:

compactness_mean=0.1,0.3,0.5,0.7,0.9 compactness_range=0.05 discount_factor=0.1,0.3,0.5,0.7,0.9

I usually run the calibration twice, first with larger range of values to get a general idea (e.g. the example above) and then look at which values perform better and run a more detailed calibration (e.g., mean=0.1,0.2,0.3).

[tgestabrook]

The attached file is from the second round of calibration; the first was done with almost identical values to those you suggested. In the first run, discount factor values of 0.5 and compactness mean of 0.1 held ranks 1-4, hence the limited values for the next run. Oddly, for that first run, area_error values were either ~0.4-0.5 or ~0.9-1.
calib2022_5.csv

[petrasovaa]

This looks fine to me. The area error is relative to the max area error so it's the larger error in the first csv comparing to the second one makes sense because in the second case you had more widespread values resulting in larger max error. At least that's my interpretation...

[tgestabrook]

Interesting - that makes sense, thank you.