Disk quota usage data is lost on restart and is not rebuilt during initialization

[j-brewer]

Issue

When using a BlobStore located outside of the default location, GeoWebCache loses all disk quota usage information on restart and fails to repopulate it during the next startup. This issue was observed using GeoWebCache integrated with GeoServer.

Steps to Reproduce

1. Create a BlobStore other than the default and locate it somewhere outside of the default GeoWebCache folder structure (i.e. /stuff/gwc/tilecache).
2. Enable disk quotas, create a cached layer, configure it to use the BlobStore from step 1, and force creation of some tiles (by seeding or by browsing the layer). Verify that cached tiles are created and that the disk quota usage for the cached layer is not zero.
3. Restart GeoWebCache
4. Review the filesystem and note that the cache tiles are still present but the quota usage for the layer is now 0.0.

Analysis

The following code in JDBCQuotaStore.java appears to delete all layers from the cache store during initialization:

https://github.com/GeoWebCache/geowebcache/blob/690be338e2b7d9f5e4fd375639c3a4c0f453a854/geowebcache/diskquota/jdbc/src/main/java/org/geowebcache/diskquota/jdbc/JDBCQuotaStore.java#L148C1-L156

I'm not sure if this is the desired intent or if I'm misunderstanding the code, but the commenting on line 153 suggests it's supposed to keep the layers that are in the configuration.

Once the existing layer(s) are removed, they get re-added on line 165. The QuotaMonitor fires up at some point and starts looking around for layers for which it needs to compute cache usage:

geowebcache/geowebcache/diskquota/core/src/main/java/org/geowebcache/diskquota/DiskQuotaMonitor.java

Line 257 in 690be33

this.cacheInfoBuilder = launchCacheInfoGatheringThreads();

That eventually winds its way to:

geowebcache/geowebcache/diskquota/core/src/main/java/org/geowebcache/diskquota/DiskQuotaMonitor.java

Line 385 in 690be33

cacheInfoBuilder.buildCacheInfo(tileLayer);

Then, we get to what appears to be the heart of the problem at:

geowebcache/geowebcache/diskquota/core/src/main/java/org/geowebcache/diskquota/LayerCacheInfoBuilder.java

Line 91 in 690be33

if (!layerDir.exists()) {

This code is looking for the existence of the layer's cache folder in the "rootCacheDir". Walking that back leads to...

geowebcache/geowebcache/diskquota/core/src/main/java/org/geowebcache/diskquota/DiskQuotaMonitor.java

Line 357 in 690be33

cacheRoot = new File(storageFinder.getDefaultPath());

...which leads to...

geowebcache/geowebcache/core/src/main/java/org/geowebcache/storage/DefaultStorageFinder.java

Line 61 in 690be33

determineDefaultPrefix();

...and...

geowebcache/geowebcache/core/src/main/java/org/geowebcache/storage/DefaultStorageFinder.java

Line 85 in 690be33

private void determineDefaultPrefix() {

Based on all that, it appears the LayerCacheInfoBuilder is looking for the layer's cache folder to exist in the "DefaultPrefix" location instead of the BlobStore location.

Disclaimer

There's a lot of complexity in how all that code works together, and I'm not going to pretend to even remotely understand all of it. It's possible I missed something in the analysis above or there may be good reasons why the code works as implemented. It's also possible this is an issue with the way GeoServer and GeoWebCache integrate. I did want to go ahead and capture what I found since I spent several hours troubleshooting why my quota usage data wasn't being retained across configuration reloads.

Workaround

I was able to work around the problem by setting the GEOWEBCACHE_CACHE_DIR environment variable to the location of the BlobStore I wanted to use and recreating the GeoWebCache configuration there. This would not work if the user needed to make use of multiple BlobStores in non-default locations.

[stijngoedertier]

We are experiencing a similar issue with a minio s3 blob store and a JDBCQuotaStore for 4 GeoServer with built-in GeoWebCache nodes running as a cluster. The blob store is used (tilepage database table), but disk quota are no longer observed (tileset database table) after modifying the disk quota parameters via the GeoServer GUI. In my case disk quota are not lost on restart.