Option to allow disabling of caching when running locally

[bstadlbauer]

So far, in most of our usecases, local caching has caused more headache as it provided benefits. Especially when running tests locally, where a cache hit might conceal an actually broken test or vice versa might hide a fix.

I would suggest something like an environment variable FLYTE_DISABLE_LOCAL_CACHE - depending on what the initial sentiment is here, this could probably go in without an RFC.

[fg91]

I agree that this feature is necessary. As a bandaid, we have a decorator to empty the cache for individual tests.

@contextmanager
def with_empty_flyte_cache():
    """Flyte has no way of easily disabling the cache of tasks just for tests.

    To do this, this decorator can be used to ensure that the local disk cache is empty for tests. This will not prevent
    caching per se (calling a task twice will still use cache if enabled), but should stop inter-task leaks.
    """
    ...

The problem is that users regularly forget to add the decorator to tests they are writing. Sometimes, they make the tests work locally but then they fail in CI because the cache is empty. This has lead to frustration sometimes.

It would be nice if engineers (and CI) could simply do FLYTE_DISABLE_LOCAL_CACHE=1 pytest ....

---

I would say this doesn't need to be an RFC though, could just be a PR.

[davidmirror-ops]

2023-11-09 Contributor's meetup notes: this a good proposal but doesn't meet the requirements for an RFC. @bstadlbauer already opened an Issue to keep track: #4395. Ok to close this entry.