Post-conditions of adding a run are not satisfied

[bluenote10]

🐛 Bug

Adding a run should intuitively have certain post-conditions like

• repo.run_exists(run.hash) should be True.
• repo.list_all_runs() should list the added run.

Unfortunately all these post-conditions are currently (temporarily) violated, as the following example demonstrates.

To reproduce

import tempfile

from aim import Repo, Run

with tempfile.TemporaryDirectory() as tmp_dir:
    repo = Repo(tmp_dir, init=True)

    # Pre-condition: No runs are there.
    assert len(repo.list_all_runs()) == 0

    # Add a run
    run = Run(repo=repo)

    # Post-conditions: All these assertions should hold.
    assert repo.run_exists(run.hash)
    assert len(repo.list_all_runs()) > 0

Both assertions are currently failing.

Expected behavior

The assertions should not fail.

Environment

• Aim Version: 3.25.1
• Python version: 3.12.7
• pip version: 24.3.1
• OS: macOS

Additional context

The reason for this behavior is that these methods internally delegate to _all_run_hashes, which unfortunately is using a ttl_cache, which does not get invalidated properly after the change has been made.

aim/aim/sdk/repo.py

Lines 412 to 413 in 9124e64

	@ttl_cache(ttl=0.5)
	def _all_run_hashes(self) -> Set[str]:

This is pretty unfortunately, because the behavior actually depends what happens before, i.e., how long ago the last call to such a function was. These "eventually satisfied post-conditions" are a pretty ugly source of WTF moments for users, because it takes quite some digging to understand why it sometimes works, and sometimes doesn't.

To fix the behavior, it might be possible to invalidate the ttl cache explicitly when a modification has been made to the repository. This should result in consistent behavior, and perhaps would also allow to increase the ttl for more aggressive caching. As per the cachetools docs, invalidating the cache explicitly should be possible by using the clear_cache() function that gets attached to the decorated function.