Expire objects based on last accessed date

[imjasonh]

As a cost-saving measure, currently, the GCS bucket is configured to expire objects one day after creation, which is fine. But it means if someone requests an image every 23h, it'll get rebuilt each time. If two images share a blob, and are created hours apart, the old object might expire before the newer image, leaving the newer image in an inconsistent/unserveable state.

Instead, we could use the lifecycle rule DaysSinceCustomTime and the Custom-Time metadata field to expire objects older than a metadata timestamp, which we can set to the last time the object was accessed.

In serve.Blob we could just also update the Custom-Time metadata value to time.Now() (or do it in a background task)

The Custom-Time metadata value can't be set to an older date than its current date, and can't be unset.

[imjasonh]

I don't remember where I saw it, but there's a Bazel remote executor that runs on Cloud Run, that caches artifacts based on last access, where the way it determines if a blob exists is to set its custom-time to now; if it succeeds, the object exists and the last-used time is updated automatically; if not, you learned the object doesn't exist.

That's a pretty good idea (it means you can't do it async though 😢 ), and kontain.me should do it too.