Split docker-clean for BuildKit cache

[kohtala]

Apt maintains two directories Dir::State "var/lib/apt" and Dir::Cache "var/cache/apt" that could benefit from BuildKit RUN --mount=type=cache.

Use of the cache requires removal of /etc/apt/apt.conf.d/docker-clean to remove definitions of DPkg::Post-Invoke and APT::Update::Post-Invoke.

Some builds have additional sources in /etc/apt/sources.list.d/, resulting in different /var/lib/apt/lists/. But regardless of that difference in lists, it is useful to share the /var/cache/apt/archives cache of .deb downloads. However since Dir::Cache::srcpkgcache "" and Dir::Cache::pkgcache "" were in the same removed /etc/apt/apt.conf.d/docker-clean, these caches get updated needlessly in /var/cache/apt/.

So it would be useful to have Dir::Cache::srcpkgcache "" and Dir::Cache::pkgcache "" in a separate docker-no-pkgcache, for example.

Since apt is not good on concurrent access to the caches, the caches need to be used with sharing=locked. Therefore this is only a minor issue.

[tianon]

Interesting! Given that this is a somewhat advanced edge use case, I'm inclined to suggest something like the following instead of just deleting the file outright:

RUN sed -ri -e '/pkgcache/d' /etc/apt/apt.conf.d/docker-clean

[kohtala]

Yes, something like that occurred to me, but I did not think further because the documentation already has established the removal of this file. I could have been too pessimistic, because it has such an obvious place to spread best practice and they surely would accept a pull request.

To think of it, the added Binary::apt::APT::Keep-Downloaded-Packages "true" could perhaps be added in the same file. Have the sed command change to something like

RUN sed -i -e '/#BuildKitRm/,/#BuildKitRm/d' -e 's/#BuildKit: *//' /etc/apt/apt.conf.d/docker-clean

would convert docker-clean like this

#BuildKitRm
DPkg::Post-Invoke { "rm -f /var/cache/apt/archives/*.deb /var/cache/apt/archives/partial/*.deb /var/cache/apt/*.bin || true"; };
APT::Update::Post-Invoke { "rm -f /var/cache/apt/archives/*.deb /var/cache/apt/archives/partial/*.deb /var/cache/apt/*.bin || true"; };
#BuildKitRm

Dir::Cache::pkgcache "";
Dir::Cache::srcpkgcache "";

#BuildKit: Binary::apt::APT::Keep-Downloaded-Packages "true";

to this for BuildKit in one command

Dir::Cache::pkgcache "";
Dir::Cache::srcpkgcache "";

Binary::apt::APT::Keep-Downloaded-Packages "true";

And it could nicely accommodate further changes to docker-clean in base images.