instructions on docker.debian.net incomplete?

[h01ger]

first, thanks a lot for debuerreotype and docker.debian.net!
second, https://bugs.debian.org/959446 made me file this issue here.

so, https://docker.debian.net has (very brief) instructions how to rebuild the docker images,
so I tried:

$ git clone https://github.com/debuerreotype/debuerreotype/
$ cd debuerreotype/
$ ./examples/debian.sh --arch amd64 out/ 'bullseye' '@1659312000'

but this fails because debuerrotype-init cannot be found. Looking at the Debian debuerreotype package
(which I havent used because it's 0.14 only and docker.debian.net suggested 0.15 which isnt packaged yet...) I see it resides in /usr/sbin, so long story short, do I need to run the above command as root?

Ultimatively I like to setup a rebuilder for docker.debian.net to independently rebuild those images
while reproducing the same checksum! :-)

[tianon]

first, thanks a lot for debuerreotype and docker.debian.net! second, https://bugs.debian.org/959446 made me file this issue here.

❤️!

so, https://docker.debian.net has (very brief) instructions how to rebuild the docker images, so I tried:

$ git clone https://github.com/debuerreotype/debuerreotype/ $ cd debuerreotype/ $ ./examples/debian.sh --arch amd64 out/ 'bullseye' '@1659312000'

but this fails because debuerrotype-init cannot be found. Looking at the Debian debuerreotype package (which I havent used because it's 0.14 only and docker.debian.net suggested 0.15 which isnt packaged yet...) I see it resides in /usr/sbin, so long story short, do I need to run the above command as root?

😅 not having 0.15 in Debian yet is because I've been away from my PGP key for a bit so couldn't test/upload, but I appreciate the reminder! I will get that uploaded to unstable today.

In rebuilding bullseye, the only difference between 0.14 and 0.15 should be the URLs in sources.list, but that's not super useful if you want to reproduce the checksums. 😄

When you run debuerrotype-init, it invokes debootstrap (and chroot), so yeah, it needs to run as root. There were some experiments around letting fakeroot/fakechroot do useful things, but the details are a little bit complicated (and I normally run the tool in a container anyhow, so it's not a huge priority for me); if you want to pull that thread more, you can find my notes/thoughts/testing in debuerreotype/debuerreotype#118. 😄 (It honestly might be easier to use rootless Docker or Podman or something similar to get a rootful environment with user namespaces if this is important to you.)

I run the tool/scripts via ./docker-run.sh which is mostly setting up an appropriate environment with the necessary permissions - the Dockerfile it builds has a few fiddly bits, but they're mostly pretty straightforward (downloading a more recent distro-info-data than bullseye has normally, for example).

Ultimatively I like to setup a rebuilder for docker.debian.net to independently rebuild those images while reproducing the same checksum! :-)

I love this! I'm happy to support you however I can. 😄

I've done a few experiments where I download not just the tarballs published to GitHub, but the actual image rootfs layers from Docker Hub and with some light repacking via debuerreotype-tar I was able to reproduce the original checksums of the tarballs I'd published (some bits of them change in the round-tripping through the build process). Eventually including something that goes that far could also be really interesting, IMO. 👀

---

Edit: 👀

debuerreotype_0.15-1_source.changes ACCEPTED into unstable

[tianon]

(I guess I should also note that you should be careful with long-EOL releases outside containers because some of them like to try to do things like SIGHUP to pid1 in order to reload the init system 😅)

[h01ger]

awesome! just saw the 0.15-1 upload, thank you! will do further tests soon and report back here. (though probably not tomorrow but rather the day after...)

[h01ger]

so, I created a podman bookworm container, upgraded to sid, installed debuerreotype from sid, cloned the git repo and as root ran

./examples/debian.sh --arch amd64 out/ 'bullseye' '@1659312000'

which did some things and then failed with

+ debuerreotype-init --arch amd64 --debian --keyring /tmp/debuerreotype.bullseye.WScm0RWKCv/debian-archive-bullseye-keyring.gpg --no-merged-usr /tmp/debuerreotype.bullseye.WScm0RWKCv/rootfs bullseye @1659312000
unshare: unshare failed: Operation not permitted

I've created the podman container using this command on bullseye:

podman run --rm -it debian:bookworm bash

These are the command I ran in the container:

apt update && apt install vim debuerreotype git
vi /etc/apt/sources.list # switched to sid
apt update && apt upgrade
git clone https://github.com/debuerreotype/debuerreotype/
cd debuerreotype/
mkdir out
./examples/debian.sh --arch amd64 out/ 'bullseye' '@1659312000'

[h01ger]

you suggested to use podman, so I any idea what went wrong? inside the container I ran everything as root...

[tianon]

I mentioned Podman for inclusivity, not because I use or test it 😅 😇

For your sake (:heart:), I spent some time messing with podman, and I've managed to make the unshare work (--cap-add SYS_ADMIN), but debootstrap continuously fails with EPERM on mknod (can reproduce with something much simpler like mknod dev-null c 1 3) -- I've tried all kinds of combinations of --cap-add, --security-opt, --privileged, etc and can't seem to get this command to work inside podman, so I'm not sure whether there's much more that can be done here without someone who knows podman better than I do. 😬 🙈
(However, containers/podman#4619 (comment) is not promising 😞)

---

The way I'd normally invoke that command would be something like this:

$ ./docker-run.sh ./examples/debian.sh --arch amd64 out/ 'bullseye' '@1659312000'

(From the host, which then docker-run.sh will build/run a container via Docker -- it also has a few flags for controlling how it works, whether to docker build, which image to use, whether to bind-mount the directory into the container, etc - it can also be run with no arguments to drop into an interactive shell.)

---

That being said, if you're willing to have a degree of trust, running as root on a host system should be reasonably safe, especially if you're only going to be trying to build recent releases (buster+ for example is totally safe and reasonable -- it's only down at the levels of slink and potato that I've seen things get really haywire, and that far down things get complicated for other reasons too like being 32bit only and balking at inodes that are too big 😂).

[h01ger]

Thanks for your testing and these suggestions and I'm sorry for the lack of reply here, I got distracted building reproducibility tests for mmdebstrap, debootstrap and cdebootstrap as I found out they all can bootstrap Debian reproducibly too. :) Will soon look into this again!