Deja-vu
This is a continuation of previous discussions. See Documenting the generation of the live images in Debian, Porting the standard image from live-wrapper to live-build and its follow-ups and Replacing live-wrapper for live images by live-build?. Some testing was discussed here: Some Debian Live testing.
Current state
The generation of live images for bookworm and sid have been turned off, see Status of weekly live builds. Sometimes questions are raised about the live images: cdimage weekly-live-builds not regenerating and #1006122
Desired state
It would be nice to have live images for Debian for Bookworm again.
However…
There are two major bottlenecks:
- Human resources
- The purpose/scope of live images
Human resources
As noted in the discussions, the live images cannot be maintained by a single person. It must be maintained by a group. So far nobody has really raised their hand and offered to push forward the support for live images.
I’ve dug into the archives, live-build has some painful history. However, it is a tool that is available at the moment and it is used for several other live-CDs as well.
At release-day the amount of testing live images is huge. It is a lot of manual work to verify the new images. Automating will reduce the work, but it cannot replace running on real hardware.
Two tools will help a lot. They have been added in the past years:
- Jenkins. Ongoing. This generates the images twice and tests whether they are reproducible.
- openQA. In progress. This will test many features of the live image. It simulates a computer from the moment it boots. It does not require special configuration/packages inside the image that is under test.
The purpose/scope of live images
If the live images are to be generated for Bookworm again, it needs to be discussed first what the expectations are for live images.
Some features:
- Installer, with full off-line installation support:
- Debian-installer, to install from the boot menu
- Calamares installer, to install from the live environment
- Internationalization/Localization support: choose your locale/language at boot
- Accessibility support: speech and high-contrast modes
- All major desktops are provided (currently Bullseye has 8)
- Security support
- Will work properly both on-line and off-line
- Easy support for persistence
- The installed software represents a workable system
- The installed software should have as few as possible pre-configured options or special tweaks
- The image must be generated reproducibly
- Unofficial support for non-free firmware
- Support for new blends like Debian Jr.
- Maintenance should be relatively low
- Which tool to use? live-build, live-wrapper, FAI, …
- Hardware: at least amd64. Bullseye also has i386
- Support several installation options:
- Support BIOS and UEFI boot (secure and non-secure)
- Other features, …
- Documentation of the the features mentioned above
Some desired features might contradict other features.
Personally I think that the following features are out-of-scope:
- A small installer -> use FAI instead
- A rescue image -> other solutions exist already
- A small image with only ‘C’ and no i18n/l10n/a18y support (but I think this might be a good addition to the list of current images)
My involvement
So far I’ve been working on:
- Documentation improvements of live-manual
- Reproducibility fixes for live-build (Wiki)
- Some functionality tests in openQA (not published yet, but running in a local, private openQA instance)
- Extracting the rebuild script from Jenkins and including a freshly-built installer (using their daily-build script)
- Gentle prodding
Personally I’m quite hesitant to volunteer, because I don’t know how much time I’ll have in the near future. However, I’m willing to help to get things started again.
Some background (kindly provided by pabs)
A list of derivatives that use live-build: https://wiki.debian.org/Derivatives/CensusFull
Some other tools that can generate live images: https://wiki.debian.org/SystemBuildTools
Note: This information is somewhat outdated.
Where are we now (2022-05-19)?
- live-wrapper is not available in Bookworm any more (#1009282), the request for fixes or alternatives dates from 2018-09.
- live-build can be used to generate reproducible live images.
- FAI has been proposed to generate live images.
- All major desktops have reproducible images (as tested by Jenkins)
- openQA testing is active:
- Activate all GRUB/syslinux menu entries to verify that they perform correctly
- Other openQA tests are planned/under review:
- Verify the correct functionality of some applications in the live environment
- Run the Debian installer
- Run the Calamares installer
- Verify the installed image (both from the boot menu and the checksum)
- I’ve reviewed only amd64. (The snapshot mirror that I use carries only amd64)
- I propose the following testing chain, which enforces reproducibility and functionality:
A) Use Jenkins to generate an ISO image based on the latest timestamp from the snapshot.debian.org/snapshot.notset.fr archive
B) Use Jenkins to generate the image again with the same timestamp
PASS/FAIL: If checksum is identical continue, otherwise abort
C) Run openQA tests
- Test all boot menu entries
- The live environment should start and the main applications should work
- The Debian installer from the boot menu should function properly
- The Calamares installer from the live environment should function properly
- After the installer is finished, the newly installed Debian should boot and the main applications should work
- etc...
PASS/FAIL: If all openQA tests pass (or have softfail) continue, otherwise abort
D) Publish the image on https://www.debian.org/CD/This means that the live images will be published after some delay. (A snapshot needs to exist and the tests also take some amount of time). However, it can be fully automated, and many trivial issues can be caught early, during daily tests.