Hi
This is a mass mail to all X2Go lists.
During the past few weeks, I have upgraded the mailing list infrastructure from Mailman 2.1 to Mailman 3.0. All subscriptions and archived messages have been imported. Users will NOT need to resubscribe and will continue to get mails.
Apologies for the long downtime, but especially the archive imports took weeks to complete and had to be done twice, since the backend used in the first run wasn't suited to the load and amount of data.
Unfortunately, the machine is struggling with the load, and web UI operations sometimes take quite long to complete or even time out completely. Just try to refresh and retry the last operation if you see this happening. We will probably want to upgrade this machine in the future.
I'll split this mail up into a user, an administrator and a technical part. Feel free to skip the parts that do not interest you.
- USER-VISIBLE CHANGES *
The old Mailman 2.1 infrastructure was quite basic - consisting of a mostly static HTML-based web interface and statically generated HTML pages for each message in the archives.
By contrast, Mailman 3.x is much more complex. The web interface is more fancy and also uses Javascript, the archiver is a separate component known as Hyperkitty and everything is plugged together via a Python-based Django uwsgi web service communicating with an internal REST service.
Most importantly, you will have to register to the Django-powered Postorius service to manage your subscriptions. Management is now centralized and you won't get password reminders every month for each individual list.
To see a list of all public lists, head to https://lists.x2go.org/postorius/lists/ ( https://lists.x2go.org/ will redirect there).
To create an account, use the "Sign Up" button/link in the upper right corner of the page.
After you have created an account, you will be able to modify all your subscriptions in one place and all your subscriptions should show up automatically (provided the account you signed up for uses the same email address as your subscriptions).
This includes setting digest options, pausing mail transfer, subscribing and unsubscribing.
If you do not wish to create an account, you can still join or leave mailing lists by sending messages to nameoflist-join@lists.x2go.org or nameoflist-leave@lists.x2go.org. nameoflist-subscribe@lists.x2go.org and nameoflist-unsubscribe@lists.x2go.org are aliases that also work. In this case, you will, however, be unable to change other options.
Archives can be found at https://lists.x2go.org/hyperkitty - note that if you're logged in to Postorius, you will also be automatically logged in to Hyperkitty. The archives for a list can also be reached by following the Archive link/button in Postorius's mailing list info page.
In order not to break old links, the old archive is also still accessible at https://lists.x2go.org/pipermail/ - albeit in a read-only state that will not be updated any longer. I have added information to the index pages to reflect that.
- ADMINISTRATION CHANGES *
Mailman 3.x allows making users administrators or owners of lists, which can then change settings, tend to held messages and the like.
Historically, in Mailman 2.1, administration was based upon a password.
I have created a super user account named listadmins@lists.x2go.org with the known password. This user is owner of all lists and can be used to manage every aspect.
As soon as you are logged in to an account that is either and administrator or owner of one or multiple lists, you can use Postorius ( https://lists.x2go.org/postorius/lists ) to manage settings, held messages and the like.
Especially the "held message" interface deserves a bit of explanation.
By default, 10 messages are shown per page. Each message can be selected/ticked individually or you can all select all messages in a page at once by using the checkbox left of "Subject" in the table header.
Clicking the buttons "Accept", "Reject" or "Discard" will carry out the appropriate action on all selected messages. Note, however, that this will not add the sender to any of the auto-accept, auto-reject or auto-discard lists.
In order to also add the sender of a message to a specific auto-action list, click on a message. A javascript-based popup will appear with the whole message text. The popup might not show up instantly, especially if the machine is slow, so please wait after clicking on message! If you get an error about the message not existing, refresh the page and retry. The popup includes two buttons at the bottom to show either the full source of the message or just the headers, which can be very handy. Above the "Rejection reason" text box, a "--- Set member moderation ---" drop down menu can be used to select the desired auto action (hold, reject, discard, accept or default processing). After setting this correctly, use any of the "Accept", "Reject" or "Discard" buttons below the text box to submit the result and carry out the desired action.
Django has a special admin interface at https://lists.x2go.org/admin that can be used to manage data in Hyperkitty (the archiver), for instance to delete messages from the archive. Usually, you will not need to use this admin interface.
- TECHNICAL INFORMATION *
Unfortunately, Mailman 3.x is horribly broken by default in Ubuntu 24.04 (c.f., https://bugs.launchpad.net/ubuntu/+source/mailman-suite/+bug/2062960 and other bug reports).
The mailman version is too old to be compatible with dependencies it needs, leading to a non-working setup out of the box.
Upstream recommends installing mailman via a Python venv, but that would essentially decouple it from the system (and system updates), which I don't like.
For this reason, I backported a lot of code to fix the Mailman 3 as installed via the distro packages.
However, I've only done so purely on the file system and haven't created modified .deb packages.
Please be extra careful when upgrading ymir and please don't update any packages related to Django, Mailman, Hyperkitty or Xapian! This will undo all of my changes and break the whole mailing list infrastructure.
I plan on backporting newer versions from Debian Testing or Unstable to Ubuntu 24.04/noble in a proper .deb-based way, but that's not something I can easily do right away.
Initially, I set up Mailman and Hyperkitty to use SQLite3 as its database backened, but that turned out to be excruciatingly slow.
It's now using the PostgreSQL 16 backend, which works better, but also uses quite a bit of system memory, so the machine's swap space is always in use.
When upgrading PostgreSQL in the future, the databases will also need to be migrated correctly.
Hyperkitty can optionally use a full-text search engine to provide better search capabilities. Initially, I used Whoosh, but it turned out to be rather resource hungry and, unfortunately, also slow. We're now using Xapian with a fix for long terms, which unfortunately has not been applied upstream yet. ( https://github.com/notanumber/xapian-haystack/pull/238 )
I've enabled Markdown rendering mode in Hyperkitty for x2go-announcements, x2go-dev, x2go-user, x2go-user and x2go-i18n. This crashed badly on some mails yesterday, but should now fall back correctly to the basic text renderer in case of failures.
Mihai