Hi
This is a mass mail to all X2Go lists.
During the past few weeks, I have upgraded the mailing list infrastructure from
Mailman 2.1 to Mailman 3.0. All subscriptions and archived messages have been
imported. Users will NOT need to resubscribe and will continue to get mails.
Apologies for the long downtime, but especially the archive imports took weeks
to complete and had to be done twice, since the backend used in the first run
wasn't suited to the load and amount of data.
Unfortunately, the machine is struggling with the load, and web UI operations
sometimes take quite long to complete or even time out completely. Just try to
refresh and retry the last operation if you see this happening. We will probably
want to upgrade this machine in the future.
I'll split this mail up into a user, an administrator and a technical part. Feel
free to skip the parts that do not interest you.
************************
* USER-VISIBLE CHANGES *
************************
The old Mailman 2.1 infrastructure was quite basic - consisting of a mostly
static HTML-based web interface and statically generated HTML pages for each
message in the archives.
By contrast, Mailman 3.x is much more complex. The web interface is more fancy
and also uses Javascript, the archiver is a separate component known as
Hyperkitty and everything is plugged together via a Python-based Django uwsgi
web service communicating with an internal REST service.
Most importantly, you will have to register to the Django-powered Postorius
service to manage your subscriptions. Management is now centralized and you
won't get password reminders every month for each individual list.
To see a list of all public lists, head to
https://lists.x2go.org/postorius/lists/ ( https://lists.x2go.org/ will redirect
there).
To create an account, use the "Sign Up" button/link in the upper right corner of
the page.
After you have created an account, you will be able to modify all your
subscriptions in one place and all your subscriptions should show up
automatically (provided the account you signed up for uses the same email
address as your subscriptions).
This includes setting digest options, pausing mail transfer, subscribing and
unsubscribing.
If you do not wish to create an account, you can still join or leave mailing
lists by sending messages to nameoflist-join(a)lists.x2go.org or
nameoflist-leave(a)lists.x2go.org. nameoflist-subscribe(a)lists.x2go.org and
nameoflist-unsubscribe(a)lists.x2go.org are aliases that also work. In this case,
you will, however, be unable to change other options.
Archives can be found at https://lists.x2go.org/hyperkitty - note that if you're
logged in to Postorius, you will also be automatically logged in to Hyperkitty.
The archives for a list can also be reached by following the Archive link/button
in Postorius's mailing list info page.
In order not to break old links, the old archive is also still accessible at
https://lists.x2go.org/pipermail/ - albeit in a read-only state that will not be
updated any longer. I have added information to the index pages to reflect that.
**************************
* ADMINISTRATION CHANGES *
**************************
Mailman 3.x allows making users administrators or owners of lists, which can
then change settings, tend to held messages and the like.
Historically, in Mailman 2.1, administration was based upon a password.
I have created a super user account named listadmins(a)lists.x2go.org with the
known password. This user is owner of all lists and can be used to manage every
aspect.
As soon as you are logged in to an account that is either and administrator or
owner of one or multiple lists, you can use Postorius (
https://lists.x2go.org/postorius/lists ) to manage settings, held messages and
the like.
Especially the "held message" interface deserves a bit of explanation.
By default, 10 messages are shown per page. Each message can be selected/ticked
individually or you can all select all messages in a page at once by using the
checkbox left of "Subject" in the table header.
Clicking the buttons "Accept", "Reject" or "Discard" will carry out the
appropriate action on all selected messages. Note, however, that this will not
add the sender to any of the auto-accept, auto-reject or auto-discard lists.
In order to also add the sender of a message to a specific auto-action list,
click on a message. A javascript-based popup will appear with the whole message
text. The popup might not show up instantly, especially if the machine is slow,
so please wait after clicking on message! If you get an error about the message
not existing, refresh the page and retry. The popup includes two buttons at the
bottom to show either the full source of the message or just the headers, which
can be very handy. Above the "Rejection reason" text box, a "--- Set member
moderation ---" drop down menu can be used to select the desired auto action
(hold, reject, discard, accept or default processing). After setting this
correctly, use any of the "Accept", "Reject" or "Discard" buttons below the text
box to submit the result and carry out the desired action.
Django has a special admin interface at https://lists.x2go.org/admin that can be
used to manage data in Hyperkitty (the archiver), for instance to delete
messages from the archive. Usually, you will not need to use this admin interface.
*************************
* TECHNICAL INFORMATION *
*************************
Unfortunately, Mailman 3.x is horribly broken by default in Ubuntu 24.04 (c.f.,
https://bugs.launchpad.net/ubuntu/+source/mailman-suite/+bug/2062960 and other
bug reports).
The mailman version is too old to be compatible with dependencies it needs,
leading to a non-working setup out of the box.
Upstream recommends installing mailman via a Python venv, but that would
essentially decouple it from the system (and system updates), which I don't like.
For this reason, I backported a lot of code to fix the Mailman 3 as installed
via the distro packages.
However, I've only done so purely on the file system and haven't created
modified .deb packages.
Please be extra careful when upgrading ymir and please don't update any packages
related to Django, Mailman, Hyperkitty or Xapian! This will undo all of my
changes and break the whole mailing list infrastructure.
I plan on backporting newer versions from Debian Testing or Unstable to Ubuntu
24.04/noble in a proper .deb-based way, but that's not something I can easily do
right away.
Initially, I set up Mailman and Hyperkitty to use SQLite3 as its database
backened, but that turned out to be excruciatingly slow.
It's now using the PostgreSQL 16 backend, which works better, but also uses
quite a bit of system memory, so the machine's swap space is always in use.
When upgrading PostgreSQL in the future, the databases will also need to be
migrated correctly.
Hyperkitty can optionally use a full-text search engine to provide better search
capabilities. Initially, I used Whoosh, but it turned out to be rather resource
hungry and, unfortunately, also slow. We're now using Xapian with a fix for long
terms, which unfortunately has not been applied upstream yet. (
https://github.com/notanumber/xapian-haystack/pull/238 )
I've enabled Markdown rendering mode in Hyperkitty for x2go-announcements,
x2go-dev, x2go-user, x2go-user and x2go-i18n. This crashed badly on some mails
yesterday, but should now fall back correctly to the basic text renderer in case
of failures.
Mihai
