Thunderbird email client update policy

Hello, hope this is the right category to post this. I am reposting from a comment of mine from this launchpad issue, since I was told this was the correct place to post this.

Original post follows.


Thanks for seeing this through, but let me raise some important points.

Why is Thunderbird not covered by the same update policy as Firefox? Sure it’s not a web browser, but it’s one of the most important and widely used GUI email clients. Also, it is a close cousin of Firefox.

These updates took way too long to come out for LTS releases, and they are/were already obsolete once they did roll out. For example, the 78.9.x and 78.10.x releases are more than a month old at this point, and both contain fixes for “high” severity vulnerabilities:

https://www.mozilla.org/en-US/security/advisories/mfsa2021-12/
https://www.mozilla.org/en-US/security/advisories/mfsa2021-14/

I think Ubuntu should make a greater effort to keep up with Thunderbird upstream.

1 Like

One fairly painless way to speed up the process: Ubuntu is volunteer-driven.

Any volunteer can help the Ubuntu Security Team triage these CVEs, help package, and test the packages.

So I do use TB snap, which is up-to-date.
image

1 Like

I can’t directly answer the question, but I would like to clarify a few things to make sure that there’s no misunderstanding or mismatched expectations here. Note that I don’t speak for Canonical or for the desktop team. I’m just making some observations that I hope will be useful to help frame any discussion about this.

1. It’s really difficult to backport Thunderbird as a deb.

This particular backport involved backporting a newer nasm and Rust toolchain, dealing with the deprecations of jsunit and enigmail, handling upgrade paths for these, and so forth. The way debs work, doing this “backwards” in an older release without disrupting existing unaffected users on that release is really difficult. Debs are tightly integrated, but this gets in the way when backporting. Dependencies have to be untangled, and putting newer stuff in has to be deconflicted from existing stuff that is there already. It takes a disproportionately large amount of time and only particularly skilled and experienced developers can do it right. This has a big bearing on my third point below.

Amongst other problems, snaps solve this problem of making the latest of something available in an old release. A Thunderbird snap is already available and is kept up-to-date. Compared to the difficulty of backporting a deb, making a snap available to users of all releases at once is easy. If users don’t want to use it, that’s up to them; my point is that what you’re asking for is essentially asking developers to do much more work.

2. “Update policy” is ambiguous; let’s not conflate technical policy with resource allocation.

In Ubuntu there was no issue getting Thunderbird approved to be updated in the same way as Firefox in this case. When the relevant leaders (myself, amongst others) were asked for the exception, they promptly approved the request. The delay was in getting the details right technically, and that happened because the task was technically difficult to do. If you want it sped up, what’s needed is more highly skilled and experienced developers with more available time. No change in policy is required, unless by policy you mean policy on priorities.

3. Anyone can work on this, and capable volunteers would be welcome and appreciated.

As Ian says, Ubuntu is volunteer-driven. The Ubuntu Code of Conduct (in my opinion a misnomer because it’s also effectively a constitution) says:

We invite anybody, from any company, to participate in any aspect of the project. Our community is open, and any responsibility can be carried by any contributor who demonstrates the required capacity and competence.

This means that one way of getting Thunderbird (deb) updates into 18.04 faster is for someone capable to volunteer to deliver them faster.

However, a warning about volunteering: dealing with the complexities of backporting Thunderbird updates is not something I expect newcomers to be able to competently. You’d need to ramp up skills and experience on easier tasks first. That’s a big ask for any volunteer. And from the other side, I don’t expect experienced developers to be prepared to spend their time helping you along diving in to something so complex if you’ve not already got a track record that demonstrates your competence[1]. In my experience, having a new volunteer dive in to something like this never ends well, and only consumes significantly more mentoring time than it would take for the mentor to do it themselves multiple times over. So as a mentor I would expect a new volunteer to start with smaller tasks and demonstrate commitment and competence that way first. I hope the community would find this reasonable.

4. The Canonical desktop team’s priorities are up to them

Reminder: I’m not speaking for Canonical or for the Desktop Team here.

In practice, a lot of the “grunt work” in maintaining a distribution is something that is difficult to find volunteers to do, and this is where Canonical comes in. However, the reality of the world is that resources are limited, and adding resource in one area means taking resource away from somewhere else. Let’s face it: most users use hosted web-based email nowadays. But everyone needs a working browser on their desktop. Firefox is the primary, default, supported web browser on Ubuntu. So, naturally, I’d expect maintenance of Firefox to get more development attention than the maintenance of Thunderbird.

It’s very easy to say “more engineering time should be spent on X”. But only the team responsible, or those funding them, are in a position to be able to determine what they have the capacity to do, and how they arrange their priorities.

Conclusions

(these are the conclusions of my framing of this discussion, not of the discussion itself)

It’s perfectly valid to ask the Canonical desktop team to adjust their priorities. My point, though is that this is what your request comes down to: you’d like the Canonical desktop team to do more of these updates quicker. There’s no other kind of policy adjustment or other type of change relevant here. It’s purely a question of engineering priority. However, this request has to be balanced with:

  • Many (most?) Ubuntu users use hosted web-based email and not Thunderbird nowadays.
  • As an upstream, Thunderbird don’t maintain stable releases, like Firefox and Chromium but unlike most other software in our ecosystem, which makes backporting Thunderbird to stable releases particularly difficult.
  • A promptly updated snap is available, because that’s much easier to do.
  • 18.04 isn’t the latest LTS.
  • The desktop team clearly has other priorities, too.

Finally, my subjective opinion: it seems to me that in this new world of some upstreams not supporting stable releases but instead shipping rolling releases with regular new dependency requirements, backporting for the life of the LTS is becoming increasingly impractical. For these particular upstreams, we’ll have to resort to the bundling of dependencies in the end. Debs are simply not designed for that; it’s like trying fit a square peg in a round hole. Sooner or later, it’s going to become too impractical to continue this way. What we need is a packaging format that allows the bundling of dependencies. And we have that: snaps. Meanwhile, volunteers who are prepared to suffer the pain to deliver deb backports will be able to continue to do so. In practice, we’ve found that nobody volunteers.


[1] This isn’t necessarily Ubuntu-specific; an experienced uploading Debian Developer would also be capable.

10 Likes

Apps that aren’t part of the core desktop should just be pushed out of main and shipped as snaps. Both Thunderbird and Mailspring are high quality open source mail apps and have been receiving regular updates.

As of this post Thunderbird has had 126(!) updates and Mailspring has had 505(!) updates, it’s unreasonable to expect volunteers to keep up with this pace to backport back to old distros. An entire class of problems would just go away and the desktop team could concentrate on doing desktop things instead of duplicating work.

6 Likes

That issue finally pushed me to Arch Linux / EndeavourOS after years of delayed package updates in Ubuntu LTS I created issue tickets for ^^

Ubuntu should at least improve its system backporting stuff into still supported ubuntu releases.
(In my case I did always jump on the newest LTS)

1 Like

I think you may have a fundamental misunderstanding here. Unlike Ubuntu, Arch Linux is a rolling release distribution. That is: it favours getting the latest upstream updates to users, including major behaviour-changing updates, instead of striving to keep the same behaviour for the lifetime of specific stable releases. So Arch Linux developers do not need to care about changing behaviour for their users, because that’s not what it’s intended for.

In contrast, Ubuntu deliberately avoids changing the stable release according to its stable release updates policy. This is what we mean by “stable”! If Arch works better for you, then Ubuntu LTS was certainly the wrong distribution to be using, because it specifically aims to do the exact opposite.

If, by “stuff” you mean the latest and greatest features, then that’s exactly the opposite of what Ubuntu LTS intends to do, and you want to be using a rolling release distribution such as Arch Linux, as you’ve found. However, as a layer on top, snaps do allow Ubuntu LTS users to get the latest and greatest of the specific thing they care about, in a safe way, without impacting general system stability.

If by “stuff” you mean important bugfixes, then we do want those in Ubuntu stable releases, but unlike rolling distributions where it’s acceptable to change behaviour for users, we have to do it carefully to ensure that we do not disrupt users who do not expect disruption. Comparing Ubuntu to Arch LInux in this respect is like apples and oranges. If you frame bugfix requests in stable Ubuntu releases as “please update package X to version Y”, you mostly won’t get anywhere, because the general principle is to land important bugfixes only (full details in the page I linked).

4 Likes

Thank you Robie for a comprehensive and well articulated response.

I do speak for the Desktop Team, and I can confirm that this particular update (from the 68 series to the 78 series) was indeed tricky to get right and disrupting. It involved people on several teams (desktop, security, SRU) and the technical board (to approve package removals), it required backporting build dependencies, and extensive testing. I acknowledge it was long in coming, it ideally should have happened earlier, but there were other priorities to deal with.

Now that this major update to 78 is (almost) done in bionic (please help test the upgrade with packages in bionic-proposed), regular security updates in the same series should flow into the security pockets much more timely.

6 Likes

Honestly in my usage I have swtiched over from traditional deb version to the snap version. Since doing that I have not noticed any difference and it works just as has always done for me. Which also meant I was up-to-date with the latest version as it was released.

1 Like

I recognize and understand the difficulty of the upgrade. But if some piece of software is both important and difficult to backport, then it follows that more resources should be allocated to deal with the task. If this is not feasible, then something else has got to change (more on this below). Now of course I’m not in a position of telling Canonical how to best allocated their resources. I’m just sharing my point of view.

I also recognize my argument (and potential refutations for it) hinges on hard data that we don’t have:

However, I would say that even if Thunderbird users are a minority relative to web-based clients users, the majority of power/serious users who have to get real work done don’t use a web-based clients. Casual users will be happy with anything, but web clients objectively suck.
Everyone I know that uses email more than once in blue moon uses a proper email client, such as Thunderbird, Outlook, etc. Email is still extremely important and widely used in professional and non-professional contexts, and telling users “just use the web interfaces because many also do so” is not a viable strategy. Plus, it creates friction for potential migrations from Outlook, for instance - users expect an up-to-date alternative at least.

Moot point IMO since what most people care about is the latest releases of Firefox/Chromium, not their ESRs.

I don’t care as much for 18.04 as for 20.04 precisely because of that. So I think it is reasonable that the update first came to 20.04. However, the update also took a heck of a long time to come to 20.04, and that’s a problem.

Sure. My post is just a piece of feedback that can be taken into account when reevaluating them. Clearly somethings could improve; first the Chromium snap debacle (not that I care about it, but at least in my mind it sets a dangerous precedent), now this. The value proposition of an LTS is that supposedly some kind of “base” remains stable for a long period of time, and efforts are directed to backporting important features and timely security fixes. I feel that the latter is not being delivered on, and even that the former may be compromising the latter (it was certainly the case for Thunderbird).

Yes. With the current paradigm, I would expect Canonical to heavily prioritize updates to a set of applications that are essential to a good full desktop experience, not just the browser.

I don’t think snaps are a viable alternative, chiefly because of the walled garden issue. However, I agree with you that the deb system, the current software landscape, and the LTS durations are not jiving well together. The fact of the matter is that the catalog of “critical software that a user expects to be regularly updated” is only growing; I think this is a rather limited view:

Everyone also needs a working and frequently updated media player (e.g. VLC, mpv), office suite, email client, screen recorder/livestream studio software (e.g. OBS), image editor (e.g. GIMP, Krita), video editor, etc. If these are not regularly updated in an LTS release, you end up cutting out large chunks of audiences that need them updated. So you either accept that, or commit to throwing an ever-increasing amount of resources at backporting efforts, that could perhaps be better spent elsewhere, such as contributing upstream and maintaining a smooth rolling release UX. Arch Linux mostly achieves this without a big corporation behind them. The breakages are few and far between and always announced in the appropriate channels.

Maybe the LTS model really is outdated and Canonical should consider investing their engineering effort into providing a smooth rolling experience, and/or embracing new packaging paradigms, like nixpkgs.

Don’t get me wrong, I have made some simple backports myself, so I have a pretty good idea of what hell it must be to backport something like Thunderbird. I know it takes people with a lot of knowledge and experience, and I really do appreciate all the effort that was put into this effort. I’m just saying that perhaps some (fundamental) changes could lead to much more efficient use of the time of all these very knowledgeable and experienced people.

I know that snaps are a valid workaround for the problem, but they’re just a band-aid over larger underlying issues. And they are not acceptable for me anyway, for the reasons outlined in the post above.

Yes, I know I can use snaps. No, I don’t want to use them, at least not while the store is proprietary and centralized. So for now, please assume that for the purposes of this discussion, snaps are not a valid workaround/solution.

That statement is anything but objective.

There are plenty of users (especially in an enterprise setting) who favour stability over new features, that’s the very reason why ESRs (and LTSes) exist.

The latter yes, not the former. Backporting features to a LTS is an exception, not the rule. Again, if what you need is the latest and greatest, using an LTS is not a good option (well that is if you choose to ignore snaps), you want a rolling release indeed.

3 Likes

Apparently there are vastly more users of Ubuntu LTS than there are users who keep up with 6-monthly releases. If a rolling release is what users want, then I’d expect it to be the other way round. Therefore I conclude the opposite: users want the feature stability in Ubuntu LTS. Thunderbird is an unavoidable exception because Thunderbird upstream don’t maintain stable releases that we can follow.

This works for Arch users because this is what they expect and selected by choosing Arch. I think the opposite applies to Ubuntu. It makes sense to have two different distribution serve the different needs of different types of user. It’s great that Arch exists to serve users who want a rolling release. I don’t think it’d serve anyone by trying to make Ubuntu more like Arch. We’d just be alienating the majority of our own users who specifically don’t want that.

Snaps do allow for interaction outside the application sandbox using well-defined interfaces. For example, see XDG portals. The problem is that typical “Linux desktop” apps are still in the dark ages when it comes to this stuff, unlike for example Android where this is an accepted and integrated part of the platform.

I’m optimistic that in time we’ll see this kind of problem go away as support for these types of mechanisms are implemented upstream. Even without the associated packaging model, these kinds of efforts are worthwhile because they improve system security regardless. It’s a revolutionary change in our ecosystem but one that needs to happen if we want these kinds of problems to be solved well.

3 Likes

All the software you’ve mentioned has been updated via snaps on the LTS over the past few years, you’re choosing to not use the solution to the problem and expecting others to spend time solving for something that’s already fixed.

Your only other option is to convince maintainers or rally people to update the software you want in the -backports pocket, which is a doable, but an uphill battle.

5 Likes

There was some issues with Thunderbird snap.
I reported some, Ubuntu team (was @oSoMoN ?) solved them. As far as I can experience, I still see one snap-related bug remaining: including an attachment makes it unavailable to open it from composer window. Not a big deal.

So I understand the concern about Snap Store proprietary software, but the snap itself is not.

And for the first time my Ubuntu story since 2006, I keep the LTS version, since I can get updated apps in a very stable system:

chromium                         91.0.4472.77                1608      latest/stable  canonical✓    -
chromium-ffmpeg                  0.1                         17        latest/stable  canonical✓    -
code                             054a9295                    65        latest/stable  vscode✓       classic
core                             16-2.50                     11081     latest/stable  canonical✓    core
core18                           20210507                    2066      latest/stable  canonical✓    base
core20                           20210429                    1026      latest/stable  canonical✓    base
firefox                          89.0-1                      543       latest/beta    mozilla✓      -
foliate                          2.6.3                       1167      latest/stable  johnfactotum  -
gimp                             2.10.24                     372       latest/stable  snapcrafters  -
gnome-3-28-1804                  3.28.0-19-g98f9e67.98f9e67  145       latest/stable  canonical✓    -
gnome-3-34-1804                  0+git.3556cb3               66        latest/stable  canonical✓    -
gnome-3-38-2004                  0+git.3d25b9b               39        latest/stable  canonical✓    -
gtk-common-themes                0.1-52-gb92ac40             1515      latest/stable  canonical✓    -
gtk2-common-themes               0.1                         13        latest/stable  canonical✓    -
inkscape                         1.1-ce6663b3b7-2021-05-25   9090      latest/stable  inkscape✓     -
kde-frameworks-5-core18          5.61.0                      32        latest/stable  kde✓          -
kde-frameworks-5-qt-5-15-core20  5.79.0                      14        latest/stable  kde✓          -
libreoffice                      7.1.3.2                     218       latest/stable  canonical✓    -
marsshooter                      latest                      221       latest/stable  diddledan     -
quadrapassel                     3.38.1+git2.e19fe71         361       latest/stable  canonical✓    -
shortwave                        1.1.1                       1         latest/stable  alexmurray    -
snap-store                       3.38.0-62-g77ca0c1          538       latest/beta    canonical✓    -
snapd                            2.50                        11841     latest/stable  canonical✓    snapd
stellarium-daily                 v0.21.0                     732       latest/stable  t4saha        -
supertuxkart                     1.2                         556       latest/stable  diddledan     -
thunderbird                      78.10.2                     126       latest/stable  canonical✓    -
vlc                              3.0.14                      2288      latest/stable  videolan✓     -
xournalpp                        1.0.20-9-gea457c9fd1        48        latest/stable  ken-vandine   -

VLC is another topic-related app (v 3.0.9 in repos).
I don’t even mention the snaps that are not available as deb.

2 Likes

This is not what I meant by “walled garden”. My problem is with the snap store being proprietary. At least until that is resolved, I’ll have to stick to flatpak or appimage when I want similar solutions. But even then (and even with snap) there are other problems, such as poor integration with the rest of the system - though admittedly this has been steadily improving over time.

Oh please… let’s not derail the thread with a discussion about how far one has to go to “prove” how much better desktop email clients are than web clients. It’s not like I’m proving a mathematical theorem here. This is one of those situations where “I know it, you know it, everybody knows it”.

So could this be an argument in favor of Canonical de-allocating engineering resources from backporting regular Firefox releases? Just because “some majority” prefers ESRs? To better show what I’m getting at: “There are plenty of users (especially in an enterprise setting) who favor desktop email clients over web clients, thats the very reason why software like Thunderbird exists”…

Maybe I did not make it clear in my previous post, but one of my main points is that nowadays an LTS must necessarily backport a bunch of features in order to provide a good experience and remain relevant for users during its lifecycle. As new video/image codecs come out, important features are implemented in user-facing software, etc, an LTS that does not backport enough of these features actually becomes difficult to use. Users end up forced to add a ton of PPAs or resorting to snaps and friends.

At what point is it not more efficient to simply keep up with upstream on the majority of software, except for a select base of baseline software (glibc, etc)?

The 6 month releases are not true rolling releases. Therefore, you can’t say “most users use LTS, which shows they prefer LTS to rolling”. Users avoid 6 month releases because they are the worst of both worlds in some sense: packages still don’t keep up with upstream, while at the same time the system is not supported for a meaningful amount of time, so users don’t think it is worth it to stick with it for just that time. So, why does Canonical commit to the “maintenance” of something nobody actually wants?

With 6 month releases, the user must upgrade the system to a new one every 6-9 months. In my (and many other people’s) anecdotal experience, Ubuntu in-place upgrades almost never work properly. There’s always some little thing that breaks in the process, so a clean install is pretty much mandatory. Apologies for not providing more constructive feedback about this upgrade issue, but it’s not the point of this discussion, and I don’t want this to derail the thread.

I understand serving different types of users. But consider this: is this (almost no feature upgrades to any application during LTS, snaps) what the type of user you want to serve with Ubuntu really wants? The chromium snap debacle and the community’s distaste for snaps in general hints that this is not the case for snaps. Similarly, judging from the fact that many, many users are already trying to workaround the poor availability of updated versions of popular applications (via PPAs, or other means) tells me that this is not the case for update “policy” either.

Again, I’ll echo my previous suggestion: perhaps maintaining smaller baseline, keeping the rest rolling, and do away with the “intermediate” releases.

“Fixed” by a walled-garden ecosystem with a proprietary store, plus with technical issues such as poor integration with the rest of the system? I don’t think so. And yes, I’m expecting Canonical, the for-profit corporation, to invest in fixing it. Of course I can’t demand anything of volunteers, if this is where you’re getting at. But I still maintain that the volunteers’ time would probably be better spent maintaining software closer to upstream. Backporting too far away from upstream doesn’t help anyone else other than Ubuntu users - it’s “wasted effort” from the perspective of everyone else, including upstream.

In order to contribute to flatpak you need to also use a proprietary web service. :smile:

4 Likes

Whataboutism, the argument is about being limited to getting applications from a proprietary store. Now let’s please stop trying to “win” arguments about snap and get the thread back on topic.

The discussion there at this point should probably be closed, the initial question got a detailed answer and now comments turned about personal opinions being repeated which isn’t useful to anyone