Hello!
Please be advised that our Matrix service will undergo an upgrade on Thursday, June 20th, from UTC 13:00 - 14:00. During this period, the service might be unavailable. We apologize for any inconvenience and appreciate your understanding.
5 Likes
Due to unforeseen infrastructure issues, the upgrade will be postponed. A new date will be announced soon.
The upgrade is rescheduled for Tuesday, June 25th, from UTC 13:00 - 14:00. During this period, the service might be unavailable. We apologize for any inconvenience and appreciate your understanding.
2 Likes
Hello!
Please be advised that our Matrix service will undergo an upgrade on Tuesday, August 20th, from UTC 18:00 - 19:00. During this period, the service might be unavailable. We apologize for any inconvenience and appreciate your understanding.
3 Likes
Hello!
Please be advised that our Matrix service will undergo an upgrade on Friday, September 20th, from 12:30 UTC to 13:30 UTC. During this period, the service might be unavailable. We apologize for any inconvenience and appreciate your understanding.
Hello everyone! Starting Dec 2nd we are moving to planned maintenance windows for Matrix. Every first and third Monday of the month, we will have a maintenance window for Matrix between 12 PM - 1 PM UTC for upgrades and new feature additions. Please reach out with any questions on the Ubuntu Matrix Ops channel!
Hello!
Please be advised that our Matrix service will undergo an upgrade on Monday, Feb 17th, from UTC 12:00 - 13:00. During this period, the service might be unavailable. We apologize for any inconvenience and appreciate your understanding.
1 Like
Hello!
Please be advised that our Matrix service will undergo an upgrade on Monday, April 14th, from 16:00 UTC to 18:00 UTC. During this period, the service might be unavailable. We apologize for any inconvenience and appreciate your understanding.
1 Like
Hi @amandahla ,
Just wondering what the status is on this as the downtime has gone beyond the two hour estimate. It’s a release week, so I know I’m not the only one wondering what’s going on.
Thanks!
3 Likes
Indeed. Please provide answers ASAP; we are unable to continue release work, this is actively blocking things.
2 Likes
@amandahla The downtime on this is well beyond your specified windows. A status update is now required for both Technical and Community reasons.
cc @ilvipero @aaronprisk because this is now a Community Team level issue - we need people on CC/TB/etc. to have a way to contact IS that actually is guaranteed for us to get responses because IRC doesn’t work for that anymore when we have things that are so mission critical to operations for the boards AND technical teams for communication and we can’t get any live updates. Which makes this an absolutely top priority issue that needs addressed.
(the above is said with both my Ubuntu Community Council and Ubuntu Technical Board hats on, which shows the severity of impact of this issue if I have to use both of those hats)
4 Likes
We are currently working on fixing some login issues with the server. The service should be up shortly!
1 Like
Thank you. I was just able to login but had to reverify my sessions.
This was two hours over the two-hour maintenance window. As @teward alluded, this was highly disruptive, especially on a release week.
We need updates on this sort of thing during the issue, not at the end of it.
1 Like
The login issues have been resolved, and the service is back up. We’re currently in the process of scaling up the database, and don’t anticipate any further downtime from that.
Context: This update was necessary as we were running out of database resources on the Synapse deployment. The backup and restore took longer than expected, and we encountered issues while scaling the service back up—this led to users being logged out and unable to log in.
Looking ahead, we’ll aim to avoid scheduling updates during release weeks or community events. We also recognize that this thread isn’t the most effective way to communicate any Matrix release or update issues, and we’ll be working with the community team to figure out a better way of sharing those updates.
Thanks again for your patience!
3 Likes
Easy solution: open a new topic in Matrix to communicate when there are issues. On the contrary to your current line of thinking, that would be the absolute most effective way to communicate this with we have at our disposal.
2 Likes
That’s a great idea. I was referring specifically to this thread, since it’s meant for general announcements about upcoming updates. We can definitely create separate threads or topics dedicated to real-time community updates, especially for things like Matrix downtime, going forward. Another option could be a public status dashboard for the service.
Use the tools you already have at your disposal and don’t reinvent the wheel. Discourse is a tool. Maybe adding it to status.canonical.com would be good too, but it’s not nearly as effective as someone just posting here.
Sounds good! Here is the topic we will use to post real-time updates for issues encountered during Matrix maintenance windows in the future. Matrix is already added to status.canonical.com, so no further action is needed there.
@varshigupta I wonder if perhaps we could arrange some “exclusion dates” for maintenance windows - eg. to deliberately adjust or cancel maintenance windows that coincide with critical events?
Ubuntu release week would be one such event. There are probably others - we’d want to establish and prioritise those with the wider Ubuntu community. Clearly there can’t be too many of these of course.
Edit: perhaps some in-person events such as the Ubuntu Summit, since Ubuntu members will generally rely on Matrix heavily during these times.
@rbasak Yes, we can. The purpose of sharing these upgrade plans in advance is to keep the community informed and to gather feedback, especially if there are specific dates that should be avoided. Feel free to let us know ahead of time about any dates you’d like us to steer clear of. We should be able to accommodate most requests, unless an urgent fix is required.
1 Like