So maybe it’s just cheap consumer routers, like mine, for instance, that cause this, because they botch PMTUD. Or, as is my suspicion, Canonical is simply null routing everything with an MTU >1320, in the assumption that botnets aren’t powered by mobile devices, possibly in a dynamic fashion based on congestion thresholds, which could explain the intermittence.
We have to assume that the DDoS attack is still ongoing, since there hasn’t been an official status update on it.
Canonical is unlikely to reply here, because this is the community Support and Help channel. But your Cloudflare summary has some hints at least. It contains references to three autonomous systems (AS) under the control of CANONICAL-AS, which pretty much means, they run their own private part of the internet, and it’s most likely powered by AWS, because that’s where all the 3rd party requests of this site go, e.g. ubuntucommunity.s3.dualstack.us-east-2.amazonaws.com.
BTW, seeing it like that, the infrastructure seems to be centered in the USA, which could also explain some issues, because I am firmly on the eastern side of the Atlantic, as are you apparently, @dmxplx. If I wanted to DoS major parts of the internet I’d go for the jugular too.
I could not find any evidence of a CloudFront setup with AWS Shield Advanced enabled. Cloudflare also does a great job against attacks, DDoS protection, and bot mitigation features (JS challenges, etc.). It might be an option worth considering (trying to help again, good luck folks).
Nobody from Canonical will read this here. They also might not take too kindly to outsiders explaining to them how to do their jobs. Just assume that everything that can be done is being done. There is no magic bullet against a DDoS, even if Cloudflare tries to convince you otherwise; I bet that they just overpower small-ish botnets to still deliver a modicum of service level when “protecting” their clients. That can only scale so far.
I’ve come across that site before in just about the same context. It seems to come up with new articles which posit solutions to problem eerily similar to topics on this very Discourse. It’s deceptively prosaic, littered with errors and whatnot.
In short, all the hallmarks of “Ai” slop. And guess how that “Ai” and all its ugly friends get trained. That in itself constitutes a DDoS attack.
@ogra Got it, that helps a lot. I’ve previously used bot mitigation with Cloudflare, but not with CloudFront, if that’s helpful. That said, I trust your team to handle it as well good luck folks
If anything, it’s worse for me today, because some content, like editor buttons, for instance, is sketchy to load. Plus I get the occasional error 500 for temporary DoS. Maybe too many people know about the secret sauce with the MTU by now?
People, you need to just be patient. For some reason, the hackers behind the DDoS seem to think that Canonical is also behind America’s attack on Iran (Canonical isn’t even a US company). We’re unlikely to see a resolution to this soon.
There’s no point in continuing to post about this until it has definitively been solved.
I’ve add an alias to my .bashrc:
alias discourse='sudo ip link set dev wlo1 mtu 1320'
So that I don’t have to keep typing my password, I’ve added this to sudoers:
paddy ALL = NOPASSWD: /usr/sbin/ip link set dev wlo1 mtu 1320
When the problem happens, I open a terminal, enter discourse, and close the terminal. Quick and easy.
I’m unsure what’s crazy about it. Probably you mean the situation that the hackers have caused, because they think that Canonical is somehow a part of the Iran invasion. I agree.
Actually, I’ve gone a step further. Instead of an alias, I’ve added discourse as a function to .bashrc, which makes it independent of the name of the interface:
function discourse
{
# Find the active internet link.
local -r INTERFACE=$( ip link show | grep -F BROADCAST | grep -Fv NO-CARRIER )
# Find its name.
local -r NAME=$( cut --delimiter=: --field=2 <<<${INTERFACE} )
# Find its current MTU.
local -ir MTU=$( grep -P --only-matching '(?<=\bmtu )\d+' <<<${INTERFACE} )
if (( MTU == 1320 ))
then
printf 'MTU for %s is already %d.\n' ${NAME} ${MTU}
else
printf 'Setting MTU for %s to 1320.\n' ${NAME}
sudo ip link set dev ${NAME} mtu 1320
fi
}
You’d have to amend the entry in sudoers if your interface name isn’t wlo1.
Thanks all
