@popey offered some suggestions as to what tests to run. @rubi1200 mentioned using time which is crude, but totally reasonable. You can simulate different systems with virtual machines, so anyone can do it. At the very least, anyone can do it on their own machine.
However, that comment was not directed at you. I understand your concern. It was directed at someone concerned about performance, where such tests are actually relevant. Assumedly such a person already knows how to measure performance because they’re complaining about seeing a difference in performance. I was merely suggesting being more scientific about it.
If all developers used this same philosophy, we’d lack a lot of software. Look at Matrix. It’s still not perfect, even after 11 years of development. But we can use the software while we wait for the changes to make it more functionally complete. Incremental improvements will ultimately lead to better software. This is very nature of open source. What you’re suggesting sounds a lot like that problem classic design/engineering problem: “the perfect gets in the way of the good.”
I have a somewhat similar concern which I’ve expressed elsewhere here:
and even on the Snapcraft Discourse:
with almost no response. The managing of reporting problems seems like a real issue. I haven’t looked at what this is like on Flatpaks but on AppImages it’s even worse since there’s no inherent app management tool and so it’s very hard to find out where to report bugs. So I guess we’re in a better state than that, but this is still a huge problem relative to Debian packages.