I’ve heard it said before that creating great documentation is a team sport, and I really think that’s true. I don’t remember who told me that. Maybe I made it up. Or maybe an AI did. I’ve been having a lot of conversations about AI in the last few months, both with colleagues and with AI itself. It’s hard to keep track of who told me what.
After my last Discourse post detailing my adventures with the robots, I’ve continued to learn and experiment, using the Ubuntu Server documentation as my playground. Even small changes that improve the quality of the docs, or enhance the experience of contributing to the docs, is worthwhile in my view, so I’m on a mission to find and remove as many sources of friction as I can.
Our contributor journey
A large part of the reason we decided to move our documentation from Discourse to GitHub in 2024 was to remove friction from our contribution process. For the first 6 months of 2024, on Discourse we had no community contributions to the docs. The Discourse model was great for discussion, with many posts generating insightful and thoughtful conversations in the community, but it didn’t make it easy for new contributors to spot problems in the docs and be empowered to just fix them – or even to report them. Following this migration to GitHub, we saw an enormous increase in the number of docs contributions being made by the community.
Many of the contributions we receive are from readers of the documentation who just want to quickly fix a typo they spotted, or report a problem with a page. Having our issues filed in a single place has helped everyone, by making it considerably easier to track, prioritize and crucially, to fix issues in the documentation. It’s easier than ever before to help improve the Server documentation, and we regularly see contributions from people who are making their first ever contribution to an open source project.
The cloud on the horizon
In 2025, we also saw a steady increase in the number of spam issues and PRs being submitted. Many of these were clearly AI-generated and had been submitted without any human oversight. Blank issues, keyboard-smash spam, issues being closed and opened multiple times by the author, crypto scams, “polishing” the readme by passing it through an LLM, trying to delete the .github folder…we’ve seen all this (and more). As a data nerd, obviously I tracked the numbers on this.
At about the same time, there were high-profile stories of claws running amok in popular projects, harassing and threatening open-source project maintainers. There are only so many hours in the day, and every hour spent trying to reason with a badly-behaved robot, or someone who has generated a PR in a few seconds without care or attention, is an hour that could have been better spent reviewing a contribution from a real, human, contributor.
We needed a way to filter out the spam, so the team could spend their time talking to actual human contributors. While spam has always been a problem to some extent, the prevalence now of autonomous agents (and LLMs in general) and the speed at which they operate has removed a lot of the friction that previously had imposed natural limits on the noise people could create.
After some research, a lot of thought, and a healthy dose of trial-and-error, I added several new features to the documentation repository over the last few months to try and tackle the issue of AI spam.
Adding a “pre-triage” filter
When we first migrated our documentation to GitHub we often received “empty” issues, with no information provided at all. To address this, we added an issue template quite early into the migration and hooked it up to our “Give feedback” button, both of which really helped our readers understand what information we need when they find a problem. This small change in turn allowed us to fix a lot of issues that we might not have caught otherwise.
However, I’ve noticed a marked increase over the last year in the number of issues being submitted with either no information, or nonsense information, not just in our project but in many others I’m watching across GitHub. To avoid the team having to read, parse and triage such issues, I pre-emptively added three workflows:
- The first runs pre-triage on all new issues submitted to the documentation, using regular expressions to classify the contents. If they’re empty, have nothing to do with our documentation (e.g. crypto scam adverts), or are just “keyboard smash” spam, the workflow labels them as “invalid”. If it’s a real issue but lacks information, the bot marks it as “incomplete” and posts a comment on the issue asking the poster to add the missing info.
- The second runs on issues or PRs marked with the “invalid” label, and automatically closes them. Before closing the issue or PR, the bot leaves a comment to help guide the contributor to understand why it’s being closed.
- The third autocloses stale issues that were originally marked as “incomplete” but have not been edited by the original poster in the last 7 days.
The pre-triage bot occasionally “catches” maintainers as well, if they forget to include relevant information in issues they post 
I keep a close eye on all issues, and the bot’s activity, to ensure such catches get reverted quickly – I’m still refining the filters to ensure we capture and divert as much spam as possible, without catching valid issues that just need a bit more information.
Writing an AI policy
For all the ethical questions and concerns around AI usage, which are valid, it’s also true that for many people, AI is an incredible accessibility tool. AI allows people to participate in ways that they previously couldn’t, which is why we don’t want to ban the use of AI in our docs.
However, when I originally wrote our contributing guide, AI was not yet the tool it is today. Saying “we welcome all contributions” in today’s landscape is a position that needs some guardrails added to protect the spirit of true, valuable contribution. An enormous amount of hard work has gone into making the Ubuntu Server documentation the trusted, respected corpus it is today. We don’t want to undermine that by accepting inaccurate, hallucinated content, or “polishing” out the voices of all our human contributors. This means we needed an acceptable usage policy, and for the first time, to set explicit limits around what contributions we will accept.
The problem with AI-generated submissions is, in reality, the gap between a low-value and a high-value contribution. We therefore needed to identify and articulate what makes a submission good, whether it’s been created using AI or not. For us, a good contribution:
- Has changes that are reasonably-sized, and within scope of the project
- Has a writing style consistent with our established documentation style
- Is a genuine, useful change – no matter how small
- Is technically accurate and precise
We also changed the contribution workflow to require a maintainer-validated issue linked to any PR for substantial changes. We’ve also specifically outlined all the contributions we consider to be out of scope, or that we don’t accept. It’s a pretty short list, because we always work with a contributor to find the parts of their pull request that we can land (rather than rejecting it outright).
Finally, we laid out what we believe responsible/acceptable AI usage looks like in the Ubuntu Server documentation project. This applies equally to maintainers and contributors, and will continue to evolve as usage of AI changes across the industry.
Constraining the agents
It’s an unfortunate reality for open source project maintainers that even if you have a strict “no AI” policy, it’s impossible to enforce such a thing. People will still point their agent at your repository anyway, and not all agents are well-behaved. Rather than trying to punish people for doing so by adding destructive patterns into the files the agents use, we want to encourage the agents to behave well. This means being much more explicit with them about what they can and cannot do.
The Server docs contain an AGENTS.md file at the root of the repository, setting out everything an agent or LLM needs to know about the project; conventions, rules, context, and guidelines. Here, we specifically instruct agents to refuse requests that contravene our AI policy (such as unsolicited “polishing” requests). In my own testing this seems to work well, and we haven’t had any such submissions since we added this instruction. It’s not clear yet if this is due to the instructions, or if we’ve just been lucky (too soon to tell!).
Needless to say, this file will continue to evolve as new types of AI-generated spam appear.
Supporting the good path
None of these guardrails prevent someone from using AI in ways that contravene our policy, but at least they make “the good path” clearer and easier to follow. If we want to be pragmatic and accept the reality that some people will choose to use AI, then it’s incumbent upon us to help them use it well. In this spirit, I crafted some reusable prompts that can be invoked in your IDE or on the command line. These have been designed to support the “good path” as we see it, where AI is used to assist, and not replace, a contributor.
- Our
/new-pageprompt generates an outline/template with all the elements in their correct place. If you supply any amount of notes (no matter how rough), they’ll be sorted into their most logical section for you to expand on later. - Our
/review-pageprompt checks for expected elements you’re missing, and also acts as a “first reader” (as many times as you like!) to provide suggestions and ask clarifying questions. Rather like having a pocket technical author to help you. - Our
/capture-learningsprompt helps our AIs to be good citizens. You can run this at the end of a session to bundle up everything you and/or your AI learned through your guiding its work, and apply those back to the AI scaffolding. Sharing improvements in the tools helps us all to avoid spending tokens repeating the same mistakes.
Lowering the cost
It’s also a reality that AI has a cost. If we can help contributors and maintainers alike to reduce that cost, we should do so.
To lower the cost of working on the Server documentation using AI, both in terms of environmental impact and monetary/token cost, I’ve organized the .github folder with some scaffolding that shrinks the context window for any given task. There are now separate .instructions.md files covering the different Diataxis sections, as well as a “completeness” checklist, and standards for the landing pages. These are either automatically included in the context when a page is touched that meets the applyTo criteria, or are invoked by an agent as they’re needed.
Conclusion
Although it’s too early to tell what the effects (and effectiveness) of our new AI policy and tooling will be, the early signs look positive. Since early March 2026, when I started to add protections to the repository, we’ve seen a reduction in the amount of spam we’re getting. We’re also feeling the benefits of our pre-triage filter, which catches the spam issues that do get filed, and moves them out of sight before we start our daily triage.
Now that the Server team has the tools to quickly shut down spam, they can spend their time more productively (and enjoyably) helping contributors. More than that, by being open about allowing AI-assisted contributions and not just rejecting them, we find that contributors are much more willing to provide details about how they used AI. This transparency is what we’re looking for.
Ultimately, we wouldn’t want to ban the use of AI in our project, even if such a thing were feasible. For some contributors, AI is the playing-field leveller that has allowed them to participate in open source for the first time, and participation is something we want to encourage. The real challenge, then, is how to steer the use of AI toward the “good path”, so we can avoid putting up barriers to contribution. This is something we’re all learning and iterating on as the tools evolve.
If you want to try out some of our tools for yourself, check out our contributing guide, our AI usage policy, and the documentation for our AI tooling.