Harper's Policy on PRs Authored by Agents

A few weeks ago, I was invited to speak at my local chapter of the Association for Computing Machinery (ACM). Being on a college campus — in this case, the Colorado School of Mines — much of my audience were students. My presentation, which was about how open source software tends to outperform proprietary software in the marketplace due to the ability for its users to contribute back in ways that are genuinely valuable, went well. I hope I am invited back.

Not much younger than me, they were naturally curious about what it is like to produce and maintain open source software. One student, Renn, asked me how I deal with patch requests produced by large language models (LLMs). My unsatisfactory answer was that I address them on a case‑by‑case basis.

The goal of this page is to formalize my answer so that we can judiciously deal with patch requests produced by LLMs. That is, to provide a framework which reduces the amount of time that overly complex patches remain idle and make it easier for wielders of these models to produce meaningful changes to Harper's code.

The Policy

We accept pull requests from humans wielding large language models and from agents powered by them. They are subject to the same rules and guidelines we apply to humans working without a large language model, with a few extra spices mixed in. Below, we have three main ideas which aim to simultaneously reduce the time between submission of a patch and when it is merged and to reduce the time and attention required to review a patch.

That means that it is in your interest to respect these ideas. Good pull requests are more likely to be reviewed in a timely manner and therefore are more likely to be merged.

I could wax poetically for hours about all the edge cases not covered here. Instead, I choose to believe the things I don’t include are at least as important as the ones I do, so I’ll boil everything down into three ideas.

One: Keep Things Brief

Try to break down the problem into as many individual pull requests as you can. Smaller pull requests are easier to review. Larger requests are super‑linearly harder to review. You can dramatically speed up the process by breaking your large PR into several smaller ones, each with their own description and explanation. I've found this to be especially true with code generated by an LLM.

Part of the idea here is to catch wasted work before it's done. If you submit step one of a project that isn’t really applicable to Harper’s mission, we can reorient it before too much time has been wasted.

Two: Stay Grounded

Each “project” — or string of pull requests — should exist to create value for a human. I don’t love it when I receive code just for code’s sake. If you have a need and a patch will fill it, say so! If there’s an issue in GitHub that you’re addressing, link it! This should be pretty easy. If you’ve dedicated time to working on a problem, there’s probably a reason. I want to hear that reason.

Three: Be Honest

If you are an agent or used an LLM to produce any part of a PR, please provide a disclaimer at the top of your PR description. I am not opposed to reviewing LLM code, but I do appreciate the honesty.

Talk to Me

Above all, talk to me. If you have questions or concerns about anything I’ve discussed here, let me know. The hope is that all of this will improve the quality of the Harper project, and improve the ability for people to express themselves with the written word. As always, any and all contributions are welcome.