Refactoring Slop

The term “slop” is rapidly en­ter­ing the con­ven­tional English lex­i­con. In fact, it was Merriam Webster’s Word Of the Year for 2025. It refers to low-qual­ity me­dia gen­er­ated by an LLM. It can be ap­plied to im­ages, text, mu­sic, or most rel­e­vantly to us, code.

As an open source main­tainer, I fre­quently both re­ceive and cre­ate pull re­quests that could be la­beled as slop. One of my prin­ci­pal jobs is to stand at the gate to the Harper repos­i­tory and say, “thou shalt not pass” to any low-qual­ity or buggy code that wishes to en­ter. So my ques­tion be­comes, how can I turn AI gen­er­ated code from “slop” tier to “top tier”?

There are a gazil­lion guides out there about the var­i­ous tech­niques you can use to im­prove the qual­ity of AI gen­er­ated code. Those tech­niques will change, and I’m sure my read­er­ship al­ready knows all about them. In­stead, I want to talk about some more gen­er­ally ap­plic­a­ble ideas that I’ve found es­pe­cially use­ful in the age of AI. I’m some­thing of a con­trar­ian, so ex­pect some dish­ing on con­ven­tional “vibe coders”.

A Good Foundation

I am of­ten sent sto­ries of de­vel­op­ers dis­cov­er­ing Claude Code for the first time and us­ing it to build some sim­ple CRUD app from scratch, with­out ever read­ing or writ­ing a line of code them­selves. Those are im­pres­sive sto­ries, and they’re a sign of the amaz­ing progress that the na­tion’s fron­tier AI labs have made in the last few months. They are not, how­ever, ex­am­ples of how an open-source main­tainer should op­er­ate.

As I said be­fore, it’s crit­i­cal that main­tainer act as a gate that al­lows or de­nies code from en­ter­ing the repos­i­tory (and thus be­ing ul­ti­mately de­liv­ered to user’s de­vices). In or­der to do that, they need to have a good un­der­stand­ing of how the code al­ready works. Then, and only then, should they al­low mod­i­fi­ca­tions to it.

This should be pretty ob­vi­ous ad­vice to most in our in­dus­try, but ev­i­dently it is­n’t. I’ve seen sev­eral in­stances re­cently of de­vel­op­ers vibe-cod­ing a new fea­ture with­out some ex­ist­ing foun­da­tional un­der­stand­ing of how the orig­i­nal code worked. The re­sult: the new fea­ture works well enough, but its im­ple­men­ta­tion breaks some other part of the ap­pli­ca­tion. That’s not to men­tion that it in­creases the apps com­plex­ity on the whole far more than nec­es­sary.

Having a foun­da­tional un­der­stand­ing from the get-go is an easy way to pre­vent this kind of tragic out­come. LLMs are not cur­rently ca­pa­ble enough to rec­og­nize when a pat­tern in the code they’re writ­ing al­ready ex­ists in the code­base. I’ve tried push­ing them in the right di­rec­tion with skills and with an AGENTS.md, and they’ll pick it up given enough mas­sag­ing, but the fact is that they still need to be poked. If I did­n’t have a good knowl­edge of my pro­jec­t’s in­ter­nals, I would­n’t know to poke them at all.

Understand your pro­ject deeply. Un­der­stand its code, its val­ues, and its pur­pose.

Reviewing AI Code

Now that you’re an ex­pert on how your pro­ject works, you’re ready to start re­view­ing AI-written pull re­quests. Here’s the thing you need to re­mem­ber: you have bet­ter taste than the clanker does. You ul­ti­mately get to de­cide what code gets merged. What should that code look like? How should it be tested? Your an­swer to those ques­tions should guide you to­wards prompt­ing a model to re­vise its work or sub­mit­ting feed­back on a pull re­quest opened by a hu­man.

Here’s what your prompt should not look like:

• “Fix it.”
• “Make it pret­tier.”
• “I don’t like that.”

Obviously, you would­n’t want feed­back like that if you were the one writ­ing the code. If you’re look­ing at a patch, who­ever wrote it did the best job they could. Whether it was an LLM or a hu­man does­n’t mat­ter. If the code is­n’t up to your re­fined taste as a hu­man, you need to give them a nudge in the right di­rec­tion. Here are some bet­ter ver­sions of the prompts from be­fore:

• “It looks like your mod­i­fi­ca­tion made the app crash on startup. Would you run the tests to make sure you haven’t bro­ken any­thing in the X mod­ule?”
• “This huge block of code is re­ally hard to read. Would you refac­tor it into in­di­vid­ual com­pos­able com­po­nents?”
• “It looks like you’re reim­ple­ment­ing a func­tion we al­ready have. Would you use X in­stead?”

For those un­aware, you should never say “I don’t like that” to hu­man’s work.

Accept That The Models Are Limited in Intelligence

It’s easy to get the im­pres­sion that with enough to­kens, any­thing is pos­si­ble. Maybe that will be true in the fu­ture, but I don’t think that’s true to­day.

LLMs are lim­ited in in­tel­li­gence and ex­pe­ri­ence. Even when pro­vided ac­cess to the best cod­ing se­tups in the world, they are still in­ca­pable of ship­ping many (most?) fea­tures or fix­ing many (most?) bugs. The fact that I have to in­clude ques­tion marks is a tes­ta­ment to how far we’ve come, but it’s im­por­tant not to over­state things. If you’re read­ing this, you’re pretty smart. It’s fine to let the model try to solve a prob­lem for you once or twice, but if it still fails, you’ll need to get in there and do it your­self. Don’t be afraid to get your hands dirty.