Markov Chains Are the Original Language Models

<h1>Hacker News, sans AI</h1>
I like Hacker News, but I do not love that so much of it has turned into dis­cus­sion of a sin­gle topic: AI. So, I have cre­ated my own ver­sion.
A ver­sion of Hacker News, with­out AI.
<a href="/hnsansai">HNSansAI</a> pulls from the of­ﬁ­cial Hacker News API, looks through the con­tent of each ar­ti­cle, and dis­cards any that men­tion a term re­lated to “AI”. That in­cludes stuff like:
<ul>
<li>“OpenAI”</li>
<li>“Anthropic”</li>
<li>“Copilot”</li>
<li>“LLM”</li>
<li>…and many more.</li>
</ul>

I like HackerNews, but I don't love that so much of it has turned into discussion of a single topic: AI. This is a version of HackerNews, filtered to remove any article focusing on __AI__. Refreshes about every ten minutes.

I like HackerNews, but I don’t love that so much of it has turned into dis­cus­sion of a sin­gle topic: AI. This is a ver­sion of HackerNews, ﬁl­tered to re­move any ar­ti­cle fo­cus­ing on AI. Refreshes about every ten min­utes.

Hacker News sans AI

<h1>Slop Paralysis</h1>
<blockquote>
slop paral­y­sis - noun
A com­plete or par­tial loss of func­tion while re­view­ing the out­put of a cod­ing agent.
</blockquote>
Let me paint you a pic­ture.
You have an idea for prod­uct.
It could be any­thing: A mo­bile app, a dash­board, or a script to au­to­mate your work.
So, you sit down with your fa­vorite LLM and de­scribe your idea.
Maybe you have a pretty good un­der­stand­ing of how it should be im­ple­mented.
You might even know the over­all struc­ture of the pro­ject in your mind.
You tell your cod­ing agent all of this.
Then, you let it loose.
It cooks and cleans and im­ple­ments your prod­uct for you.
In the end, you have some idea of how the end re­sult works, but it is not com­plete.
Since this is a pro­ject that is im­por­tant, which you in­tend to main­tain, you go to read the code.
That’s when it hits you.
Slop paral­y­sis.
<h2>The Problem</h2>
Slop paral­y­sis is the com­plete lack of de­sire to re­view the out­put of an LLM.
It could show up for any num­ber of rea­sons.
The most com­mon, for me, are:
<ul>
<li>The sheer amount of code de­mand­ing con­sump­tion.</li>
<li>Missing con­text (that the agent was aware of, but not you).</li>
<li>A fear of break­ing some­thing.</li>
</ul>
In the end, slop paral­y­sis slows you down and makes your life more stress­ful.
So, what can we do about it?
<h2>Some Solutions</h2>
I will de­scribe some so­lu­tions that have worked for me.
As al­ways, your mileage may vary.
The ob­vi­ous so­lu­tion is to sim­ply not use an agent.
Some­times cod­ing agents make your job eas­ier.
Some­times they do not.
<a href="./when_to_use_an_agent">Knowing which is which is dif­ﬁ­cult, but ex­tremely valu­able.</a>
When you do de­cide to use an agent, you can have the agent sim­ply pro­duce less code.
I have heard of <a href="https://github.com/DietrichGebert/ponytail">pro­jects</a> that help you do this, but I have not ac­tu­ally tried them.
In­stead, I ask the agent to make a plan ﬁrst, then re­work that plan my­self to min­i­mize the num­ber of changes.
In the end, a plan which was orig­i­nally writ­ten by the agent be­comes one that has been mer­ci­lessly edited by my­self.
In this way, I min­i­mize the amount of code which de­mands con­sump­tion.
The side-ef­fect of this is that I then have an ex­cel­lent un­der­stand­ing of what the agent ac­tu­ally does, which helps with the con­text prob­lem.
The ﬁ­nal strat­egy, which I use when all else fails, is to refac­tor the code by hand.
When the agent has proven the tech­ni­cal fea­si­bil­ity of the pro­ject, but the code is sim­ply too ex­pan­sive to prop­erly grok all at once, I sit down with a ﬁne-tooth comb and refac­tor it, mod­ule by mod­ule.
I do not need to care­fully read each func­tion and eval­u­ate its mer­its, but I do force my­self to, at min­i­mum, lay eyes on them all.
<h2>Stay Steady Out There</h2>
I ﬁnd my­self be­com­ing more over­whelmed by the dis­course every sin­gle day.
It’s ex­haust­ing how much is hap­pen­ing in the world, and even worst, how much you are ex­pected to care about it all.
Take some time for your­self.
Save your­self from slop paral­y­sis.

A complete or partial loss of function while reviewing the output of a coding agent.

A com­plete or par­tial loss of func­tion while re­view­ing the out­put of a cod­ing agent.

Slop Paralysis

<h1>Everything Is a Leaky Abstraction*</h1>
*in ma­chine learn­ing
One of my fa­vorite blog posts of all time was writ­ten by Andrej Karpathy nearly ten years ago.
In it, he de­tails how he would re­quire the stu­dents of his CS231n Deep Learning class at Stanford to im­ple­ment the for­ward and back­ward passes of neural net­works by hand (that is, in Python with NumPy).
Why?
Because he wanted his stu­dents to leave his class with a vis­ceral un­der­stand­ing of back­prop­a­ga­tion.
They should know how it works, why it works, and thus how to ﬁx it when it stops work­ing.
It’s a great read, and I highly rec­om­mend that any­one and every­one in­ter­ested in ML pe­ruse it.
<a href="https://karpathy.medium.com/yes-you-should-understand-backprop-e2f06eab496b#.lbzzq2acs">You can ﬁnd it here.</a>
<h2>Some Sealed Abstractions</h2>
There are some things that are sealed ab­strac­tions (that is, they are not leaky).
But I do not think any of them ex­ist in the se­ri­ous study of ma­chine learn­ing.
Some ex­am­ples in­clude:
<ul>
<li>Lossless com­pres­sion al­go­rithms.</li>
<li>TCP/IP</li>
<li>TLS</li>
<li>PostgreSQL</li>
</ul>
Sure, with some of these things you oc­ca­sion­ally need to un­der­stand the un­der­ly­ing sys­tem for per­for­mance tun­ing.
But knowl­edge of the un­der­ly­ing sys­tem is rarely the dif­fer­ence be­tween suc­cess and fail­ure.
<h2>ML Is Not like Normal Engineering</h2>
ML is not that way.
To suc­cess­fully train a model, you need to have a clear un­der­stand­ing of every as­pect of the sys­tem, all the way down to how in­di­vid­ual weights and bi­ases are up­dated.
It is not a kind dis­ci­pline.
Obviously, there is Karpathy’s ex­am­ple of back­prop­a­ga­tion, but it goes fur­ther than that.
Every as­pect of model ar­chi­tec­ture can in­ﬂu­ence the shape of the cost func­tion op­ti­mized via back­prop­a­ga­tion, which means every as­pect of model ar­chi­tec­ture is leaky.
I have got the sense that lay­men be­lieve that any prob­lem can be solved by throw­ing a <a href="https://en.wikipedia.org/wiki/Transformer_(deep_learning)">trans­former</a> at it.
If only it were so sim­ple.
Successful use of a trans­former (or any model ar­chi­tec­ture, for that mat­ter) de­pends on a good un­der­stand­ing of how it works and how it af­fects the cost func­tion.
<h2>Take Your Time</h2>
As with any kind of sys­tem with a Rube Goldberg–level com­plex­ity, it is best to take your time.
Go slow.
Skip that ex­tra pot of cof­fee.
It’s bet­ter to be right slowly than wrong very fast.
Un­der­stand every as­pect of your model and train­ing reg­i­men be­fore you start, not af­ter.

Elijah Potter

Everything Is a Leaky Abstraction*

Some Sealed Abstractions

ML Is Not like Normal Engineering

Take Your Time

Comments

Other Stuff

Markov Chains Are the Original Language Models

Hacker News sans AI

Slop Paralysis