Richard Hesketh

Elijah Potter

<h1>Local-First Software is Easier to Scale</h1>
<img src="/images/flatirons.webp" alt="The Flatirons">
The ti­tle of this post is some­what mis­lead­ing. Local-ﬁrst soft­ware rarely needs to be scaled at all.
<a href="https://writewithharper.com">Harper</a> re­cently re­ceived a mas­sive in­crease in both traf­ﬁc and user count.
How?
By mak­ing it to the front page of Hacker News.
If you could­n’t tell by the ex­tent to which I talk about this, I’m thrilled.
This amounted to a tremen­dous amount of free ad­ver­tis­ing, if noth­ing else.
I’ve been think­ing a lot about the won­drous ben­e­ﬁts of work­ing at the edge.
That’s just one term of many that peo­ple use to de­scribe Harper: “edge-ﬁrst”.
Others in­clude “local-ﬁrst” or sim­ply “on-device”.
They are all just jar­gon for a sim­ple idea: We run all the rel­e­vant gram­mar check­ing code as phys­i­cally close to the users as pos­si­ble.
That way, we can serve up sug­ges­tions faster than any­one else.
To achieve our goal, we make sure that every­thing is well-op­ti­mized and lean.
Harper barely uses any CPU or mem­ory at all.
What does all this mean in the con­text of the re­cent uptick in user ac­qui­si­tion? Imagine, for a mo­ment, we are LanguageTool.
Our soft­ware is writ­ten in Java, with ques­tion­able at­ten­dance to the ef­ﬁ­ciency of our code.
Since our soft­ware re­quires a big server to run, we have a cer­tain num­ber of servers on standby to han­dle a cer­tain amount of ex­pected load.
If sud­denly the num­ber of re­quests we re­ceived were to dou­ble (as it did for the Chrome ex­ten­sion), we would be big trou­ble.
To avoid in­creas­ing the la­tency on re­quests (or drop­ping re­quests all to­gether), we would need to scale up the num­ber of run­ning servers.
This not only takes hir­ing an ex­pert in cloud ar­chi­tec­ture, but also ad­di­tional fund­ing to pay the AWS (or what­ever) bill at the end of the month.
Because Harper runs at the edge (no server re­quired), we don’t have to worry about that.
In fact, I only no­ticed that our user count had spiked af­ter I vis­ited Hacker News the morn­ing af­ter.
No hic­cups at all.
Lots of cloud providers like to brag about be­ing able to scale with their users. I like to brag about not hav­ing to scale at all.

The title of this post is somewhat misleading. Local-first software rarely needs to be scaled at all.

The ti­tle of this post is some­what mis­lead­ing. Local-ﬁrst soft­ware rarely needs to be scaled at all.

Local-First Software Is Easier to Scale

It didn't work for me, and if you reading this, it probably won't work for you either.

It did­n’t work for me, and if you read­ing this, it prob­a­bly won’t work for you ei­ther.

Do Not Type Your Notes

<h1>Slop Paralysis</h1>
<blockquote>
slop paral­y­sis - noun
A com­plete or par­tial loss of func­tion while re­view­ing the out­put of a cod­ing agent.
</blockquote>
Let me paint you a pic­ture.
You have an idea for prod­uct.
It could be any­thing: A mo­bile app, a dash­board, or a script to au­to­mate your work.
So, you sit down with your fa­vorite LLM and de­scribe your idea.
Maybe you have a pretty good un­der­stand­ing of how it should be im­ple­mented.
You might even know the over­all struc­ture of the pro­ject in your mind.
You tell your cod­ing agent all of this.
Then, you let it loose.
It cooks and cleans and im­ple­ments your prod­uct for you.
In the end, you have some idea of how the end re­sult works, but it is not com­plete.
Since this is a pro­ject that is im­por­tant, which you in­tend to main­tain, you go to read the code.
That’s when it hits you.
Slop paral­y­sis.
<h2>The Problem</h2>
Slop paral­y­sis is the com­plete lack of de­sire to re­view the out­put of an LLM.
It could show up for any num­ber of rea­sons.
The most com­mon, for me, are:
<ul>
<li>The sheer amount of code de­mand­ing con­sump­tion.</li>
<li>Missing con­text (that the agent was aware of, but not you).</li>
<li>A fear of break­ing some­thing.</li>
</ul>
In the end, slop paral­y­sis slows you down and makes your life more stress­ful.
So, what can we do about it?
<h2>Some Solutions</h2>
I will de­scribe some so­lu­tions that have worked for me.
As al­ways, your mileage may vary.
The ob­vi­ous so­lu­tion is to sim­ply not use an agent.
Some­times cod­ing agents make your job eas­ier.
Some­times they do not.
<a href="./when_to_use_an_agent">Knowing which is which is dif­ﬁ­cult, but ex­tremely valu­able.</a>
When you do de­cide to use an agent, you can have the agent sim­ply pro­duce less code.
I have heard of <a href="https://github.com/DietrichGebert/ponytail">pro­jects</a> that help you do this, but I have not ac­tu­ally tried them.
In­stead, I ask the agent to make a plan ﬁrst, then re­work that plan my­self to min­i­mize the num­ber of changes.
In the end, a plan which was orig­i­nally writ­ten by the agent be­comes one that has been mer­ci­lessly edited by my­self.
In this way, I min­i­mize the amount of code which de­mands con­sump­tion.
The side-ef­fect of this is that I then have an ex­cel­lent un­der­stand­ing of what the agent ac­tu­ally does, which helps with the con­text prob­lem.
The ﬁ­nal strat­egy, which I use when all else fails, is to refac­tor the code by hand.
When the agent has proven the tech­ni­cal fea­si­bil­ity of the pro­ject, but the code is sim­ply too ex­pan­sive to prop­erly grok all at once, I sit down with a ﬁne-tooth comb and refac­tor it, mod­ule by mod­ule.
I do not need to care­fully read each func­tion and eval­u­ate its mer­its, but I do force my­self to, at min­i­mum, lay eyes on them all.
<h2>Stay Steady Out There</h2>
I ﬁnd my­self be­com­ing more over­whelmed by the dis­course every sin­gle day.
It’s ex­haust­ing how much is hap­pen­ing in the world, and even worst, how much you are ex­pected to care about it all.
Take some time for your­self.
Save your­self from slop paral­y­sis.

A complete or partial loss of function while reviewing the output of a coding agent.

A com­plete or par­tial loss of func­tion while re­view­ing the out­put of a cod­ing agent.

Slop Paralysis

<h1>Naming Harper</h1>
<a href="https://github.com/elijah-potter/harper/issues/103">Someone</a> re­cently asked me where the name <a href="https://writewithharper.com">Harper</a> came from.
When I ﬁrst sat down to start work on Harper, I had one goal in mind.
I wanted to cre­ate a gram­mar checker that would­n’t im­pinge on the artis­tic free­dom of au­thors.
Part of that meant that I did not want lit­er­ary de­vices to be ﬂagged as gram­mat­i­cal er­rors.
Harper should speed up the writ­ing process rather than slow it down as Grammarly tends to.
To em­body this orig­i­nal goal, I wanted to make the pro­ject a name­sake of an au­thor known for their lit­er­ary de­vices and prose. I also had three other (practical) cri­te­rion:
<ol>
<li>It could not be oc­cu­pied on <a href="https://crates.io/">crates.io</a>.</li>
<li>It could not be oc­cu­pied on GitHub.</li>
<li>It should not be more than two syl­la­bles.</li>
</ol>
I asked around for some ideas and some­one sug­gested I name the pro­ject af­ter <a href="https://en.wikipedia.org/wiki/Harper_Lee">Harper Lee</a>.
It ﬁt the bill, so I went with it.

Elijah Potter

Naming Harper

Comments

Other Stuff

Local-First Software Is Easier to Scale

Do Not Type Your Notes

Slop Paralysis