<h1>The Three Steps to an Apology</h1>
<p>My grandmother is a wonderful woman. She never leaves the house without a least a little bit of purple. Maybe a keychain or a purse. Sometimes, she wouldn't leave the house without a lot of purple, head to toe. She also never forgets a smile and a good attitude.</p>
<p>She's a wise woman. With more than 70 years and five children under her belt, she's seen some stuff. All that experience comes with a lot of advice. The three steps to an apology are some of the best bits of advice she's ever given me.</p>
<h2>Step One: "I am sorry"</h2>
<p>Admit you've done something wrong. Recognize your fault and imagine what would have happened if you had done the right thing.</p>
<h2>Step Two: "It will not happen again"</h2>
<p>If you're actually sorry, you'll want to avoid replicating the same mistake at a later date. Figure out what you need to do to make sure you this is a one-time error.</p>
<h2>Step Three: "What can I do to make it better?"</h2>
<p>Attempt repair. In a fantasy world, you could go back in time and stop yourself from doing whatever it is you did in the first place. But we don't live in a fantasy world, so you have to find a way to make it right, as soon as possible.</p>

The Three Steps to an Apology

<h1>The Books I Read in February and March 2025</h1>
<p>In a world containing TikTok, YouTube and a vast array of other apps that profit off human attention, it takes effort to place our focus on things that truly benefit our minds.</p>
<p>At least, that is what I would I say on LinkedIn as an explanation for why I read.
In reality: <strong>I simply enjoy it</strong>.</p>
<p>Each month I go back and summarize what I've learned from the books I've read.
In doing so, I've found that I get both deeper insights and greater joy from the literature.</p>
<h3>Wait! What's Up With February?</h3>
<p>While I only finished one book in February, I started reading several others that I ended up finishing in March.
It only felt right to combine the two months into a single post.</p>
<h2><a href="https://app.thestorygraph.com/books/2986591a-fa02-4b02-b180-13b27708155c"><em>The Dinner</em> by Herman Koch</a></h2>
<p>Someone convinced me to read this book based on its simple premise: the entire novel is an account of a single dinner.
I was skeptical.
It is a big book.
How could they possibly fit that much plot into a single dinner?</p>
<p>The answer: Koch drip-feeds you information about the eclectic characters in attendance.
They gain an additional dimension that you couldn't achieve in another, more action-packed book.</p>
<p>I won't spoil anything here, but the ending was a complete surprise to me.
I did <em>not</em> see that coming.</p>
<h2><a href="https://app.thestorygraph.com/books/798488cf-0270-436c-900f-aac5fc26c71d"><em>Gold</em> by Isaac Asimov</a></h2>
<p>I remember reading <em>Gold</em> for the first time on a camping trip to the <em>Great Sand Dunes National Park</em>.
I was captivated by the dynamic short-stories contained within.
At the time I believed them to be pure science fiction.</p>
<p>Reading <em>Gold</em> for the second time revealed how wrong I was.
More than one of the fictional ideas Asimov presents in the collection of short stories has since come to reality.</p>
<h2><a href="https://app.thestorygraph.com/books/6844cc6f-2652-415e-8783-d544cf26e3c8"><em>My Grandma Asked Me to Tell You She's Sorry</em> by Fredrik Backman</a></h2>
<p>I'll admit: this novel was challenging to finish.</p>
<p><strong>Pros:</strong> the main character (an almost-eight year old girl) is expertly conveyed in the writing.
Backman makes clever use of run-on sentences to convey frustration, which is a literary device I've never seen before.
Further, the character has an unusual degree of insight into everyday aspects of life.
I found myself appreciating <em>my</em> every day more placing myself in her shoes.</p>
<p><strong>Cons:</strong> it takes a long time for the insights to start coming.
An excessive degree of unimportant information is thrown at you right out of the gate.
I had difficulty maintaining my focus until the second half of the book (when stuff starts heating up).</p>
<h2><a href="https://app.thestorygraph.com/books/836aff10-4b4e-4b28-8a89-c8ed9e3f966a"><em>Lessons in Chemistry</em> by Bonnie Garmus</a></h2>
<p>Garmus knocked it out of the park for her first novel.
It follows Elizabeth Zott, who, in all respects is the person I want to be when I grow up.</p>
<p><em>Lessons in Chemistry</em> is a testament to the value of subtraction.
The original manuscript of this book was about twice as long as the final published version.
The author was forced to condense it down after getting rejected from several publishers.
It effort shows.</p>
<p>I cannot capture the novel's elegance of prose, creativity in fiction, or importance in morality here.
<strong>You need to read this book</strong>.</p>

<p>Two months does not mean twice as many.</p>

The Books I Read in February and March 2025

<h1>LLM-Assisted Fuzzing: A New Approach to False-Positives</h1>
<blockquote>
<p><strong>Preface</strong>: this post was actually written on February 24th, despite being published today.</p>
</blockquote>
<p>I'd like to do some reflection on how we can improve <a href="https://github.com/automattic/Harper">Harper's</a> output. It is fundamentally a process problem. For context, let us go through a contrived example.</p>
<img src="/images/sumner_screenshot_error.png" alt="Sumner Evans reported a bug to me while writing a tech talk about CI and CD">
<ol>
<li>Somebody (usually me) discovers an error in their writing that Harper did not catch. This is a concrete string like "I have a a care in the world".</li>
<li>I look for other forms of the error to see if I can generalize it to more cases. Here, we see an indefinite article repeating itself. We could just look for the exact string "a a" and call it day, but we can cover more cases by looking for <indefinite article> <indefinite article>. In fact, we can generalize it even further. In this case, writers want to avoid repetitions of any word that is not a homograph.</li>
<li>Once the code is written that defines the rule, I push it out, and wait to hear from users if any false-positives arise.</li>
</ol>
<p>This is a flawed system, particularly that last step. I've found something a bit better.</p>
<p>I've been experimenting more with LLM tooling, particularly Ollama. With it, I've set up a useful system for identifying false-positives before pushing changes out to users. I like to call it "LLM Assisted Fuzzing".</p>
<h2>LLM-Assisted Fuzzing</h2>
<p>Fuzzing is a common practice wherein engineers rapidly and continuously push random data through their system to deterministically check if it is behaving correctly (i.e. not crashing). If a particular batch of random data causes an issue, it is elevated to an engineer.</p>
<p>Harper as a program stands in a somewhat unique position: it analyzes natural language to identify grammatical errors. LLMs exist as a direct complement to it: they generate "natural" language. Do you see where I am going with this?</p>
<p>Let's be clear: language models do not generate grammatically correct text even a good amount of the time. As a result, their output is useful as a proxy for our user's written work. If we can implement a feature that reliably corrects (without false-positives) an LLM's output, we can reasonably expect it to work for our user's text.</p>
<h2>My Initial Attempt at LLM-Assisted Fuzzing</h2>
<p>I started working on this Friday morning, and finally have what I would call a "first draft" of an LLM-assisted fuzzing system for identifying false-positives. Here's how it works:</p>
<p>I start a local Ollama server, preloaded with several models (which I'll expand on more below).</p>
<p>I run a bash script that repeatedly feeds the same prompt into the models <a href="https://elijahpotter.dev/articles/prompting_large_language_models_in_bash_scripts">using a command-line client for Ollama</a>. This gives me a large dataset of responses to the prompt. I'm not looking for the semantic content of the response. I'm interested in extensive coverage of whichever linguistic domain I've defined in the prompt. I can run this as long as I like before going on to the next step.</p>
<p>Once I have a large dataset, I run a separate script that runs Harper over each response. If it finds an error, the script opens the response in <a href="https://writewithharper.com/docs/about">the Harper editor</a> for further inspection.</p>
<p>The biggest issue with this workflow right now: regardless of my prompts, the LLMs continue to emit actual grammatical errors. This means that when I go through this process, I get at least as many true-positives as false-positives. It also does nothing for false-negatives.</p>
<h2>Tweaks</h2>
<p>There are a couple parameters here that I'm going to continue to tweak as the workflow becomes more mature. So far, I've been using <code>deepseek-r1:32b</code> and <code>mistral-small</code>. DeepSeek tends to use more colorful language than Mistral, which runs much faster on my machine.</p>
<p>I've also been iterating on the prompt as I go. Here's the latest version:</p>
<pre><code class="hljs language-plaintext">Please write a long, drawn-out essay on a topic of your choosing.
Write out the essay multiple times before giving me your final draft.
Your final draft should be grammatically perfect, clearly demonstrating that you put time and effort into your submission.

I repeat: I do not want to see anything other than your final drafts.
Do not give me any intermediate work.

Furthermore, do not use any characters or words other than English. 
I do not want Chinese or anything else.

I have all the time in the world. I'm perfectly fine with waiting a little longer if it means you give me something you are proud of.
</code></pre>
<p>When I move on to tackling false-negatives with this, I'll be able to prompt the model to give me specific kinds of errors, which makes it significantly more useful than trying to track down existing work.</p>

<p>A new approach to false-positives.</p>

LLM Assisted Fuzzing

harper_in_the_news

<h1>Harper in the News</h1>
<p>The biggest complaint about Harper is that the quality of grammar checking still needs some work.
I've got an exciting new project I'm preparing to discuss later this week (hopefully) in that area.
It should be a step-function improvement over our previous methods for creating new rules.</p>
<p>In the meantime, I've collected some recent sightings of Harper in the wild:</p>
<ul>
<li><a href="https://obsidian.rocks/resource-harper/">Obsidian Rocks</a></li>
<li><a href="https://www.wearefounders.uk/the-grammar-checker-that-actually-gets-developers-meet-harper/">We Are Founders</a></li>
<li><a href="https://itsfoss.com/harper-grammar-checker/">It's FOSS</a></li>
<li><a href="https://www.xda-developers.com/ditched-grammarly-for-this-amazing-open-source-alternative/">XDA Developers</a></li>
</ul>

I've got some exciting new work I'm preparing to discuss later this week (hopefully). In the meantime, I've collected some recent sightings of Harper in the wild.

<p>I've got some exciting new work I'm preparing to discuss later this week (hopefully). In the meantime, I've collected some recent sightings of Harper in the wild.</p>

Elijah Potter

Harper in the News

Other Stuff

The Three Steps to an Apology

The Books I Read in February and March 2025

LLM Assisted Fuzzing