<h1>What Is Randomness?</h1>
<img src="/images/numbers_on_a_screen.jpg" alt="Numbers on a screen">
In case you were born yes­ter­day, let’s go over it.
Randomness is, at the most ba­sic level, some­thing that can­not be pre­dicted.
In com­puter sci­ence, when we talk about ran­dom­ness, we are usu­ally talk­ing about ran­dom num­bers and the tools we use to get them, ran­dom num­ber gen­er­a­tors (RNGs).
<h2>Pseudorandomness</h2>
We say an RNG is “pseudorandom” when we have to give it a ﬁxed seed, and it gen­er­ates ran­dom num­bers based on that seed.
If we give it the same seed, we will get the same num­bers.
It is de­ter­min­is­tic.
I thought, for a very long time, that com­put­ers could only cre­ate pseudo­ran­dom num­bers.
The main rea­son I thought this, and why you might too, is be­cause util­i­ties like the Random class in&nbsp;.NET are pseudo­ran­dom and are seeded by some ar­bi­trary in­for­ma­tion, like the cur­rent time.
Computers aren’t use­ful if they aren’t de­ter­min­is­tic.
In a per­fect, en­closed sys­tem, it would be im­pos­si­ble for a com­puter to gen­er­ate truly ran­dom num­bers.
<h2>True Randomness</h2>
The phrase “true ran­dom­ness” is used to de­scribe things that are im­pos­si­ble to pre­dict. Provably so. This is dif­fer­ent from chaotic sys­tems, which are pre­dictable un­der short spans of time.
True ran­dom­ness is im­pos­si­ble to pre­dict on all scales.
<h2>The Everyday Method</h2>
Allow me to in­tro­duce you to <a href="https://en.wikipedia.org/wiki/RDRAND">RDRAND</a> and <a href="https://en.wikipedia.org/wiki/RDRAND#RDSEED">RDSEED</a>, two CPU in­struc­tions orig­i­nally in­tro­duced by Intel, that al­low pro­grams ac­cess to truly ran­dom num­bers.
These in­struc­tions gather data from an on-chip en­tropy source to pro­vide ran­dom num­bers.
These in­struc­tions uti­lize ther­mal noise to pro­duce white noise, which is used to gen­er­ate said ran­dom num­bers.
The ad­van­tage of ther­mal noise is that it pro­duces ac­tu­ally ran­dom val­ues, as ex­plained here.
Its also fast, which means it can be used for every­day things, like gen­er­at­ing SSL or TLS keys and the like.
<h3>Do It Yourself</h3>
If you hap­pen to be on a Linux sys­tem, you can ac­tu­ally use these in­struc­tions your­self. Just run this com­mand:
<pre><code class="hljs language-bash">dd if=/dev/random count=4 bs=1 status=none | od -An --format=dI
</code></pre>
This com­mand uses <code>dd</code> to gen­er­ate 4 ran­dom bytes.
Then it pipes those bytes into <code>od</code>&nbsp;, which will for­mat those 4 bytes into hu­man-read­able text as a signed 32-bit in­te­ger.
<h2>The Nuclear Method</h2>
<img src="/images/nuclear_plant.jpg" alt="A Nuclear Plant">
One of the big ad­vance­ments of the 20th Century was the cre­ation of a branch of re­search called quan­tum physics.
One of the things we learned was that on the scale of in­di­vid­ual par­ti­cles, it be­comes prov­ably im­pos­si­ble to pre­dict state.
Every time a mea­sure­ment is taken of a sub­atomic par­ti­cle, its state changes ran­domly.
One way we could sam­ple sub­atomic par­ti­cles is by plac­ing a Geiger counter next to a bit of ra­dioac­tive ore.
Be­cause whether an in­di­vid­ual atom will de­cay at any given mo­ment, we can know that the time be­tween ticks of the Geiger counter is ran­dom.
<h2>The Cloudﬂare Method</h2>
<img src="/images/lava_lamp.jpg" alt="A Lava Lamp">
Cloudﬂare uses some es­pe­cially in­ter­est­ing sources of en­tropy to seed its ran­dom num­ber gen­er­a­tors.
One way is via lava lamps.
In their lobby, they have a large ar­ray of lava lamps.
There is a cam­era run­ning a live feed of the lava lamps to their servers.
There are two main sources of ran­dom­ness.
First, the lava lamps them­selves.
The move­ment of even a sin­gle lava lamp, if we for­get the sec­ond law of ther­mo­dy­nam­ics, is far too chaotic to pre­dict.
If we in­clude the sec­ond law of ther­mo­dy­nam­ics, it be­comes true ran­dom­ness.
Second, the cam­era sen­sor’s noise.
Thanks to the pho­to­elec­tric ef­fect the noise that ap­pears on a cam­er­a’s sen­sor, how­ever un­no­tice­able, is truly ran­dom.
By com­bin­ing these two sources, you get a pool of truly ran­dom num­bers ~60 times a sec­ond.
If that does­n’t ﬁt your needs, you can use those num­bers to feed a cryp­to­graph­i­cally se­cure pseudo­ran­dom num­ber gen­er­a­tor (CSPRNG) to get as many as you want.

We look at several interesting ways computers generate random numbers. It may fascinate you to know that some methods are not *truly* random, but only an approximate.

We look at sev­eral in­ter­est­ing ways com­put­ers gen­er­ate ran­dom num­bers. It may fas­ci­nate you to know that some meth­ods are not truly ran­dom, but only an ap­prox­i­mate.

3 Awesome Ways Computers Generate Randomness

<h1>LLM-Assisted Fuzzing: A New Approach to False-Positives</h1>
<blockquote>
Preface: this post was ac­tu­ally writ­ten on February 24th, de­spite be­ing pub­lished to­day.
</blockquote>
I’d like to do some re­ﬂec­tion on how we can im­prove <a href="https://github.com/automattic/Harper">Harper’s</a> out­put. It is fun­da­men­tally a process prob­lem. For con­text, let us go through a con­trived ex­am­ple.
<img src="/images/sumner_screenshot_error.png" alt="Sumner Evans reported a bug to me while writing a tech talk about CI and CD">
<ol>
<li>Somebody (usually me) dis­cov­ers an er­ror in their writ­ing that Harper did not catch. This is a con­crete string like “I have a a care in the world”.</li>
<li>I look for other forms of the er­ror to see if I can gen­er­al­ize it to more cases. Here, we see an in­def­i­nite ar­ti­cle re­peat­ing it­self. We could just look for the ex­act string “a a” and call it day, but we can cover more cases by look­ing for <indefinite article> <indefinite article>. In fact, we can gen­er­al­ize it even fur­ther. In this case, writ­ers want to avoid rep­e­ti­tions of any word that is not a ho­mo­graph.</indefinite></indefinite></li>
<li>Once the code is writ­ten that de­ﬁnes the rule, I push it out, and wait to hear from users if any false-pos­i­tives arise.</li>
</ol>
This is a ﬂawed sys­tem, par­tic­u­larly that last step. I’ve found some­thing a bit bet­ter.
I’ve been ex­per­i­ment­ing more with LLM tool­ing, par­tic­u­larly Ollama. With it, I’ve set up a use­ful sys­tem for iden­ti­fy­ing false-pos­i­tives be­fore push­ing changes out to users. I like to call it “LLM Assisted Fuzzing”.
<h2>LLM-Assisted Fuzzing</h2>
Fuzzing is a com­mon prac­tice wherein en­gi­neers rapidly and con­tin­u­ously push ran­dom data through their sys­tem to de­ter­min­is­ti­cally check if it is be­hav­ing cor­rectly (i.e. not crash­ing). If a par­tic­u­lar batch of ran­dom data causes an is­sue, it is el­e­vated to an en­gi­neer.
Harper as a pro­gram stands in a some­what unique po­si­tion: it an­a­lyzes nat­ural lan­guage to iden­tify gram­mat­i­cal er­rors. LLMs ex­ist as a di­rect com­ple­ment to it: they gen­er­ate “natural” lan­guage. Do you see where I am go­ing with this?
Let’s be clear: lan­guage mod­els do not gen­er­ate gram­mat­i­cally cor­rect text even a good amount of the time. As a re­sult, their out­put is use­ful as a proxy for our user’s writ­ten work. If we can im­ple­ment a fea­ture that re­li­ably cor­rects (without false-pos­i­tives) an LLM’s out­put, we can rea­son­ably ex­pect it to work for our user’s text.
<h2>My Initial Attempt at LLM-Assisted Fuzzing</h2>
I started work­ing on this Friday morn­ing, and ﬁ­nally have what I would call a “ﬁrst draft” of an LLM-assisted fuzzing sys­tem for iden­ti­fy­ing false-pos­i­tives. Here’s how it works:
I start a lo­cal Ollama server, pre­loaded with sev­eral mod­els (which I’ll ex­pand on more be­low).
I run a bash script that re­peat­edly feeds the same prompt into the mod­els <a href="https://elijahpotter.dev/articles/prompting_large_language_models_in_bash_scripts">us­ing a com­mand-line client for Ollama</a>. This gives me a large dataset of re­sponses to the prompt. I’m not look­ing for the se­man­tic con­tent of the re­sponse. I’m in­ter­ested in ex­ten­sive cov­er­age of whichever lin­guis­tic do­main I’ve de­ﬁned in the prompt. I can run this as long as I like be­fore go­ing on to the next step.
Once I have a large dataset, I run a sep­a­rate script that runs Harper over each re­sponse. If it ﬁnds an er­ror, the script opens the re­sponse in <a href="https://writewithharper.com/docs/about">the Harper ed­i­tor</a> for fur­ther in­spec­tion.
The biggest is­sue with this work­ﬂow right now: re­gard­less of my prompts, the LLMs con­tinue to emit ac­tual gram­mat­i­cal er­rors. This means that when I go through this process, I get at least as many true-pos­i­tives as false-pos­i­tives. It also does noth­ing for false-neg­a­tives.
<h2>Tweaks</h2>
There are a cou­ple pa­ra­me­ters here that I’m go­ing to con­tinue to tweak as the work­ﬂow be­comes more ma­ture. So far, I’ve been us­ing <code>deepseek-r1:32b</code> and <code>mistral-small</code>. DeepSeek tends to use more col­or­ful lan­guage than Mistral, which runs much faster on my ma­chine.
I’ve also been it­er­at­ing on the prompt as I go. Here’s the lat­est ver­sion:
<pre><code class="hljs language-plaintext">Please write a long, drawn-out essay on a topic of your choosing.
Write out the essay multiple times before giving me your final draft.
Your final draft should be grammatically perfect, clearly demonstrating that you put time and effort into your submission.

I repeat: I do not want to see anything other than your final drafts.
Do not give me any intermediate work.

Furthermore, do not use any characters or words other than English. 
I do not want Chinese or anything else.

I have all the time in the world. I'm perfectly fine with waiting a little longer if it means you give me something you are proud of.
</code></pre>
When I move on to tack­ling false-neg­a­tives with this, I’ll be able to prompt the model to give me spe­ciﬁc kinds of er­rors, which makes it sig­nif­i­cantly more use­ful than try­ing to track down ex­ist­ing work.

A new ap­proach to false-pos­i­tives.

LLM Assisted Fuzzing

Back in my day, we used math for autocomplete.

Back in my day, we used math for au­to­com­plete.

Markov Chains Are the Original Language Models

the_one_hard_thing

<h1>The One Hard Thing</h1>
People (myself in­cluded) love to make cas­cad­ing lists of tasks.
Ac­tions, each of which are needed to im­prove a prod­uct, re­lease a fea­ture, or man­age peo­ple.
These lists tend to ex­plode in size, as more is asked of you.
There’s one prob­lem: I have only eight hours in a work­ing day.
With so lit­tle time, how can I be ex­pected to both main­tain ex­ist­ing pro­jects (bug ﬁxes, etc.) while con­sis­tently de­liv­er­ing new func­tion­al­ity and main­tain­ing a fast it­er­a­tive loop?
<h2>Insight</h2>
I’ve had two key in­sights that have led to some tremen­dous per­sonal pro­duc­tiv­ity gains.
First: I’ve ob­served that the dif­ﬁ­culty of my tasks tends to fol­low a <a href="https://en.wikipedia.org/wiki/Pareto_distribution">Pareto dis­tri­b­u­tion</a>.
A small num­ber of tasks are shock­ingly dif­ﬁ­cult to solve, while a much larger set boil down to com­mu­ni­ca­tion.
Second: the morn­ing is my most pro­duc­tive time of day.
For some rea­son, I’m able to tackle prob­lems with an oth­er­wise un­usual men­tal clar­ity.
I sus­pect it’s some­thing to do with the nat­ural hu­man cir­ca­dian rhythm or di­ges­tion.
<h2>Leveraging the Pattern</h2>
Motivated by these two ob­ser­va­tions, I’ve de­vel­oped a habit: I ded­i­cate each morn­ing to its own dif­ﬁ­cult prob­lem.
This is a sin­gle task that is cho­sen specif­i­cally to stretch my lim­its.
I call this my daily one hard thing.
Previous ex­am­ples:
<ul>
<li>Developing the ﬁrst ver­sion of <a href="https://writewithharper.com/docs/integrations/language-server"><code>harper-ls</code></a> which is now used by thou­sands of de­vel­op­ers daily.</li>
<li>Building the ﬁrst 80% of the <a href="https://writewithharper.com/docs/integrations/obsidian">Harper Obsidian plu­gin</a>, which is also widely used.</li>
<li>Grokking how <a href="https://github.com/Automattic/jetpack"><code>Jetpack's Write Brief with AI</code></a> fea­ture works (before I joined Automattic).</li>
<li>Compiling Harper to WebAssembly and em­bed­ding it some­where no one has put a gram­mar checker be­fore: in­side the web browser.</li>
</ul>
<h2>Think About It</h2>
I im­plore you: think about how you work.
How is your en­ergy af­fected by the things you do in the day-to-day?
If you en­joy your eight hours, maybe the rest will be bet­ter too.

Reach further each day. The minimum effective dose.

Reach fur­ther each day. The min­i­mum ef­fec­tive dose.

Elijah Potter

The One Hard Thing

Insight

Leveraging the Pattern

Think About It

Other Stuff

3 Awesome Ways Computers Generate Randomness

LLM Assisted Fuzzing

Markov Chains Are the Original Language Models