Dealing with Deadlines

Harper will soon be do­ing more work on your de­vice than it ever has be­fore. We are prepar­ing to re­lease a se­ries of fan­tas­tic gram­mar check­ing mod­els (trans­form­ers, in fact) that will sig­nif­i­cantly en­hance the kinds of things Harper can do for you. These mod­els are go­ing to be quite help­ful for every kind of writer, but they come with two ma­jor down­sides: They take time to run, and they con­sume up a lot of com­pute re­sources.

We want to be mind­ful of our user’s com­put­ers, so we want to run these mod­els only when nec­es­sary and we want the process to avoid in­ter­rupt­ing the user’s flow. To that end, I am slightly mod­i­fy­ing how our var­i­ous rules check your text.

In ad­di­tion to our de­sire to avoid wast­ing CPU cy­cles, there’s an­other prob­lem. We aim to of­fer the best cor­rec­tions pos­si­ble in the time be­tween each key­stroke. This al­lows au­thors to min­i­mize the time be­tween when they write text and when they may re­view our sug­ges­tions. Af­ter all, Harper’s ul­ti­mate goal is to bring edit­ing time and en­ergy to 0. To achieve that, we need to be able to do as much analy­sis as pos­si­ble in the time be­tween when the user writes a pas­sage and when they are ready to re­view it.

Before The Deadline

To al­low Harper to do the most work pos­si­ble in the time be­tween when a user fin­ished typ­ing and when they’re ready for re­view, I am in­tro­duc­ing a new idea into Harper’s core gram­mar checker: the dead­line. Here’s how it works.

Harper checks a va­ri­ety of in­di­vid­ual gram­mat­i­cal and styl­is­tic rules. Each one of these rules is pretty self con­tained. You can imag­ine them to be in­di­vid­ual lit­tle pro­grams. Every time Harper “checks” a doc­u­ment for er­rors (or for ways to im­prove it), each one of these pro­grams is run.

To set up a dead­line, Harper will keep track of the through­put (in char­ac­ters per sec­ond) of every gram­mar pro­gram we run and there­fore every gram­mat­i­cal rule we check. When­ever we check a user’s doc­u­ment, Harper will keep track of ex­actly how much CPU time each rule con­sumes, then di­vide it by the doc­u­ment length. If we do this many times with a va­ri­ety of doc­u­ments, we can get a rough es­ti­mate of how long on av­er­age it takes a given rule to process and of­fer sug­ges­tions to im­prove or gram­mar check a sin­gle char­ac­ter

I call this process cal­i­bra­tion. An ini­tial cal­i­bra­tion process will need to run when Harper first starts up. That will hap­pen quickly in the back­ground, so the user will not even no­ticed it hap­pened. Af­ter that, the through­put es­ti­mates will con­tinue to get more ac­cu­rate as the user uses Harper.

Once Harper has been cal­i­brated, we can as­sign dead­lines to spe­cific rules based on when the user might ex­pect them to be avail­able. Some things are most use­ful while the user is ac­tively typ­ing.

For in­stance, I of­ten al­ready know if I have made a typo be­fore I even fin­ish typ­ing a word. When that hap­pens, I ex­pect the spell checker to al­ready be avail­able to fix my prob­lem. With a quick tap of Ctrl+e I can fix it with­out even read­ing what the spell checker thinks the word is. I trust it im­plic­itly to be both fast and cor­rect. Rules like this will be given a very fast dead­line, un­der 16 mil­lisec­onds.

Other types of sug­ges­tions, like full sen­tence or para­graph rewrites, are not as time sen­si­tive. I do not need to read through a sug­ges­tion to rewrite a sen­tence un­til af­ter I am ac­tu­ally fin­ished writ­ing the sen­tence. Rules like these which need more time to process any­way will be given longer dead­lines mea­sured in sec­onds.

Evolution

These dead­lines are only a frame­work to com­mu­ni­cate to Harper’s core how long an op­er­a­tion is al­lowed to take. Fine tun­ing the specifics of what the dead­lines should be is an­other, big­ger prob­lem.

I can imag­ine a fu­ture where we train a small neural net­work to in­fer these dead­lines from past key­strokes, but we are not quite there yet. To start, we will use very coarse es­ti­ma­tions of what a “good” dead­line looks like for var­i­ous sit­u­a­tions. If the course dead­lines turn out to work well enough, great! If not, we can al­ways it­er­ate un­til we at­tain per­fec­tion.

Of course, if any read­ers have any in­put on what a good dead­line looks like, let me know!

Published November 20, 2026 at 10:31 PM

Proofread by Harper.

Follow me on X.

Comments