LaTeX Support Is Coming to Harper

It’s been a long time com­ing, which is why I’m thrilled to say that Harper will soon sup­port LaTeX. This is some­thing our users have been plead­ing for since the be­gin­ning of the pro­ject. LaTeX sup­port feels like a fea­ture that is com­par­a­tively low‑ef­fort, but un­locks a mas­sive new con­sumer seg­ment with lit­tle ex­ist­ing com­pe­ti­tion. It has the po­ten­tial to be a huge boon to our user base and our of­fer­ing over­all.

For con­text, LaTeX is a doc­u­ment markup lan­guage (similar in soul to Markdown), pri­mar­ily used by STEM pro­fes­sions and stu­dents. It’s ex­tra­or­di­nar­ily good at for­mat­ting and ren­der­ing com­plex math­e­mat­i­cal equa­tions and chem­i­cal for­mu­las.

The Market

Neither Grammarly nor LanguageTool sup­port LaTeX doc­u­ments, yet sites like Overleaf (the most pop­u­lar LaTeX ed­i­tor) reg­u­larly see an ex­cess of 400,000 daily ac­tive users. That’s an enor­mous mar­ket of peo­ple who tend to be tech­ni­cal and who care about their pri­vacy. Since the two most pop­u­lar gram­mar check­ers (Grammarly and LanguageTool) don’t sup­port sites like Overleaf in any way, we don’t have much com­pe­ti­tion to worry about.

The story is sim­i­lar (albeit to a lesser scale) for Typst, an­other doc­u­ment markup lan­guage. We haven’t even merged sup­port for Typst yet, and we’re al­ready see­ing peo­ple pick up Harper ex­plic­itly for use in the Typst on­line ed­i­tor.

Previous Attempts

On two sep­a­rate oc­ca­sions, mem­bers of the Harper com­mu­nity have opened (and sub­se­quently closed) pull re­quests which tried to add LaTeX sup­port to Harper. I’ve put links to that work be­low for pos­ter­ity.

• PR by Grant Lemons
• PR by Shreyas Minocha

The fun­da­men­tal prob­lem with LaTeX is that its un­der­ly­ing lan­guage, TeX, is not reg­u­lar. In or­der to parse the lan­guage, you need to ac­tu­ally run it. This is why LaTeX doc­u­ments can take as many as sev­eral full min­utes to com­pile and ren­der to a PDF.

Rather than “correctly” read and run the TeX code con­tained within LaTeX doc­u­ments, these pre­vi­ous at­tempts (including mine, which we will dis­cuss in a sec­ond) choose to as­sume that LaTeX com­mands are key­words in a more ab­stract lan­guage. Some edge‑cases will nat­u­rally arise when we ig­nore the low­est level of ab­strac­tion, but do­ing so al­lows us to parse most real doc­u­ments with less ef­fort.

Implementation Details

It was very im­por­tant to me that our LaTeX sup­port was fast, small, and could be com­piled to run in the browser. That last bit is crit­i­cal, be­cause pre­vi­ous at­tempts (by the com­mu­nity) to add LaTeX sup­port tried to use Tree‑sitter, which can­not be eas­ily com­piled to run in the browser. Almost all LaTeX users (which is the vast ma­jor­ity of STEM un­der­grad­u­ate and grad­u­ate stu­dents) use Overleaf, an on­line LaTeX com­piler and ed­i­tor. Developing LaTeX sup­port with­out mak­ing it pos­si­ble to use Overleaf would be like de­vel­op­ing a text ed­i­tor that can’t be used with a key­board. Simply il­log­i­cal.

To make this hap­pen, I wrote my parser from scratch in Rust. Actually, call­ing it a parser would be an in­sult to all of parser‑kind. My so­lu­tion pro­duces a char­ac­ter mask. That is, it iden­ti­fies which parts of the doc­u­ment are real English text, and which parts are LaTeX code. From there, I can just mark all the LaTeX sec­tions as “unlintable” and thus un­de­serv­ing of Harper’s dis­cern­ing gaze.

The cu­ri­ous can check out my PR.

What’s Next

While we’re si­mul­ta­ne­ously work­ing on rolling out our new Weirpack sys­tem and our small lan­guage model, I plan to work with some of our un­der­grad­u­ate STEM stu­dent users. I find it quite likely there’s a use‑case for our LaTeX and Typst sup­port that I’m not con­sid­er­ing. It’s a huge mar­ket, and I’m ex­cited to work with those in it.

A Quick Correction

A few days af­ter writ­ing this post, I found ev­i­dence that LanguageTool sup­ports an older ver­sion of the Overleaf ed­i­tor. While it does­n’t seem to sup­port the lat­est ver­sion, this new in­for­ma­tion makes my pre­vi­ous state­ment that it “doesn’t sup­port sites like Overleaf in any way” false. I apol­o­gize for the mis­in­for­ma­tion.