Adding a Programming Language to Harper

When I started the Harper pro­ject I knew I wanted to be able to use it for the com­ments in my code. First, be­cause I knew these com­ments would be­come part of our of­fi­cial doc­u­men­ta­tion over time, and be­cause I hoped it would en­cour­age me to write more. Over time, this has be­come one of the most prized fea­tures of the soft­ware, at­tract­ing tens of thou­sands of de­vel­op­ers.

The com­mon prob­lem, how­ever, is that there have al­ways been pro­gram­ming lan­guages that our LSP does­n’t sup­port. One of the old­est is­sues on GitHub is about this.

This post is a guide for adding a new pro­gram­ming lan­guage to the Harper lan­guage server.

Why is­n’t it in the of­fi­cial doc­u­men­ta­tion? While the in­for­ma­tion con­tained within this guide will re­main rel­e­vant to the pro­ject for a long time, I don’t imag­ine each iden­ti­fier or file path to re­main the same. If you think it would bet­ter serve po­ten­tial con­trib­u­tors to place this guide on the main site, let me know.

Introduction to Tree-sitter

Tree-sitter is fan­tas­tic frame­work for build­ing fault-tol­er­ant lan­guage parsers. That means it is still able to parse the ma­jor­ity of a doc­u­ment, even if it con­tains por­tions of in­valid syn­tax.

This is im­por­tant for Harper, since we ex­pect peo­ple to use Harper while their pro­gram­ming. It should be OK if some of their code is in­cor­rect, since we only care about their com­ments.

There are also a wide va­ri­ety of Tree-sitter parsers avail­able on crates.io, ripe for our con­sump­tion. If you want to add a lan­guage to Harper, this is the eas­i­est way to do so.

Step 0: Avoid Duplicating Work

You’re in­ter­ested in adding sup­port for a pro­gram­ming lan­guage. If that’s the case, it’s pos­si­ble other peo­ple are too. Make sure no one else has opened a PR or has al­ready merged sup­port for the lan­guage you have in mind.

Step 1: Find a Grammar

Look for an ex­ist­ing gram­mar on crates.io. By con­ven­tion, they tend to be named tree-sitter-<language>, where <language> is the lan­guage you’re look­ing for. For ex­am­ple, tree-sitter-java is for Java and tree-sitter-rust is for Rust.

If you would rather write your own gram­mar, make sure it is even­tu­ally pub­lished on crates.io. harper-ls bi­na­ries are of­ten con­sumed from crates.io, which re­quires that all up­stream de­pen­den­cies come from the same source.

Step 2: Import and Wire In

Harper’s com­ment sup­port lies in the harper-comments crate in the monorepo. Import the gram­mar’s crate into the pro­ject with Cargo.

cargo add <CRATE-NAME>

Then, add lines to the rel­e­vant func­tions in harper-comments/src/comment_parser.rs. Make sure you visit the Language Server Protocol Specification to ob­tain the cor­rect lan­guage ID.

pub fn new_from_language_id(
    language_id: &str,
    markdown_options: MarkdownOptions,
) -> Option<Self> {
    let language = match language_id {
        "cmake" => tree_sitter_cmake::LANGUAGE,
        "cpp" => tree_sitter_cpp::LANGUAGE,
        "csharp" => tree_sitter_c_sharp::LANGUAGE,
        "c" => tree_sitter_c::LANGUAGE,
        "dart" => harper_tree_sitter_dart::LANGUAGE,
        "go" => tree_sitter_go::LANGUAGE, // Add a line here

/// Convert a provided path to a corresponding Language Server Protocol file
/// type.
///
/// Note to contributors: try to keep this in sync with
/// [`Self::new_from_language_id`]
fn filename_to_filetype(path: &Path) -> Option<&'static str> {
    Some(match path.extension()?.to_str()? {
        "bash" => "shellscript",
        "c" => "c",
        "cmake" => "cmake",
        "cpp" => "cpp",
        "cs" => "csharp", // Add a line here

Step 3: Testing

To make sure every­thing be­haves cor­rectly, we need to add some in­te­gra­tion tests. You’ll find all the ex­ist­ing ones un­der harper-comments/tests/language_support_sources.

Find or write sev­eral new files un­der this di­rec­tory in the lan­guage you’ve added sup­port for. Add in­ten­tional gram­mat­i­cal er­rors to these file in syn­tac­ti­cally rel­e­vant places. We want to make sure that Harper can de­tect the er­rors we want and will ig­nore the er­rors we do not want. For ex­am­ple, we might put an er­ror in­side an @param tag in JSDoc. That way we’ll know if Harper is not prop­erly ig­nor­ing those el­e­ments.

Add new en­tries to the bot­tom of harper-comments/tests/language_support.rs. The sec­ond pa­ra­me­ter of the create_test! macro is the num­ber of gram­mat­i­cal er­rors that Harper should de­tect in that file.

create_test!(ignore_shebang_3.sh, 0);
create_test!(ignore_shebang_4.sh, 1);
create_test!(common.mill, 1);
create_test!(basic_kotlin.kt, 0); // Add a line here

From there, you can run cargo test to make sure every­thing passes.

Step 4: Document

To ad­ver­tise sup­port for the lan­guage, there are a cou­ple ad­di­tion places that need mod­i­fi­ca­tion. No­tably:

• The sup­ported lan­guages table in packages/web/src/routes/docs/integrations/language-server/+page.md
• The GitHub Issue
• The activationEvents key in the VS Code plug­in’s man­i­fest: packages/vscode-plugin/package.json

Done!

That should be every­thing. Open a draft pull re­quest while you work and ping me (eli­jah-pot­ter) if you have any ques­tions.

Additional Resources

• PR for adding Solidity sup­port
• PR for adding Scala sup­port