Building Software That Is Good for Humans

Earlier to­day, the European Commission re­leased a re­port de­tail­ing their ini­tial find­ings that TikTok’s core rec­om­men­da­tion al­go­rithm is in breach of the Digital Services Act. This is only the most re­cent episode in the ex­tended saga where west­ern gov­ern­ments try to reg­u­late the Chinese com­pany. The var­i­ous rea­sons and the meth­ods they em­ploy in this reg­u­la­tion are far out­side of my do­main of ex­per­tise, but the re­port got me think­ing. Why does so much of the soft­ware we use feel harm­ful?

There are few peo­ple who can re­port that an in­ter­ac­tion with Microsoft Word, Teams, or TikTok has left them feel­ing em­pow­ered, health­ier, or hap­pier. On the other hand, I can think of plenty of soft­ware that has left me feel­ing great. What is that? How can we make sure our soft­ware sits in the lat­ter group?

Traits of Offensive Software

There are a few fun­da­men­tal things that I’ve found make us­ing cer­tain soft­ware feel like a chore. Soft­ware feels “icky” when it is some com­bi­na­tion of slow, un­pre­dictable, or when it gets in the way of the mean­ing­ful pur­suits in life.

It Is Slow

Computers are fast. Peo­ple un­der­stand that. So if they’re left wait­ing for a page to load or a process to fin­ish, they feel frus­trated.

Slow soft­ware makes it dif­fi­cult to it­er­ate and ex­per­i­ment. Imag­ine if WordPress took 20 min­utes to pub­lish a post, only to dis­cover af­ter­ward that you left a typo in the ti­tle. That would be pretty frus­trat­ing, no?

Steve Jobs imag­ined Apple Computers to be “bicycles for the mind.” A bi­cy­cle whose chain hitched or whose steer­ing lagged not be very help­ful. It might not get you to your des­ti­na­tion at all.

I think that this idea of slow­ness is es­pe­cially im­por­tant in re­gards to AI agents. If an as­sis­tant spends even a few sec­onds more than what is crit­i­cally nec­es­sary for the query, and they’re mes­saged fre­quently (say, to it­er­ate on a blog post), users can quickly get tired or “brainrotted”, as the kids say.

It Is Random

If some soft­ware works some­times, but not al­ways, there’s a good chance it’s ad­dic­tive. This pre­sents most ob­vi­ously in apps like TikTok, but it also ap­pears in more in­sid­i­ous places. The in­fer­ence of large lan­guage mod­els in­volves ran­dom­ness in its very na­ture.

Fundamentally, this from of ad­dic­tion comes from a quirk of hu­man psy­chol­ogy. Ran­domly ap­plied re­wards (the soft­ware some­times works, but not al­ways) is a form of op­er­ant con­di­tion­ing. It feels good to roll the dice, which makes us want to roll it more of­ten. Over time, rolling the dice starts to be less of a choice, and more of a habit.

When some­one chooses to use soft­ware not be­cause it’s gen­uinely help­ful, but be­cause it feels good to roll the dice, a sense of agency is lost.

In my ex­pe­ri­ence, Microsoft Word is like this, as are most agen­tic cod­ing tools and im­age gen­er­a­tors.

It Distracts from That Which Makes Life Meaningful

When soft­ware is slow, or oth­er­wise re­moves your agency with vari­able re­ward sys­tems, it can start to chip away at your abil­ity to make rea­son­able de­ci­sions on where to spend your time. Peo­ple who use “social” me­dia sites like Facebook or Instagram can at­test to this. They sim­ply don’t feel like they have agency over their own ac­tions any­more. They can’t stop us­ing soft­ware they ac­tively hate.

Our Goals

I think that one of Automattic’s great­est strengths is that none of our soft­ware does any one of these things. I re­mem­ber the first time I used Pocket Casts: I was thrilled. It was eas­ily the best pod­cast soft­ware I had ever used. The same things hap­pened with Day One and WordPress.com.

As we look to build­ing Harper, I think it’s im­por­tant that we re­mem­ber these core prin­ci­ples. Our soft­ware should be fast, pre­dictable, and sup­port our cus­tomers in find­ing mean­ing in their lives.