<h1>The Books I Read in February and March 2025</h1>
<p>In a world containing TikTok, YouTube and a vast array of other apps that profit off human attention, it takes effort to place our focus on things that truly benefit our minds.</p>
<p>At least, that is what I would I say on LinkedIn as an explanation for why I read.
In reality: <strong>I simply enjoy it</strong>.</p>
<p>Each month I go back and summarize what I've learned from the books I've read.
In doing so, I've found that I get both deeper insights and greater joy from the literature.</p>
<h3>Wait! What's Up With February?</h3>
<p>While I only finished one book in February, I started reading several others that I ended up finishing in March.
It only felt right to combine the two months into a single post.</p>
<h2><a href="https://app.thestorygraph.com/books/2986591a-fa02-4b02-b180-13b27708155c"><em>The Dinner</em> by Herman Koch</a></h2>
<p>Someone convinced me to read this book based on its simple premise: the entire novel is an account of a single dinner.
I was skeptical.
It is a big book.
How could they possibly fit that much plot into a single dinner?</p>
<p>The answer: Koch drip-feeds you information about the eclectic characters in attendance.
They gain an additional dimension that you couldn't achieve in another, more action-packed book.</p>
<p>I won't spoil anything here, but the ending was a complete surprise to me.
I did <em>not</em> see that coming.</p>
<h2><a href="https://app.thestorygraph.com/books/798488cf-0270-436c-900f-aac5fc26c71d"><em>Gold</em> by Isaac Asimov</a></h2>
<p>I remember reading <em>Gold</em> for the first time on a camping trip to the <em>Great Sand Dunes National Park</em>.
I was captivated by the dynamic short-stories contained within.
At the time I believed them to be pure science fiction.</p>
<p>Reading <em>Gold</em> for the second time revealed how wrong I was.
More than one of the fictional ideas Asimov presents in the collection of short stories has since come to reality.</p>
<h2><a href="https://app.thestorygraph.com/books/6844cc6f-2652-415e-8783-d544cf26e3c8"><em>My Grandma Asked Me to Tell You She's Sorry</em> by Fredrik Backman</a></h2>
<p>I'll admit: this novel was challenging to finish.</p>
<p><strong>Pros:</strong> the main character (an almost-eight year old girl) is expertly conveyed in the writing.
Backman makes clever use of run-on sentences to convey frustration, which is a literary device I've never seen before.
Further, the character has an unusual degree of insight into everyday aspects of life.
I found myself appreciating <em>my</em> every day more placing myself in her shoes.</p>
<p><strong>Cons:</strong> it takes a long time for the insights to start coming.
An excessive degree of unimportant information is thrown at you right out of the gate.
I had difficulty maintaining my focus until the second half of the book (when stuff starts heating up).</p>
<h2><a href="https://app.thestorygraph.com/books/836aff10-4b4e-4b28-8a89-c8ed9e3f966a"><em>Lessons in Chemistry</em> by Bonnie Garmus</a></h2>
<p>Garmus knocked it out of the park for her first novel.
It follows Elizabeth Zott, who, in all respects is the person I want to be when I grow up.</p>
<p><em>Lessons in Chemistry</em> is a testament to the value of subtraction.
The original manuscript of this book was about twice as long as the final published version.
The author was forced to condense it down after getting rejected from several publishers.
It effort shows.</p>
<p>I cannot capture the novel's elegance of prose, creativity in fiction, or importance in morality here.
<strong>You need to read this book</strong>.</p>

<p>Two months does not mean twice as many.</p>

The Books I Read in February and March 2025

<h1>Continuations on Transformation-based Learning</h1>
<img src="/images/clear_creek_steps.webp" alt="Steps near Clear Creek. Taken by me.">
<p>The most common type of machine learning out there takes the form of some kind of neural network. Inspired by how our own brains work, these systems act as function approximations. They are great, but they come with a few key pitfalls.</p>
<p>First and foremost, they start out with very little baked-in understanding of the context they live in. This is fine—usually enough data can be provided to bridge the gap. It does mean, however, that they spend an inordinate amount of time learning the fundamentals of their field. This translates to a larger model size and longer inference time (especially at <a href="https://en.wikipedia.org/wiki/Edge_computing">the edge</a>).</p>
<p>Secondly, most neural networks are initialized with randomness, which results in extremely high entropy. High entropy means that these models cannot be compressed easily (if at all).</p>
<p>This disqualifies neural networks from many aspects of <a href="https://writewithharper.com/">Harper's architecture</a>. Harper tries to be fast and small, so it can be shipped and run wherever our users are. Neural networks (especially in the world of natural language processing) are neither fast nor small.</p>
<p>This is why we've taking an alternative approach to machine learning, as evidenced by last week's post on <a href="https://elijahpotter.dev/articles/transformation-based_learning">transformation-based learning</a>.</p>
<h2>Transformation-Based Learning: A Refresher</h2>
<p>Transformation-based learning is remarkably simple. It boils down to just four steps:</p>
<ul>
<li>Use a simple, stochastic model t label your data. This can be as simple as tagging each token (or other discrete component) with that variant’s most common tag. It doesn’t need to super accurate, just enough to establish a baseline.</li>
<li>Identify the errors between the tags in your canonical data and that which produced by your baseline model.</li>
<li>Using a finite list of human-defined templates, generate candidate rules that transform the output of the baseline model into something else. This is where the term “transformation-based” comes from.</li>
<li>Apply each of the candidate rules to the baseline model’s output. Check if the result is more accurate than before. If so, save the rule for future use.</li>
</ul>
<p>These saved candidates become your model.</p>
<p>If you're interested how this could be applied to POS tagging, I've since updated my original post on the subject to better explain the process. I'd recommend taking a look.</p>
<h2>Nominal Phrase Chunking</h2>
<p>It's often useful, especially when building a grammar checker, to be able to identify the subjects and objects of sentences. Suppose, for example, that we want to insert the missing Oxford comma in a list of fruits: "I like apples, bananas and oranges". In this trivial example, this can be done with POS tagging. If we have more complex subjects, like in the phrase "I like green apples, deliciously pernicious bananas and fresh oranges," POS tagging starts to fall apart. Identifying multi-token subjects is the job of a nominal phrase chunker.</p>
<p>I've been wanting to build a nominal phrase chunker for a while, but haven't had the tools to do so. Now that I have pipeline in place (from last week), it should be relatively straightforward.</p>
<p>For the purposes of this model, we'll be tagging each token with a boolean; it is either a member of a noun phrase, or it is not.</p>
<p>I started by assigning each token to a nominal phrase if a POS tagger marks it as a noun. This is our baseline model. It performs poorly because the resulting nominal phrases do not include determiners or any adjectives.</p>
<p>Similar to our POS tagging model, I used a Universal Dependencies treebank to determine the accuracy of our baseline. After generating candidate rules using the same patch templates as the POS tagging system and running them against the treebank, I have a model with a 90% accuracy.</p>
<p>I feel as though there should be more details to share, but that was pretty much it. I spent a good amount of time optimizing the training code. There's still a lot of work left to do to incorporate it into the rest of Harper. I am also unsatisfied with the model's current accuracy. To get it closer to 100%, I suspect I'll need to do a good amount of data cleaning.</p>

The most common type of machine learning out there takes the form of some kind of neural network. Inspired by how our own brains work, these systems act as function approximations. They are great, but they come with a few key pitfalls.

<p>The most common type of machine learning out there takes the form of some kind of neural network. Inspired by how our own brains work, these systems act as function approximations. They are great, but they come with a few key pitfalls.</p>

More Transformation Based Learning

How I built a software render engine from scratch, and you can too.

<p>How I built a software render engine from scratch, and you can too.</p>

Building a Software Render Engine from Scratch

the_three_steps_to_an_apology

<h1>The Three Steps to an Apology</h1>
<p>My grandmother is a wonderful woman. She never leaves the house without a least a little bit of purple. Maybe a keychain or a purse. Sometimes, she wouldn't leave the house without a lot of purple, head to toe. She also never forgets a smile and a good attitude.</p>
<p>She's a wise woman. With more than 70 years and five children under her belt, she's seen some stuff. All that experience comes with a lot of advice. The three steps to an apology are some of the best bits of advice she's ever given me.</p>
<h2>Step One: "I am sorry"</h2>
<p>Admit you've done something wrong. Recognize your fault and imagine what would have happened if you had done the right thing.</p>
<h2>Step Two: "It will not happen again"</h2>
<p>If you're actually sorry, you'll want to avoid replicating the same mistake at a later date. Figure out what you need to do to make sure you this is a one-time error.</p>
<h2>Step Three: "What can I do to make it better?"</h2>
<p>Attempt repair. In a fantasy world, you could go back in time and stop yourself from doing whatever it is you did in the first place. But we don't live in a fantasy world, so you have to find a way to make it right, as soon as possible.</p>

Elijah Potter

The Three Steps to an Apology

Step One: "I am sorry"

Step Two: "It will not happen again"

Step Three: "What can I do to make it better?"

Other Stuff

The Books I Read in February and March 2025

More Transformation Based Learning

Building a Software Render Engine from Scratch