<h1>The Easiest Way to Run LLMs Locally</h1>
<img src="/images/llama.webp" alt="A Goofy Lookin&#x27; Llama">
<h2>LLMs</h2>
<p>Unless you've been living under a rock for the past year, you already know what LLMs are.
If you <em>do</em> happen to be one of the lucky few unaware of the current hype around these things, I'll go through it real quick.</p>
<p>A large language model (or LLM) is a statistical model capable of "predicting" a subsequent word or letter, given a body of text.
Essentially, it is a computer program capable of filling in the blank.
If you let it predict the next word, then feed the result back in, you can get some pretty human-looking text.</p>
<h2>Let's Be Clear</h2>
<p>I hold a lot of skepticism on the practical applications of LLMs as a tool.
As a blanket rule, I never use LLMs or any similar technology in my education.</p>
<p>I know some people ask LLMs questions like "explain the fundamental theorem of calculus to me like I'm five."
While they may get good results for questions, I do not want to lean on them as a crutch.
College is not only an opportunity to learn the raw material, but also an opportunity to learn how to learn.
If we know anything about LLMs, it's that its ability to answer complex questions break down as you move to more specialized classes.</p>
<p>Which is all to say: I did not investigate this with the intention of using it as a tool, I just wanted to play around.</p>
<h2>My Circumstance</h2>
<p>I use <a href="https://archlinux.org/">arch, btw.</a>
While I enjoy the level of control it provides, I don't think it's for everybody.
This is partly because some things are quite difficult to set up.</p>
<p>For example, GPU support is limited and finicky, especially if you run an Intel Arc card, like I do.
While it works perfectly for some apps, like <a href="https://www.blender.org/">blender</a>, it doesn't work so well for other things.
My card only has 3 GB of VRAM, so it wouldn't be able to fit most models anyway.</p>
<p>So when I took on the task of running an LLM on my local machine, I started at looking at CPU-only solutions.</p>
<p>Initially, I tried to raw-dog <a href="https://github.com/ggerganov/llama.cpp">llama.cpp</a>.
That <em>worked</em> but only so.
The command-line interface left a lot to be desired, and the process of downloading and loading various models was tedious and confusing.</p>
<h2>Ollama</h2>
<p>That's when I discovered <a href="https://ollama.ai/">Ollama</a>.
Installing it was as easy as running:</p>
<pre><code class="hljs language-bash">sudo pacman -S ollama
</code></pre>
<p>To avoid wasting resources on multiple instances of each model, Ollama uses a server architecture.
You can start the server by running</p>
<pre><code class="hljs language-bash">ollama serve
</code></pre>
<p>Then, you can download an start chatting with a model with:</p>
<pre><code class="hljs language-bash">ollama run llama2
<span class="hljs-comment"># Or:</span>
ollama run mistral
</code></pre>
<h2>That's It</h2>
<p>That's it!
It really is that simple.</p>
<p>Again, you might have no reason to do any of this.
Especially if you are happy with the privacy nightmare that is OpenAI, Google or Anthropic, or if you already have a system that works for you.</p>

It saved me enough time, I had some to share about it.

<p>It saved me enough time, I had some to share about it.</p>

The Easiest Way to Run Llms Locally

<h1>What Is Randomness?</h1>
<img src="/images/numbers_on_a_screen.jpg" alt="Numbers on a screen">
<p>In case you were born yesterday, let's go over it.</p>
<p>Randomness is, at the most basic level, something that cannot be predicted.
In computer science, when we talk about randomness, we are usually talking about random numbers and the tools we use to get them, random number generators (RNGs).</p>
<h2>Pseudorandomness</h2>
<p>We say an RNG is “pseudorandom” when we have to give it a fixed seed, and it generates random numbers based on that seed.
If we give it the same seed, we will get the same numbers.
It is deterministic.</p>
<p>I thought, for a very long time, that computers could only create pseudorandom numbers.
The main reason I thought this, and why you might too, is because utilities like the Random class in .NET are pseudorandom and are seeded by some arbitrary information, like the current time.</p>
<p>Computers aren't useful if they aren't deterministic.
In a perfect, enclosed system, it would be impossible for a computer to generate truly random numbers.</p>
<h2>True Randomness</h2>
<p>The phrase “true randomness” is used to describe things that are impossible to predict. Provably so. This is different from chaotic systems, which are predictable under short spans of time.
True randomness is impossible to predict on all scales.</p>
<h2>The Everyday Method</h2>
<p>Allow me to introduce you to <a href="https://en.wikipedia.org/wiki/RDRAND">RDRAND</a> and <a href="https://en.wikipedia.org/wiki/RDRAND#RDSEED">RDSEED</a>, two CPU instructions originally introduced by Intel, that allow programs access to truly random numbers.
These instructions gather data from an on-chip entropy source to provide random numbers.</p>
<p>These instructions utilize thermal noise to produce white noise, which is used to generate said random numbers.
The advantage of thermal noise is that it produces actually random values, as explained here.
Its also fast, which means it can be used for everyday things, like generating SSL or TLS keys and the like.</p>
<h3>Do It Yourself</h3>
<p>If you happen to be on a Linux system, you can actually use these instructions yourself. Just run this command:</p>
<pre><code class="hljs language-bash"><span class="hljs-built_in">dd</span> <span class="hljs-keyword">if</span>=/dev/random count=4 bs=1 status=none | <span class="hljs-built_in">od</span> -An --format=dI
</code></pre>
<p>This command uses <code>dd</code> to generate 4 random bytes.
Then it pipes those bytes into <code>od</code> , which will format those 4 bytes into human-readable text as a signed 32-bit integer.</p>
<h2>The Nuclear Method</h2>
<img src="/images/nuclear_plant.jpg" alt="A Nuclear Plant">
<p>One of the big advancements of the 20th Century was the creation of a branch of research called quantum physics.
One of the things we learned was that on the scale of individual particles, it becomes provably impossible to predict state.
Every time a measurement is taken of a subatomic particle, its state changes randomly.</p>
<p>One way we could sample subatomic particles is by placing a Geiger counter next to a bit of radioactive ore.
Because whether an individual atom will decay at any given moment, we can know that the time between ticks of the Geiger counter is random.</p>
<h2>The Cloudflare Method</h2>
<img src="/images/lava_lamp.jpg" alt="A Lava Lamp">
<p>Cloudflare uses some especially interesting sources of entropy to seed its random number generators.
One way is via lava lamps.
In their lobby, they have a large array of lava lamps.
There is a camera running a live feed of the lava lamps to their servers.
There are two main sources of randomness.</p>
<p><strong>First</strong>, the lava lamps themselves.
The movement of even a single lava lamp, if we forget the second law of thermodynamics, is far too chaotic to predict.
If we include the second law of thermodynamics, it becomes true randomness.</p>
<p><strong>Second</strong>, the camera sensor's noise.
Thanks to the photoelectric effect the noise that appears on a camera's sensor, however unnoticeable, is truly random.</p>
<p>By combining these two sources, you get a pool of truly random numbers ~60 times a second.
If that doesn't fit your needs, you can use those numbers to feed a cryptographically secure pseudorandom number generator (CSPRNG) to get as many as you want.</p>

We look at several interesting ways computers generate random numbers. It may fascinate you to know that some methods are not *truly* random, but only an approximate.

<p>We look at several interesting ways computers generate random numbers. It may fascinate you to know that some methods are not <em>truly</em> random, but only an approximate.</p>

3 Awesome Ways Computers Generate Randomness

How I built a software render engine from scratch, and you can too.

<p>How I built a software render engine from scratch, and you can too.</p>

Building a Software Render Engine from Scratch

photography_as_meditation

<h1>Photography as Meditation</h1>
<p>The goal of meditation is to become more mindful and aware.
For me, it is often an attempt to get an appreciation for my surroundings—to see how utterly complex the world is.</p>
<img src="/images/pink_flower_tree.webp" alt="Spring has sprung in Colorado. That means pink flowers everwhere.">
<p>Photography does something similar for me.
The heft of the camera in my hands reminds me to slow down and take in the view.</p>
<p>My mother is a photographer—a great one at that.
She's been in the business for at least as long as I've been alive.
She tells me something similar: the act of framing a picture is itself a way to understand your subject better.</p>
<img src="/images/puddle.webp" alt="Spring brings rain and rain brings puddles.">

Reflecting on the clarity photography brings.

<p>Reflecting on the clarity photography brings.</p>

Elijah Potter

Photography as Meditation

Other Stuff

The Easiest Way to Run Llms Locally

3 Awesome Ways Computers Generate Randomness

Building a Software Render Engine from Scratch