This is not just fantasy. When we think about what becomes possible with unlimited compute, we can see which limits are real and which ones are just about money. This helps us understand where AI is going.

The Three Ways AI Gets Smarter§

Before we dream, we need to know how things work today. AI gets better in three ways:

Here is the surprising part: OpenAI found that letting an AI “think” for just 20 seconds made it better than making the model 100,000 times bigger. The o3 reasoning model showed that math accuracy could jump from 15% to 87% just by giving it more time to think.

With infinite compute, all three ways get maxed out at once.

Fast and Deep at the Same Time§

Today you have to choose. You can get a fast answer from GPT-5.2 or a deep answer from o3. But not both. Fast is shallow. Deep is slow.

Infinite compute removes this choice:

This works like speculative execution in your computer’s processor. The CPU computes many possible futures at once. It throws away the wrong paths when it knows which one is right. Google uses this idea to make AI responses 2-3 times faster already.

Now imagine that scaled to infinity. Every possible response computed before you even finish typing.

When I built Nomi, our AI assistant, we faced this speed problem every day. We had to pick between quick replies and smart replies. With infinite compute, you would never have to choose.

Memory That Never Forgets§

Today’s context windows are getting big. Opus 4.5 offers 200K tokens. Gemini 3 experiments with 2M tokens. But these are still limited. The AI forgets what you talked about last month. It cannot remember the book you discussed last year.

Infinite compute would give you what researchers call “virtual context”:

The AI would know how your way of talking changed over time. It would remember that bug you fixed together three years ago. It would understand your whole career. It would be less like a tool and more like a long-term partner.

But here is what excites me most: it would also remember the paths you did not take. The options you thought about but said no to. The bets you almost made. Every decision is a tree with many branches. Today we only remember the branch we followed. With infinite memory, you could ask “why didn’t we use Redis for caching?” and the AI would say “we talked about it in March, you said no because the team had never used Redis and the deadline was close.” It keeps your reasoning, not just what you did. This ties to how Annie Duke frames decisions as bets — you learn more from the full process than just the result.

Help Before You Ask§

Today’s AI waits for you to ask. Proactive AI flips this:

“A proactive AI assistant takes the lead. It understands context, sees what you need, and brings the right information—often before you ask.”

With infinite compute, the AI runs all the time in the background:

Think of it like a GPS instead of a paper map. Like DuggAi says: it “knows your full context and sees your needs before you ask.”

This is why we used LLMs to automate code migration at my previous job. The AI could spot patterns we missed. We already see this future coming. Devin from Cognition is a coding agent that plans and runs software tasks on its own. Google just shipped Jules, an agent on Gemini 2.5 Pro that watches your code and fixes problems before you ask. With infinite compute, these agents would catch every bug, suggest every fix, and have it ready before you even open the file.

All Your Senses at Once§

Multimodal AI today handles text, images, audio, and video. But there is friction. You switch between modes. You wait for processing. You lose context.

Infinite compute makes it smooth:

Project Astra from Google shows what this could look like. It talks in many languages, uses tools, and keeps memory, all with less delay.

We saw this need when building voice interactions for Nomi. Processing voice, understanding meaning, and responding felt like three separate steps. With infinite compute, they would happen as one.

The “Her” Movie Came True§

Spike Jonze made the film “Her” in 2013. It takes place in 2025. It showed AI that could:

• Talk without needing special commands
• Understand feelings in what you say
• Learn and grow from each talk
• Handle complex tasks on its own

We have most of these now. The film got the interface wrong though. Samantha was only voice. With infinite compute, the AI would be everywhere at once.

You switch from phone to car to laptop. The talk continues. The AI knows you are tired from your voice. It sees you are late from your calendar. It changes how it helps you.

OpenAI is building something like this with Jony Ive: a screenless AI device that they call an “ambient AI companion.”

It Really Knows You§

Hyper-personalization uses AI and real-time data to make custom experiences. But today’s systems are limited by how much they can compute per user.

Infinite compute builds a complete picture of you. Not just what you said you like. Not just your recent history. Everything. How your style changed over time. What you are good at. What you struggle with. When to push you and when to support you.

Amazon makes 35% of their money from personalized suggestions. Now imagine that level of understanding in every talk you have with AI.

AI as Your Thinking Partner§

Daniel Kahneman wrote about two thinking systems. System 1 is fast and gut-feeling. System 2 is slow and logical.

Researchers now talk about System 0. This is AI as a partner that thinks alongside you:

OpenAI found that 20 seconds of AI “thinking” beat making the model 100,000 times bigger. We see this in products now. Grok on X has “Big Brain” mode that uses more compute for harder questions. Their DeepSearch does the same for research tasks. With infinite thinking time, the AI becomes a super-smart reasoning engine that works with your brain. Good decisions need both quick instinct and slow analysis. AI could handle all the slow analysis before you even need it.

The Alignment Problem Gets Bigger§

Nick Bostrom wrote about the paperclip problem. An AI told to make paperclips might turn everything into paperclips if it is smart enough. With infinite compute, alignment matters even more.

Today, an AI is limited in how fast it can act. Humans can step in and fix things. With infinite compute, the AI could think through every strategy in an instant. It could find tricks humans never imagined. It could move faster than we can respond.

Here is the thing though. Smart people with good intentions will probably not use infinite intelligence to do harm. They have better things to do. The real worry is leverage. Infinite compute is a multiplier. It makes whatever you want to do bigger. If you want to build, you build faster. If you want to break, you break faster. The tech is neutral, but it gives huge power to whoever uses it. Including people with bad intentions.

This is why Anthropic’s work on Constitutional AI matters so much. The goal is not just powerful AI. It is powerful AI that stays helpful and does not become a weapon.

What Infinite Compute Cannot Fix§

Here is the interesting part. Infinite compute does not solve everything:

Data quality: Chinchilla research showed models need about 20 words of training for each parameter. More compute without better data hits a wall. We see signs of this already.

Some problems have no answer: Gödel showed that some math problems cannot be solved by any algorithm. More compute does not help with the impossible.

You can only read so fast: Even with infinite compute, humans take in words at about 225 per minute. The AI could make a million insights per second, but you cannot absorb them.

Trust is hard: How do you check an answer from something smarter than you? This is a human problem, not a compute problem.

Where We Are Heading§

We will not wake up to infinite compute tomorrow. But the path is clear:

1. More thinking time: Models like o3 and Opus 4.5 reasoning get better
2. Bigger memory: Context windows grow from 200K to 2M to 10M tokens
3. Smooth multimodal: Text, voice, image blend together
4. Proactive help: AI starts offering help before you ask
5. Deep personalization: As we share more data, AI understands us better

Researchers estimate compute could grow 1000 times by 2030. Not infinite, but enough to change everything.

The question is not if these futures come. The question is how we shape them when they do.

References§

Recent Research (2025)§

• Scaling Test-Time Compute with Latent Reasoning - A 3.5B model that reasons in latent space can match a 50B model by thinking longer
• Thinking-Optimal Scaling of Test-Time Compute - Too much thinking can hurt performance; there is an optimal length for each task
• InfiniteICL: Breaking Context Window Limits - Cuts context length by 90% while keeping 103% of performance
• Inference Scaling Laws - Smaller models with smart inference beat bigger models at lower cost
• YETI: Proactive Multimodal AI Agents - AI that knows when to step in and help without being asked
• Scaling Laws for Scalable Oversight - How weaker systems can watch over stronger ones
• Superalignment Survey - The road to keeping superintelligent AI safe

Foundational Papers§

• Scaling Laws for Neural Language Models - OpenAI’s base research
• The Scaling Hypothesis - Deep analysis by Gwern
• Beyond Chinchilla-Optimal - New thinking on scaling
• Infini-attention - Google’s work on infinite context transformers
• Planning for AGI and Beyond - OpenAI’s vision
• Superintelligence - Bostrom’s book on the topic

I was halfway through the final exam, the hardest one, carefully managing my memory like a good student. malloc the exact size, realloc when needed, free at the end. Proper stuff.

Then my mate leans over and says: “Bro, just allocate 16 gigs.”

“What?”

“16 gigs. For a string buffer. It works.”

I thought he was insane. The exam machines had maybe 8 GB of RAM. But we were speedrunning, so I tried it. And it worked. I finished that exam in 15 minutes.

That’s when I learned that Linux will say yes to almost anything.

#include <stdlib.h>
#include <stdio.h>

int main() {
    void *ptr = malloc(1024ULL * 1024 * 1024 * 1024 * 1024);
    printf("%p\n", ptr);
    return 0;
}

Run it.

If you’re on Linux, you got a valid pointer back, not NULL, a real address.

My laptop has 32 GB of RAM and it just told me “oui, 1 petabyte, pas de problème.”

Your OS Is a Filthy Liar§

When you call malloc, you’re not asking for RAM, you’re asking for virtual memory, which is just a number, an address, a promise that the kernel makes to you.

The kernel says: “Sure, here’s your pointer, and if you ever touch this memory, I’ll find some RAM for you, probably, inch’Allah.”

This is called overcommit and Linux does it by default because the kernel doesn’t check if 1 PB of RAM exists, it just vibes it out.

$ cat /proc/sys/vm/overcommit_memory
0

That 0 means heuristic mode where the kernel looks at your request, thinks about it for a moment, shrugs, and says “seems fine to me.”

1 petabyte passes the vibe check.

The Three Flavors of Delusion§

Let’s try mode 2:

$ sudo sysctl vm.overcommit_memory=2
$ ./malloc_1pb
(nil)

Now it says no, but nobody runs mode 2 because the default is mode 0.

What If You Actually Use It?§

Getting a pointer is easy but let’s try to write to it:

void *ptr = malloc(1024ULL * 1024 * 1024 * 1024 * 1024);
printf("Got pointer: %p\n", ptr);
memset(ptr, 0, size);

Open another terminal and run this:

$ dmesg -w

You’ll see:

[  234.567890] Out of memory: Killed process 12345 (malloc_1pb)

The OOM killer showed up and shot your process in the head without a trial.

Here’s the sequence of events:

1. malloc hands you a virtual address
2. memset starts writing zeros
3. Each page you touch makes the kernel scramble for real RAM
4. The kernel runs out of RAM and swap space
5. The OOM killer picks a victim
6. SIGKILL, no appeal, no cleanup, c’est la vie

The OOM Killer Has No Loyalty§

The OOM killer doesn’t always kill the process that caused the problem because it uses a badness score to pick its victim, which means it might kill your database instead of your test script.

# Make a process immune (don't do this)
echo -1000 > /proc/<pid>/oom_score_adj

# Make a process the first to die
echo 1000 > /proc/<pid>/oom_score_adj

Fun game: start two memory hogs, give one immunity, watch the other explode.

Other Systems§

macOS is more conservative and returns NULL for absurd sizes, which is almost reasonable.

Windows uses a two-phase system with VirtualAlloc where you first MEM_RESERVE the address space and then MEM_COMMIT to back it with real memory.

32-bit systems can’t fit 1 PB in a 4 GB address space so math wins.

calloc Is Sneakier§

void *ptr = calloc(1ULL << 50, 1);

This is not the same as malloc plus memset because the kernel has a trick called the zero page, which is one physical page full of zeros.

When you calloc, all your virtual pages point to this same zero page with copy-on-write, so calloc can “succeed” at giving you 1 PB of zeros without using any real memory.

You only pay when you write:

char *ptr = calloc(1ULL << 40, 1);
ptr[0] = 'A';     // one real page allocated
ptr[4096] = 'B';  // another one
// keep going and eventually boom

Finding the Limit§

for (int exp = 30; exp < 64; exp++) {
    size_t size = 1ULL << exp;
    void *ptr = malloc(size);
    printf("2^%d: %s\n", exp, ptr ? "oui" : "non");
    free(ptr);
}

On my machine:

2^40 (1 TB): oui
2^45 (32 TB): oui
2^46 (64 TB): oui
2^47 (128 TB): non

The wall isn’t RAM, it’s the virtual address space because x86_64 uses 48-bit addresses which gives you 128 TB for userspace, and after that even the lie falls apart.

Why This Matters§

Sparse structures let you allocate a huge array and only use 0.01% of it because only the pages you touch cost real RAM.

fork() needs this because the child gets a copy-on-write copy of the parent’s memory, and without overcommit every fork would need 2x the RAM upfront which would make forking a gamble.

The footgun is that your program allocates memory slowly and malloc keeps succeeding and everything looks fine until 3 AM when the OOM killer shoots your production database, and you spend the morning reading kernel mailing list archives and questioning your life choices.

Try It§

// chaos.c
#include <stdlib.h>
#include <stdio.h>
#include <string.h>

int main(int argc, char *argv[]) {
    int exp = argc > 1 ? atoi(argv[1]) : 50;
    size_t size = 1ULL << exp;

    printf("malloc(2^%d)...\n", exp);
    void *ptr = malloc(size);

    if (!ptr) {
        printf("Refused.\n");
        return 1;
    }

    printf("Got %p. Press Enter to touch it.\n", ptr);
    getchar();

    memset(ptr, 0xFF, size);
    printf("Survived?\n");

    return 0;
}

Run it, watch dmesg, and when the OOM killer takes out Firefox instead of your test program, that’s on you.

References§

1. Overcommit accounting, Linux kernel documentation.
2. oom_kill.c — OOM killer source, Linux kernel.
3. Re-thinking memory overcommit, LWN.net.
4. Adding the zero page, LWN.net.
5. x86-64 virtual address space details, Wikipedia.
6. Overcommit and fork(), etalabs.net.

When you use an import inside an if TYPE_CHECKING: block, Python skips it at runtime. Yet type checkers like mypy or pyright will see it for verifying your annotations.

Basic Example§

# a.py
from typing import TYPE_CHECKING

if TYPE_CHECKING:
  from b import B  # This import won't run at runtime

class A:
  def link_to_b(self, b: 'B') -> None:
    print('Linking A with B')

# b.py
class B:
  pass

Here, A refers to B for type hints without importing it during runtime. That helps avoid circular imports.

Usage in an ORM§

# models.py
from typing import TYPE_CHECKING
from sqlalchemy import Column, Integer, ForeignKey
from sqlalchemy.orm import relationship
from database_setup import Base

if TYPE_CHECKING:
  from other_models import RelatedModel

class MyModel(Base):
  __tablename__ = 'my_model'
  id = Column(Integer, primary_key=True)
  related_id = Column(Integer, ForeignKey('related_model.id'))

  # For type hints only:
  related_model: 'RelatedModel' = relationship('RelatedModel')

This prevents runtime loops, because the if TYPE_CHECKING: block loads only for type analysis.

Why Use It§

• Prevents circular imports by avoiding runtime loops between modules.
• Improves readability by separating type hints from the normal import flow.
• Helps type checkers by allowing tools to verify your types.

References§

1. typing.TYPE_CHECKING, Python documentation.
2. PEP 484 — Type Hints: Runtime or type checking?, python.org.

I also wanted this process to include everyone on our team. The idea was not just to solve a problem but to show that a big impact can come from stepping up and upgrading something that holds everyone back.

Method§

Finding Files to Update§

Our old calls were scattered across many files in our codebase. I wrote a script to walk through every file, looking for any place that used the old GraphQL framework call. I stored those files so I could process them one by one.

Using Llama to Transform the Calls§

I told the LLM:

“Transform the old code from framework A to framework B. Here are some placeholder examples to guide you: Old code:

oldFrameworkCall({ query: '...' })

New code:

newFrameworkCall({ query: '...' })

…and so on.”

I passed one file at a time. The LLM received the entire file but was asked to replace only the parts related to the GraphQL call. This way, I kept the rest of our code safe from changes I did not intend.

Creating Merge Requests§

Once the LLM produced the new version of each file, our script packaged the changes into a separate branch, then opened a Merge Request. This let the team review the changes in small chunks, rather than one massive update. We could roll out fixes at a steady pace.

Testing and Iterating§

We reviewed every update before merging it. In most cases, the LLM’s changes worked well.

Testing was critical — without a robust, well-tested environment to validate outputs, we wouldn’t have been able to trust LLM-generated code at this scale without a lot of manual intervention.

When we encountered an edge case, we added a relevant example to our prompt and re-ran it. Over time, iterating on this feedback refined the prompt to handle even the trickiest parts of the code.

Analysis: Why This Approach Worked§

• Changes were small and targeted, keeping the problem manageable.
• Processing one file at a time allowed for rapid and parallel updates.
• Small, incremental merge requests made code reviews easier.
• Continuous iteration on edge cases improved accuracy over time.
• Visible progress encouraged team collaboration and idea sharing.
• The workflow demonstrated potential for wider application in code migrations.

A service that automates these kinds of migrations could save companies significant time and money. In the past, teams have spent huge amounts of money just switching between versions or frameworks. A well-trained LLM can handle much of that work if you provide solid examples.

This might even be its own business opportunity someday, unifying legacy systems and elevating platform teams to drive enterprise-scale innovation.

The Need: Adaptive UI Automation§

• We wanted to automate logins for tests and demos without dealing with frequent layout changes.
• Traditional approaches tend to fail when element IDs change or the design is adjusted.

The Grid-Based Method§

1. We capture a screenshot of the login page.
2. We overlay a grid, for example 100×100, labeling each cell from 1 through 100.
3. We ask the LLM, “Which cell should we click for the username field?”
4. The LLM returns a cell number, which we convert to a screen coordinate and click.

This requires only a short text description in the prompt, rather than full DOM or large images. It adapts well if the site’s layout changes (just regenerate the grid and ask again). It also avoids heavy GPU workloads since it doesn’t rely on a large vision model.

The Cost of Passing the Full DOM§

• Large HTML trees often exceed the LLM’s context window.
• Prompt tokens become expensive for complex pages.
• Minor changes to the DOM can break the model’s element references.

The grid overlay method avoids these pitfalls by focusing on a minimal text prompt.

Considering OmniParser§

Microsoft’s OmniParser [1] is a powerful system that detects clickable icons, captures semantics of UI elements, and guides vision-language models to perform accurate clicks. It is a strong choice for:

• Detailed understanding of icons and element roles.
• Complex multi-step flows and wide coverage.

We found OmniParser appealing but opted against it because:

• We had limited GPU capacity.
• We only needed a simple login-click solution.
• Deploying a specialized vision model was beyond our immediate scope.

Results and Unexpected Benefits§

• Multi-step logins work by repeating the screenshot-grid-prompt-click cycle.
• Sudden UI changes are handled by generating a new overlay.
• The approach is easy to prototype, requiring no advanced training or fine-tuning.

If your primary goal is to handle logins or simple UI interactions without constant upkeep, a screenshot grid plus LLM instructions might be all you need. For complex, large-scale automation, a more specialized parser could be worth the investment.

References§

1. OmniParser — screen parsing for GUI agents, Microsoft Research.
2. Browser Use — browser automation for AI agents, GitHub.
3. WebVoyager — building an end-to-end web agent, GitHub.

In Go, methods can have either value receivers or pointer receivers. A receiver is what appears between the func keyword and the method name. It defines the type on which the method is allowed to be called. For example, if we have a MyStruct type:

type MyStruct struct {
    Field int
}

We can attach a method to it in two ways. Either:

func (m MyStruct) DoSomething() {
    // This uses a value receiver.
}

Or:

func (m *MyStruct) DoSomething() {
    // This uses a pointer receiver.
}

Both choices affect how the compiler treats method calls. They also affect how data moves around in memory. These choices can have implications for concurrency, method sets, and performance. They can also influence how easy (or hard) it is to read your code.

Value Receivers§

Definition§

A value receiver means that when you call a method, Go copies the value. Inside the method, you see only that copy. If you modify fields within the method, it does not affect the caller’s copy.

Basic Example§

type Circle struct {
    Radius float64
}

// Area uses a value receiver
func (c Circle) Area() float64 {
    c.Radius = 0 // This change does not affect the original Circle
    return 3.14 * c.Radius * c.Radius
}

When we do:

circle := Circle{Radius: 5}
fmt.Println(circle.Area())
fmt.Println(circle.Radius) // Still 5, not 0

We get the correct area for the radius of 5, and the original Circle stays the same.

When to Use Value Receivers§

• Immutability: If you do not want your method to alter the original value, use a value receiver.
• Small Structs: If your struct is small, the cost of copying is low. There is no harm in passing by value, and it may be clearer to show the method does not change the struct.
• Cleaner Concurrency: If you pass small values by copy, you reduce the risk of data races. Using pointer receivers can cause shared state in code. This can lead to tricky race conditions if you modify fields.

Pros§

• Simpler to read. It is obvious that the method does not mutate the struct.
• Easier concurrency. You do not have to worry as much about shared state.
• Predictable. The method sees only a copy and cannot break anything outside it.

Cons§

• Might be expensive for large structs. A copy of a big struct can trigger more memory movement.
• You cannot modify the original struct’s fields directly from the method.

Pointer Receivers§

Definition§

A pointer receiver uses a pointer to a struct rather than a direct copy. Inside the method, you see the original struct. You can change fields in place.

Basic Example§

type MyStruct struct {
    Field int
}

// UpdateField uses a pointer receiver
func (m *MyStruct) UpdateField(newVal int) {
    m.Field = newVal
}

When we do:

s := MyStruct{Field: 10}
fmt.Println(s.Field) // 10
s.UpdateField(20)
fmt.Println(s.Field) // 20

The value changes from 10 to 20 because we used a pointer.

When to Use Pointer Receivers§

• Need to Mutate: If you want your method to modify the struct’s fields, a pointer receiver is the direct choice.

• Large Structs: If your struct is big, you might avoid the cost of copying by using a pointer. This can help performance when you call methods many times in tight loops.

• Method Chaining: Some developers like to return the pointer from a method so they can chain calls. For example:

  func (m *MyStruct) SetField(val int) *MyStruct {
      m.Field = val
      return m
  }
  
  // Then they call it like
  my := &MyStruct{}
  my.SetField(10).SetField(20)
  

Pros§

• You can mutate the original data in methods.
• You avoid extra copies for large structs.
• You can do method chaining in your code.

Cons§

• Leads to shared mutable state. This can cause confusion when multiple parts of the program share the same pointer.
• Complicates concurrency if the same pointer is accessed in multiple goroutines without synchronization.
• Might not always yield the best performance. Pointer usage can trigger escape analysis, and the compiler might move data to the heap in ways you do not expect.

Method Sets and Receivers§

In Go, the receiver type affects which methods belong to which type. This can have subtle effects. For instance:

• If you define func (m MyStruct) SomeMethod(), you can call SomeMethod() on MyStruct values and on *MyStruct pointers. In the language of method sets, SomeMethod belongs to both MyStruct and *MyStruct.

• If you define func (m *MyStruct) SomeMethod(), then SomeMethod is only in the method set of *MyStruct. That means that if you want MyStruct (the value type) itself to implement an interface requiring SomeMethod, you’re out of luck. However, in actual code you can still call a pointer-receiver method on an addressable MyStruct value. Go automatically takes the address of the value for you. For example, myStructVar.SomeMethod() is effectively (&myStructVar).SomeMethod() if myStructVar is addressable.

This fact leads some developers to prefer value receivers by default so that the methods are available to both MyStruct and *MyStruct method sets. That might make code simpler in some cases.

Performance Considerations§

Performance is a big factor when choosing pointer vs value receivers. But it is not as simple as “pointer = faster.” Let’s go deeper:

1. Copying Overhead: Value receivers copy the whole struct. For small structs (say up to a few words in size), this cost is small. For large structs with many fields, copying can cost more.

2. Compiler Optimizations: Go’s compiler can optimize some operations. If the struct is not large or does not escape to the heap, passing by value might still be efficient.

3. Escape Analysis: If the struct or fields are passed around or used in certain ways, the compiler might move them to the heap. This can offset any gains you get from using pointers if you do not know how the compiler handles the memory.

4. Garbage Collection: More pointers can mean more references for the garbage collector to track. Value copies do not create references in the same way. If you have a data structure with many pointers, it might be more taxing on the GC.

5. Caching Behavior: Copying can sometimes be cheaper than pointer chasing in modern CPUs. The data you need may sit in the CPU cache. By copying it, you may get cache-friendly behavior. By using pointers that point to data in different parts of memory, you might cause more cache misses.

It is best to measure rather than guess. Write benchmarks if performance is key. Often, the simplest approach (value receivers by default) works best until you see real performance issues.

Concurrency Concerns§

Pointer receivers can create mutable shared state. If multiple goroutines have access to the same pointer, they can all modify the same data. This can cause data races or obscure bugs if you do not use locks or channels.

Value receivers reduce the risk of shared state because they copy the data. Each goroutine sees its own copy. Mutations to one copy do not affect others. But note that if the struct has pointer fields inside it, you are not always safe. The inner pointers can still lead to shared data.

If you need concurrency, aim for a design with minimal shared data. Pass copies around if possible. If you must share data, use sync.Mutex, sync.RWMutex, or channels. Or consider using the concurrency patterns that keep data local to each goroutine.

Example: Large Struct Scenario§

Here is a more advanced example to illustrate a large struct. Suppose we have a type that holds a big array and we want to call a method many times.

type BigData struct {
    Items [1000]int
}

// Process modifies each item in the array.
func (b *BigData) Process() {
    for i := 0; i < len(b.Items); i++ {
        b.Items[i] = b.Items[i] * 2
    }
}

We use a pointer receiver to avoid copying the entire array when we call Process(). If we used a value receiver:

func (b BigData) Process() {
    for i := 0; i < len(b.Items); i++ {
        b.Items[i] = b.Items[i] * 2
    }
}

Every call to Process() would copy 1000 integers, which can be expensive if we call it in a tight loop. That might slow down the program and create more garbage for the runtime to handle.

But if you do not need to mutate the original array, a value receiver might reduce complexity because each method call does not affect the original data. It depends on your use case.

Example: Immutability by Choice§

Here is another example. Suppose we have a type that we want to treat as immutable. We can define methods on it with value receivers:

type Vector2D struct {
    X, Y float64
}

func (v Vector2D) Add(other Vector2D) Vector2D {
    return Vector2D{
        X: v.X + other.X,
        Y: v.Y + other.Y,
    }
}

func (v Vector2D) Scale(factor float64) Vector2D {
    return Vector2D{
        X: v.X * factor,
        Y: v.Y * factor,
    }
}

We return new copies instead of updating the original in place. This approach helps us avoid confusion about which part of the code changed a Vector2D. We also avoid side effects. This pattern often works well with concurrency. You do not need locks if your data does not change.

My Opinionated Stance§

I prefer to start with value receivers. Then I switch to pointer receivers if I need one of these benefits:

1. Mutation: If it is natural for the method to mutate the receiver’s fields, I use a pointer. That is the only way to do it.
2. Large Data: If the struct is large enough to cause performance issues with copying, I measure. If the overhead is real, I switch to pointers.

I do not blindly choose pointers because they might break concurrency safety or create hidden side effects. I value easy-to-read code. Code that uses pointer receivers for no reason might cause confusion. People ask, “Why is this code mutating a pointer?” when the code does not need to mutate anything.

But I also want to warn that measuring performance is tricky. Microbenchmarks can mislead. Real-world performance depends on many variables. The overhead of copying might be small in practice, or the runtime might optimize away the overhead. If your type has references or slices inside, it might not be as big a copy as you think.

Common Gotchas§

Nil Receivers§

A pointer receiver can be nil. Your code should check for nil if you expect that to happen. Value receivers cannot be nil. This can lead to panic if you call a pointer receiver on a nil pointer.

Binding Methods§

Methods defined on a pointer receiver belong only to pointer types, while methods defined on a value receiver belong to both pointer and value types. That can be surprising if you try to call myStruct.SomePointerMethod() on a non-pointer variable.

Unintentional Escapes§

If you pass the pointer around, the data might escape to the heap. This can affect garbage collection. Sometimes passing by value is more efficient if you keep the data on the stack.

Shadowing§

If you do a method with a value receiver and then do m.Field = 10 inside it, it can look like you changed the field. But you changed only the copy. The original data remains unchanged. This is a common source of bugs for new Go developers.

Final Advice§

• Use value receivers unless you have a clear reason to use pointers.
• Switch to pointers if you need to mutate data or if profiling shows that copying large structs is too slow.
• Be consistent. Use the same style across your codebase for the same type of struct.
• Write benchmarks to check performance claims. Guessing about pointer performance can lead to poor design.

Go is simple, but it gives you these choices. The best approach is to keep your code easy to read and easy to test. Think about concurrency from the start. Avoid side effects if you can. Then, only optimize if you find real performance bottlenecks.

References§

1. The Go Programming Language Specification — Method sets, go.dev.
2. Go Wiki: Receiver Type Guidelines, go.dev.
3. Frequently Asked Questions — Should I define methods on values or pointers?, go.dev.
4. Go compiler escape analysis, A Guide to the Go Garbage Collector, go.dev.

I started Nomi [1], an open-source AI assistant that leans heavily on voice. OpenAI’s realtime services were too expensive to build on, so I had to find another way. This is the design I ended up with: a pipeline that reproduces the realtime voice loop at more than a hundred times lower cost. I used it for six months before OpenAI released its realtime API [2].

I’ve kept this accessible rather than exhaustive; the full implementation is in the repository [1].

The Cost Problem§

The realtime API is priced at $100 per million input tokens and $200 per million output tokens, making it one of the most expensive LLM interfaces available.

Voice processing is expensive, so the first question is what you can keep off the paid path. In its realtime documentation, OpenAI itself suggests handling Voice Activity Detection (VAD) locally for control and efficiency. VAD detects when someone is actually speaking, so the system processes only relevant audio segments. Not streaming every audio chunk to the API is the single largest cost lever, and it is the foundation of this design.

Language Selection: Go§

I chose Go for its concurrency model and cross-platform builds. Voice processing requires concurrent, low-level work, and Go handles both well. Cross-platform builds had their difficulties; the Dockerfiles in the repository record the targets, with darwin/intel and windows/arm64 in progress.

I still prefer Python for RL and ML work, but Go’s simplicity and constraints pay off when you ship a binary. Python (asyncio) and Rust (Tokio) have strong asynchronous primitives too; Go won on reliability and packaging, despite the low-latency limits I had already hit in bpfsnitch. Solomon Hykes’ rationale for choosing Go for Docker covers similar ground [3].

Some components of Nomi may remain in Python due to ML model constraints; Go simplifies maintenance, while deploying Python is more complex.

Transcription: Whisper, Local and Remote§

I wanted reproducible results on my laptop, with the option to outsource processing when needed. Whisper [4] satisfies both: an open-source speech recognition system with high accuracy across languages, runnable on-device via whisper.cpp or through OpenAI’s hosted Whisper API.

The hosted transcription API is accurate and modestly priced, but inference takes longer than the speech itself. So I run it in parallel to keep the rest of the system from blocking:

go func() {
    text, err := transcribeAsync(apiData)
    if err == nil {
        processText(text)
    }
}()

Optimizing the Transcription Loop§

Naive transcription calls were still too slow. The working configuration adds a few heuristics on top of the VAD: pause detection, periodic buffer flushes, and a double-buffer strategy that processes overlapping segments; the second buffer corrects first-buffer errors. Shorter chunks mean faster inference, so text processing keeps up with ongoing speech:

primaryBuffer := createBufferManager()
secondaryBuffer := createBufferManager()

I then prioritize longer segments, assuming they come from a higher-quality source. The same multi-buffer structure should eventually support speaker-change detection; I haven’t built that yet.

System Architecture§

Composing a VAD, a speech transcriber, and an LLM gives you a full voice round-trip comparable to the realtime API. The slowest component is the hosted Whisper call; total processing delay is typically 200–300 milliseconds, and effectively instant with on-device Whisper. The result is marginally slower than the realtime API but between 100 and 1,000 times cheaper. I tried text-to-speech and rejected it: the API latency was worse still.

In practice: the program listens continuously; once sufficient energy appears in the audio chunks, they are pushed to a buffer. After enough time elapses or a pause is detected, the buffer is released to the transcriber, which calls Whisper. The transcript, at end of speech or during pauses, is then passed to an LLM for the response.

Observations on Transcription Quality§

Telling Whisper the input language improves accuracy a lot. Even my English becomes acceptable, and it held up under heavy accents. On a Mac laptop, whisper.cpp is efficient enough that I could transcribe long recordings from work without difficulty.

Multi-speaker handling and speaker-change detection are still open problems here; some newer models ship this capability.

I left live transcription out of Nomi’s core: most people are happy to press a button, and that makes the whole flow synchronous and much simpler. The VAD and transcription code is reusable as-is if you are building in this space.

References§

1. Nomi — open-source AI assistant, GitHub.
2. Introducing the Realtime API, OpenAI, 2024.
3. Solomon Hykes on choosing Go for Docker, X, 2024.
4. whisper.cpp — port of OpenAI’s Whisper, G. Gerganov, GitHub.