Software 3.0 is the Wrong Framing

I just finished reading Project Hail Mary. Late to the party, I know. Regardless, it’s a great book!

So naturally I Googled showtimes for this weekend and this is what popped up:

Seems fine, right? Wait, where’s the link to Totem Lake Cinemark? And now that I look closer, those are typical showtimes. Why is it guessing??? Just tell me the times. And there’s a disclaimer at the bottom? Feels like a pretty benign search, I don’t really need a disclaimer. Oh, the disclaimer is telling me to verify the times in the above links. Wait…the link is to IMDb, not Cinemark. I’ll just do a separate search and go directly to the Cinemark site. Oh wait, Project Hail Mary is no longer showing at Totem Lake Cinemark.

I tried the search again and got this breath of fresh air:

---

The Software 3.0 Mental Model

Andrej Karpathy has been talking about the idea of Software 3.0 for a while now. The basic gist is this:

• Software 1.0: explicit rules → programming = writing code
• Software 2.0: learned weights → programming = creating datasets and training neural networks
• Software 3.0: generative AI → programming = prompting / managing context

I 100% agree that there are new, jaw-dropping possibilities. Where I diverge is the idea that this is an evolution. He seems to suggest that Software 1.0 will eventually be superseded by the later generations. He uses loaded words, like saying tool use will become a “historical appendage” for deterministic tasks.

Tactile Controls 2.0

The 2020 Subaru Outback introduced a massive infotainment screen. You can’t adjust climate controls without looking down and tapping on a touchscreen. The 2026 model reintroduced physical climate control and everyone is happy about it.

Of course screens are great, but no one would describe buttons as “Tactile Controls 1.0”. They have different strengths and serve different purposes. And importantly there are tradeoffs. I’m still yet to see anyone successfully type on a smartphone…but we know a keyboard would take up too much precious real estate in your pocket, so we accept the design decision.

An Alternative Mental Model: Probabilistic and Deterministic Systems

Am I just being closed-minded? To his credit, Karpathy is being asked to predict the future. And AI is making breakneck progress.

But LLMs are probabilistic systems. As the models get better, the probability of the desired outcome increases, but it will never be 100%.

So instead:

• Software 1.0 explicit rules = deterministic systems ⭤ buttons
• Software 3.0 GenAI = probabilistic systems ⭤ touchscreens

Many systems are too complex to make deterministic. This is why GOFAI failed in the first place. There are innumerable cases where fuzziness is needed; as such, LLMs have unlocked incredible potential. But I cannot think of a case where I would not choose the deterministic system were it an available option. So, probabilistic systems don’t supersede deterministic systems, they complement them.

And as AI improves and the infrastructure surrounding it matures, the answer will increasingly be a combination of the two. Google’s movie showtimes is a great example of this: the fact that I can loosely talk to my computer and have it pull up the correct program to display showtimes is the perfect balance of deterministic and probabilistic functionality.

I would add that there are other tradeoffs. For example, it is quite hard to reprogram “software 3.0”. Yes, you can prompt engineer, but as Andrej says himself, we are “at the mercy of whatever the labs are doing, whatever they happen to put into the mix”.

Dreamers and Pragmatists

In a recent interview, Karpathy was asked what the future would bring — what would seem obvious and natural looking back. He answered with MenuGen:

His point was that MenuGen is an example of Software 3.0: image in, image out. He then discussed the idea of a “neural computer”, where the primary engine is the LLM.

It takes creativity and skill to dream up what has never been done before. But it takes a different and equally important set of skills to discern between can and should.

Google Lens is a technological marvel — sci-fi turned reality:

MenuGen is AI slop.

Here are some suggested alternatives to MenuGen:

• Use your brain and visualize something
• Learn something: if you don’t know a word, Google it
• Talk to a living, breathing human being: ask the staff, ask the people at the table next to you

Karpathy is a dreamer. But let’s keep some pragmatists in the room when discussing real products. Or else we might be seeing this with the next software update:

macOS can make mistakes, please double-check everything your computer does.