We’re now about 18 months into the AI revolution. One thing that was uncertain in late 2022 was whether prompt engineering would be around to stay, or whether better AI would quickly obviate it.

I now think it’s around to stay and I have an explanation that makes sense to me, at least: prompt engineering is just a subset of software engineering.

That is, prompts are tiny programs written in natural language. But the API isn’t specified and varies between models. So guessing the right “function calls” with clever use of vocabulary is a huge part of the game. On the other hand, even if you don’t guess *exactly* the right words to use, the model will often do what you mean.

This is different from how we normally think of an API, which is both more legible and more fragile. The exact words to make an API do what you want are written down, but if you don’t say those exact words it won’t do what you want.

Even given this difference, the concept of prompts as tiny programs using hidden APIs helps explain the bizarre magic associated with specific phrases. I’m reminded of Quake3’s fast inverse square root[1,2], which has a famously obscure incantation in C that just so happened to deliver a 4X speedup.

More code now looks like that, and it makes sense. C is how you talk to machines and English is how you talk to humans. So, just like you write part of a large application in C for performance, you’ll also write part of it in English for dealing with unstructured data.

You can go further with this analogy. Once you think of prompts as code, you can probably generate model-aware syntax highlighters for favored keywords. You can maybe automatically generate API-like docs from a model for the most common use cases.

And you can think of every new model you add to your codebase as roughly analogous to adding a new programming language — because just as it takes time for someone to ramp up on the idioms of Rust, they’ll need to play around with the latest Mistral to get the hang of how to talk to it.

Anyway — this is all probably obvious to folks spending 100% of their time in the field, and is similar to some of the things Andrej Karpathy has posted about, but at least for me it was a useful articulation of why prompts are around to stay: prompts are tiny programs.