Which Way Will Models Go

The so-called foundation models grab all of the headlines at present. And small wonder. They progress at a dizzying pace. The major providers keep leapfrogging one another with new capabilities or improvements in speed, accuracy, and capacity. Where will this lead us over time?

Someone quipped that the main use of large models is to produce small models. Alas I have lost the reference. There is some sense in this. Until I encountered magika the idea of wrapping a model into a command-line executable had never occurred to me. Yes, there is still a high start-up cost, and maybe it pays that back only when used as a UNIX filter with a large number of input paths. Nevertheless, the concept of command-line utility is forever changed.

What might this mean for the future of models generally? I speculate that they will go the way of microprocessors. There will be a limited number of large models that we interact with directly, analagous to the laptop or desktop CPUs that we use daily. And there will be thousands of small models embedded everywhere, focused on narrow and specific tasks, analogous to the chips embedded in your car. Who knows how many chips are in a typical new car today? Very few of us, and why should we care?