chatGPT is a potato
Updated: Apr 5
During the past weeks, our feeds have been full with chatGPT prompts. The world is shifting under our feet, and it's unclear whether this is good, bad, or how much of both. I believe a huge change relates to being able to ask as many questions as we might want for the first time in our lives. We can all remember how it felt to want to ask for something and how annoying it was when people would stop answering. As our disillusionment grew, we learned to search for information ourselves and built tools to learn. These tools grew in quality as information technologies came to our fingertips. Now, this is all changing as we can ask and search as we scavenge for knowledge.
Machu Pichu is an incredible place. If you visit and take part in a tour, you will learn that Machu Pichu was built as a research center for the bioengineering of potatoes. Potatoes were, after all designed with careful and precise attention.
Potatoes did not just appear. They did not just grow. Nor were they just improved by craving the faster-growing ones and planting them next. Potatoes were plants that could only live in absurdly high and cold lands. The Incas hated that and built stupidly complex systems (see here) to try out ways of creating new varieties of potatoes that could be cultivated anywhere in their empire.
Trust me, Machu Pichu was a research center. In it, scientists could determine the time of the year and precisely adjust the temperature and humidity of their lands by using a stepped crop system. On the side of the mountain were dozens of levels, the coldest at the top but increasing around half a degree as one went down the ladder. New species would grow first on the top-most level and would be taken down year after year.
Eventually, when they reached the bottom, they would be exported to faraway lands and consumed by the masses. This process led to incredible riches in the South American empire, not only in the form of over 5000 species of potatoes. But in the form of alternative foodstuff to compete against the typical staple of the American continent, corn. In contrast to any other American empire, the Inca had two foodstuffs that served to hedge risk and increase the chances of surviving a bad season.
Imagine a world without potatoes
Please do. No. I am sorry. You failed. Without potatoes, the world would suck. Really. As Nunn and Qian (2011:593) show in QJE, "the introduction of the potato accounts for approximately one-quarter of the growth in Old World population and urbanization between 1700 and 1900". A quarter of the growth that, if removed, might have delayed the industrial revolution a few further centuries. Compounding effects are a thing after all.
Our history books are unfortunately fraught with mistakes about potatoes. They talk about the Great Famine. The Great Famine being the name given to the 30th largest famine of all times; a famine that happened in Ireland in the mid-1900s. The story's moral is that over-reliance on one food staple (i.e. monoculture) leads to collapse when said staple gets sick. But it forgets why a whole island of non-potato eaters switched to become an island of only potato eaters in a bit more than ten generations (1600-1850).
They did not choose potatoes because potato lovers colonized them (e.g., wheat in Latin America). They did not do it because potatoes were fancy (e.g., the quinoa or kale of the 1600s). No, they did it because they were easier to cultivate than anything else. As farmers were thrown out of the commons, they emigrated to the cities, and in the little land they had, they would cultivate potatoes to eat and survive. The nonpotato eaters died (or failed to reproduce, but I am not sure what the difference is).
Sorry for the long post; here's a potato
We are well into our four century of eating potatoes in Europe. As a form of appreciation, we honored our potato Gods by ending our conversation with images of them. Or at least, we did for a while in 9gag. Whenever a post was long, the creator would add a potato, and all their sins would be absolved. Who does not love potatoes anyways?
As such, for over two decades, potatoes have been signs of appreciation on the internet. New ideas were made and shared, keeping the potato as a mediator of said interactions. Countless laughs have been saved from famine by a well-placed starchy potato.
Don't get me wrong. Long 9gag or Reddit posts can be grueling experiences. One goes over dozens of fo figures, thousands of words, countless meta commentaries, and Easter eggs that one ends up drained. Crucially though, long posts are a chance to get to know others and what they want. And if I am honest, I have zero interest in learning new things. But I am always interested in learning what makes other people's hearts tick. No tl:dr for me. Not always at least.
Where potatoes meet chatGPT
In the past two decades, there have been three or four important machine learning papers. Cockburn, Henderson, and Stern (2018) cite two, the IMAGENET 2012 paper and the creation of convolutional neural networks (CNNs). Adversarial networks were the third, in my opinion. And I always believed that CNNs were a kind of breaking news. Not real news but something transient.
I say this because, unlike deep neural networks, CNNs were just a hack, a somewhat contrived manner of getting more data points from a datum. CNNs were not GIGO per se, but they were not beautiful either. They just work. Similar to k-means or support vector clustering, a kind of way of getting something our brain does intuitively but in a weird, clunky manner.
Along came paper number three, or actually two: generative adversarial networks. It became clear after a few years that overfitting worked well, even when one took measures to avoid it. One-pixel attacks would defeat DNNs. Small differences in the seed used to train a neural network would lead to marked changes in the behavior of the networks. It was stupid and wrong. Adversarial attacks were the way to go, and we built so much from GANs.
Paper three was the one that let me throw CNNs out off the podium. Attention is all you need, was the title of the first paper discussing the use of a transformer architecture. This architecture is something I still do not understand. But the crux of it is the use of a multi-head attention mechanism that takes a structured view of the inputs and controls the way information is processed within the tool.
This was genius. In one blow, we could remove loads of the artifacts we had built to make neural networks, which are essentially decision-making tools, into systems with short-term memory. Things like clunky recursive neural networks, LSTM models, and other ill-formed ideas could be thrown out, and instead, we kept a relatively simple attention system. A system that moved around over time and shifted the focus of the network over the parts of the prompt it was given. A system not dissimilar to how we interact with the world.
With these two last papers, our world changed. We could do so much more than the high-quality predictions from the good old neural networks. We can now hallucinate via transformers and reign down chaos via adversarial networks. Our lives changed when these two papers were published, and it took us years until chatGPT came out to get a killer app that laymen like me could understand the geological shifts that happened under our feet.
Potatoes are beautiful. They grow in the mud and pack so much energy that be them fried, pureed, made into gnocchi, or backed.; they bring joy to our bellies. The same is true for long posts in 9gag, Reddit, and now even chatGPT rabbit holes. They enable us to learn what others love.
Since the release of chatGPT, everyone's feeds are full of very intimate, quirky stories of discovery. Stories that make us wonder why some would ask that specific question. But also stories that let us learn what the people around them like. Stories that let us learn in a very different way than we had during the last two decades.
I find these stories beautiful and keep me wondering what is next. Just as in the early days of the internet when we had no idea so many people cared about one topic, and we found camaredary and companionship from the community of nerds out there. Today, chatGPT users share their insights and tools. We can see what prompts others use and adapt them to our needs. It is exhilarating and a little bit crazy.
Ask don't search
Since the advent of Google's Pagerank algorithm, we have been able to scavenge for knowledge in one of the two forms agreeable to our brains: search. The idea that the more something is linked to is intrinsically the same way our neurons search for information. The more neurons connect to one concept, the easier it is to recall. The more often one recalls something, the more connections are built to said something. Pagerank helped us search the depths of the web.
The other way agreeable for our brains to learn is by asking questions. This form has not worked very well in the past. Don't get me wrong. You can ask questions to Google, even more so to the weirdly name AI assistants we all have now. But the answers were not great. They were just the output of a query, just as clicking "I am feeling lucky" in Google after typing a string of characters. I.e., not something I would ever trust.
We did not trust what our computer told us. We did not. For decades we would search and navigate. We would find pockets of trust and ask questions there. Be them in Reddit, Quora, or StackExchange. We all found ways of uncovering communities that would answer one's questions in a caring way. In other words, we all learned that it was mostly unsafe to ask questions online. We learned that unless extreme measures were made, we would get garbage.
Now comes OpenAI, and within a few weeks, all changed. Well. That is false. Some have been using GPT, GPT2, DALLE or DALLE-2 in the past. But this was similar to the web 1.0 situation in which only firms had websites. No one really could understand why one would use the internet as a private citizen. Then came web 2.0, with its blogs, social media, and user-centric platforms that made us share and be online all the time. ChatGPT is IMHO the web 2.0 moment for these technologies.
What I find exhilarating about all of this is that, after all, the super users share their views, tips, and tricks. We are getting now to a point in which it feels ok to ask. OK in the sense that I feel ok searching for information in Google. I start to feel ok to ask naive questions to chatGPT. Naive questions are unaskable in Google. If Wikipedia does not have a thread, there is no way. But now we do not need to care. Even if we do not know where to ask, we can ask chatGPT and get dozens of answers.
Dozens of answers are the crucial point. One can tailor and modify one's queries. One can adapt and tinker. One can learn how to scavenge for knowledge in a completely different way. A way that we are intrinsically capable of using. We were all two years old once. We all asked why things worked the way they worked. But independently of our parents' patience, we were told to stop asking questions. NOT ANYMORE.
What will be of our lives when we can reliably ask questions and search for information?
Clearly, no one knows. Clearly, we are years away from being able to ask reliably for answers to our questions. Clearly, it is too early. But clearly, we are in no AI winter. We are all sweet summer children today, and God how I love this sunshine!
Potatoes led to the advent of the industrial revolution. They fed us enough to be able to change our worlds and create knowledge. They fed Gutenberg and relieved monks from their copying of books. They fed former farmers and helped them survive as factories grew. They enabled us to live in a world where few of us touch the dirt. A world so disconnected from itself that we need to search for information about potatoes from an interconnected network of copper, silicon, and glass. A network that we no longer need to search if we don't want to. Where facts won't matter anymore, where navigation is devalued. Where we can learn as we all wished we would when we were two, from the quirkiness and wonderful insights of others.
ChatGPT is a potato, but not one that feeds on earth. ChatGPT is a potato that feeds on data, on text, images, videos, and source code. Like every other potato, it can serve as a source of energy for a growing population. It can lead to growth and prosperity. Just like any potato, it will flounder if it becomes a monoculture. As we start letting chatGPT out of the box, its output starts to poison the through it drinks from. It will come a day in which so much of the Internet is chatGPT-generated content that it might implode. Like the Irish potatoes, there might be a point in which chatGPT will learn spuriously and poison our world. Where we eat and regurgitate its outputs without knowing we are just spewing death. ChatGPT is a potato and as all potatoes, it is fine if eaten as part of a healthy diet.