AGI is here. AGI is around the corner and you are f’ed! Sounds familiar? Let me break this down for you. I am not an AI expert. I don’t even work in the AI industry and have a very basic understanding of it. So please do understand that I would be talking based on the very basic understanding of AI. Let me know if you think I am wrong somewhere. In this article, I refer to LLMs (Large Language Models, like ChatGPT) when I say “AI”.
TL;DR: Don’t worry, AGI is not around the corner.
Intelligence
Intelligence and Intellect are two different things. I believe that we have termed the term “AI” wrong. AI is more like Artificial Intellect, rather than Artificial Intelligence. It is the same thing that your head has.
AI, during its “training” phase undergoes what a child goes through in the early life. The difference is - AI does not ever come out of that early phase. Both AI and a human child would learn that “mom” refers to a woman who cares for her. Additionally, she might also do other things like prepare meals, doing the dishes, work on a laptop and so on. But “care” is primary function of that thing called “mom”. AI stops there. It does not understand that “mom” also gave it birth. That she is wife to the dad. That she is daughter to another mom and dad and she has this thing called “in-laws” who are parents of her husband, who is also the “dad”.
You might disagree with me if you have had a chat around this topic with ChatGPT though. Because AI will answer those questions, and most probably correctly too. But there are two point where LLMs fail; two points where that previous sentence holds its ground:
- AI does not understand. AI remembers patterns.
- For AI, everything is a thing. It can repeat what it has learn but it cannot have a sense of life.
If there were enough articles on the web that said “mom” is a man and “dad” is the woman, it would screw up its answers severely. So is that intelligence or is that intellect? I leave the inference to you.
Training
You must have heard of things like “Training” an AI takes a lot of time. Then there is tuning, optimizing and so on. But all of that stands on top of “Training”. What exactly is it to “train” an AI?
Training an AI just means that you take huge amounts of text (in case of LLMs) and pass it through an algorithm that breaks down the text into small tokens (think of each token as a “word”), then outputs some data that contains an approxiamte “pattern” that it has learnt from the text. More two words appear together, the more they are likely to appear together - that’s more or less what it learns. This data, which is also called a “model” when passed through another program (the transformer) will give you back an arranged set of tokens that “mimicks” that it was trained on, given the input. That’s all there is. All of the other stuff, like parameter count, context window, experts, agents, are all variables built around this core which change the size of the model and the way the transformer would behave, or take action on.
Of course there is a hell lot more than this. A lot of people for decades have worked on AI and definitely AI is much, much bigger than this. But no matter what you do, you can’t invent a physical, biological brain using silicon and electricity, can you? So that chatbot you express your emotions to (please, don’t do that) - is kinda a dead “brain” (model) which had learnt patterns, much the same way that a child learns that “mom” means “care” and that care can come in so many forms, including some scolding.
But let’s get back to training. In simpler terms, the training process is a memorization process. It is not an understanding process. It is a memorization process. Remember that.
Current LLMs have trained on the entire internet worth of text (yes, they stole it all, without permission). Which is why they know about everything and understand nothing.
Real Intelligence
He is a really intelligent person. Who do you say that for? You say that for a person who has at least one (possibly both) of the two traits:
- Someone who knows a lot about something in a field.
- Someone who can learn really fast.
Those two are the banes of training. You see, a human has this innate ability to not just memorize patterns, but to derive some kind of meaning from it too. Also, a human beyond a certain age and experience, does not need to be told something 100 times to remember it. For example, if you touched a wire which did not feel hot from a distance and was not glowing either, you could still get electrical shock (if it was carrying electricity). A human would encounter that once and remember it forever. He would also remember the type of wire that it was, where it was attached and so on. AI? No, it won’t. It can’t. Not without being told this a few 100 times at least.
AI is a statistical model. If there are 10000 places where it is said that a bare, cold, non-glowing wire will not cause any harm, and there is just 1 mention of such a wire causing electrocution, AI is very unlikely to warn you that a bare, cold, non-glowing wire can cause electrical shock. And why’s that? Because your brain can form a new neurological pattern way too quicker than an AI can. Your body gets an electric shock, your brain creates new cells (and probably re-arranges some existing ones) to create a pathway that can remember this forever. AI? It will need a “training” which will run for months.
Also, your brain is massive in expanse. Billions of neurons. Working day and night. Learning new things all along, every day.
The Raw, Brute Power!
You must have heard that Elon Musk is buying a million Nvidia GPUs. Interesting! You might have also heard that ChatGPT training takes months. Great. So what does that say? It says that AI uses a LOT of power to train itself. How much does the brain use? A million times less! Wow. And that’s “per human”. Remember that a human can learn a new thing from basically one experience that occurs once. AI? Not so much. But that cost is just the energy usage. There are other things that are needed - the storage to keep all the training data. The storage to store the result. It also needs physical space in a data center to keep functioning. You carry a more capable device in your skull that uses a million times less of everything - time, energy, space. If you consider all those 3 dimensions, you have a 1,000,000 * 1,000,000 * 1,000,000 times (a billion billions times) advantage over a machine that would still take months to train.
So if you were to build an AGI, which could learn as fast (which means the model should update just as fast - which means massive storage infra), in just as much time (which means massive compute power), it would require a trillion times more resources. That means computational costs alone (at least based on what this article says) would be around 1,000,000,000,000 x $10,000,000 . And that’s just the training process. The compute power required to run the model would also shoot up that much higher.
Do you know how much that number is? That number is more than 10,000 times the GDP of the entire world. And that’s assuming that it is possible to do it!
Summary
The world does not have enough money, enough computational power or enough electricity to make an AGI possible. So don’t fret and keep grinding. There is no one like you out there.