From barbaric society to artificial intelligence, computer scientist Wu Jun explains to you the ins and outs of ChatGDP.
On the evening of April 3, Wu Jun, a computer scientist and natural language model expert, was invited to the live broadcast room to conduct a live broadcast on currently hot topics such as artificial intelligence and ChatGPT.
Q1：
Why does the emergence of ChatGPT cause panic?
I know that ChatGPT is very popular in China recently, and many people are discussing it. But what’s very interesting is that actually not many people in the United States talk about this topic anymore. In fact, it’s not just ChatGPT. Looking back ten years, when many new technologies emerged, I found that the discussion in the Chinese media was much higher than that in the United States. Although that technology actually mainly appears in the United States, the Chinese people are more concerned about it. I think that's a good thing, but it's also a bad thing.
The "bad thing" is that these technologies have actually been over-hyped, and in the process, many people are making money from it. For example, blockchain was so hot at the time, but now few people discuss it, right? This is the first one. The second one is the Metaverse. Currently, only Facebook in the United States is still insisting on doing it. When we arrived in China, many people were discussing whether we would live in a completely virtual world in the future. Finally, from the end of last year to the beginning of this year, Facebook invested tens of billions of dollars in this field without hearing a single word, and finally began large-scale layoffs. Nowadays, one of the hottest topics is ChatGPT. Some people are excited, some are afraid, and I also see that there are still many people in China fishing in troubled waters, trying to cut everyone's leeks again.
Before telling you what ChatGPT is, let me tell you a historical story. This historical story will make you laugh, but if you look back, many people behave the same way today.
In 1503, Columbus's son recorded this incident. Columbus sailed west to the New World. As a result, halfway through the voyage, he arrived at Jamaica, and there was no food on the ship. Therefore, Columbus and his crew could only hope that the locals would provide food and drinks. However, after a few days of provision, the crew had conflicts with the locals - some crew members stole things from the locals, so the locals cut off the food supply.
In order to get out of this predicament, Columbus came up with a clever idea. Columbus carried a perpetual calendar with him at the time, and marked on the calendar that there would be a solar eclipse, a lunar eclipse, and all this information on a certain year, month, and day. Columbus called the local tribal leaders at that time and said that if you don't provide me with food, you have offended God. God will be angry, the moon will turn red, and then God will take the moon away. Of course, we basically all know now that when a total lunar eclipse occurs, that is, when the earth has not completely blocked the moon, the moon is indeed red, which is what we call a "blood moon." However, Jamaicans at the time did not know this. As a result, at night, the Jamaicans discovered that the moon turned red and then slowly disappeared little by little. The locals fell into panic, and everyone said that God was going to punish them.
The tribal leader hurriedly went to Columbus and promised to agree to all Columbus's conditions. Columbus said, OK, I will go to the tent and pray to God so that He will not punish you, but I need a little time, and then Columbus walked into the tent. In fact, after entering the tent, Columbus was holding an hourglass and looking at the time.
Today we have knowledge of astronomy, and we definitely know the time of the total lunar eclipse, which will probably last for about 48 minutes, and the moon will reappear by then. But these Jamaicans don't know that. What they saw was that when Columbus came out of his tent, the moon came out. Then Columbus said, God has listened to my advice and promised to forgive you, but you must provide us with good food. Therefore, the local people are grateful and continue to provide them with food.
What does this story mean? There is a reason behind the occurrence of a total lunar eclipse, but when people don't know the reason, they often can only attribute this natural phenomenon to the action of a god. And this god himself was created by man. In other words, after man creates a god himself, he then lies at the feet of the god and becomes his slave.
This is why I want to teach you the course "History of World Civilizations".
In fact, the development process of this civilization is the process of human beings constantly understanding the laws of nature. We have made little progress so that we are no longer like the local indigenous people who blindly believe that praying to God can really prevent the moon from disappearing. We now know that behind the eclipses of the sun and the moon, it is actually Kepler's three laws of planets that are at work, and then behind Kepler's three laws of planets is Newton's law of universal gravitation. After humans understand this reason, they are no longer just afraid of nature. We can use the laws of nature to do many, many things.
Q2：
What is the technical basis of ChatGPT?
Looking back from history to the present, the situation of ChatGPT is actually similar. Behind it is a mathematical model called a language model at work. In other words, behind ChatGPT is a mathematical model. Today, this technology is powerful for three main reasons:
First, it requires a lot of calculations;
Second, it has a large amount of data;
Third, today’s methods for training language models are much better than before. So, what is a language model? Or is it a product of what era?
It is a technology developed by a team led by my mentor Fred Jelinek in 1972. Specifically, it was a technology he led people to complete at IBM, which was used to measure how likely a sentence or a language phenomenon is to occur. So what's the use? Its initial use was for speech recognition, later for machine translation, and later for computer question and answer, which is the answer question we are familiar with today.
At that time, it could be used as a summary. For example, if there is an article of 10,000 words, how can you summarize the content of the article in ten sentences? For people who do this natural language processing, this is a mathematical problem. In other words, what are your conditions? The condition is these 10,000 words, and then what is the result you want to get? The result may be ten sentences or a hundred words, and there are many combinations here. You can pick a few sentences at random, or you can split some sentences into two paragraphs and add the less important modifications or descriptions at the end. Partially removed. Then, you can also combine two sentences into one sentence. Then when you combine a piece of text, the computer will calculate a probability. Which sentences are more likely to be combined together, and it will help you combine them according to the probability.
The ChatGPT we see today is this large language model. It will select a text with the highest probability and the most likely occurrence to show you. So in general, the process of generating results by ChatGPT is a process that uses a lot of computing resources. It requires a very large amount of data to support, and there are many GPUs (computer processors). Without these things, ChatGPT cannot be built.
And today's ChatGTP is actually not only technology, but also has a lot of manpower behind it. They also hired a company to audit the results produced by ChatGPT. For example, ChatGPT has generated a hundred abstracts, all of which are quite good, but I can no longer distinguish them. Then these people are responsible for helping me distinguish which one is more accurate.
In fact, you can see that behind Chat GPT is a language model, and the technology of this language model has been around since 1972. Now, after fifty years, in the industry, people don’t actually think it is a big deal. Before this, this language model has actually done a lot of things.
When it comes to language model, this term was originally proposed by my mentor Jarinick. He arrived at Johns Hopkins University in about 1993, and I arrived at the university in 1996 and became his student. So the Chinese version of this word, which is the four words "language model" you see, was created by me when I published a paper in the 1990s. At that time, only those of us in the circle knew that it could do a lot of things, but you didn't think of saying, Huh? This matter will be hotly debated later.
You can understand it this way, "language model" is to ChatGPT what Kepler's three laws of planets are to lunar eclipses.
Q3：
What was the situation when the “language model” was born?
So what was the situation of the language model at the time of its invention?
In fact, in the 1990s, models obtained using simple statistical methods were already very inaccurate. This is equivalent to, let me use an analogy, you observe the planets, but using Ptolemaic geocentric theory to predict them is very inaccurate. Therefore, at that time we began to introduce a lot of information about grammar, themes, and semantics. Then, this language model becomes very complex. The complexity brings another big problem.
what is the problem?
For example, I made a very complex language model. How many parameters did this language model have at that time? 6 million parameters, that is to say, the size of the language model of this song is basically determined by these parameters. What I was working on at that time was already the largest and most complex language model that could be built at that time. At that time, I was not using PCs, but 20 super servers, and it took about 3 months to train such a language model. So you see, the calculation amount is very large. So, what are the language model parameters used in the first version of ChatGPT? There are about 200 billion parameters, and you can see the changes over the years. Therefore, many people ask today that ChatGPT has appeared in the United States. When will Chinese research institutions be able to make ChatGPT? In fact, most research institutions in China cannot do it, not because of the research level, but because Chat GPT consumes too many resources. Today's ChatGPT may cost almost 1 billion US dollars in hardware alone. This does not include electricity costs, so the cost and expenditure are very huge. So, if you finish joking and ask what the biggest contribution of ChatGPT is, I think it has made a great contribution to global warming. So, what I want to say is that the principle of ChatGPT is very simple, but it is actually quite difficult to achieve it in engineering.
Q4:
What questions are computers good at answering?
Around 2010, that is, 13 years ago, to what extent could language models achieve? Let me show you two examples. Both of these examples were done before I left Google in 2014. At that time, I was responsible for Google's automatic question and answer system, which let computers answer questions. However, because this product is in English, it basically does not show much face in the Chinese world.
Let me show you a question answered by Google, why is the sky blue?
You can take a look at this. The answer is this: Sunlight is refracted when it reaches the earth through the atmosphere. The gases in the air scatter light of different colors to various places. Blue light has a shorter wavelength and has a higher refractive index than other colors. High, so the sky looks blue. This was an answer generated by computers at the time. To be fair, this answer is better than my own paragraph answer, because to explain this phenomenon, you need to know a lot of physics knowledge, and the sentence seems quite reasonable. One of the purposes for people using ChatGPT today is to let him answer questions.
Here, I will break it down for you.
In fact, the questions we ask computers can be divided into two categories. The first category is called simple questions, and the second category is called complex questions. Simple questions are about facts, such as where a certain star is from and what year he was born. These are all easy questions. Because it is a fact and there is a clear answer.
The second category is complex problems, which is why everyone thinks ChatGPT is so amazing. It can integrate information and answer why the sky is blue, as if it has its own logic. Another question is about the process. For example, how do I bake a cake? Can you write it down step by step? Today we asked ChatGPT how to bake eggs high. It can write you the process in great detail. It can tell you how many cups of water, how many eggs to add, how much flour to add, etc. Then based on the answers it provides, you can really bake a cake, and it may be pretty good. This is something that everyone thinks is amazing.
But you have to know that computers have actually done this in 2014, and they have done it very well. So, there’s not much mystery about the technology itself.
Q5:
Computers or humans, who is better at writing?
Nowadays, everyone is talking about ChatGPT hotly, and another reason is that they think it can write. For example, writing a work presentation is where Americans use ChatGPT most today. I did 1,234,567 things this week, these seven things, hey? You see, I don’t have to write it myself. I let ChatGPT generate one and then edit it.
However, computer writing is actually difficult or easy. I can give you an example.
After I left Google in 2014, I didn't do much programming at that time, but I still had some computing resources at that time, so I would write some programs and play around in my free time. At that time, I asked the computer to write two poems. You can read these two poems.
The first poem is a five-character poem. In my words, it is a poem in the style of Li Bai. You can read it. This poem was written by the computer itself. In fact, if you read it, this poem does have some of Li Bai's characteristics.
As for the second poem, I also put the picture below, you can take a look.
First of all, because ancient poems all have the same meaning of ping and lei, but our current pronunciation was different from the pronunciation at that time, so we don’t care whether this ping and lei is consistent with the ancient times, but we only look at its content and artistic conception. As you read It will feel very smooth.
Okay, so back to that.
How did you make the first poem? In fact, it couldn't be simpler, just put Li Bai's poems into the computer. There are more than 1,000 poems by Li Bai, which is only about 10,000 sentences. This is too simple for computers. When writing it, I split the sentences into groups of two or three characters. For example, "Kongchou" is a group and "Recalling Chang'an" is a group of three characters. Then it goes to put together the language model I just talked about and calculates the probability. Which one has the highest probability? After dismantling it, I made a request to him, saying that I wanted to write a poem about recalling Chang'an, and the poems were arranged and combined to produce "Remembering Chang'an". This is actually how it was put together. The second poem is a little more complicated.
But do you know how long it took me to write these two programs? Two days. What is this indicating? It means that it is not very difficult for you to ask the computer to write something decent. It is not as mysterious as you think, or computer writing itself is not as mysterious as you think.
So why do these two poems seem so good? Because this is a Tang poem, the format of Tang poetry is fixed. By the same token, why is it good to use ChatGPT to write weekly reports? Because the format of the weekly report is basically a list, it is also a fixed format. Including, if you read the Chinese version of the Wall Street Journal, let me tell you that 90% of the content here is written by computers, but you don’t know it. After writing it, of course people have to give it a theme, and then write an introduction to the first paragraph of it, and then give a summary and a title. This is what people have to do. Why is it better to write financial articles? Because it has a lot of facts in it and the format is fixed, it does this very well.
I spent so long talking about the background of ChatGPT. Actually, I just want to say that it is not mysterious, and it is not a very sophisticated machine behind it. On the one hand, ChatGPT relies on a mathematical model, and this mathematical model has been around since 1972, but today its computing power is very strong and it relies on brute force calculations. So, how much power does a ChatGPT training consume? It may be 3,000 Tesla electric cars, each running 200,000 miles, and running it to death. Such a large power consumption is enough to train once. This is a very expensive thing.
Q6:
What impact does ChatGPT have on us?
So let’s talk about what impact ChatGPT has on people.
Let’s go back to history. Every technological revolution actually has some impact on people. However, ChatGPT is not a new technological revolution, because as I just said, this process is very long. From the 1970s to the 1990s, we did a lot of things, and from the 1990s to now, many people have done a lot. thing. The biggest progress here is not actually the language model itself. In fact, it is the deep learning that came about around 2000, which made training language models more accurate than before. It is not just about doing statistics.
Today, training language models is no longer simply about statistics. This is one of the reasons why ChatGP T can produce better results. As for what impact ChatGPT can have on people?
I won’t answer your question directly. Let me ask you first. I just showed you these two Tang poems. Did you notice any characteristics? By the way, these two poems are well written, but your original understanding of the Tang Dynasty will not be updated because of these two poems. Because ChatGPT is a bit like a parrot in a way. You have to say something first, and then it can follow. It may sound nice, but it doesn't provide any more information. 90% of the content on the Internet today falls into this category - it does not provide more new information, nor is it original content, nor is it my own insights. It is nothing more than copying and piecing together. At present, I think 99% of the content of short videos such as Douyin and Kuaishou fall into this category and are not nutritious. You may find it interesting after reading it, but in fact, no matter how much you read on it, it actually has no benefit to you. Any help. If ChatGPT really threatens anyone, I think it threatens the jobs of this type of people, that is, those on Douyin who make short videos or publish some content. ChatGPT will do much better than them. Just think about this. Suppose there is a group of people who turn over and over the sentences in the three hundred Tang poems every day, and they can also compose some poems. Then ChatGPT will definitely be able to compose them much faster than humans. So this technology will have an impact on this group of people.
So, who won’t be affected? The people who created the content will not be affected.
Why do I say this? Remember the question I just said, "Why is the sky blue?" Why can Google answer this question?
Because when Google was answering, it probably analyzed almost all decent sentences in English at that time, which was about 100 billion English sentences. So actually you will find that on some university websites and on the NASA website, it has this answer, but we pieced it together, deleted and deleted it, and picked it out. But the earliest physicists did this research and figured out this truth. This work is meaningful and cannot be replaced by ChatCPT. So, what does the work of Chat GPT amount to? For example, after Ptolemy created this model, every once in a while, they in Europe would compile a calendar for several decades, and then mark on it which day there would be a solar eclipse, how the planets would move on that day, etc. . Then according to these rules, people print many copies of this book. This ChatGPT is equivalent to many books. If you take it and look at it later, you will say, eh, a lunar eclipse will occur on a certain year, month and day, and the answer will be very clear. However, the really meaningful work behind it is not printing this book, but doing research on Ptolemy. So I think that historically speaking, ChatGPT is not actually a technological revolution. It only affects lazy people who are too lazy to use their brains and create new things. Those who truly explore the mysteries of human knowledge will never be replaced.
Q7:
What new opportunities can ChatGPT bring?
Many people ask, what new opportunities does ChatGPT have? Frankly speaking, you have no chance, because it consumes too many resources and you can't afford it. So who can benefit? Those are the people who sell resources.
I can make an analogy, that is, during the gold rush in California, many people flocked to dig for gold. To this day, we still don’t know which gold digger really made money, and no one left his name. Come down. But who makes the money in the end? It’s the water seller and the jeans seller. The same is true for ChatGPT. If everyone goes to pan for gold, you actually won't make any money, but in the process, you still have to buy water and jeans to wear. In the end, it is these two groups of people who make money. Levi's was a company that was born at that time. It made jeans.
Then in the end you may end up paying money to several large cloud computing companies, which may be a result. Okay, now that I’ve finished talking about the history of ChatGPT, I’ll give you a brief summary.
First, don't be afraid.
Today, many people are afraid of ChatGPT, just like the indigenous people of Jamaica who Columbus encountered were afraid of the lunar eclipse.
Second, don’t force yourself to find so-called opportunities. Work is how you should work.
I saw some students asked me why Apple doesn’t do ChatGPT, and I said that’s right! This is why Apple is the richest company in the world, with the highest profits and the largest market capitalization. Currently, many so-called companies that do this kind of artificial intelligence are still losing money. Therefore, this is why many students sometimes ask questions that are too out of the ordinary, so I jokingly ask them, have you paid off your mortgage? If you haven't paid it off, just go back to work and do your job well. This is the most meaningful thing for everyone, and this is also true historically.
Third, you have to see through the tricks of these so-called conspirators or people who want to cut you off.
That is to say, if another person pretends to be Columbus and says that he is the representative of God, and then he can pray to God to make the moon come out, don't believe it. So you need to understand some of the science behind ChatGPT. You still need to understand some of the simplest principles, like the ones I’m talking about today.
