10 Shameful Questions About Neural Networks: Machine Learning Specialist Igor Kotenkov Answers

In the new series Articles by well-known experts answer questions that are usually embarrassing to ask: it seems that everyone already knows about it, and the questioner will look stupid.

This time we talked with artificial intelligence specialist Igor Kotenkov. You will learn whether you can save your digital copy for your great-grandchildren, why neurons cannot be trusted 100%, and whether the world is in danger of a machine uprising.

1. How do neural networks work? It's some kind of magic. How could ChatGPT be made at all? And Midjourney or DALL-E?

A neural network is a mathematical model invented with an eye to understanding how the brain of a living organism works. True, the most basic ideas of the beginning of the second half of the 20th century were taken as a basis, which can now be called irrelevant or too simplified.

Even the name "neural network" comes from the word "neuron" - this is the name of one of the main functional units of the brain. Neural networks themselves consist of nodes - artificial neurons. So we can say that many ideas of modern architectures were "peeped" from nature itself.

But more importantly, the neural network is a mathematical model. And since this is something related to mathematics, then we can use the full power of the mathematical apparatus in order to find out or evaluate the properties of such a model. You can consider a neural network as a function, and a function is also a mathematical object. The simplest and most understandable example: a function that, say, takes any number as input and adds 2 to it: f (4) = 6, f (10) = 12.

But such a function is very easy to program, even a child can handle it after a couple of hours of learning languages. programming. And the reason is that such a function is very easily formalized, described in detail in a simple and understandable language.

However, there are some tasks that we do not even know how to approach. For example, I can give you photos of cats and dogs mixed up, and you can sort them into two piles without any problems. But what exactly are you guided by when determining the answer? Both of them are fluffy. Both species have a tail, ears, two eyes. Maybe the size? But there are very small dogs, there are big cats.

We cannot describe many tasks of the real world, we do not know the dependence of our observation and some conditional “correct” answer.

We just know how to give this answer - and that's it, without thinking about how it turns out.

This is where neural networks come to the rescue. These mathematical functions are trained from the data. You don't need to describe the relationship between input and output. You simply prepare two stacks of photos and the model trains to give correct answers. She herself learns to find this connection, she finds it herself, relying on mistakeswho does. Confused a Bengal cat and a Rottweiler? Well, it'll be better next time!

The process of learning a neural network is such an adjustment of “neurons” in order to learn how to solve a problem and give the correct answer. And what is most remarkable: there is a theoretical proof that a sufficiently large neural network with a sufficiently large data set can learn any complex function. But the most important thing here is the computing power (because the neuron can be very large) and the availability of labeled data. Namely marked, that is, they have the class “dog”, cat or whatever.

We do not fully understand how models work - the most complex and large models like ChatGPT almost unanalysable.

The best researchers are working on the challenge of "understanding" the inner workings of their processes right now.

But we know what task the models were trained for, what error they tried to minimize during training. For ChatGPT, the task consists of two. The first is the prediction of the next word according to its context: “mom washed ...” What? This is what the model should predict.

The second task is to ensure that the answers are not offensive, but at the same time remain useful and understandable. That's why the model went viral - it is directly trained to generate the kind of text that people like!

You can read more about how ChatGPT works in my article.

2. Can neurons think?

Scientists still do not understand what it means to "think" or "reason" and how the intellect works in general. Therefore, it is difficult to judge whether a model like ChatGPT has such properties.

Let's imagine a situation: you approach the door of your apartment. Do you have the idea that you need to get the key from the left pocket of your backpack to open the door? Can we say that the description and presentation of actions is a thought process? In essence, we have established a relationship between the current state and the target desired (open door). If you think the answer to the question above is yes, then my answer would be the same. 🙂

Another thing is when it comes to innovative thoughts that have not been expressed before or are not so common. After all, for example, you can easily find fault with the example above: “Yes, I read this model 100500 times on the Internet and in books. Of course she knows it! Nothing surprising." By the way, how did you know? Is it because your parents showed you in childhood, and you watched the process for hundreds of days in a row?

In this case, there is no exact answer. And the point here is that we do not take into account one important component: probability.

How likely is it that the model will generate a thought that fits your specific definition of "thought"?

After all, a neuron like ChatGPT can be made to generate a million different responses to the same request. For example, "come up with an idea for scientific research». If one generation in a million is really interesting and new, does that count as proof that a model can give birth to an idea? But how will it differ from a parrot that shouts out random words that no-no and add up to something understandable?

On the other hand, people also do not always give out correct thoughts - some phrases lead to a dead end and end in nothing. Why can't neural networks forgive this? Well, one new idea out of a million generated is really bad... But what if 100 out of a million? Thousand? Where is this border?

This is what we don't know. The trend is that at first we think that it will be difficult for machines to solve problem X. For example, to pass the Turing test, where you just need to chat with a person for half an hour. Then, with the development of technology, people come up with ways to solve, or rather, train models for a task. And we say: “Well, it was actually the wrong test, here’s a new one for you, neurons will definitely not be able to pass it!” And the situation repeats itself.

Those technologies that are now, 80 years ago, would have been perceived as a miracle. And now we are trying with all our might to push the border of "reasonableness" so as not to admit to ourselves that machines already know how to think. In fact, it is even possible that we first invent something, and then post factum and retrospectively define it as AI.

3. If neurons can draw and write poetry, then they can be creative and almost like people?

The answer actually relies heavily on the information above. What is creativity? How much creativity is in the average person? Are you sure that a janitor from Siberia knows how to create? And why?

What if a model can produce a poem or a painting that, conditionally, will reach the finals of a city competition for amateur writers or children's artists? And if this happens not every time, but one out of a hundred?

Most of these questions are debatable. If it seems to you that the answer is obvious, try interviewing your friends and relatives. With a very high probability, their point of view will not coincide with yours. And here the main thing is not quarrel.

4. Is it possible to trust the answers of neural networks and no longer google?

It all depends on how the models are used. If you ask them a question without context, without accompanying information in the prompt, and expect an answer on topics where factual accuracy is important, and not the general tone of the answer (for example, a sequence of events within a certain period, but without an exact mention of places and dates), then the answer is No.

By domestic estimated OpenAI, in such situations, the best model to date, GPT-4, answers correctly in about 70-80% of cases, depending on the topic of the questions.

It may seem that these numbers are very far from the ideal 100% actual "accuracy". But in fact, this is a big leap compared to the previous generation of models (ChatGPT, based on the GPT-3.5 architecture) - those had an accuracy of 40-50%. It turns out that such a jump was made within the framework of 6-8 months of research.

It is clear that the closer we get to 100%, the more difficult it will be to make some corrections so as not to “break” anything in the understanding and knowledge of the model.

However, all of the above refers to questions without context. For example, you can ask: “When was Einstein? The model should rely only on internal knowledge that was "hardwired" into it at the stage of long-term training on data from all over the Internet. So the person will not be able to answer! But if they gave me a page from Wikipedia, then I could read it and answer according to the source of information. Then the correctness of the answers would be close to 100% (adjusted for the correctness of the source).

Accordingly, if the model is provided with a context in which information is contained, then the answer will be much more reliable.

But what if we let the model google and find sources of information on the Internet? So that she herself finds the source and builds an answer based on it? Well, this has already been done! So you can not google yourself, but delegate part of the Internet search to GPT‑4 itself. However, this requires a paid subscription.

As for further progress in developing the reliability of factual information within the model, OpenAI CEO Sam Altman gives an estimate of 1.5–2 years to solve this problem by a team of researchers. We will be very much looking forward to it! But for now, keep in mind that you don’t need to trust what is written by a neuron 100%, and check-recheck at least the sources.

5. Is it true that neural networks steal drawings of real artists?

Yes and no - both sides of the conflict are actively arguing about this in courts around the world. It can be said for sure that the images are not directly stored in the models, just “watchfulness” appears.

In this plan neurons very similar to people who first study art, different styles, look at the work of the authors, and then try to imitate.

However, models learn, as we have already found out, according to the principle of error minimization. And if during training the model sees the same (or very similar) image hundreds of times, then, from her point of view, the best strategy is to remember the picture.

Let's take an example: your teacher at art school chose a very strange strategy. You draw two pictures every single day: the first is always unique, in a new style, and the second is the Mona Lisa. After a year, you try to evaluate what you have learned. Since you have drawn the Mona Lisa over 300 times, you remember almost all the details and now you can reproduce it. It will not be the exact original, and you will certainly add something of your own. Colors will be slightly different.

And now you are asked to draw something that was 100 days ago (and that you saw once). You will reproduce what is required much less accurately. Just because the hand is not stuffed.

It’s the same with neurons: they learn the same way in all pictures, just some are more common, which means that the model is also fined during training more often. This applies not only to paintings by artists - to any image (even advertising) in the training sample. Now there are methods for eliminating duplicates (because training on them is at least inefficient), but they are not perfect. Research shows that there are images that occur 400-500 times during a workout.

My verdict: neural networks do not steal images, but simply consider drawings as examples. The more popular the example, the more accurately the model reproduces it.

People do the same during training: they look at the beauty, study the details, the styles of different artists. But for artists or photographers who have spent half their lives learning a craft, the point of view is often radically different from the one described above.

6. Is it true that “everything is lost” and neural networks will take away work from people? Who cares the most?

It is important to separate just “neural networks” that do certain tasks from general purpose neural networks like ChatGPT. The latter are very good at following instructions and able to learn from examples in context. True, now the size of their "memory" is limited to 10-50 pages of text, as are the skills of reflection and planning.

But if someone's work comes down to the routine execution of instructions and this is easy to learn in a couple of days by reading articles (or if the entire Internet is filled with this information), and the cost of labor is above average - then soon such work automate.

But by itself, automation does not mean a complete replacement of people. Only part of routine work can be optimized.

A person will begin to get more interesting and creative tasks that the machine (so far) cannot cope with.

If we give examples, then to the group of changeable or replaceable professions I would include, say, tax assistants-consultants who help prepare a declaration and check for typical errors, identify inconsistencies. Changes are possible in such a specialty as a clinical trial data manager - the essence of work is in filling out reports and reconciling them with a table of standards.

But a cook or a bus driver will be in demand much longer simply because they can connect neural networks and a real the physical world is quite complicated, especially in terms of legislation and regulations - thanks to the bureaucrats for moving away Crisis AI!

Big changes are expected in the industries associated with printed materials and textual information: journalism, education. With a very high probability for the first, neurons will very soon write drafts with a set of theses, in which people will already make point changes.

I am most pleased with the changes in the field of education. Eat research, which show that the quality of education directly depends on the "personality" of the approach and how much time the teacher devotes to a particular student. The simplest example: teaching in groups of 30 people using a textbook is much worse than individual tutor for specific needs (albeit according to the same program as in the textbook). With the development of AI, humanity will have the opportunity to provide a personalized assistant to each student. It's just incredible! The role of the teacher will shift, as I see it, to a strategic and controlling one: determining the general program and sequence of study, testing knowledge, and so on.

7. Is it possible to upload your consciousness into a computer, make a digital twin and live forever?

In the sense in which it is imagined on the basis of sci-fi, no. You can only teach the model to imitate your communication style, learn your jokes. Perhaps GPT-4 level models will even be able to invent new ones framed in your unique style and manner of presentation, but this clearly does not mean a complete transfer of consciousness.

We as humanity, again, do not know what consciousness is, where it is stored, how it differs from others, what makes me - me, and you - you. If it suddenly turns out that all this is just a set of memories and experiences, multiplied by individual characteristics perception, then, most likely, it will be possible to somehow transfer knowledge to neural networks so that they simulate future life on their basis.

8. Is it dangerous to upload your voice, appearance, your text style of speech in a neural network? It seems that such a digital identity can be stolen.

You can't literally download anything into them. You can train them (or re-train them) in such a way that the results are more like your appearance, voice or text. And such a trained model can really be stolen, that is, simply copy the script and a set of parameters to run on another computer.

You can even generate a video with a request transfer money at someone else's expense, which your relative will believe in: the best deepfake and voice cloning algorithms have already reached this level. True, thousands of dollars and tens of hours of recording are required, but nevertheless.

In general, with the development of technology, the issue of identification and confirmation of identity becomes more important.

And they are trying to solve it one way or another. For example, there is a startup WorldCoin (in fact, it makes a cryptocurrency), in which the head of OpenAI, Sam Altman, invested. The meaning of the startup is that each piece of information about a person will be signed by his own key for subsequent identification. The same will apply to the mass media, in order to know for sure whether this news is true or fake.

But, unfortunately, while all this is at the stage of prototypes. And I don’t consider the deep introduction of systems in all industries to be implemented on the horizon of the next decade, simply because it is too complicated and large-scale.

9. Can neurons start to harm and take over the world?

The danger is not current developments, but what will follow them with further development. Currently, no methods have been invented to control the operation of neural networks. Take, for example, a very simple task: to make sure that the model does not swear. Never ever. There is no method that will allow you to follow such a rule. So far, you can find different ways of how to “breed” it all the same.

Now imagine that we are talking about GPT-8 conditionally, whose skills will be comparable to the skills of the most capable and smart people. The neural network can program, use the Internet, knows psychology and understands how people think. If you give it free rein and do not set a specific task, then what will it do? What if she finds out she can't be controlled?

The likelihood of a bad turn of events is not so great, according to estimates. By the way, there is no generally accepted assessment - although everyone argues about the details, about the detrimental consequences, and so on. Now they call approximate figures from 0.01% to 10%.

In my view, these are huge risks, assuming that the most negative scenario is the destruction of humanity.

Interestingly, ChatGPT and GPT-4 are products that were created by teams working on the problems of "aligning" the intentions of people and neurons (details can be found here). That is why models listen to instructions so well, try not to be rude, ask clarifying questions, but this is still very far from ideal. The problem of control is not even half solved. And while we do not know whether it is being solved at all, and if so, by what methods. This is the hottest research topic for today.

10. Can a neural network fall in love with a person?

With current approaches and architectures of neurons, no. They only generate text that is most plausible as a continuation of the input text. If you throw in the first chapter of a love story, rewriting it under your personality, and ask the model to answer your love letter, she will cope with it. But not because I fell in love, but because it most accurately fits the context and the request “write me a letter!”. Remember that models learn to generate text that follows instructions.

Moreover, neural networks in the basic version do not have memory - between two different launches, they forget everything and roll back to the "factory settings". Memory can be added artificially, as if from the side, so that, say, 10 pages of the most relevant "memories" are fed into the model. But then it turns out that we simply feed a set of events into the original model and say: “How would you behave under such conditions?” The model does not have any feelings.