Neural Networks and Humans- Exploring Similarities
Machine Learning, Computer Vision, Artificial Intelligence, Natural Language Processing are all buzz words in the tech world right now. At the heart of all these buzz words lies one concept that is the base for many solutions- Neural Networks. But what exactly is a neural network and why is it called so?
Let’s start with a bit of Biology. The diagram given below is of a neuron that we have all have seen in our 10th grade biology textbook. In simpler words, it is the basic unit of the nervous system. Neurons make nerves, nerves along with the brain and the spinal cord make the nervous system. The nervous system is responsible for all our decisions, actions and thoughts. But what is the function of a neuron at its core? It is to receive and send information. Just like it is shown in the diagram, the neuron receives and further sends an electrical or chemical signal to other neurons. Thus, acting like an f(x) function.
It has been a long time since we humans started our attempts to make computers think, perceive and react like humans do. Whether it is the automation of various jobs in a factory or your search engine recommending you the best document according to your information need. One of the attempts which has worked brilliantly is the concept of Neural Networks. What is so special about it? It is the fact that neural networks imitate (or at least try to) the human nervous system.
The basic unit of a neural network is also called a neuron. But it does actual math. It takes an input value and gives an output value to the next “neuron” just like in humans. This diagram beautifully presents the analogy between the parts of a human neuron and the neural network neuron.
Another interesting thing I want to point out is how there is similarity in their interconnections too. This figure clearly brings out the similarity in this aspect too- one is connected to many which is further connected to many and so on.
The “math” here is the multiplication of the inputs (x_i) with some weights (w_i) and adding a bias (b). “Why is it done?” is another question to which the answer is a bit long and since I aim to answer only the basic question of “What is a neural network at its core?” I shall not be diving into the answer for the former question.
How are these weights decided? The answer is simple. Let me take an example, let’s say the expected output for a specific set of inputs is y_e and I start with the weights of w_i=0. Let’s say that I got an output of y_a using these weights. Now, I know that there is an error of y_a- y_e. If the absolute error is high, I know that my weight values are deviating from the right one by a huge margin. If the absolute error is small, I know that I am very close to the right values! The core principle being that I use the information of “How far is my output value from the desired output?” and modify my weight values accordingly. This technique is called “Back-propagation.”
For electrical engineers like me it would be more appropriate to understand this technique with the help of this control systems diagram which also makes use of the error to give the desired output. The function of the feedback mechanism is to guide the value of the output in the right direction and so is the function of weights- to push the output value in the right direction! Note that this is just an analogy and not the exact way that neural networks function.
One question I want to put across is “How do humans learn?” We experience something, learn from that experience and then try to apply it in similar situations later. This is another aspect that is common between neural networks and humans. Neural networks learn from experience or what is called the “training set.” It finds the suitable values for the weights by “experiencing” the training set and trying to achieve the desired output. This way it has “learnt” from its experiences and then applies it on a “testing set” to understand if the solution is right and robust.
This resemblance of neural networks to the human brain is not coincidental. It was a deliberate attempt to mimic how the brain works by Warren McCulloch (neurophysiologist and cybernetician) and Walter Pitts (a computational neuroscientist).
The similarities between human nervous system and neural networks are huge. Yet, computers haven’t been able to exactly mimic humans. There is more to be explored and done in order to achieve the human-like-computers dream. But there is one idea to still ponder upon: just like a neuron cannot be replaced if one dies, the success of neural networks is irreplaceable at least in the near future.
Hi there! Thank you so much for giving this blog a read. You can reach out to me via mail (miloni.mittal@gmail.com) for any queries or for a small chit-chat. :)
