“The human brain, then, is the most complicated organization of matter that we know.”
Isaac Asimov

If someone asks what you recall when you look at the following pictures, I can hear you say ‘President Obama’ and ‘Statue of Liberty’. You just see a fragment of the pictures and remember them.

So, how does it happen? This is just a simple task for the brain. It stores an image and retrieves it whenever a part of it is seen. Amazing features of the brain, especially its power to learn and make decisions, inspires computer scientists in the field of artificial intelligence.

In computer science, an artificial neuron is a simple computational model of a neuron in the brain that excludes biological properties. Artificial neural networks (ANNs) are composed of artificial neurons, and they are utilized to solve specific problems, especially those that require learning and decision-making. ANNs may not be the best solutions in various machine learning problems; however, they are accepted as strong alternatives. Although ANNs don’t claim to be so, currently they are not even close to producing a simple model of the brain. Let’s take a short journey into the world of ANNs to experience the extreme difficulty of modeling the brain.

Some Applications of ANNs

Figure 1: Overview of ALVINN structure. Images obtained from the camera installed on the vehicle are provided to the ANN, and ANN decides the steering angle.

(adapted from: http://virtuallab.kar.fei.stuba.sk/robowiki/images/e/e8/Lecture_ALVINN.pdf).

ANNs have various applications in very large spectrum of problems that require learning, such as the Autonomous Land Vehicle in a Neural Network (ALVINN). The structure of ALVINN is shown in Figure 1. The ALVINN project by Carnegie Mellon University started in 1986 and aims to make a vehicle without a driver (Mitchell, 1997). In this project, ANN learns the steering habits of a driver. A camera is mounted on the vehicle to capture the images of the road. With respect to the continuous images provided, ALVINN determines the steering level with 45 different angle positions from sharp left to sharp right. Steering is updated 15 times per second so that it allows real-time control while driving at 55 mph. The system is trained by the data obtained from a human driver in a simulator and a real vehicle. ALVINN was able to speed up to 70 mph and successfully drive at 55 mph for 90 miles.

Figure 2: Handwritten zip codes (LeCun, et al., 1989)

Another example is handwritten zip code recognition (LeCun, et al., 1989). Zip codes from US Mail written by various people with large variety of styles and sizes were used in the experiments. Figure 2 presents some examples of zip codes in the experiment database. After the ANN was trained with more than 7,000 digits in the zip codes, it was 99% successful in recognizing around 2,000 digits in new zip codes.

Learning and Decision-making in ANNs

In order to understand the challenges better, we will first examine learning and decision-making in neurons and ANNs on simple examples.

Figure 3: (a) A typical neuron (adopted from http://commons.wikimedia.org/wiki/File:Neuron_-_annotated.svg), (b) artificial neuron in computer

Figure 3(a) illustrates a typical neuron which is the constituent of brain’s complicated network structure. Each neuron receives information as signals via dendrites, then evaluates it and generates a signal that is transmitted through its axon. A neuron has many connections between its dendrites and the axons of various other neurons. Figure (b) demonstrates an artificial neuron in computer science. It is considered a function: dendrites as the inputs of the function and generated signal via the axon as the output of the function.

Let’s see an example of an artificial neuron that understands if a given produce is a red apple or not. Think about how you understand whether a produce is a red apple or not. You see the shape and the color. However, it might be an artificial one for decoration. Then you can taste it and you get the sweetness of the apple. Similarly, our neuron receives three pieces of information as the input; ‘has circular shape?’, ‘is sweet?’, and ‘has red color?’. If the output is ‘yes’, that means the neuron recognizes the produce as a red apple. Otherwise, it will be ‘no’, which means the produce is not a red apple (Figure 4). Here, the neuron’s function is defined in such a way that it only generates ‘yes’ when all the inputs are ‘yes’.

Figure 4: Example inputs and outputs for the artificial neuron.

In an artificial neuron, some of the information can be more important than the others. For instance, to have red color may be more valuable in determining the price of produce. Assume that round shape and sweetness has equal value of $1; however, having red color is $2 – twice as valuable as the other features (Figure 5).

Figure 5: Artificial neuron with different input weights. Arrow thickness indicates the importance.

So, what is the big fuss about artificial neurons if they are only functions? In fact, the main feature of artificial neurons is learning. Considering the last example above, the neuron initially does not know the importance of the dendrites, i.e. the weights of inputs are all the same. If not trained, the neuron will generate the following answers which are sometimes wrong as indicated in Table 1.

Table 1: Artificial neuron before training; highlighted answers are wrong.

In real life, a teacher trains students. For instance, the teacher asks a question and if the received answer is not correct, she provides the right answer. Students learn the right answer and use this correct information in their lives. It is similar in artificial neurons as depicted in Figure 6. When the response of the neuron is incorrect, it adjusts the importance of the dendrites with respect to the correct answer hence it answers the same question correctly next time. During the training session, the neurons will be continuously asked the values of all the produce until it learns them all.

Figure 6: The learning process of the artificial neuron.

What if the problem gets complicated? Then one artificial neuron will not be sufficient, and we will need a network of neurons; ANNs. A more complex problem, ‘learning the digits’ is indicated in Figure 7.

Figure 7: (a) Digit learning problem, (b) ANN structure that learns digits. Due to the difficulty, only the connections between the input layer and first / last neurons in the middle layer are shown.

ANN has 3 x 5 = 15 input units like receptors of an eye retina. Each input unit corresponds to one square in the digits; either filled or blank. Each input unit is connected to the dendrites of all neurons in the middle layer. The output of each cell in the middle is connected to the dendrites of all neurons in the output layer. There are 10 output neurons corresponding to the digits from 0 to 9. After the ANN is trained, it provides a correct answer to the given digit as input. When digit ‘3’ is provided to the network, the neuron labeled with number ‘3’ in Figure 7(b) is triggered and outputs ‘yes’ whereas the rest of the neurons output ‘no’.

Figure 8: Faulty digit ‘3’ with a missing black square on the top right side.

Initially, all neurons in the network have equally weighted dendrites. After a reasonable amount of training, neurons adjust their weights, and ANN is able to identify digits. Here, we have some major challenges: what does ‘reasonable amount of training’ mean? When the ANN is undertrained, it will not always answer correctly to the digits given in Figure (a). In the other case, when the ANN is overtrained, it will memorize the digits provided during the training and will not recognize the faulty ones such as the one in Figure 8.

Challenges of ANNs

Beyond the mentioned the overtraining / undertraining problems, ANNs have a bigger challenge – how to determine the structure of ANN that fits the problem? In the digit learning example, we’re lucky because the structure is provided in Figure 7(b). However, the outcome of the solution may drastically depend on the number of neurons and the connections among them which is indeed a hard problem for ANNs.

The huge capability of the brain in learning and decision making comes from the huge number of neurons – around 100 billion – and the enormous amount of connections among them – from 100 to 500 trillion. The challenge to design such a huge network requires huge computation power. With the increasing number of neurons, ANN dramatically slows down especially during the learning process. Here, our example is a simple learning task of 3×5 pixel digits compared to the brain’s acquisition capacity of hundreds of images in our daily life. 

When the number of neurons gets larger, the reliability of network also reduces. Small adjustments in weights may change the entire behavior of the network hence it is easy to lose control of ANN. In contrast, the brain has a robust system, and its fault tolerance is admirable. Although neurons die every day, this doesn’t affect its performance significantly. The training method and how to update the weights are other hard problems leading to many different approaches in the neural computation field.

We have presented some simple tasks that can be solved using a few neurons and their challenges. On the other hand, consider the thousands of problems, various and incredible amount of information we have learned, and the thousands of decisions we make. The brain is truly amazing from the computer science perspective.

Bibliography

• Hertz, J. A., Krogh, A. S., & Palmer, R. G. (1991). Introduction To The Theory Of Neural Computation. Reading, MA: Addison-Wesley.
• Hopfield, J. J. (1982). Neural networks and physical systems with emergent collective computational properties. Proceedings of the National Academy of Sciences of the USA , 79, 2554-2588.
• LeCun, Y., Boser, B., Denker, J. S., Henderson, D., Howard, R. E., Hubbard, W., et al. (1989). Backpropagation applied to handwritten zip code recognition. 1 (4), 541-551.
• Mitchell, T. M. (1997). Machine Learning. McGraw-Hill.

Programs and documents on the computer are held in two areas: software and hardware. For scientists, one of many goals is to make the processors or chips, which are like the human brain that consists of nerves, much smaller, but still powerful enough to process many more calculations. There are many differences between the current chips and earlier ones. Chips which will be produced in the future will be smaller and probably process more calculations more quickly.

Programs, which can be seen as being the mechanical counterpart of the human mind, bring the above-mentioned improvements into daily life. New programs boost the capability of the computer in parallel with the capability of their chips. Without these programs, computers would be no more than ordinary electronic machines.

Developed technologies in fields like industry, communication, or the military bring us face to face with new developments and have made the computer an undeniable part of our lives. Mobile phones equipped with new features, medical machines which can easily make a large number of analyses and provide ease in diagnosis and treatment, robots that can operate with minimum error and have a low cost when used in production, and weapons that automatically focus on the target are all part of this progress.

With time, software and hi-tech sensors have enabled computers to communicate with people; there are now systems that are controlled by the voice, which are able to recognize a person from their iris or fingerprints, control systems which are carried out by touching a screen, etc. Such systems are operated with the help of special sensors or by some signals that carry messages from the person or the environment to the computer. The most important feature of these sensors is that the signals produced at the output are very weak and there are few differences between them. An ATM can recognize a particular person’s iris, thanks to the ability of its computer to compare the signals from the ATM’s iris scanner with previously recorded data. In this process, the computer uses the small differences that one person’s iris has to another’s. In this or similar systems, complicated programs are used, called “expert systems” or “artificial intelligence”. These programs imitate human senses, but they aim to operate with an even keener sensitivity and clearer criteria.

A question that is a subject of fiction comes to mind; “Will computers vie with or even fight with human beings?” In the mid-term, the rapid development of technology will create computers which can communicate with humans, which can understand them, and put forward ideas. A negative outcome of such a situation depends, once again, on man. Such a horrific situation could be the result of technology that can cause environmental disasters; this technology is almost identical to the one that we have described above. If we are able to establish an understanding of “civilization” which does not ignore human values for the sake of technological development, then such fears will be groundless.

Artificial Nervous Systems

As we all know, people have imitated nature in many of their inventions. In a way, artificial nervous systems imitate how a nerve cell learns and how it works. Fuzzy systems however imitate how the judgment of a human being works, rather than the nerve cells of the brain. In these systems people try to form a decision making criterion by assuming that there are endless grey tones between white and black or by assuming that there are infinite values between zero and one.

The purpose of the research on artificial nervous systems is to understand how the brain operates, then to make a system that imitates it and carries out the same operations. Artificial nervous systems are made of simple nerve cells which are bound in parallel, called process elements; these allow for real objects to be seen as if they were biological systems.

Here, the program that resembles the nerve cell operates in the same way as a nerve cell. The main part of a nerve cell is formed from the body, called a “soma”, an “axon” that is bound to the body and many “dendrites”. There are many “roots”, or synapses, on the dendrite of a cell which make contact with the dendrites of other cells. A nerve cell either transmits the electrical stimulus that comes through the axon to the other nerve cells through the synapse, or it does not transmit it, depending on whether or not the signal is over or below the threshold value. So a nerve works by itself, but its activity becomes meaningful when working as a part of a nervous system. It would be useful if we consider how the learning process occurs here. It is thought that the required data are stored in the memory center and this fact is taken as a model for some artificial nervous system software that has been successfully developed to date.

A nerve cell and the process of transporting signals from one cell to another can be written as software. It is clear that a natural nerve cell is more complex and that it is bound to more cells than an artificial one can be. The number of communication ports (synapses) of a natural nerve can vary from between 1,000 to 10,000.

An artificial cell produces output if the input value is over the cell’s threshold value; if this is not the case then there is no production. If there is output – as in natural cells – then this output is transported to the next cell group. Each cell produces its output as an input for the next cell.

A cell is separated into three groups: input, the hidden layer and output. Each group is considered to be made up of one layer, while the hidden layer can consist of more than one, according to the complexity of the job. As can be seen, the placement of the layers is similar in the process of the human body. We can compare the cells on the input layer with human senses. In this way we can teach a robot to avoid heat and cold, we can make them see and act according to this information. (Do not forget that a robot is in fact a computer.) It is natural that some sensors must be bound to the cells on the input layer. For instance, a sensor which is sensitive to heat can make the robot react to heat when the temperature is over the limit value or when the temperature is dramatically low it can move closer to a heat source. Or if pictures received from a video-camera are similar to an object that has been fed into the robot such data input can cause the robot to move to that object.

Artificial nervous systems are not only used in robot applications. They are commonly used in making clinical diagnoses, determining market-customer profiles, recognizing voices or pictures, classifications such as determining micro-structures, like germs and cell materials, economic profiles, energy sources, the futures of market shares, some predictive sciences, such as weather forecast, zipping data for computers, process control in industry, checking resources and some other matters in technological areas. As can be seen, there are many application areas for artificial nervous systems, all of which differ from one another.

The Basic Features of Artificial Nervous Systems

The features of artificial nervous systems can be simplified as follows: firstly, they can learn how to solve problems. In order to do this they use sample data and learning styles and while doing this they do not require any special help. Secondly, they can recognize important features and relations to help them distinguish different data forms.

When an artificial nervous system is operated, the first thing to be carried out is the training process. In order to do this, the program needs to have two alternating operations. It may obtain information concerning some results to be achieved, using results that come from the user, or the program is itself asked to produce some results. These two types of learning are not very different from how a human learns. One shows a young child an animal, and repeats the name. Now the child has learned the name of the animal and correlates it with the picture of the same. If no one teaches a child what a bird is, the child will all the same classify all animals that have wings and beaks and that have a certain physical shape, maybe even creating a name for the animal by him/herself. The difference between the computer and the human in this process is that a human has the ability to judge, while computers classify the animals according to their shapes and groups them thus. Naming and giving a naming feature to the computer is again a decision that a human will make. When the training process is finished, the data can be entered into the computer and the desired results can be attained.

Artificial nervous systems are changing and developing day by day. With each new development they become closer to the human nervous system; they are able to recognize different characteristics of different people and they are learning to make sorting decisions, even limited judgments. Whether or not these machines may one day enact a nightmare scenario, taking over from us is not a great threat, as whatever they are capable of doing is up to us to decide, as their “masters”. We should not fear these systems, but try to develop more of them; such systems help us in every day tasks, from drawing money out of the bank to our annual check-up at the doctor’s.

Synchronization is a problem in many areas of science, notably in electrical and electronics engineering. In synchronization, there must be at least two events, one of which serves as the reference for the other. Synchronized events always follow each other in a regular manner. In electrical engineering at the instant of synchronization of two busbar voltages, both voltages must be equal in magnitude and period and they must be in phase so that they can be switched in parallel if desired.

In my research I was synchronizing output voltage with line voltage; more recently I was trying to add certain further features into my program like time delay. At this stage while trying to generate synchronized outputs with a delay I failed to allow a few microseconds to the related registers (necessary because of some time delay caused by a few instructions) and also (as I later realized) I was putting some instructions in the wrong sequence. Maybe the beauty of the micro controller design is that it does not allow you to generate (actually you command the microcontroller to generate at the related outputs what you want it to generate) just anything you may happen to have in mind. The input has to be correctly ordered. If you give the right instructions in the right order, it generates (of course, within its limitations) the correct result, otherwise it generates the wrong result or just rubbish.

I spent a whole week looking for the reason for the problem which I have very roughly described. The program ought to have worked correctly because every instruction looked to be all right. But I didn’t see far enough into just how important a few microseconds and the sequences of instructions are. So, I got very frustrated and annoyed at not being able to find the reason for the failure of what ought to have been a simple program.

At night, while thinking about the problem, I realized some of the reasons for the problem, with the help of God. It was only a matter of a few microseconds in every cycle. I did not think that the program could be affected that much by that little. The outputs appeared quite stable for a time but then, after a while, the program would suddenly crash.

Ordinarily we might think: What can a few microseconds matter or the sequence of instructions? When we ask such questions, actually we are starting to think about the complexity of the universe.

The cause of the problem I was having was a few microseconds in every cycle (one cycle is 20,000 microsecond). A few microseconds in one cycle may seem nothing, but in fact the few microseconds are out in a continuous system, every cycle is affected. As a result, the program was causing the wrong outputs to be generated.

If, at this juncture, we think about the magnificence and/or complexity of the universe or for that matter of human beings, we begin to appreciate the greatness of God. In reality, it seems to me, it is impossible to imagine fully or to realize exactly the greatness of God since we cannot even grasp fully how complex the organization of the universe is. Take my problem as an example: it was a simple system with single input and single output, and yet neglecting to compensate a few microsecond of delays caused my output to crash. In the universe, every action and event in every bodily process in every plant and animal, must take place with the most minute exactness in real time, and not in the microsecond range but maybe in many times more or less than that range. Any oversight, be it ever so small, any error of sequence, any delay however small in any event in the universe, will affect all the other events in a chain of effects causing the system to crash suddenly, locally or, maybe, entirely. In short, the existence of the universe depends upon the correct instructions being minutely programmed in the correct sequence.

When we look at either the universe or at an individual creature in it, a human.being or plant or animal, we see that each operates as a large, separate system. We cannot even imagine how many inputs and outputs these systems have, we cannot imagine the complexity of the innumerable problems that are solved in such a way that life has been going on for millions of years. Whenever we look with open mind at any living organism within the universe or at the universe as a single, whole system, our sight returns to us, dazzled and overwhelmed-exactly as is described in the beautiful words of sura al-Mulk:

Then look again and yet again, your sight will return to you weakened and made dim. (67.4)

We see in the sky billions of stars turning in synchronization with each other according to some extraordinary law of harmony. And this harmony has been operative for millions of years, so effectively that its continuance is not in doubt. The same extraordinary miracle of harmony can be studied at the microscopic level: Within a single atom huge numbers of particles whizz past each other around the nucleous at unimaginable speeds in a continually renewed and vital process of creation.

Our understanding cannot fathom, nor our researches exhaust, the wonder in which we live and which we behold. And when we realize the complexity of the innumerable systems which compose the universe and whose inter-related functions have been managed not for seconds or hours, but for hundreds of millions of years, can we do otherwise than humbly acknowledge the wisdom and power of God? Equally, when we accept, as logically we must, that in the universe as a whole, everything is organized in the right way for its continued operation, are we not bound to conclude that every event, seemingly good or bad, has occurred at its own time and place, precisely, as pre-ordained (or ‘programmed’ we might say) by God? Thus, we are led to acknowledge the Creator, to marvel in humility at His grandeur, and His greatness.