ARTIFICIAL NEURAL
NETWORK
…The Emerging Technology
An Artificial Neural Network (ANN) is an information processing paradigm that is inspired by the way biological nervous systems, such as the brain, process information. The key element of this paradigm is the novel structure of the information processing system. It is composed of a large number of highly interconnected processing elements (neurons) working in unison to solve specific problems. . An ANN is configured for a specific application, such as pattern recognition or data classification, through a learning process.
Historical background
The study of the human brain is thousands of years old. With the advent of modern electronics, it was only natural to try to harness this thinking process. The first step toward artificial neural networks came in 1943 when Warren McCulloch, a neurophysiologist, and a young mathematician, Walter Pitts, wrote a paper on how neurons might work. They modeled a simple neural network with electrical circuits.
As computers advanced into their infancy of the 1950s, it became possible to begin to model the rudiments of these theories concerning human thought. Nathanial Rochester from the IBM research laboratories led the first effort to simulate a neural network. That first attempt failed. But later attempts were successful.
It was during this time that traditional computing began to flower and, as it did, the emphasis in computing left the neural research in the background.
How the Human Brain Learns?
Much is still unknown about how the brain trains itself to process information, so theories abound. In the human brain, a typical neuron collects signals from others through a host of fine structures called dendrites. The neuron sends out spikes of electrical activity through a long, thin stand known as an axon, which splits into thousands of branches. At the end of each branch, a structure called a synapse converts the activity from the axon into electrical effects that inhibit or excite activity from the axon into electrical effects that inhibit or excite activity in the connected neurons. When a neuron receives excitatory input that is sufficiently large compared with its inhibitory input, it sends a spike of electrical activity down its axon. Learning occurs by changing
finally, a biological neuron receives inputs from other sources, combines them in some way, performs a generally nonlinear operation on the result, and then outputs the final result. Figure shows the relationship of these four parts. (dendrites, soma, axon, and synapses)
From Human Neurons to Artificial Neurons
We conduct these neural networks by first trying to deduce the essential features of neurons and their interconnections. We then typically program a computer to simulate these features. However because our knowledge of neurons is incomplete and our computing power is limited, our models are necessarily gross idealizations of real networks of neurons.
A Basic Artificial Neuron
consists various inputs to the network are represented by the mathematical symbol, x(n). Each of these inputs are multiplied by a connection weight. These weights are represented by w (n)
Artificial Neurons working principle
In the simplest case, these products are simply summed, fed through a transfer function to generate a result, and then output. This process lends itself to physical implementation on a large scale in a small package. This electronic implementation is still possible with other network structures which utilize different summing functions as well as different transfer functions.
Some applications require "black and white," or binary, answers. These applications include the recognition of text, the identification of speech, and the image deciphering of scenes. These applications are required to turn real-world inputs into discrete values. These potential values are limited to some known set, like the ASCII characters or the most common 50,000 English words. Because of this limitation of output options, these applications don't always utilize networks composed of neurons that simply sum up, and thereby smooth, inputs. These networks may utilize the binary properties of ORing and ANDing of inputs. These functions, and many others, can be built into the summation and transfer functions of a network.
Other applications might simply sum and compare to a threshold, thereby producing one of two possible outputs, a zero or a one. Other functions scale the outputs to match the application, such as the values minus one and one. Some functions even integrate the input data over time, creating time-dependent networks.
1.Electronic Implementation of Artificial Neurons
In currently available software packages these artificial neurons are called "processing elements" and have many more capabilities than the simple artificial neuron described above. Those capabilities will be discussed later in this report
In above Figure, inputs enter into the processing element from the upper left. The first step is for each of these inputs to be multiplied by their respective weighting factor (w(n)). Then these modified inputs are fed into the summing function, which usually just sums these products. Yet, many different types of operations can be selected. These operations could produce a number of different values which are then propagated forward; values such as the average, the largest, the smallest, the ORed values, the ANDed values, etc. Furthermore, most commercial development products allow software engineers to create their own summing functions via routines coded in a higher level language (C is commonly supported). Sometimes the summing function is further complicated by the addition of an activation function which enables the summing function to operate in a time sensitive way.
Either way, the output of the summing function is then sent into a transfer function. This function then turns this number into a real output via some algorithm. It is this algorithm that takes the input and turns it into a zero or a one, a minus one or a one, or some other number. The transfer functions that are commonly supported are sigmoid, sine, hyperbolic tangent, etc. The result of the transfer function is usually the direct output of the processing element.
This sigmoid transfer function takes the value from the summation function, called sum in the below Figure, and turns it into a value between zero and one.
Finally, the processing element is ready to output the result of its transfer function. This output is then input into other processing elements, or to an outside connection, as dictated by the structure of the network.
2. Artificial Network Operations
It is variety and the fundamental differences in these building blocks which partially cause the implementing of neural networks to be an "ART". The other part of the "art" of using neural networks revolves around the myriad of ways these individual neurons can be clustered together. This clustering occurs in the human mind in such a way that information can be processed in a dynamic, interactive, and self-organizing way
This clustering occurs by creating layers which are then connected to one another. How these layers connect is the other part of the "art" of engineering networks to resolve real world problems.
Basically, all artificial neural networks have a similar structure or topology as shown in above Figure. One of the easiest ways to design a structure is to create layers of elements. It is the grouping of these neurons into layers, the connections between these layers, and the summation and transfer functions that comprises a functioning neural network.
Most applications require networks that contain at least the three normal types of layers - input, hidden, and output. The layer of input neurons receives the data either from input files or directly from electronic sensors in real-time applications. The output layer sends information directly to the outside world, to a secondary computer process, or to other devices such as a mechanical control system. Between these two layers can be many hidden layers. These internal layers contain many of the neurons in various interconnected structures. The inputs and outputs of each of these hidden neurons simply go to other neurons.
In most networks each neuron in a hidden layer receives the signals from all of the neurons in a layer above it, typically an input layer. After a neuron performs its function it passes its output to all of the neurons in the layer below it, providing a feed forward path to the output.
These lines of communication from one neuron to another are important aspects of neural networks. There are two types of these connections. One causes the summing mechanism of the next neuron to add while the other causes it to subtract.
Some networks want a neuron to inhibit the other neurons in the same layer. This is called lateral inhibition. The most common use of this is in the output layer.
Another type of connection is feedback. This is where the output of one layer routes back to a previous layer.
Once a network has been structured for a particular application, that network is ready to be trained.
There are two approaches to training - supervised and unsupervised. Supervised training involves a mechanism of providing the network with the desired output either by manually "grading" the network's performance or by providing the desired outputs with the inputs.
Major Components of an Artificial Neuron
This section describes the seven major components which make up an artificial neuron.
Component 1. Weighting Factors: A neuron usually receives many simultaneous inputs. Each input has its own relative weight which gives the input the impact that it needs on the processing element's summation function.
Component 2. Summation Function: The first step in a processing element's operation is to compute the weighted sum of all of the inputs. Mathematically, the inputs and the corresponding weights are vectors which can be represented as (i1, i2 . . . in) and (w1, w2 . . . wn). Input1 = i1 * w1, input2 = i2 * w2, etc., are added as input1 + input2 + . . . + input. The result is a single number, not a multi-element vector.
Component 3. Transfer Function: The result of the summation function, almost always the weighted sum, is transformed to a working output through an algorithmic process known as the transfer function. In the transfer function the summation total can be compared with some threshold to determine the neural output.
Component 4. Scaling and Limiting: After the processing element's transfer function, the result can pass through additional processes which scale and limit. This scaling simply multiplies a scale factor times the transfer value, and then adds an offset. Limiting is the mechanism which insures that the scaled result does not exceed an upper or lower bound.
Component 5. Output Function (Competition): Each processing element is allowed one output signal which it may output to hundreds of other neurons. This is just like the biological neuron, where there are many inputs and only one output action..
Neural Networks Vs Traditional Computing and Expert Systems
Characteristics | Von Neumann Architecture Used for Expert Systems | Artificial Neural Networks |
Processing Approach | Separate | The same |
Processors | VLSI (traditional processors) | Artificial Neural Networks; variety of technologies; hardware development is on going |
Neurobiology in design | None | Moderate |
Connections | Externally programmable | Dynamically self programming |
Self learning | Only algorithmic parameters modified | Continuously adaptable |
APPLICATIONS OF NEURAL NETWORKS
Language Processing : Language processing encompasses a wide variety of applications. These applications include text-to-speech conversion, auditory input for machines, automatic language translation, secure voice keyed locks.
Character Recognition: Character recognition is another area in which neural networks are providing solutions. for example-This product can take cards, like a credit card application form, and put those recognized characters into a data base. This product has been out for two and a half years. It is 98% to 99% accurate for numbers.
Image (data) Compression: These networks are auto associative in that they can reduce eight bits of data to three and then reverse that process upon restructuring to eight bits again. However, they are not lossless.
Pattern Recognition: Another application involves the grading of rare coins. Digitized images from an electronic camera are fed into a neural network. These images include several angles of the front and back. These images are then compared against known patterns which represent the various grades for a coin.
Financial: Neural networks are making big inroads into the financial worlds. Banking, credit card companies, and lending institutions deal with decisions that are not clear cut. They involve learning and statistical trends.
Present "Ann"
Basically there are 2 important fields where our vendors have concentrated. In choosing a development tool a software engineer needs to beware of this emerging field. Most products are not evolved into the user friendly routines that draw raves.
Development Systems: Good development systems allow a user to prototype a network, train it, tweak it, and use it. These systems run on the standard range of computers. These packages usually don't run on specialized hardware, although some vendors have packaged fast RISC processors into special neural processing boards.
Hardware Accelerators : The key to the continued evolution of neural networking lies in the hardware. Traditional hardware does not enable the massive parallelism that is required by neural networks. There are several approaches one of them is, One is to develop a processor which is specifically tailored to performing the tasks of individual artificial neurons.
Dedicated Neural Processors : Dedicated neural processors are processors with specific capabilities that enable their use in neural networks. Several manufacturers have developed neural processors. Some of these processors were created specifically for the development system vendors. Others incorporate proprietary concepts, such as creating a specific type of fuzzy neuron. These chips come in many broad technologies - analog, digital, hybrid, and optical.
Emerging Technology
If the 21st Century is to be the age of intelligent machines, then artificial neural networks will become an integral part of life.
Fuzzy logic: The greatest interest is on merging fuzzy logic with neural networks. Fuzzy logic incorporates the inexactness of life into mathematics. In life most pieces of data do not exactly fit into certain categories. For instance, a person is not just short or tall. He can be kinda short, pretty tall, a little above average, or very tall. Fuzzy logic takes these real-world variations into account. Systems built with fuzzy neurons may be initialized to what an expert thinks are the rules and the weights for a given application.
Conclusion
An artificial neural network (ANN) or commonly just neural network (NN) is an interconnected group of artificial neurons that uses a mathematical model or computational model for information processing based on a connectionist approach to computation. In most cases an ANN is an adaptive system that changes its structure based on external or internal information that flows through the network. (The term "neural network" can also mean biological-type systems.)
A neural network is an interconnected group of nodes, akin to the vast network of neurons in the human brain.
Finally, I would like to state that even though neural networks have a huge potential we will only get the best of them when they are integrated with computing, AI, fuzzy logic and related subjects.
Recent Comments