Basic Concepts of Artificial Neural Network – ANN

A neural network is just an attempt to make computer models like a brain because if computers were more like the brain they could be good at some of the things humans are good at, like pattern recognition. So a neural network simulate a collection of neurons just as done in the brain and these simulated neurons take inputs and give outputs through their connections.

Whole neural network is just a function that takes in a few input numbers and outputs a few output numbers and in between it does some computation with the hidden layers.

So, neural network, in general, is a highly interconnected network of billions of artificial neuron with trillion of interconnections between them which are are configured to perform specific tasks.

And artificial neural networks match the human brain in two ways –
  • A neural network acquires knowledge through learning.
  • A neural network’s knowledge is stored within inter-neuron connection strengths known as synaptic weights.

An Artificial Neural Network(ANN) has several advantages but one of the most recognized of these is the fact that it can actually learn from observing data sets and it takes data samples rather than entire data sets to arrive at solutions, which saves both time and money.

The training of neural network to learn is performed using defined set of rules also known as the learning algorithm. Some of those algorithms are–

  • Gradient Descent— This is the simplest training algorithm used in case of supervised training model. The gradient
    descent algorithm changes the weights of network accordingly to minimize the error.
  • Back propagation — It can be taken as an extension of delta learning rule based on gradient. Here, the error is
    propagated backward from output layer to the input layer through hidden layer. It is used in case
    of multi-layer neural network.
Architecture of Artificial Neural Network

A typical neural network contains a large number of artificial neurons called units which are arranged in a series of layers like-

  • Input layer — It contains the artificial neurons which receive input from the outside world on which network will learn,recognize about or otherwise process.
  • Output layer — It contains units that respond to the information about how it’s learned any task.
  • Hidden layer— These units are in between input and output layers. It transform the input into something that can be used by output unit in some way.
Types of Learning in Neural Network
  • Supervised Learning — In supervised learning, the training data is input to the network, and the desired output known weights, are adjusted until output yields desired value.
  • Unsupervised Learning — The input data is used to train the network whose output is known. The network classifies the input data and adjusts the weight by feature extraction in input data.
  • Reinforcement Learning— Here the value of the output is unknown, but the network provides the feedback whether the output is right or wrong.
  • Offline Learning — The adjustment of the weight vector and threshold is done only after all the training set is presented to the is also called batch learning.
  • Online Learning— The adjustment of the weight and threshold is done after presenting each training sample to the network.

Some characteristics of Artificial Neural Networks are-

  • Massive Parallelism
  • Distributed representation
  • Learning ability
  • Generalization ablity
  • Fault tolerance
Neural Networks for Machine Learning
  1. Multilayer Perceptron (supervised classification)
  2. Back Propagation Network (supervised classification)
  3. Hopfield Network (for pattern association)
  4. Deep Neural Networks (unsupervised clustering)
Neural Networks for Deep Learning
  1. Feed-forward neural networks
  2. Recurrent neural network
  3. Multi-layer perceptrons (MLP)
  4. Convolutional neural networks
Applications of Artificial Neural Network
  • Pattern Classification
  • Clustering/Categorization
  • Function approximation
  • Prediction/Forecasting
  • Optimization
  • Content-addressable Memory
  • Control

Join Our Forum discussion on various CS topics and subscribe our Youtube channel to get updates on webinars.