20-06-2011, 04:33 PM
Submitted By :-
Vikas Sharma
[attachment=14177]
Introduction to Artificial Neural Networks
Neural networks : Introduction
Neural network: information processing paradigm inspired by biological nervous systems, such as our brain
Structure: large number of highly interconnected processing elements (neurons) working together
Like people, they learn from experience (by example)
Neural networks : Introduction
Neural networks are configured for a specific application, such as pattern recognition or data classification, through a learning process
In a biological system, learning involves adjustments to the synaptic connections between neurons
same for artificial neural networks (ANNs)
A new sort of computer
What are (everyday) computer systems good at... and not so good at?
Where can neural network systems help
when we can't formulate an algorithmic solution.
when we can get lots of examples of the behavior we require.
‘learning from experience’
when we need to pick out the structure from existing data.
Inspiration from Neurobiology
A neuron: many-inputs / one-output unit
output can be excited or not excited
incoming signals from other neurons determine if the neuron shall excite ("fire")
Synapse concept
The synapse resistance to the incoming signal can be changed during a "learning" process [1949]
Mathematical representation
The neuron calculates a weighted sum of inputs and compares it to a threshold. If the sum is higher than the threshold, the output is set to 1, otherwise to -1.
A simple perceptron
It’s a single-unit network
Change the weight by an amount proportional to the difference between the desired output and the actual output.
Δ Wi = η * (D-Y).Ii
Example: A simple single unit adaptive network
The network has 2 inputs, and one output. All are binary. The output is
1 if W0I0 + W1I1 > 0
0 if W0I0 + W1I1 ≤ 0
We want it to learn simple OR: output a 1 if either I0 or I1 is 1.
Artificial Neural Networks
Adaptive interaction between individual neurons
Power: collective behavior of interconnected neurons
Evolving networks
Continuous process of:
Evaluate output
Adapt weights
Take new inputs
ANN evolving causes stable state of the weights, but neurons continue working: network has ‘learned’ dealing with the problem
Learning
From experience: examples / training data
Strength of connection between the neurons is stored as a weight-value for the specific connection
Learning the solution to a problem = changing the connection weights
Learning performance
Learning Paradigms:
Unsupervised
Competitive Learning
Reinforcement learning
Backpropagation
Unsupervised learning
No help from the outside
No training data, no information available on the desired output
Learning by doing
Example : -
Competitive learning: example
In this type, it is generally Winner takes all concept.
only update weights of winning neuron
Back propagation
Desired output of the training examples
Error = difference between actual & desired output
Change weight relative to error size
Calculate output layer error , then propagate back to previous layer
Improved performance, very common!
Where are NN used?
Recognizing and matching complicated, vague, or incomplete patterns
Data is unreliable
Problems with noisy data
Prediction
Classification
Data association
Data conceptualization
Filtering
Planning
Applications
Prediction: learning from past experience
pick the best stocks in the market
predict weather
identify people with cancer risk
Classification
Image processing
Predict bankruptcy for credit card companies
Risk assessment
Applications
Recognition
Pattern recognition: SNOOPE (bomb detector in U.S. airports)
Character recognition
Handwriting: processing checks
Data association
Not only identify the characters that were scanned but identify when the scanner is not working properly
Applications
Data Conceptualization
infer grouping relationships e.g. extract from a database the names of those most likely to buy a particular product.
Data Filtering
e.g. take the noise out of a telephone signal, signal smoothing
Planning
Unknown environments
Sensor data is noisy
Fairly new approach to planning
Strengths of a Neural Network
Power: Model complex functions, nonlinearity built into the network
Ease of use:
Learn by example
Very little user domain-specific expertise needed
Intuitively appealing: based on model of biology, will it lead to genuinely intelligent computers/robots?
Neural networks cannot do anything that cannot be done using traditional computing techniques, BUT they can do some things which would otherwise be very difficult.
General Advantages
Advantages
Adapt to unknown situations
Robustness: fault tolerance due to network redundancy
Autonomous learning and generalization
Disadvantages
Not exact
Large complexity of the network structure
Future of Neural Networks
Most of the reported applications are still in research stage
No formal proofs, but they seem to have useful applications that work
Vikas Sharma
[attachment=14177]
Introduction to Artificial Neural Networks
Neural networks : Introduction
Neural network: information processing paradigm inspired by biological nervous systems, such as our brain
Structure: large number of highly interconnected processing elements (neurons) working together
Like people, they learn from experience (by example)
Neural networks : Introduction
Neural networks are configured for a specific application, such as pattern recognition or data classification, through a learning process
In a biological system, learning involves adjustments to the synaptic connections between neurons
same for artificial neural networks (ANNs)
A new sort of computer
What are (everyday) computer systems good at... and not so good at?
Where can neural network systems help
when we can't formulate an algorithmic solution.
when we can get lots of examples of the behavior we require.
‘learning from experience’
when we need to pick out the structure from existing data.
Inspiration from Neurobiology
A neuron: many-inputs / one-output unit
output can be excited or not excited
incoming signals from other neurons determine if the neuron shall excite ("fire")
Synapse concept
The synapse resistance to the incoming signal can be changed during a "learning" process [1949]
Mathematical representation
The neuron calculates a weighted sum of inputs and compares it to a threshold. If the sum is higher than the threshold, the output is set to 1, otherwise to -1.
A simple perceptron
It’s a single-unit network
Change the weight by an amount proportional to the difference between the desired output and the actual output.
Δ Wi = η * (D-Y).Ii
Example: A simple single unit adaptive network
The network has 2 inputs, and one output. All are binary. The output is
1 if W0I0 + W1I1 > 0
0 if W0I0 + W1I1 ≤ 0
We want it to learn simple OR: output a 1 if either I0 or I1 is 1.
Artificial Neural Networks
Adaptive interaction between individual neurons
Power: collective behavior of interconnected neurons
Evolving networks
Continuous process of:
Evaluate output
Adapt weights
Take new inputs
ANN evolving causes stable state of the weights, but neurons continue working: network has ‘learned’ dealing with the problem
Learning
From experience: examples / training data
Strength of connection between the neurons is stored as a weight-value for the specific connection
Learning the solution to a problem = changing the connection weights
Learning performance
Learning Paradigms:
Unsupervised
Competitive Learning
Reinforcement learning
Backpropagation
Unsupervised learning
No help from the outside
No training data, no information available on the desired output
Learning by doing
Example : -
Competitive learning: example
In this type, it is generally Winner takes all concept.
only update weights of winning neuron
Back propagation
Desired output of the training examples
Error = difference between actual & desired output
Change weight relative to error size
Calculate output layer error , then propagate back to previous layer
Improved performance, very common!
Where are NN used?
Recognizing and matching complicated, vague, or incomplete patterns
Data is unreliable
Problems with noisy data
Prediction
Classification
Data association
Data conceptualization
Filtering
Planning
Applications
Prediction: learning from past experience
pick the best stocks in the market
predict weather
identify people with cancer risk
Classification
Image processing
Predict bankruptcy for credit card companies
Risk assessment
Applications
Recognition
Pattern recognition: SNOOPE (bomb detector in U.S. airports)
Character recognition
Handwriting: processing checks
Data association
Not only identify the characters that were scanned but identify when the scanner is not working properly
Applications
Data Conceptualization
infer grouping relationships e.g. extract from a database the names of those most likely to buy a particular product.
Data Filtering
e.g. take the noise out of a telephone signal, signal smoothing
Planning
Unknown environments
Sensor data is noisy
Fairly new approach to planning
Strengths of a Neural Network
Power: Model complex functions, nonlinearity built into the network
Ease of use:
Learn by example
Very little user domain-specific expertise needed
Intuitively appealing: based on model of biology, will it lead to genuinely intelligent computers/robots?
Neural networks cannot do anything that cannot be done using traditional computing techniques, BUT they can do some things which would otherwise be very difficult.
General Advantages
Advantages
Adapt to unknown situations
Robustness: fault tolerance due to network redundancy
Autonomous learning and generalization
Disadvantages
Not exact
Large complexity of the network structure
Future of Neural Networks
Most of the reported applications are still in research stage
No formal proofs, but they seem to have useful applications that work
