Simulation of Single and Multilayer of Artificial Neural Network using Verilog
1 like538 views
This document discusses the simulation of single layer and multilayer artificial neural networks using Verilog. It begins with an introduction to artificial neural networks and their application in VLSI circuit fault diagnosis. It then provides details on the algorithm and design methodology for simulating a single layer neural network to model an AND gate, showing the calculation of error over iterations in Matlab and time taken using Verilog code. For a multilayer network modeling an XOR gate, it similarly discusses the backpropagation algorithm, showing error reduction over iterations in Matlab and time taken using Verilog. It concludes that neural networks can help minimize time to find faults in digital circuits.
![IJSRD - International Journal for Scientific Research & Development| Vol. 1, Issue 5, 2013 | ISSN (online): 2321-0613
All rights reserved by www.ijsrd.com 1082
Abstract — Artificial neural network play an important role
in VLSI circuit to find and diagnosis multiple fault in digital
circuit. In this paper, the example of single layer and multi-
layer neural network had been discussed secondly
implement those structure by using verilog code and same
idea must be implement in mat lab for getting number of
iteration and verilog code gives us time taken to adjust the
weight when error become almost equal to zero. The
purposed aim at reducing resource requirement, without
much compromises on the speed that neural network can be
realized on single chip at lower cost.
Keywords:--Neural Network, Verilog, Matlab, architecture
I. INTRODUCTION
As we know rapid increase in demand of digital circuit and
industry need there to launch their product as earlier as
possible without sacrificing integrated circuit (IC) quality,
so testing of digital circuit in Very Large Scale Integration
(VLSI) has become challenge for that it is also very
important to develop more powerful algorithms for
diagnosis more multiple fault in digital circuit. Diagnosis
defines as task of identifying the cause and location of a
manifested by some observation behavior. This is often
considering being a two stage process: first the fact that fault
has occurred must be recognized-this is referred to as fault
detection that is general achieved by testing. Secondly the
nature and location should be determined such that
appropriate remedial action may be initiated. To come
across this problem a test engineer has devolved a parallel
algorithm knows as ARTIFIAL NEURAL NETWORK. [4]
The application of ANN are expanding because neural
network are good at solving problem not just in engineering
but in medicine, control system, signal processing ,science
etc., because of faster algorithm and faster computer have
made it possible to use neural network to solve complex
industrial problem that formerly required too much
computation. Artificial neural networks (ANN) are parallel
algorithms. Their inherent parallelism makes them
particularly suited to parallel VLSI implementations.
The idea of neural network based on characteristic of brain
function, the brain consist of many highly connected
element knows as neuron and this is connected to axon.
Even though biological neuron is very slow when compared
to electrical circuit, the brain must perform many tasks
faster than any computer. ANN work on two network single
input neuron network and multi-layer input neuron network.
II. ALGORITHM
A. SINGLE LAYER NEURON NETWORK
Fig. 1: Single layer Neural network
Above fig-1 shown is single layer neuron network in which
there is one input layer where multiple input is being applied
and all applied input and one bias voltage are summed at
one neuron and at last output is passed through nonlinear
activation function that can be sigmoid function, there are
many other non-linear activation function hard for those
refer [4] For single input equation can be return as
A=F (w1*P1+Bias voltage) (1)
For n number of input the equation-1 can be return [1]
A ∑ (2)
Weight in single layer neural network can be adjusted by
different algorithm such as supervised learning,
unsupervised and reinforcement learning here our example
falls under category of supervised learning because to
adjust weight automatically we have to find mean square
error and tries to minimize the average squared error
between network output and targeted value. To minimize
those errors we use gradient descent algorithm.[4]
Error= (target out – network output)2
(3)
Weight=weight + alpha*Error*neural network input (4)
Continue adjusting weight from are equation-4 up to Error
reach zero here alpha is learning rate which always between
0 and 1. [4]
Simulation of Single and Multilayer of Artificial Neural Network
using Verilog
Shreyas J. Patel1
Rajesh Vasdadiya2
Shaktivel S.M3
1
PG Student 2
PG Student 3
Associate Professor
1,2,3
Department of Electronics and communication Engineering.
1,2,3
VIT UNIVERSITY, Chennai-600 048 (Chennai-Tamil Nadu-INDIA)](https://image.slidesharecdn.com/ijsrdv1i5011-140805034905-phpapp02/85/Simulation-of-Single-and-Multilayer-of-Artificial-Neural-Network-using-Verilog-1-320.jpg)
![Simulation of Single and Multilayer of Artificial Neural Network using VERILOG
(IJSRD/Vol. 1/Issue 5/2013/0011)
All rights reserved by www.ijsrd.com 1083
B. MULTILAYER NEURAL NETWORK
Fig. 2: Multilayer network
From fig-2 Multilayer consists three layer input (I), hidden
(j) and output layer (k). To adjust the weight between two
different layer automatically in an multi-layer neural
network there are so many algorithm but one of
advantageous for this network is batch gradient negative
that is nothing but back propagation method in which weight
are update in direction of negative gradient of the
performance. The learning factor is multiplied the negative
gradient to determine the changes to weight and bias. The
larger the learning rates the bigger step. If the learning rate
is made too bigger the algorithm becomes unstable. Back
propagation is fastest algorithm than the other technique.
For multilayer neural network we are using sigmoid
function [4].where x is your input of any neuron.
(5)
Error= (target out – network output) 2
(6)
From equation-6
Wjk = Wjk+ alpha*Δjk*input (input of hidden neuron)
Wij = Wij+ alpha*Δij*input (input of input neuron)
Where
Δjk=error*dE/dWjk and wjk is weight between hidden
layer to output layer
Δij=∑wjk* Δjk and wij is weight between input and hidden
layer
III. DESIGN METHODOLOGY/DESIGN DETAILS
Simulation for single layer and multilayer example for And
Gate and X-or Gate is given.
A. And gate using single neural network
The single layer neural network first of all weight is being
assigned manually and by neural network rule weight is
being multiplied by input and summing all input with weight
and last neural network output compared to targeted output.
if difference is there that is error by using gradient decent
algorithm decrease error and at same time weight is being
adjusted and when error reach to desired output we can say
that our neural network is being trained. In an and gate we
had applied two input (1, 0) and w0, w1 are synaptic weigh
and b is bias voltage (b is not necessary that depends upon
user)[4] these three value are summed together .
How many iteration training algorithm take to reduce error
zero that can be seen by implementing the idea or program
in mat lab and how many second the algorithm is taking to
reduce error approximately up to zero can be observed by
implementing your Verilog code in model sim or Xilinx
simulator.
Fig. 3: And gate using single neural network
For example consider you are applying input and weight
randomly.
A=1 and B=0;
W0=0.5; W1=0.7;
N=(a*w0+b*w1)
=1 * 0.5 + 0 * 0.7
= 0.5
Error = target output –(desired output)
=-0.5
Here by summing input and weight (eq-3) at last output
compare with targeted output, and continuously updating
weight as per (eq-4). By using Gradient decent algorithm
error will decrease and how much iteration been taken to
adjust the weight can be easily seen by mat lab.(fig-4)
Fig. 4: Calculation of error using gradient decent algorithm
Number of iteration ==7000
Now same algorithm been implemented in model simulator
to see how much second take to adjust the weight.](https://image.slidesharecdn.com/ijsrdv1i5011-140805034905-phpapp02/85/Simulation-of-Single-and-Multilayer-of-Artificial-Neural-Network-using-Verilog-2-320.jpg)
![Simulation of Single and Multilayer of Artificial Neural Network using VERILOG
(IJSRD/Vol. 1/Issue 5/2013/0011)
All rights reserved by www.ijsrd.com 1084
Fig. 5: Calculation of error using Verilog code
Time taken to adjust weight:-754ns
Fig. 6: Dataflow structure
This is weight w1 and w2 value when error is zero can be
seen by model simulator.
IV. MULTILAYER NEURAL NETWORK
Fig. 6: X-OR gate using multilayer neural network
In an 2-2-1 multilayer there are 2 input neuron,2 hidden
neuron and single output,the weight between input and
hidden layer is w1,w2,w3,w4 and weight between hidden to
output is w5 and w6,here we had used back propagation
algorithm the main advantage of this method to reduce
number of iterrationand time[3].the simulation result shown
below
Fig. 7: Number of itteration to adjust weight using matlab
Number of iteration:-1500
Fig 8: Time taken to adjust weight using verilog
Time taken to adjust weight:-2904ps
Fig. 9: Dataflow structure
V. CONCLUSION
All digital circuit consist of universal and logical gate and
last step in industry to test those circuit if there is fault again
they have to go for some adjustment it will take more time
to reach product to market, so if we replace those circuit by
neural network maximum time to for finding fault can be
minimized.](https://image.slidesharecdn.com/ijsrdv1i5011-140805034905-phpapp02/85/Simulation-of-Single-and-Multilayer-of-Artificial-Neural-Network-using-Verilog-3-320.jpg)
![Simulation of Single and Multilayer of Artificial Neural Network using VERILOG
(IJSRD/Vol. 1/Issue 5/2013/0011)
All rights reserved by www.ijsrd.com 1085
REFERENCES
[1]. Karthikeyan. A , Rajeswaran. N, “DESIGN AND
IMPLEMENTATION OF MULTIPLE FAULT
DIAGNOSIS ON VLSI CIRCUITS USING
ARTIFICIAL NEURAL” IJAET, May 2012
[2]. AL-Jumah. A.A. Arslan. T,” ARTIFICIAL NEURAL
NETWORK BASED MULTIPLE FAULT
DIAGNOSIS IN DIGITAL CIRCUIT” IEEE, may
1998.
[3]. Rafid Ahamed Khali ,“HARDWARE
IMPLEMTATION OF BACK PRAPAGATION
NEURAL NETWORK ON FPGA”, university of
Mosul, sep 2007
[4]. Hagan Demuth dearle, “NEURAL NETWORK
DESIGN”, book
[5]. Aydoğan Savran, Serkan Ünsal “HARDWARE
IMPLEMTATION OF FEEDFORWARD NEURAL
NETWORK ON FPGA” Ege University, Department of
Electrical and Electronics Engineering
[6]. Alan .N.Willson, “ONE-NEURON CIRCUITARY
FOR CARRY GENERATION IN 4-BIT ADDER ”,
IEEE,1992
[7]. Samir palnitkar “VERILOG HDL”,IEEE 1364-2001](https://image.slidesharecdn.com/ijsrdv1i5011-140805034905-phpapp02/85/Simulation-of-Single-and-Multilayer-of-Artificial-Neural-Network-using-Verilog-4-320.jpg)
Simulation of Single and Multilayer of Artificial Neural Network using Verilog
- 1. IJSRD - International Journal for Scientific Research & Development| Vol. 1, Issue 5, 2013 | ISSN (online): 2321-0613 All rights reserved by www.ijsrd.com 1082 Abstract — Artificial neural network play an important role in VLSI circuit to find and diagnosis multiple fault in digital circuit. In this paper, the example of single layer and multi- layer neural network had been discussed secondly implement those structure by using verilog code and same idea must be implement in mat lab for getting number of iteration and verilog code gives us time taken to adjust the weight when error become almost equal to zero. The purposed aim at reducing resource requirement, without much compromises on the speed that neural network can be realized on single chip at lower cost. Keywords:--Neural Network, Verilog, Matlab, architecture I. INTRODUCTION As we know rapid increase in demand of digital circuit and industry need there to launch their product as earlier as possible without sacrificing integrated circuit (IC) quality, so testing of digital circuit in Very Large Scale Integration (VLSI) has become challenge for that it is also very important to develop more powerful algorithms for diagnosis more multiple fault in digital circuit. Diagnosis defines as task of identifying the cause and location of a manifested by some observation behavior. This is often considering being a two stage process: first the fact that fault has occurred must be recognized-this is referred to as fault detection that is general achieved by testing. Secondly the nature and location should be determined such that appropriate remedial action may be initiated. To come across this problem a test engineer has devolved a parallel algorithm knows as ARTIFIAL NEURAL NETWORK. [4] The application of ANN are expanding because neural network are good at solving problem not just in engineering but in medicine, control system, signal processing ,science etc., because of faster algorithm and faster computer have made it possible to use neural network to solve complex industrial problem that formerly required too much computation. Artificial neural networks (ANN) are parallel algorithms. Their inherent parallelism makes them particularly suited to parallel VLSI implementations. The idea of neural network based on characteristic of brain function, the brain consist of many highly connected element knows as neuron and this is connected to axon. Even though biological neuron is very slow when compared to electrical circuit, the brain must perform many tasks faster than any computer. ANN work on two network single input neuron network and multi-layer input neuron network. II. ALGORITHM A. SINGLE LAYER NEURON NETWORK Fig. 1: Single layer Neural network Above fig-1 shown is single layer neuron network in which there is one input layer where multiple input is being applied and all applied input and one bias voltage are summed at one neuron and at last output is passed through nonlinear activation function that can be sigmoid function, there are many other non-linear activation function hard for those refer [4] For single input equation can be return as A=F (w1*P1+Bias voltage) (1) For n number of input the equation-1 can be return [1] A ∑ (2) Weight in single layer neural network can be adjusted by different algorithm such as supervised learning, unsupervised and reinforcement learning here our example falls under category of supervised learning because to adjust weight automatically we have to find mean square error and tries to minimize the average squared error between network output and targeted value. To minimize those errors we use gradient descent algorithm.[4] Error= (target out – network output)2 (3) Weight=weight + alpha*Error*neural network input (4) Continue adjusting weight from are equation-4 up to Error reach zero here alpha is learning rate which always between 0 and 1. [4] Simulation of Single and Multilayer of Artificial Neural Network using Verilog Shreyas J. Patel1 Rajesh Vasdadiya2 Shaktivel S.M3 1 PG Student 2 PG Student 3 Associate Professor 1,2,3 Department of Electronics and communication Engineering. 1,2,3 VIT UNIVERSITY, Chennai-600 048 (Chennai-Tamil Nadu-INDIA)
- 2. Simulation of Single and Multilayer of Artificial Neural Network using VERILOG (IJSRD/Vol. 1/Issue 5/2013/0011) All rights reserved by www.ijsrd.com 1083 B. MULTILAYER NEURAL NETWORK Fig. 2: Multilayer network From fig-2 Multilayer consists three layer input (I), hidden (j) and output layer (k). To adjust the weight between two different layer automatically in an multi-layer neural network there are so many algorithm but one of advantageous for this network is batch gradient negative that is nothing but back propagation method in which weight are update in direction of negative gradient of the performance. The learning factor is multiplied the negative gradient to determine the changes to weight and bias. The larger the learning rates the bigger step. If the learning rate is made too bigger the algorithm becomes unstable. Back propagation is fastest algorithm than the other technique. For multilayer neural network we are using sigmoid function [4].where x is your input of any neuron. (5) Error= (target out – network output) 2 (6) From equation-6 Wjk = Wjk+ alpha*Δjk*input (input of hidden neuron) Wij = Wij+ alpha*Δij*input (input of input neuron) Where Δjk=error*dE/dWjk and wjk is weight between hidden layer to output layer Δij=∑wjk* Δjk and wij is weight between input and hidden layer III. DESIGN METHODOLOGY/DESIGN DETAILS Simulation for single layer and multilayer example for And Gate and X-or Gate is given. A. And gate using single neural network The single layer neural network first of all weight is being assigned manually and by neural network rule weight is being multiplied by input and summing all input with weight and last neural network output compared to targeted output. if difference is there that is error by using gradient decent algorithm decrease error and at same time weight is being adjusted and when error reach to desired output we can say that our neural network is being trained. In an and gate we had applied two input (1, 0) and w0, w1 are synaptic weigh and b is bias voltage (b is not necessary that depends upon user)[4] these three value are summed together . How many iteration training algorithm take to reduce error zero that can be seen by implementing the idea or program in mat lab and how many second the algorithm is taking to reduce error approximately up to zero can be observed by implementing your Verilog code in model sim or Xilinx simulator. Fig. 3: And gate using single neural network For example consider you are applying input and weight randomly. A=1 and B=0; W0=0.5; W1=0.7; N=(a*w0+b*w1) =1 * 0.5 + 0 * 0.7 = 0.5 Error = target output –(desired output) =-0.5 Here by summing input and weight (eq-3) at last output compare with targeted output, and continuously updating weight as per (eq-4). By using Gradient decent algorithm error will decrease and how much iteration been taken to adjust the weight can be easily seen by mat lab.(fig-4) Fig. 4: Calculation of error using gradient decent algorithm Number of iteration ==7000 Now same algorithm been implemented in model simulator to see how much second take to adjust the weight.
- 3. Simulation of Single and Multilayer of Artificial Neural Network using VERILOG (IJSRD/Vol. 1/Issue 5/2013/0011) All rights reserved by www.ijsrd.com 1084 Fig. 5: Calculation of error using Verilog code Time taken to adjust weight:-754ns Fig. 6: Dataflow structure This is weight w1 and w2 value when error is zero can be seen by model simulator. IV. MULTILAYER NEURAL NETWORK Fig. 6: X-OR gate using multilayer neural network In an 2-2-1 multilayer there are 2 input neuron,2 hidden neuron and single output,the weight between input and hidden layer is w1,w2,w3,w4 and weight between hidden to output is w5 and w6,here we had used back propagation algorithm the main advantage of this method to reduce number of iterrationand time[3].the simulation result shown below Fig. 7: Number of itteration to adjust weight using matlab Number of iteration:-1500 Fig 8: Time taken to adjust weight using verilog Time taken to adjust weight:-2904ps Fig. 9: Dataflow structure V. CONCLUSION All digital circuit consist of universal and logical gate and last step in industry to test those circuit if there is fault again they have to go for some adjustment it will take more time to reach product to market, so if we replace those circuit by neural network maximum time to for finding fault can be minimized.
- 4. Simulation of Single and Multilayer of Artificial Neural Network using VERILOG (IJSRD/Vol. 1/Issue 5/2013/0011) All rights reserved by www.ijsrd.com 1085 REFERENCES [1]. Karthikeyan. A , Rajeswaran. N, “DESIGN AND IMPLEMENTATION OF MULTIPLE FAULT DIAGNOSIS ON VLSI CIRCUITS USING ARTIFICIAL NEURAL” IJAET, May 2012 [2]. AL-Jumah. A.A. Arslan. T,” ARTIFICIAL NEURAL NETWORK BASED MULTIPLE FAULT DIAGNOSIS IN DIGITAL CIRCUIT” IEEE, may 1998. [3]. Rafid Ahamed Khali ,“HARDWARE IMPLEMTATION OF BACK PRAPAGATION NEURAL NETWORK ON FPGA”, university of Mosul, sep 2007 [4]. Hagan Demuth dearle, “NEURAL NETWORK DESIGN”, book [5]. Aydoğan Savran, Serkan Ünsal “HARDWARE IMPLEMTATION OF FEEDFORWARD NEURAL NETWORK ON FPGA” Ege University, Department of Electrical and Electronics Engineering [6]. Alan .N.Willson, “ONE-NEURON CIRCUITARY FOR CARRY GENERATION IN 4-BIT ADDER ”, IEEE,1992 [7]. Samir palnitkar “VERILOG HDL”,IEEE 1364-2001