![International Journal of Electrical and Computer Engineering (IJECE)
Vol. 12, No. 4, August 2022, pp. 4373~4379
ISSN: 2088-8708, DOI: 10.11591/ijece.v12i4.pp4373-4379 4373
Journal homepage: http://ijece.iaescore.com
Application of optimization algorithms for classification
problem
Alaa Eleyan, Mohammad Shukri Salman, Bahaa Al-Sheikh
College of Engineering and Technology, American University of the Middle East, Egaila, Kuwait
Article Info ABSTRACT
Article history:
Received Dec 23, 2020
Revised Jan 29, 2022
Accepted Feb 25, 2022
The work presented in this paper investigates the use of metaheuristic
optimization algorithms for the face recognition problem. In the first setup, a
face recognition system is implemented using particle swarm optimization
(PSO) and firefly optimization algorithms, separately. PSO and firefly are
used for forming the feature vectors in the feature selection stage. These
feature vectors serve as the new representation for the face images that will be
fed to the classifier. In the second setup, selected features from both PSO and
firefly algorithms are fused to form one single feature vector for each face
image before the classification stage. Extensive simulations are conducted
using Poznan University of Technology (PUT) and face recognition
technology (FERET) face databases. Optimal values for population size and
maximum iterations number were selected before conducting the experiments.
The effect of using different numbers of selected features on the performance
is investigated for feature selection using PSO, firefly, and feature fusion of
both.
Keywords:
Face recognition
Firefly optimization
Particle swarm optimization
Swarm intelligence
This is an open access article under the CC BY-SA license.
Corresponding Author:
Alaa Eleyan
College of Engineering and Technology, American University of the Middle East
Egaila, Kuwait
Email: alaa.eleyan@aum.edu.kw
1. INTRODUCTION
We live in a world where analyzing an enormous amount of diversified data is becoming a crucial and
fundamental necessity. Therefore, in the analysis, we can bring out proper conclusions from the data we possess
and classify it into positive, negative, or neutral sets. It is evident that people around us affect our daily life.
People tend to change their minds and decisions based on other people’s opinions and decisions. To some
extent, we use other people’s ideas to come up with our own decisions. Based on this idea swarm intelligence
concept was introduced [1]–[3]. It is a nature-inspired artificial intelligence based on the communication
models of social insects such as ants [4], [5], fireflies [6], [7], and bees [8], [9].
A swarm is a large number of agents interacting locally with themselves [10]–[13]. In a swarm, there
is no supervisor or central control to give orders on how to behave. Swarm-based algorithms are popular in this
era with the thirst for nature-inspired, population-based algorithms that can generate low-cost, fast, and correct
solutions to complex and hard-to-solve problems. For that reason, swarm intelligence is becoming a golden
ticket in the era of artificial intelligence metaheuristic optimization [14]–[16]. Swarm intelligence optimization
techniques have been applied successfully in numerous applications where they helped to improve the system
performance such as proportional integral (PI) controller tuning in wind turbines [17], the maximum power
point tracking (MPPT) enhancement in photovoltaic systems [18], and cervical cancer classification [19]. One
of the applications that the optimization algorithms can further improve its performance is face recognition.
Face recognition is a very hot research topic especially with the increasing demand for security in different](https://image.slidesharecdn.com/102157067734627113esr14apr2223dec20n-220629034802-82375165/85/Application-of-optimization-algorithms-for-classification-problem-1-320.jpg)
![ ISSN: 2088-8708
Int J Elec & Comp Eng, Vol. 12, No. 4, August 2022: 4373-4379
4374
areas such as businesses, public transportations, borders, and airports. Since the start of using computers to
recognize human faces around sixty years ago [20], many algorithms and techniques have been introduced to
improve face recognition performance in challenging scenarios such as different poses, lighting, orientations,
and facial expressions [21]–[24]. Hermosilla et al. [25] proposed to fuse thermal and visible descriptors where
the system obtained the optimal weights using the particle swarm optimization (PSO) algorithm to maximize
the face recognition rates obtained from different combinations of local descriptor methods. PSO was applied
to coefficients extracted by the discrete cosine transforms (DCT) and the discrete wavelet transform (DWT)
algorithms where it searched the feature space for the optimal feature subset based on a discrimination
criterion [26]. In this paper, we propose to use two well-known optimization algorithms namely; PSO and
firefly optimization algorithms for improving face recognition accuracy. In this paper, these two algorithms
will be applied at the feature selection stage of the face recognition module in two different setups. In the first
setup, the optimization algorithms are applied to face images for features selection and their performances are
evaluated separately, whereas in the second setup the features selected from both optimization algorithms are
fused before evaluation. The performance of the proposed approach will be tested using two popular face
databases.
The rest of the paper is organized as follows: section 2 presents the optimization algorithms used for
feature selection. Section 3 describes the methodology and setup. Simulation results are shown and discussed
in section 4. Finally, the paper is concluded in the last section.
2. RESEARCH METHOD
2.1. Optimization algorithms
In this work, two swarm intelligence metaheuristic optimization algorithms are employed for feature
selection. The selected salient features are used for class description and discrimination. The working
mechanism and detailed explanations of both particle swarm optimization [3], [27]–[30] and firefly
optimization [6], [7] algorithms are presented briefly below. For more details, reader is encouraged to refer to
respective references.
2.1.1. Particle swarm optimization (PSO)
This algorithm has been introduced by Kennedy and Eberhart [3]. It can be summarized as: given that
the swarm has a certain number of particles, every single particle in a population has a current location, a
current speed, and a personal best location in a search area. The personal best location is related to the location
in the search area where an objective function provides the minimum calculated error for the corresponding
particle. The location corresponds to minimum error among all the personal best locations is declared to be the
global best location. Both, personal and global best locations and speeds are updated for every particle in the
population at each iteration using the equation in [3].
An inertia weight, adapted by PSO, is linearly decreased during training to control the convergence
rate of the algorithm. The next position of the particle is determined by accumulating the new speed to the
particle’s current location. Limiting the speed vector value to a predefined range will guarantee the particle
does not leave the search area. A detailed description of the algorithm can be found at [3]–[5], [31].
2.1.2. Firefly optimization
The firefly optimization algorithm is a global metaheuristic optimization algorithm, proposed by
Yang [6], which imitates the behavior of firefly insects. Fireflies use the flashing behavior to attract other
fireflies, usually for sending signals to the opposite sex. However, in the mathematical model, used inside the
firefly algorithm, simply the fireflies are unisex, and any firefly can attract other fireflies.
Firefly brightness amount is the key factor for its attraction by other fireflies. For a couple of fireflies;
the brighter one will attract the other; so, the less bright one will move in the direction of the brighter one. This
is done on every iteration of the algorithm for any binary combination of fireflies in the population. For more
details about the algorithm, readers should refer to [6], [7], [32], [33].
2.2. Methodology and setup
At the first setup, we implemented our system using PSO and firefly optimization algorithms
separately. PSO and firefly helped in the feature selection stage of the system. These selected features were
used as a new representation for the images. They are fed to the classifier to evaluate the performance of the
system as shown in Figure 1. At the second setup, after separately applying PSO and firefly to the image,
selected features from both algorithms were fused to form one feature vector, as shown in Figure 2. The rest
of the system is similar to the first setup.](https://image.slidesharecdn.com/102157067734627113esr14apr2223dec20n-220629034802-82375165/85/Application-of-optimization-algorithms-for-classification-problem-2-320.jpg)
![Int J Elec & Comp Eng ISSN: 2088-8708
Application of optimization algorithms for classification problem (Alaa Eleyan)
4375
Two face databases were used to evaluate the proposed approaches. The first database is Poznan
University of Technology (PUT) database [34]. It contains images of 100 people. Each person has 100 images
covering varied facial expressions and orientations of size 128×128 pixels. 10 images per person with a total
of 1,000 images are used in our experiments. The second database is face recognition technology (FERET)
database [35]. A subset of FERET database with 194 people is used in our experiments. This subset contains
10 images per person of size 96×64 pixels. Examples of faces from both databases are shown in Figure 3.
Figure 1. Block diagram of the proposed PSO/firefly-based texture classification
Figure 2. Block diagram of the proposed feature fusion face recognition system
Figure 3. Face examples from PUT database (on the left) and FERET database (on the right)
The optimal swarm size (population) is problem-dependent which describes the social interaction
within the swarm. While smaller populations are slower in convergence, they are less likely to fail into local
minima and have more reliable convergence to optimal solutions [3]. Starting the search with small populations
and increase population size proportionally to increase in iterations number ensures an initial high diversity
with faster convergence as particles move towards a probable search area [36].](https://image.slidesharecdn.com/102157067734627113esr14apr2223dec20n-220629034802-82375165/85/Application-of-optimization-algorithms-for-classification-problem-3-320.jpg)
![ ISSN: 2088-8708
Int J Elec & Comp Eng, Vol. 12, No. 4, August 2022: 4373-4379
4376
The number of iterations to reach a good solution is also problem-dependent. Too few iterations may
terminate the search prematurely. On the other hand, a too large number of iterations adds avoidable
computational complexity (valid if the iterations number is the only stopping criteria) [3]. In this study,
different population sizes and number of iterations were evaluated before deciding their optimal values for the
rest of the simulations.
3. RESULTS AND DISCUSSION
As mentioned earlier, two face databases are used for performance evaluation. We investigated
different population sizes and number of iterations to find the optimal values for the rest of the simulations.
Optimal values for all simulations are chosen as 20 for population and 100 for the maximum number of
iterations. k-nearest neighbors (k-NN) is used with three different distance metrics namely; Manhattan,
Euclidean, and Cosine. 10-fold cross-validation was applied in all the experiments. Results recorded in all
figures and tables are averaged ones. Tables 1 and 2 show performances by applying different numbers of
selected features for PSO and firefly algorithms, using PUT and FERET databases, respectively.
Firefly algorithm is superior to PSO algorithm in all scenarios in both tables with different numbers
of training images and selected features. According to the results in Tables 1 and 2, Manhattan distance gave
the best performance. Based on this, results in Figures 4 and 5 are reported only using this distance.
Table 1. Recognition rates with different number of selected features of PSO algorithm using PUT/FERET
face databases
Features
Distanc
e
# Training Images
1 2 3 4 5 6 7 8 9
10
l1 45.56/20.25 55.84/26.84 64.96/33.57 71.95/41.28 75.00/38.33 77.30/39.51 80.73/46.55 79.40/41.52 79.80/45.57
l2 40.78/16.13 52.17/21.91 60.76/28.83 67.52/36.05 71.12/32.28 74.22/33.94 77.67/39.98 76.20/36.78 76.80/39.64
Cos 33.31/12.61 43.69/17.32 52.83/21.88 59.08/29.29 64.00/25.66 66.45/26.53 68.40/32.49 68.45/28.84 68.50/32.63
20
l1 59.46/36.55 74.88/45.23 82.17/55.41 87.62/64.24 87.90/65.35 90.45/72.28 92.67/71.98 94.55/72.01 94.80/74.69
l2 51.74/27.29 66.72/34.28 75.53/44.79 80.72/51.22 81.92/53.48 86.13/58.38 88.90/58.80 91.20/58.80 91.30/61.08
Cos 45.43/23.52 60.61/30.01 70.19/39.96 75.88/46.39 76.94/48.31 81.22/53.79 84.33/53.80 87.30/52.63 88.00/57.47
50
l1 70.10/51.32 83.16/62.74 90.30/75.71 93.10/79.75 94.78/86.88 95.67/88.44 97.37/87.94 97.65/89.87 97.90/91.24
l2 62.72/40.35 76.81/50.39 84.54/64.27 88.47/69.45 90.78/76.40 92.15/78.98 95.00/78.33 95.90/80.93 96.60/82.58
Cos 55.48/34.64 71.47/44.76 79.91/58.84 85.03/64.47 87.36/72.00 89.67/74.30 93.13/73.77 93.75/77.16 94.70/78.35
100
l1 74.86/64.42 86.34/74.46 91.67/84.22 94.85/88.38 96.36/90.39 97.17/93.65 98.20/94.40 98.30/94.51 98.20/94.69
l2 68.02/50.93 80.60/62.07 87.67/73.73 92.18/80.11 94.20/83.77 94.78/87.50 97.17/88.83 96.85/88.22 97.30/87.94
Cos 61.90/44.97 75.20/56.92 83.91/69.42 89.27/76.14 92.06/80.03 93.30/84.11 95.23/85.00 95.40/84.90 96.90/85.26
250
l1 79.13/71.00 90.36/79.79 93.71/88.82 95.97/92.27 97.00/94.09 98.08/95.77 98.90/96.49 98.35/96.78 98.90/97.27
l2 73.40/56.95 86.80/67.35 91.16/79.06 93.87/85.38 95.38/88.09 96.95/91.60 98.00/92.13 97.90/92.37 98.30/93.61
Cos 67.59/51.40 83.14/62.30 88.49/75.41 91.52/82.44 93.62/85.67 95.40/89.30 97.13/89.52 97.20/90.26 97.60/91.91
450
l1 82.51/72.40 92.19/81.60 95.51/89.65 97.20/93.63 97.70/95.25 98.38/96.30 99.23/96.92 98.85/97.35 99.10/97.99
l2 76.88/58.12 88.59/69.32 92.79/80.50 95.52/86.86 96.60/90.06 97.10/92.82 98.57/93.51 98.25/93.45 98.40/94.38
Cos 71.64/52.71 86.10/64.88 90.76/77.13 94.10/84.20 95.36/87.68 96.03/90.82 97.77/91.32 97.85/91.49 98.40/92.94
Table 2. Recognition rates with different number of selected features of firefly algorithm using PUT/FERET
face databases
Features
Distanc
e
# Training Images
1 2 3 4 5 6 7 8 9
10
l1 49.30/23.24 63.44/29.36 73.07/37.94 76.50/43.69 80.36/45.03 80.20/45.57 86.87/47.04 84.95/51.55 87.90/51.75
l2 43.10/18.54 57.64/23.80 67.97/30.60 71.35/36.03 75.08/40.18 76.17/39.48 82.23/40.79 80.75/44.69 84.80/43.45
Cos 37.09/15.83 51.39/20.14 60.41/26.50 64.22/30.51 69.12/38.06 68.75/36.29 75.23/33.56 73.60/38.87 77.90/37.32
20
l1 62.71/34.94 77.05/43.05 85.17/52.17 88.88/60.84 89.84/66.45 92.20/70.67 94.77/68.78 95.10/69.97 96.00/69.12
l2 54.17/26.08 68.60/33.80 77.96/42.06 82.08/49.33 84.18/53.99 87.90/58.30 90.50/56.79 91.60/58.81 91.70/57.99
Cos 48.54/22.12 62.41/29.50 71.96/36.69 77.67/43.99 79.52/49.09 83.83/51.87 86.90/51.94 88.30/53.12 88.10/51.75
50
l1 73.98/52.69 85.53/63.36 91.56/75.53 94.08/82.77 95.46/85.42 97.10/89.79 97.97/87.37 97.90/89.30 98.10/90.72
l2 64.48/40.18 78.71/50.05 85.97/62.50 90.00/71.45 91.92/75.40 93.63/80.05 94.97/76.70 95.25/79.05 97.20/81.08
Cos 58.66/35.69 73.55/44.99 81.87/57.26 86.25/66.44 88.84/70.95 91.20/76.16 93.23/72.32 94.05/74.95 95.10/77.84
100
l1 78.34/63.12 89.97/75.21 93.74/83.59 95.40/88.89 96.98/92.05 97.88/93.07 98.83/94.24 98.50/93.92 98.90/94.64
l2 70.94/48.91 85.16/62.47 90.26/72.64 92.53/80.25 94.82/84.54 95.72/87.04 97.33/88.33 97.40/87.27 98.20/87.78
Cos 64.81/43.39 81.16/57.02 87.19/68.45 90.08/76.49 93.12/80.81 94.15/83.72 96.50/85.15 96.20/84.05 97.50/84.95
250
l1 81.94/69.24 92.36/79.00 94.81/87.95 97.13/92.66 97.48/94.02 98.08/96.16 99.03/96.43 99.10/96.42 99.10/96.96
l2 76.34/55.14 88.42/66.57 92.10/78.34 95.25/85.37 95.78/88.10 96.75/91.47 98.10/92.49 97.90/92.76 98.10/93.20
Cos 70.62/49.81 85.33/61.44 90.01/74.65 93.67/82.18 94.70/85.54 95.65/88.85 97.27/89.48 97.25/90.64 97.50/91.19
450
l1 82.34/71.68 92.21/81.37 95.37/89.79 97.32/93.49 97.96/94.98 98.35/96.47 99.20/97.16 99.20/97.37 99.10/97.89
l2 77.28/58.05 88.81/69.76 92.67/80.66 95.37/86.77 96.46/89.55 97.40/92.86 98.53/93.26 98.45/93.81 98.70/94.85
Cos 72.22/52.35 86.41/64.95 90.87/77.22 94.08/84.04 95.02/87.38 96.45/90.57 97.77/90.91 97.85/92.27 98.00/92.84](https://image.slidesharecdn.com/102157067734627113esr14apr2223dec20n-220629034802-82375165/85/Application-of-optimization-algorithms-for-classification-problem-4-320.jpg)
![ ISSN: 2088-8708
Int J Elec & Comp Eng, Vol. 12, No. 4, August 2022: 4373-4379
4378
the number of fused features from both PSO and firefly beyond 100 features each, will not have a noticeable
effect on the performance. It will only increase computational complexity.
4. CONCLUSION
We investigated the use of two of the metaheuristic population-based optimization algorithms namely;
PSO and firefly, for the face recognition problem. PSO and firefly optimization algorithms are utilized in the
feature selection stage for generating the feature vectors. The application of such algorithms showed
enhancement in the recognition performance of the system. Firefly algorithm provided better performance than
PSO algorithm. In addition, the fusion of selected features from both algorithms forming one single feature
vector further improved the recognition performance. Results indicated that using optimization algorithms for
feature selection is a good choice for improving the performance with much fewer features. Furthermore,
feature fusion of the selected features generated using both optimization algorithms helped to boost the
performance with fewer selected features.
REFERENCES
[1] A. P. Engelbrecht, Computational intelligence, an introduction. England: 2nd Ed. John Wiley & Sons Ltd, 2007.
[2] R. C. Eberhart and Y. Shi, Computational intelligence: Concepts to implementations. Morgan Kaufmann; 1st edition, 2007.
[3] R. C. Eberhart, Y. Shi, and J. Kennedy, Swarm intelligence (the morgan kaufmann series in evolutionary computation). Morgan
Kaufmann; 1st edition, 2001.
[4] M. Dorigo, V. Maniezzo, and A. Colorni, “Ant system: optimization by a colony of cooperating agents,” IEEE Transactions on
Systems, Man, and Cybernetics, Part B (Cybernetics), vol. 26, no. 1, pp. 29–41, Feb. 1996, doi: 10.1109/3477.484436.
[5] M. Dorigo, G. Di Caro, and L. M. Gambardella, “Ant algorithms for discrete optimization,” Artificial Life, vol. 5, no. 2, pp. 137–
172, Apr. 1999, doi: 10.1162/106454699568728.
[6] X.-S. Yang, Nature-inspired metaheuristic algorithms, 2nd Editio. Luniver Press, 2010.
[7] I. Fister, I. Fister, X.-S. Yang, and J. Brest, “A comprehensive review of firefly algorithms,” Swarm and Evolutionary Computation,
vol. 13, pp. 34–46, Dec. 2013, doi: 10.1016/j.swevo.2013.06.001.
[8] D. Karaboga, “Technical Report TR06: An idea based on honey bee swarm for numerical optimization,” 2005.
[9] D. Karaboga and B. Basturk, “Artificial bee colony (ABC) optimization algorithm for solving constrained optimization problems,”
in Foundations of Fuzzy Logic and Soft Computing, Berlin, Heidelberg: Springer Berlin Heidelberg, 2007, pp. 789–798.
[10] S. Bandyopadhyay, S.-J. Chung, and F. Y. Hadaegh, “Probabilistic and distributed control of a large-scale swarm of autonomous
agents,” IEEE Transactions on Robotics, vol. 33, no. 5, pp. 1103–1123, Oct. 2017, doi: 10.1109/TRO.2017.2705044.
[11] D. Messie and J. C. Oh, “SWARMs of self-organizing polymorphic agents,” in Swarm Intelligent Systems, Springer Berlin
Heidelberg, 2005, pp. 75–90.
[12] B. Doerr and R. Linares, “Control of large swarms via random finite set theory,” in 2018 Annual American Control Conference
(ACC), Jun. 2018, pp. 2904–2909, doi: 10.23919/ACC.2018.8430968.
[13] J. A. Nasiri, A. M. Fard, M. Naghibzadeh, and M. Rouhani, “High dimensional problem optimization using distributed multi-agent
PSO,” in 2009 Third UKSim European Symposium on Computer Modeling and Simulation, 2009, pp. 245–250, doi:
10.1109/EMS.2009.107.
[14] M. Masood, M. M. Fouad, and I. Glesk, “Analysis of artificial intelligence-based metaheuristic algorithm for MPLS network
optimization,” in 2018 20th International Conference on Transparent Optical Networks (ICTON), Jul. 2018, pp. 1–4, doi:
10.1109/ICTON.2018.8473751.
[15] L. Penghui et al., “Metaheuristic optimization algorithms hybridized with artificial intelligence model for soil temperature
prediction: novel model,” IEEE Access, vol. 8, pp. 51884–51904, 2020, doi: 10.1109/ACCESS.2020.2979822.
[16] D. Ekmekci, “A pheromonal artificial bee colony -pABC-algorithm for optimization problems,” in 2019 16th International Multi-
Conference on Systems, Signals & Devices (SSD), Mar. 2019, pp. 452–456, doi: 10.1109/SSD.2019.8893259.
[17] A. H. Sule, A. S. Mokhtar, J. J. Bin Jamian, A. Khidrani, and R. M. Larik, “Optimal tuning of proportional integral controller for
fixed-speed wind turbine using grey wolf optimizer,” International Journal of Electrical and Computer Engineering (IJECE), vol.
10, no. 5, pp. 5251–5261, Oct. 2020, doi: 10.11591/ijece.v10i5.pp5251-5261.
[18] Y. A. P. and A. G. S., “A modified particle swarm optimization algorithm to enhance MPPT in the PV array,” International Journal
of Electrical and Computer Engineering (IJECE), vol. 10, no. 5, pp. 5001–5008, Oct. 2020, doi: 10.11591/ijece.v10i5.pp5001-
5008.
[19] R. Sagban, H. A. Marhoon, and R. Alubady, “Hybrid bat-ant colony optimization algorithm for rule-based feature selection in health
care,” International Journal of Electrical and Computer Engineering (IJECE), vol. 10, no. 6, pp. 6655–6663, Dec. 2020, doi:
10.11591/ijece.v10i6.pp6655-6663.
[20] K. de Leeuw and J. Bergstra, The history of information security: A comprehensive handbook, 1st Ed. Amsterdam: Elsevier, 2007.
[21] A. K. Jain, B. Klare, and U. Park, “Face recognition: Some challenges in forensics,” in Face and Gesture 2011, Mar. 2011, pp.
726–733, doi: 10.1109/FG.2011.5771338.
[22] A. Eleyan, H. Özkaramanli, and H. Demirel, “Complex wavelet transform-based face recognition,” EURASIP Journal on Advances
in Signal Processing, vol. 2008, no. 1, Dec. 2009, doi: 10.1155/2008/185281.
[23] M. Hassaballah and S. Aly, “Face recognition: challenges, achievements and future directions,” IET Computer Vision, vol. 9, no. 4,
pp. 614–626, Aug. 2015, doi: 10.1049/iet-cvi.2014.0084.
[24] J. Shan, C. Hou, H. Tao, W. Zhuge, and D. Yi, “Full representation data embedding via nonoverlapping historical features,” IEEE
Transactions on Cybernetics, vol. 49, no. 8, pp. 3006–3019, Aug. 2019, doi: 10.1109/TCYB.2018.2837675.
[25] G. Hermosilla et al., “Particle swarm optimization for the fusion of thermal and visible descriptors in face recognition systems,”
IEEE Access, vol. 6, pp. 42800–42811, 2018, doi: 10.1109/ACCESS.2018.2850281.
[26] R. M. Ramadan and R. F. A.- Kader, “Face recognition using particle swarm optimization-based selected features,” Face recognition
using particle swarm optimization-based selected features, vol. 2, no. 2, pp. 51–66, 2009.](https://image.slidesharecdn.com/102157067734627113esr14apr2223dec20n-220629034802-82375165/85/Application-of-optimization-algorithms-for-classification-problem-6-320.jpg)
![Int J Elec & Comp Eng ISSN: 2088-8708
Application of optimization algorithms for classification problem (Alaa Eleyan)
4379
[27] A. Eleyan, “Particle swarm optimization based feature selection for face recognition,” in 2019 Seventh International Conference on
Digital Information Processing and Communications (ICDIPC), May 2019, pp. 1–4, doi: 10.1109/ICDIPC.2019.8723831.
[28] A. Eleyan and M. Anwar, “Multiresolution edge detection using particle swarm optimization,” International Journal of Engineering
Science and Application, vol. 1, no. 1, pp. 11–17, 2017.
[29] A. Taieb and A. Ferdjouni, “A New design of fuzzy logic controller optimized By PSO-SCSO applied To SFO-DTC induction
motor drive,” International Journal of Electrical and Computer Engineering (IJECE), vol. 10, no. 6, pp. 5813–5823, Dec. 2020,
doi: 10.11591/ijece.v10i6.pp5813-5823.
[30] Y. Jiang, T. Hu, C. C. Huang, X. Wu, “An improved particle swarm optimization algorithm,” Applied Mathematics and
Computation, vol. 193, no. 1, pp. 231–239, Oct. 2007, doi: 10.1016/j.amc.2007.03.047.
[31] M. A. Meziane, Y. Mouloudi, and A. Draoui, “Comparative study of the price penalty factors approaches for Bi-objective dispatch
problem via PSO,” International Journal of Electrical and Computer Engineering (IJECE), vol. 10, no. 4, pp. 3343–3349, Aug.
2020, doi: 10.11591/ijece.v10i4.pp3343-3349.
[32] V. Z. Manusov, P. V. Matrenin, and L. S. Atabaeva, “Firefly algorithm to opmimal distribution of reactive power compensation
units,” International Journal of Electrical and Computer Engineering (IJECE), vol. 8, no. 3, pp. 1758–1765, Jun. 2018, doi:
10.11591/ijece.v8i3.pp1758-1765.
[33] M. Sababha and M. Zohdy, “Linear phase FIR Low pass filter design based on firefly algorithm,” International Journal of Electrical
and Computer Engineering (IJECE), vol. 8, no. 6, pp. 4356–4365, Dec. 2018, doi: 10.11591/ijece.v8i6.pp4356-4365.
[34] A. Kasinski, A. Florek, and A. Schmidt, “The PUT face database,” Image Processing and Communications, vol. 13, no. 3–4, pp.
59–64, 2008.
[35] P. J. Phillips, Hyeonjoon Moon, S. A. Rizvi, and P. J. Rauss, “The FERET evaluation methodology for face-recognition algorithms,”
IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 22, no. 10, pp. 1090–1104, 2000, doi: 10.1109/34.879790.
[36] P. N. Suganthan, “Particle swarm optimiser with neighbourhood operator,” in Proceedings of the 1999 Congress on Evolutionary
Computation-CEC99 (Cat. No. 99TH8406), 1999, pp. 1958–1962, doi: 10.1109/CEC.1999.785514.
BIOGRAPHIES OF AUTHORS
Alaa Eleyan received the B.Sc. and M.Sc. degrees in Electrical & Electronics
Engineering from Near East University, Northern Cyprus, in 2002 and 2004, respectively. In
2009, He finished his Ph.D. degree in Electrical and Electronics Engineering from Eastern
Mediterranean University, Northern Cyprus. Dr. Eleyan did his post doctorate studies at
Bilkent University in 2010. He has nearly two decades of working experience at various
universities in different countries. Currently, he is with the Electrical Engineering Department
at the American University of Middle East in Kuwait. His research interests are computer
vision, signal & image processing, pattern recognition, machine learning, and robotics. He has
more than 60 published journal articles and conference papers in these research fields. Dr.
Eleyan served as general chair for many international conferences such as ICDIPC2019,
DIPECC2018, TAEECE2018, and DICTAP2016. Tel: +965 2225 1400 Ext: 2679, Email:
Alaa.Eleyan@aum.edu.kw.
Mohammad Shukri Salman received the B.Sc., M.Sc., and Ph.D. degrees in
Electrical and Electronics Engineering from Eastern Mediterranean University (EMU), in
2006, 2007, and 2011, respectively. From 2006 to 2010, he was a teaching assistant of
Electrical and Electronics Engineering department at EMU. In 2010, he has joined the
Department of Electrical and Electronic Engineering at the European University of Lefke
(EUL) as a senior lecturer. For the period 2011-2015, he has worked as an Assist. Prof. in the
Department of Electrical and Electronics Engineering, Mevlana (Rumi) University, Turkey.
Currently, he is an Assoc. Prof. with the Electrical Engineering Department at the American
University of Middle East in Kuwait. He has served as a general chair, program chair, and
TPC member for many international conferences. His research interests include signal
processing, adaptive filters, image processing, sparse representation of signals, control
systems, and communications systems. Tel: +965 2225 1400 Ext: 1765, Email:
Mohammad.Salman@aum.edu.kw.
Bahaa Al-Sheikh received the B.Sc. degree in Electronics Engineering from
Yarmouk University, Jordan, MSc in Electrical Engineering from Colorado State University,
Colorado, USA, and Ph.D. in Biomedical Engineering degree from the University of Denver,
Colorado, USA, in 2000, 2005, and 2009, respectively. Between 2009 and 2015, he worked
for Yarmouk University as an assistant professor in the department of Biomedical Systems
and Medical Informatics Engineering and served as the department chairman between 2010
and 2012. He served as a part-time consultant for Sand-hill Scientific Inc., Highlands Ranch,
Colorado, USA in Biomedical Signal Processing field between 2009 and 2014. Currently, he
is an Associate Professor at the Electrical Engineering Department at the American University
of the Middle East in Kuwait. His research interests include digital signal and image
processing, biomedical systems modeling, medical instrumentation, and sound source
localization systems. Tel: +965 2225 1400 Ext: 1856, Email: Bahaa.Al-Sheikh@aum.edu.kw.](https://image.slidesharecdn.com/102157067734627113esr14apr2223dec20n-220629034802-82375165/85/Application-of-optimization-algorithms-for-classification-problem-7-320.jpg)
Application of optimization algorithms for classification problem
- 1. International Journal of Electrical and Computer Engineering (IJECE) Vol. 12, No. 4, August 2022, pp. 4373~4379 ISSN: 2088-8708, DOI: 10.11591/ijece.v12i4.pp4373-4379 4373 Journal homepage: http://ijece.iaescore.com Application of optimization algorithms for classification problem Alaa Eleyan, Mohammad Shukri Salman, Bahaa Al-Sheikh College of Engineering and Technology, American University of the Middle East, Egaila, Kuwait Article Info ABSTRACT Article history: Received Dec 23, 2020 Revised Jan 29, 2022 Accepted Feb 25, 2022 The work presented in this paper investigates the use of metaheuristic optimization algorithms for the face recognition problem. In the first setup, a face recognition system is implemented using particle swarm optimization (PSO) and firefly optimization algorithms, separately. PSO and firefly are used for forming the feature vectors in the feature selection stage. These feature vectors serve as the new representation for the face images that will be fed to the classifier. In the second setup, selected features from both PSO and firefly algorithms are fused to form one single feature vector for each face image before the classification stage. Extensive simulations are conducted using Poznan University of Technology (PUT) and face recognition technology (FERET) face databases. Optimal values for population size and maximum iterations number were selected before conducting the experiments. The effect of using different numbers of selected features on the performance is investigated for feature selection using PSO, firefly, and feature fusion of both. Keywords: Face recognition Firefly optimization Particle swarm optimization Swarm intelligence This is an open access article under the CC BY-SA license. Corresponding Author: Alaa Eleyan College of Engineering and Technology, American University of the Middle East Egaila, Kuwait Email: [email protected] 1. INTRODUCTION We live in a world where analyzing an enormous amount of diversified data is becoming a crucial and fundamental necessity. Therefore, in the analysis, we can bring out proper conclusions from the data we possess and classify it into positive, negative, or neutral sets. It is evident that people around us affect our daily life. People tend to change their minds and decisions based on other people’s opinions and decisions. To some extent, we use other people’s ideas to come up with our own decisions. Based on this idea swarm intelligence concept was introduced [1]–[3]. It is a nature-inspired artificial intelligence based on the communication models of social insects such as ants [4], [5], fireflies [6], [7], and bees [8], [9]. A swarm is a large number of agents interacting locally with themselves [10]–[13]. In a swarm, there is no supervisor or central control to give orders on how to behave. Swarm-based algorithms are popular in this era with the thirst for nature-inspired, population-based algorithms that can generate low-cost, fast, and correct solutions to complex and hard-to-solve problems. For that reason, swarm intelligence is becoming a golden ticket in the era of artificial intelligence metaheuristic optimization [14]–[16]. Swarm intelligence optimization techniques have been applied successfully in numerous applications where they helped to improve the system performance such as proportional integral (PI) controller tuning in wind turbines [17], the maximum power point tracking (MPPT) enhancement in photovoltaic systems [18], and cervical cancer classification [19]. One of the applications that the optimization algorithms can further improve its performance is face recognition. Face recognition is a very hot research topic especially with the increasing demand for security in different
- 2. ISSN: 2088-8708 Int J Elec & Comp Eng, Vol. 12, No. 4, August 2022: 4373-4379 4374 areas such as businesses, public transportations, borders, and airports. Since the start of using computers to recognize human faces around sixty years ago [20], many algorithms and techniques have been introduced to improve face recognition performance in challenging scenarios such as different poses, lighting, orientations, and facial expressions [21]–[24]. Hermosilla et al. [25] proposed to fuse thermal and visible descriptors where the system obtained the optimal weights using the particle swarm optimization (PSO) algorithm to maximize the face recognition rates obtained from different combinations of local descriptor methods. PSO was applied to coefficients extracted by the discrete cosine transforms (DCT) and the discrete wavelet transform (DWT) algorithms where it searched the feature space for the optimal feature subset based on a discrimination criterion [26]. In this paper, we propose to use two well-known optimization algorithms namely; PSO and firefly optimization algorithms for improving face recognition accuracy. In this paper, these two algorithms will be applied at the feature selection stage of the face recognition module in two different setups. In the first setup, the optimization algorithms are applied to face images for features selection and their performances are evaluated separately, whereas in the second setup the features selected from both optimization algorithms are fused before evaluation. The performance of the proposed approach will be tested using two popular face databases. The rest of the paper is organized as follows: section 2 presents the optimization algorithms used for feature selection. Section 3 describes the methodology and setup. Simulation results are shown and discussed in section 4. Finally, the paper is concluded in the last section. 2. RESEARCH METHOD 2.1. Optimization algorithms In this work, two swarm intelligence metaheuristic optimization algorithms are employed for feature selection. The selected salient features are used for class description and discrimination. The working mechanism and detailed explanations of both particle swarm optimization [3], [27]–[30] and firefly optimization [6], [7] algorithms are presented briefly below. For more details, reader is encouraged to refer to respective references. 2.1.1. Particle swarm optimization (PSO) This algorithm has been introduced by Kennedy and Eberhart [3]. It can be summarized as: given that the swarm has a certain number of particles, every single particle in a population has a current location, a current speed, and a personal best location in a search area. The personal best location is related to the location in the search area where an objective function provides the minimum calculated error for the corresponding particle. The location corresponds to minimum error among all the personal best locations is declared to be the global best location. Both, personal and global best locations and speeds are updated for every particle in the population at each iteration using the equation in [3]. An inertia weight, adapted by PSO, is linearly decreased during training to control the convergence rate of the algorithm. The next position of the particle is determined by accumulating the new speed to the particle’s current location. Limiting the speed vector value to a predefined range will guarantee the particle does not leave the search area. A detailed description of the algorithm can be found at [3]–[5], [31]. 2.1.2. Firefly optimization The firefly optimization algorithm is a global metaheuristic optimization algorithm, proposed by Yang [6], which imitates the behavior of firefly insects. Fireflies use the flashing behavior to attract other fireflies, usually for sending signals to the opposite sex. However, in the mathematical model, used inside the firefly algorithm, simply the fireflies are unisex, and any firefly can attract other fireflies. Firefly brightness amount is the key factor for its attraction by other fireflies. For a couple of fireflies; the brighter one will attract the other; so, the less bright one will move in the direction of the brighter one. This is done on every iteration of the algorithm for any binary combination of fireflies in the population. For more details about the algorithm, readers should refer to [6], [7], [32], [33]. 2.2. Methodology and setup At the first setup, we implemented our system using PSO and firefly optimization algorithms separately. PSO and firefly helped in the feature selection stage of the system. These selected features were used as a new representation for the images. They are fed to the classifier to evaluate the performance of the system as shown in Figure 1. At the second setup, after separately applying PSO and firefly to the image, selected features from both algorithms were fused to form one feature vector, as shown in Figure 2. The rest of the system is similar to the first setup.
- 3. Int J Elec & Comp Eng ISSN: 2088-8708 Application of optimization algorithms for classification problem (Alaa Eleyan) 4375 Two face databases were used to evaluate the proposed approaches. The first database is Poznan University of Technology (PUT) database [34]. It contains images of 100 people. Each person has 100 images covering varied facial expressions and orientations of size 128×128 pixels. 10 images per person with a total of 1,000 images are used in our experiments. The second database is face recognition technology (FERET) database [35]. A subset of FERET database with 194 people is used in our experiments. This subset contains 10 images per person of size 96×64 pixels. Examples of faces from both databases are shown in Figure 3. Figure 1. Block diagram of the proposed PSO/firefly-based texture classification Figure 2. Block diagram of the proposed feature fusion face recognition system Figure 3. Face examples from PUT database (on the left) and FERET database (on the right) The optimal swarm size (population) is problem-dependent which describes the social interaction within the swarm. While smaller populations are slower in convergence, they are less likely to fail into local minima and have more reliable convergence to optimal solutions [3]. Starting the search with small populations and increase population size proportionally to increase in iterations number ensures an initial high diversity with faster convergence as particles move towards a probable search area [36].
- 4. ISSN: 2088-8708 Int J Elec & Comp Eng, Vol. 12, No. 4, August 2022: 4373-4379 4376 The number of iterations to reach a good solution is also problem-dependent. Too few iterations may terminate the search prematurely. On the other hand, a too large number of iterations adds avoidable computational complexity (valid if the iterations number is the only stopping criteria) [3]. In this study, different population sizes and number of iterations were evaluated before deciding their optimal values for the rest of the simulations. 3. RESULTS AND DISCUSSION As mentioned earlier, two face databases are used for performance evaluation. We investigated different population sizes and number of iterations to find the optimal values for the rest of the simulations. Optimal values for all simulations are chosen as 20 for population and 100 for the maximum number of iterations. k-nearest neighbors (k-NN) is used with three different distance metrics namely; Manhattan, Euclidean, and Cosine. 10-fold cross-validation was applied in all the experiments. Results recorded in all figures and tables are averaged ones. Tables 1 and 2 show performances by applying different numbers of selected features for PSO and firefly algorithms, using PUT and FERET databases, respectively. Firefly algorithm is superior to PSO algorithm in all scenarios in both tables with different numbers of training images and selected features. According to the results in Tables 1 and 2, Manhattan distance gave the best performance. Based on this, results in Figures 4 and 5 are reported only using this distance. Table 1. Recognition rates with different number of selected features of PSO algorithm using PUT/FERET face databases Features Distanc e # Training Images 1 2 3 4 5 6 7 8 9 10 l1 45.56/20.25 55.84/26.84 64.96/33.57 71.95/41.28 75.00/38.33 77.30/39.51 80.73/46.55 79.40/41.52 79.80/45.57 l2 40.78/16.13 52.17/21.91 60.76/28.83 67.52/36.05 71.12/32.28 74.22/33.94 77.67/39.98 76.20/36.78 76.80/39.64 Cos 33.31/12.61 43.69/17.32 52.83/21.88 59.08/29.29 64.00/25.66 66.45/26.53 68.40/32.49 68.45/28.84 68.50/32.63 20 l1 59.46/36.55 74.88/45.23 82.17/55.41 87.62/64.24 87.90/65.35 90.45/72.28 92.67/71.98 94.55/72.01 94.80/74.69 l2 51.74/27.29 66.72/34.28 75.53/44.79 80.72/51.22 81.92/53.48 86.13/58.38 88.90/58.80 91.20/58.80 91.30/61.08 Cos 45.43/23.52 60.61/30.01 70.19/39.96 75.88/46.39 76.94/48.31 81.22/53.79 84.33/53.80 87.30/52.63 88.00/57.47 50 l1 70.10/51.32 83.16/62.74 90.30/75.71 93.10/79.75 94.78/86.88 95.67/88.44 97.37/87.94 97.65/89.87 97.90/91.24 l2 62.72/40.35 76.81/50.39 84.54/64.27 88.47/69.45 90.78/76.40 92.15/78.98 95.00/78.33 95.90/80.93 96.60/82.58 Cos 55.48/34.64 71.47/44.76 79.91/58.84 85.03/64.47 87.36/72.00 89.67/74.30 93.13/73.77 93.75/77.16 94.70/78.35 100 l1 74.86/64.42 86.34/74.46 91.67/84.22 94.85/88.38 96.36/90.39 97.17/93.65 98.20/94.40 98.30/94.51 98.20/94.69 l2 68.02/50.93 80.60/62.07 87.67/73.73 92.18/80.11 94.20/83.77 94.78/87.50 97.17/88.83 96.85/88.22 97.30/87.94 Cos 61.90/44.97 75.20/56.92 83.91/69.42 89.27/76.14 92.06/80.03 93.30/84.11 95.23/85.00 95.40/84.90 96.90/85.26 250 l1 79.13/71.00 90.36/79.79 93.71/88.82 95.97/92.27 97.00/94.09 98.08/95.77 98.90/96.49 98.35/96.78 98.90/97.27 l2 73.40/56.95 86.80/67.35 91.16/79.06 93.87/85.38 95.38/88.09 96.95/91.60 98.00/92.13 97.90/92.37 98.30/93.61 Cos 67.59/51.40 83.14/62.30 88.49/75.41 91.52/82.44 93.62/85.67 95.40/89.30 97.13/89.52 97.20/90.26 97.60/91.91 450 l1 82.51/72.40 92.19/81.60 95.51/89.65 97.20/93.63 97.70/95.25 98.38/96.30 99.23/96.92 98.85/97.35 99.10/97.99 l2 76.88/58.12 88.59/69.32 92.79/80.50 95.52/86.86 96.60/90.06 97.10/92.82 98.57/93.51 98.25/93.45 98.40/94.38 Cos 71.64/52.71 86.10/64.88 90.76/77.13 94.10/84.20 95.36/87.68 96.03/90.82 97.77/91.32 97.85/91.49 98.40/92.94 Table 2. Recognition rates with different number of selected features of firefly algorithm using PUT/FERET face databases Features Distanc e # Training Images 1 2 3 4 5 6 7 8 9 10 l1 49.30/23.24 63.44/29.36 73.07/37.94 76.50/43.69 80.36/45.03 80.20/45.57 86.87/47.04 84.95/51.55 87.90/51.75 l2 43.10/18.54 57.64/23.80 67.97/30.60 71.35/36.03 75.08/40.18 76.17/39.48 82.23/40.79 80.75/44.69 84.80/43.45 Cos 37.09/15.83 51.39/20.14 60.41/26.50 64.22/30.51 69.12/38.06 68.75/36.29 75.23/33.56 73.60/38.87 77.90/37.32 20 l1 62.71/34.94 77.05/43.05 85.17/52.17 88.88/60.84 89.84/66.45 92.20/70.67 94.77/68.78 95.10/69.97 96.00/69.12 l2 54.17/26.08 68.60/33.80 77.96/42.06 82.08/49.33 84.18/53.99 87.90/58.30 90.50/56.79 91.60/58.81 91.70/57.99 Cos 48.54/22.12 62.41/29.50 71.96/36.69 77.67/43.99 79.52/49.09 83.83/51.87 86.90/51.94 88.30/53.12 88.10/51.75 50 l1 73.98/52.69 85.53/63.36 91.56/75.53 94.08/82.77 95.46/85.42 97.10/89.79 97.97/87.37 97.90/89.30 98.10/90.72 l2 64.48/40.18 78.71/50.05 85.97/62.50 90.00/71.45 91.92/75.40 93.63/80.05 94.97/76.70 95.25/79.05 97.20/81.08 Cos 58.66/35.69 73.55/44.99 81.87/57.26 86.25/66.44 88.84/70.95 91.20/76.16 93.23/72.32 94.05/74.95 95.10/77.84 100 l1 78.34/63.12 89.97/75.21 93.74/83.59 95.40/88.89 96.98/92.05 97.88/93.07 98.83/94.24 98.50/93.92 98.90/94.64 l2 70.94/48.91 85.16/62.47 90.26/72.64 92.53/80.25 94.82/84.54 95.72/87.04 97.33/88.33 97.40/87.27 98.20/87.78 Cos 64.81/43.39 81.16/57.02 87.19/68.45 90.08/76.49 93.12/80.81 94.15/83.72 96.50/85.15 96.20/84.05 97.50/84.95 250 l1 81.94/69.24 92.36/79.00 94.81/87.95 97.13/92.66 97.48/94.02 98.08/96.16 99.03/96.43 99.10/96.42 99.10/96.96 l2 76.34/55.14 88.42/66.57 92.10/78.34 95.25/85.37 95.78/88.10 96.75/91.47 98.10/92.49 97.90/92.76 98.10/93.20 Cos 70.62/49.81 85.33/61.44 90.01/74.65 93.67/82.18 94.70/85.54 95.65/88.85 97.27/89.48 97.25/90.64 97.50/91.19 450 l1 82.34/71.68 92.21/81.37 95.37/89.79 97.32/93.49 97.96/94.98 98.35/96.47 99.20/97.16 99.20/97.37 99.10/97.89 l2 77.28/58.05 88.81/69.76 92.67/80.66 95.37/86.77 96.46/89.55 97.40/92.86 98.53/93.26 98.45/93.81 98.70/94.85 Cos 72.22/52.35 86.41/64.95 90.87/77.22 94.08/84.04 95.02/87.38 96.45/90.57 97.77/90.91 97.85/92.27 98.00/92.84
- 5. Int J Elec & Comp Eng ISSN: 2088-8708 Application of optimization algorithms for classification problem (Alaa Eleyan) 4377 Figure 4 shows a comparison among PSO, firefly, and PSO+firefly algorithms using PUT database. Fifty percent of images per person were used as a training set and the rest were used as a testing set (500 training images/500 testing images). A similar comparison was conducted using FERET database with 970 training images (Fifty percent of images per person) against 970 testing images. Figure 5 shows the results of this comparison among PSO, firefly, and PSO+firefly algorithms. Figure 4. Performance comparison with different number of selected features among PSO, firefly, and PSO+firefly (feature fusion) algorithms using PUT face database (500 training images/500 testing images) In both figures, it is clear that using more selected features helped to improve the performance using either PSO or firefly. For the fusion of PSO-generated features and firefly-generated features, improvement was obvious with a fewer number of selected features. For example, Figure 4 shows that with 20 selected features using PSO and firefly algorithms, performances reached 87.9% and 89.84% respectively. The performance increased to 94.44% with feature fusion/concatenation (20+20=40 fused features). The increasing number of fused features above 200 fused features (100 from PSO and 100 from firefly) did not show much performance improvement as shown in Figure 4. Figure 5. Performance comparison with different number of selected features among PSO, firefly, and PSO+firefly (feature fusion) algorithms using FERET face database (970 training images/970 testing images) Figure 5 shows similar results using FERET database even though the whole performance was lower due to the use of 194 people with 1940 images in total. Using 20 selected features from PSO and firefly algorithms, performances reached 65.35% and 66.45%, respectively. The performance increased to 82.25% with feature fusion/concatenation (20 + 20 = 40 fused features). Similar to the results of PUT database, it is clear that increasing 0 20 40 60 80 100 10 20 50 100 250 450 Recognition Rate Number of Features PSO Firefly Feature Fusion 0 20 40 60 80 100 10 20 50 100 250 450 Recognition Rate Number of Features PSO Firefly Feature Fusion
- 6. ISSN: 2088-8708 Int J Elec & Comp Eng, Vol. 12, No. 4, August 2022: 4373-4379 4378 the number of fused features from both PSO and firefly beyond 100 features each, will not have a noticeable effect on the performance. It will only increase computational complexity. 4. CONCLUSION We investigated the use of two of the metaheuristic population-based optimization algorithms namely; PSO and firefly, for the face recognition problem. PSO and firefly optimization algorithms are utilized in the feature selection stage for generating the feature vectors. The application of such algorithms showed enhancement in the recognition performance of the system. Firefly algorithm provided better performance than PSO algorithm. In addition, the fusion of selected features from both algorithms forming one single feature vector further improved the recognition performance. Results indicated that using optimization algorithms for feature selection is a good choice for improving the performance with much fewer features. Furthermore, feature fusion of the selected features generated using both optimization algorithms helped to boost the performance with fewer selected features. REFERENCES [1] A. P. Engelbrecht, Computational intelligence, an introduction. England: 2nd Ed. John Wiley & Sons Ltd, 2007. [2] R. C. Eberhart and Y. Shi, Computational intelligence: Concepts to implementations. Morgan Kaufmann; 1st edition, 2007. [3] R. C. Eberhart, Y. Shi, and J. Kennedy, Swarm intelligence (the morgan kaufmann series in evolutionary computation). Morgan Kaufmann; 1st edition, 2001. [4] M. Dorigo, V. Maniezzo, and A. Colorni, “Ant system: optimization by a colony of cooperating agents,” IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), vol. 26, no. 1, pp. 29–41, Feb. 1996, doi: 10.1109/3477.484436. [5] M. Dorigo, G. Di Caro, and L. M. Gambardella, “Ant algorithms for discrete optimization,” Artificial Life, vol. 5, no. 2, pp. 137– 172, Apr. 1999, doi: 10.1162/106454699568728. [6] X.-S. Yang, Nature-inspired metaheuristic algorithms, 2nd Editio. Luniver Press, 2010. [7] I. Fister, I. Fister, X.-S. Yang, and J. Brest, “A comprehensive review of firefly algorithms,” Swarm and Evolutionary Computation, vol. 13, pp. 34–46, Dec. 2013, doi: 10.1016/j.swevo.2013.06.001. [8] D. Karaboga, “Technical Report TR06: An idea based on honey bee swarm for numerical optimization,” 2005. [9] D. Karaboga and B. Basturk, “Artificial bee colony (ABC) optimization algorithm for solving constrained optimization problems,” in Foundations of Fuzzy Logic and Soft Computing, Berlin, Heidelberg: Springer Berlin Heidelberg, 2007, pp. 789–798. [10] S. Bandyopadhyay, S.-J. Chung, and F. Y. Hadaegh, “Probabilistic and distributed control of a large-scale swarm of autonomous agents,” IEEE Transactions on Robotics, vol. 33, no. 5, pp. 1103–1123, Oct. 2017, doi: 10.1109/TRO.2017.2705044. [11] D. Messie and J. C. Oh, “SWARMs of self-organizing polymorphic agents,” in Swarm Intelligent Systems, Springer Berlin Heidelberg, 2005, pp. 75–90. [12] B. Doerr and R. Linares, “Control of large swarms via random finite set theory,” in 2018 Annual American Control Conference (ACC), Jun. 2018, pp. 2904–2909, doi: 10.23919/ACC.2018.8430968. [13] J. A. Nasiri, A. M. Fard, M. Naghibzadeh, and M. Rouhani, “High dimensional problem optimization using distributed multi-agent PSO,” in 2009 Third UKSim European Symposium on Computer Modeling and Simulation, 2009, pp. 245–250, doi: 10.1109/EMS.2009.107. [14] M. Masood, M. M. Fouad, and I. Glesk, “Analysis of artificial intelligence-based metaheuristic algorithm for MPLS network optimization,” in 2018 20th International Conference on Transparent Optical Networks (ICTON), Jul. 2018, pp. 1–4, doi: 10.1109/ICTON.2018.8473751. [15] L. Penghui et al., “Metaheuristic optimization algorithms hybridized with artificial intelligence model for soil temperature prediction: novel model,” IEEE Access, vol. 8, pp. 51884–51904, 2020, doi: 10.1109/ACCESS.2020.2979822. [16] D. Ekmekci, “A pheromonal artificial bee colony -pABC-algorithm for optimization problems,” in 2019 16th International Multi- Conference on Systems, Signals & Devices (SSD), Mar. 2019, pp. 452–456, doi: 10.1109/SSD.2019.8893259. [17] A. H. Sule, A. S. Mokhtar, J. J. Bin Jamian, A. Khidrani, and R. M. Larik, “Optimal tuning of proportional integral controller for fixed-speed wind turbine using grey wolf optimizer,” International Journal of Electrical and Computer Engineering (IJECE), vol. 10, no. 5, pp. 5251–5261, Oct. 2020, doi: 10.11591/ijece.v10i5.pp5251-5261. [18] Y. A. P. and A. G. S., “A modified particle swarm optimization algorithm to enhance MPPT in the PV array,” International Journal of Electrical and Computer Engineering (IJECE), vol. 10, no. 5, pp. 5001–5008, Oct. 2020, doi: 10.11591/ijece.v10i5.pp5001- 5008. [19] R. Sagban, H. A. Marhoon, and R. Alubady, “Hybrid bat-ant colony optimization algorithm for rule-based feature selection in health care,” International Journal of Electrical and Computer Engineering (IJECE), vol. 10, no. 6, pp. 6655–6663, Dec. 2020, doi: 10.11591/ijece.v10i6.pp6655-6663. [20] K. de Leeuw and J. Bergstra, The history of information security: A comprehensive handbook, 1st Ed. Amsterdam: Elsevier, 2007. [21] A. K. Jain, B. Klare, and U. Park, “Face recognition: Some challenges in forensics,” in Face and Gesture 2011, Mar. 2011, pp. 726–733, doi: 10.1109/FG.2011.5771338. [22] A. Eleyan, H. Özkaramanli, and H. Demirel, “Complex wavelet transform-based face recognition,” EURASIP Journal on Advances in Signal Processing, vol. 2008, no. 1, Dec. 2009, doi: 10.1155/2008/185281. [23] M. Hassaballah and S. Aly, “Face recognition: challenges, achievements and future directions,” IET Computer Vision, vol. 9, no. 4, pp. 614–626, Aug. 2015, doi: 10.1049/iet-cvi.2014.0084. [24] J. Shan, C. Hou, H. Tao, W. Zhuge, and D. Yi, “Full representation data embedding via nonoverlapping historical features,” IEEE Transactions on Cybernetics, vol. 49, no. 8, pp. 3006–3019, Aug. 2019, doi: 10.1109/TCYB.2018.2837675. [25] G. Hermosilla et al., “Particle swarm optimization for the fusion of thermal and visible descriptors in face recognition systems,” IEEE Access, vol. 6, pp. 42800–42811, 2018, doi: 10.1109/ACCESS.2018.2850281. [26] R. M. Ramadan and R. F. A.- Kader, “Face recognition using particle swarm optimization-based selected features,” Face recognition using particle swarm optimization-based selected features, vol. 2, no. 2, pp. 51–66, 2009.
- 7. Int J Elec & Comp Eng ISSN: 2088-8708 Application of optimization algorithms for classification problem (Alaa Eleyan) 4379 [27] A. Eleyan, “Particle swarm optimization based feature selection for face recognition,” in 2019 Seventh International Conference on Digital Information Processing and Communications (ICDIPC), May 2019, pp. 1–4, doi: 10.1109/ICDIPC.2019.8723831. [28] A. Eleyan and M. Anwar, “Multiresolution edge detection using particle swarm optimization,” International Journal of Engineering Science and Application, vol. 1, no. 1, pp. 11–17, 2017. [29] A. Taieb and A. Ferdjouni, “A New design of fuzzy logic controller optimized By PSO-SCSO applied To SFO-DTC induction motor drive,” International Journal of Electrical and Computer Engineering (IJECE), vol. 10, no. 6, pp. 5813–5823, Dec. 2020, doi: 10.11591/ijece.v10i6.pp5813-5823. [30] Y. Jiang, T. Hu, C. C. Huang, X. Wu, “An improved particle swarm optimization algorithm,” Applied Mathematics and Computation, vol. 193, no. 1, pp. 231–239, Oct. 2007, doi: 10.1016/j.amc.2007.03.047. [31] M. A. Meziane, Y. Mouloudi, and A. Draoui, “Comparative study of the price penalty factors approaches for Bi-objective dispatch problem via PSO,” International Journal of Electrical and Computer Engineering (IJECE), vol. 10, no. 4, pp. 3343–3349, Aug. 2020, doi: 10.11591/ijece.v10i4.pp3343-3349. [32] V. Z. Manusov, P. V. Matrenin, and L. S. Atabaeva, “Firefly algorithm to opmimal distribution of reactive power compensation units,” International Journal of Electrical and Computer Engineering (IJECE), vol. 8, no. 3, pp. 1758–1765, Jun. 2018, doi: 10.11591/ijece.v8i3.pp1758-1765. [33] M. Sababha and M. Zohdy, “Linear phase FIR Low pass filter design based on firefly algorithm,” International Journal of Electrical and Computer Engineering (IJECE), vol. 8, no. 6, pp. 4356–4365, Dec. 2018, doi: 10.11591/ijece.v8i6.pp4356-4365. [34] A. Kasinski, A. Florek, and A. Schmidt, “The PUT face database,” Image Processing and Communications, vol. 13, no. 3–4, pp. 59–64, 2008. [35] P. J. Phillips, Hyeonjoon Moon, S. A. Rizvi, and P. J. Rauss, “The FERET evaluation methodology for face-recognition algorithms,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 22, no. 10, pp. 1090–1104, 2000, doi: 10.1109/34.879790. [36] P. N. Suganthan, “Particle swarm optimiser with neighbourhood operator,” in Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406), 1999, pp. 1958–1962, doi: 10.1109/CEC.1999.785514. BIOGRAPHIES OF AUTHORS Alaa Eleyan received the B.Sc. and M.Sc. degrees in Electrical & Electronics Engineering from Near East University, Northern Cyprus, in 2002 and 2004, respectively. In 2009, He finished his Ph.D. degree in Electrical and Electronics Engineering from Eastern Mediterranean University, Northern Cyprus. Dr. Eleyan did his post doctorate studies at Bilkent University in 2010. He has nearly two decades of working experience at various universities in different countries. Currently, he is with the Electrical Engineering Department at the American University of Middle East in Kuwait. His research interests are computer vision, signal & image processing, pattern recognition, machine learning, and robotics. He has more than 60 published journal articles and conference papers in these research fields. Dr. Eleyan served as general chair for many international conferences such as ICDIPC2019, DIPECC2018, TAEECE2018, and DICTAP2016. Tel: +965 2225 1400 Ext: 2679, Email: [email protected]. Mohammad Shukri Salman received the B.Sc., M.Sc., and Ph.D. degrees in Electrical and Electronics Engineering from Eastern Mediterranean University (EMU), in 2006, 2007, and 2011, respectively. From 2006 to 2010, he was a teaching assistant of Electrical and Electronics Engineering department at EMU. In 2010, he has joined the Department of Electrical and Electronic Engineering at the European University of Lefke (EUL) as a senior lecturer. For the period 2011-2015, he has worked as an Assist. Prof. in the Department of Electrical and Electronics Engineering, Mevlana (Rumi) University, Turkey. Currently, he is an Assoc. Prof. with the Electrical Engineering Department at the American University of Middle East in Kuwait. He has served as a general chair, program chair, and TPC member for many international conferences. His research interests include signal processing, adaptive filters, image processing, sparse representation of signals, control systems, and communications systems. Tel: +965 2225 1400 Ext: 1765, Email: [email protected]. Bahaa Al-Sheikh received the B.Sc. degree in Electronics Engineering from Yarmouk University, Jordan, MSc in Electrical Engineering from Colorado State University, Colorado, USA, and Ph.D. in Biomedical Engineering degree from the University of Denver, Colorado, USA, in 2000, 2005, and 2009, respectively. Between 2009 and 2015, he worked for Yarmouk University as an assistant professor in the department of Biomedical Systems and Medical Informatics Engineering and served as the department chairman between 2010 and 2012. He served as a part-time consultant for Sand-hill Scientific Inc., Highlands Ranch, Colorado, USA in Biomedical Signal Processing field between 2009 and 2014. Currently, he is an Associate Professor at the Electrical Engineering Department at the American University of the Middle East in Kuwait. His research interests include digital signal and image processing, biomedical systems modeling, medical instrumentation, and sound source localization systems. Tel: +965 2225 1400 Ext: 1856, Email: [email protected].