Web Service QoS Prediction Based on Adaptive Dynamic Programming Using Fuzzy Neural Networks for Cloud Services

SPECIAL SECTION ON CHALLENGES FOR SMART WORLDS
Received September 3, 2015, accepted October 5, 2015, date of publication November 6, 2015, date of current version
November 19, 2015.
Digital Object Identifier 10.1109/ACCESS.2015.2498191
Web Service QoS Prediction Based on Adaptive
Dynamic Programming Using Fuzzy Neural
Networks for Cloud Services
XIONG LUO, YIXUAN LV, RUIXING LI, AND YI CHEN
School of Computer and Communication Engineering, University of Science and Technology Beijing, Beijing 100083, China
Beijing Key Laboratory of Knowledge Engineering for Materials Science, Beijing 100083, China
Corresponding author: X. Luo (xluo@ustb.edu.cn)
This work was supported in part by the Fundamental Research Funds for the Central Universities, in part by the National Natural Science
Foundation of China under Grant 61174103, Grant 61272357, and Grant 61300074, and in part by the National Key Technologies
Research and Development Program of China under Grant 2015BAK38B01.
ABSTRACT Recently, more and more traditional services are being migrated into a cloud computing
environment that makes the quality of service (QoS) becomes an important factor for service selection
and optimal service composition while forming cross-cloud service applications. Considering the nonlinear
and dynamic property of QoS data, it is so difficult to achieve dynamic prediction while designing a QoS
prediction method with unsatisfactory prediction accuracy. It is thus desirable to explore how to design an
effective approach by incorporating some intelligent techniques into the QoS prediction method to improve
prediction performance. In this paper, motivated by the adaptive critic design and Q-learning technique,
we propose a novel QoS prediction approach to serve this purpose through the combination of fuzzy
neural networks and adaptive dynamic programming (ADP), i.e., an online learning scheme. This approach
extracts fuzzy rules from QoS data and employs the ADP method to parameter learning of the fuzzy rules.
Moreover, we provide a convergence boundedness result for our proposed approach to guarantee the stability.
Experimental results on a large-scale QoS service data set verify the prediction accuracy of our proposed
approach.
INDEX TERMS Quality of service (QoS), QoS prediction, fuzzy neural network, adaptive dynamic
programming, cloud services.
I. INTRODUCTION
Recent years have seen the massive migration of traditional
services to the cloud computing environment. These cloud
services on different cloud platforms are often invoked in
web services either deployed in the cloud by companies or
deployed in accordance with cloud applications. It is clearly
crucial to form cross-cloud service applications. There has
been a tremendous surge of interest on web services in
cloud environment [1]–[4]. Nowadays, these ever-increasing
web services could provide more functions and features
in cloud environment to enhance the user experience [1].
Within this trend, the performance of cloud services can be
improved when constructing the personalized cloud appli-
cations through the integration of web services in cloud
environment. Meanwhile, it poses a challenge in selecting
qualified web services for developing web service-based inte-
gration cloud applications. The quality of service (QoS) is
becoming more and more crucial in describing web services
during the implementation of service selection and optimal
service composition. Generally, QoS is a set of nonfunctional
performance indices of web services, including popularity,
response time, failure probability, and many others [5], [6].
Also, more and more attention has been paid to QoS which
has been seen as a very hot topic in cloud computing [7], [8].
Among those available performance indices, response time
that users can feel directly is one of the most important QoS
properties used to evaluate web services. The predicted QoS
values have been widely used in service selection, service
ranking, and service composition under dynamic environ-
ment [9]–[11]. Here, we discuss the prediction method for
response time in QoS.
QoS is fundamental for cloud users, who expect providers
to offer services of the advertised quality, and for service
providers, who need to find the right tradeoffs between
2260
2169-3536
2015 IEEE. Translations and content mining are permitted for academic research only.
Personal use is also permitted, but republication/redistribution requires IEEE permission.
See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
VOLUME 3, 2015
www.redpel.com +917620593389

X. Luo et al.: Web Service QoS Prediction Based on Adaptive Dynamic Programming ADP Using FNNs
QoS levels and operational costs [12]. The predicted
response time is valuable for service composition, component
selection, task scheduling, and so on. At present, a number
of QoS prediction approaches for web services have been
developed. Generally speaking, the QoS prediction
approaches can be decomposed in two families. The first one
is the static prediction method. These methods (e.g. arith-
metic average value method) are simple and easy to
implement, ignoring the individual factors of users, the ser-
vice status, and the network environment. These prediction
methods do not reflect the nonlinear relationship of the
QoS data [13], which may lead to a great error between the
predicted value and the actual value. The second one is
the dynamic prediction method, e.g. prediction via collabora-
tive filtering, similarity measure, multi-dimensional weight-
ing [14]–[19]. The main idea behind these methods is that
web users who have similar historical QoS data on some
services, would have similar experiences on other services.
But the difficulty in getting sufficient QoS data for similarity
calculation may lead to a large error under some cloud
computing environment.
Considering the nonlinear and dynamic property of
QoS data as well as the lack of QoS data for calculation
in some cases, we present a prediction approach based on
adaptive dynamic programming (ADP) to avoid the
above limitations. ADP is a novel optimization tech-
nique which combines concepts of reinforcement learning
(e.g. Q-learning) and dynamic programming [20], [21].
While dealing with complex systems, ADP uses a function
approximation structure such as neural network (NN) to
approximate cost function, which avoids the curse of dimen-
sionality and reduces the computation time. The approximate
optimal solution is obtained by using the offline iteration
or the online update algorithms. In other words, ADP can
learn from a dynamic environment with less information. The
application of ADP is focused on nonlinear and complex
systems, such as aircraft system [22], power system [23],
energy management system [24], and many others [25], [26].
In view of this, due to its potential scalability of the adaptive
critic designs and the intuitiveness of Q-learning, ADP may
play an important role in dealing with the nonlinear prediction
for the QoS data.
Meanwhile, we propose to use fuzzy logic to express the
inner relationship of the QoS data that users who have sim-
ilar QoS data on some services would have similar expe-
riences on other services. Fuzzy logic can deal with the
QoS of web service in context-aware environment, which
can be expressed with linguistic variables and membership
functions. Thus, fuzzy logic has been used in QoS man-
agement system [27] and QoS prediction with fuzzy clus-
tering [28]. But, on the other hand, fuzzy logic lacks the
learning ability and the adaptability while extracting fuzzy
rules from QoS data. Then, it can be combined with NN to
avoid such limitation. By combining fuzzy logic with NN,
fuzzy neural network (FNN) provides a mathematical tool to
deal with the nonlinear and dynamic property of QoS data
in context-aware environment. Specifically, FNN not only
extracts fuzzy rules from the data to express the dynamic
property of the system, but also simply adjusts the parameters
of the network during the learning process to enhance the
adaptability of the network [29]. Therefore, FNN shows great
adaptability and flexibility in developing a nonlinear model
for QoS prediction.
In consideration of all the issues in QoS prediction
mentioned above, we may conduct a QoS prediction though
the combination of FNN and ADP. In our previous research
work of [30], by using FNN, the direct heuristic dynamic pro-
gramming as a special ADP was presented in a longitudinal
control of hypersonic vehicles. Although the control system
framework was implemented, there is only a single action
output under its scheme. Moreover, there is no stability proof
related to the convergence of proposed ADP approach, which
limited its application. Motivated by it, the purpose of this
paper is to present a novel QoS prediction service scheme
which fuses ADP with FNN. Within the proposed scheme,
FNN is employed to approximate the cost-to-go function
and the action network in ADP. In addition to novel
QoS prediction scheme, we also provide a associated
Lyapunov stability proof and its convergence result to guar-
antee a uniform ultimate boundedness (UUB) property for
the weight errors of NN in the proposed FNN-based ADP
scheme. These results are better than the previous one in [30].
Therefore, due to its unique features of ADP as well as
strong adaptability and flexibility of FNN in context-aware
environment, our proposed scheme may serve the purpose of
effectively predicting those QoS data for cloud services.
The rest of this paper is organized as follows. Section 2
presents the background of QoS prediction. Section 3
describes the characteristics of FNN. Section 4 presents
the architecture and the convergence analysis of the
FNN-based ADP prediction algorithm. The QoS prediction
experiments using our proposed scheme are illustrated in
section 5. Finally, section 6 provides a conclusion.
II. PROBLEM STATEMENT
Since many web services have the similar functions with
cloud applications, consumers are not possible to use every
web service. Therefore, QoS data of unused web services
plays an important role in providing suitable web services to
consumers.
TABLE 1. A user-component matrix.
Let us first consider an example in Table 1. Each entry
presents the response time of a web service invoked by
a user, which is a particular QoS property value. The
numeric entries in the table mean the response time for users
VOLUME 3, 2015 2261

X. Luo et al.: Web Service QoS Prediction Based on ADP Using FNNs
who have invoked the services. The blank entries correspond
to the users who have not invoked the services.
For the purpose of service composition or service ranking,
we need to know all the QoS data of web services related to
users. However, the real QoS data is similar to the example we
present here. Therefore, it is significant to predict the blank
entries before any QoS-based service selection or service
ranking. As a result, the problem we study in this paper is
how to precisely predict the blank entries in accordance with
the existing entries.
III. FUZZY NEURAL NETWORK
With the development of intelligent techniques, fuzzy logic
and NN have made remarkable achievements in their respec-
tive fields. With the improvement of the fuzzy theory and the
new upsurge of NN research in 1980s, many researchers paid
more attention to the fusion of those two fields. They utilized
the superiority of fuzzy theory and NN, forming the concept
of FNN.
Adaptive-network-based fuzzy inference system (ANFIS)
is a NN implementation of Takagi-Suguno (T-S) fuzzy infer-
ence system [31]. ANFIS constructs a set of fuzzy if-then
rules automatically and optimizes the rules through the self-
learning of NN. This avoids such limitation that fuzzy if-then
rules mainly come from the experience of experts. Compared
with back propagate (BP) network, ANFIS has stronger self-
learning ability, robustness, and adaptability while it can
approximate any nonlinear function.
FIGURE 1. ANFIS architecture.
Fig. 1 shows an architecture of ANFIS. The ANFIS con-
sists of five layers. Here, we assume that ANFIS has n inputs
(i.e., x1, · · · , xn) and 1 output (i.e., y). There are N fuzzy sets.
Suppose that the rule base contains N fuzzy if-then rules of
T-S type:
Rule j : IF x1 = A1j and · · · and xn = Anj,
THEN wj =
n
i=1
zijxi + zn+1,j,
z1j, · · · , zn+1,j ∈ R; j = 1, 2, · · · , N.
Here, Aij represents the fuzzy variable where i = 1, · · · , n
and j = 1, · · · , N.
Layer 1 is the fuzzification layer. This layer defines the
membership function of the nodes. Normally the function is
a Guassian function:
µAij (xi) = exp −
(xi − cij)2
σ2
ij
. (1)
where cij and σij are the excepted value and standard deviation
of Gaussian function, respectively.
Layer 2 is the rule layer. Each node multiplies those inputs
and generates the outputs as follows:
vj =
n
i=1
µAij (xi). (2)
The outputs of this layer stand for the firing strength of a rule.
Layer 3 is the normalization layer. Nodes of layer 3 gener-
ate the weights according to:
¯vj =
vj
N
j=1
vj
. (3)
Layer 4 is the defuzzification layer. Nodes in this layer are
calculated by:
¯vjwj = ¯vj
n
i=1
zijxi + zn+1,j . (4)
Layer 5 is the output layer. The node stands for the ANFIS
output y by calculating the sum of outputs of defuzzification
layer.
y =
N
j=1
¯vjwj. (5)
Due to the combination of BP algorithm and least square
estimation (LSE) algorithm in its implementation, ANFIS
achieves good performance with fast learning speed. In the
training process, the parameters of if-then rules can be iden-
tified via LSE in the forward pass. In the backward pass, the
error rates propagate backward while the expectations and
variances of Guassian function are updated via BP algorithm.
Therefore it is more accurate and efficient through the use of
that hybrid algorithm.
IV. QoS PREDICTION SCHEME
In the past decades, the ADP method was proposed through
the use of NN to approximate the cost function. According
to the critic’s output, the ADP can be categorized as:
heuristic dynamic programming (HDP), dual heuristic pro-
gramming (DHP), and globalized dual heuristic program-
ming (GDHP) [21], [32]. The action dependent version of
HDP and DHP are formed when the critic’s inputs are
augmented with the controller’s output. In this paper, our
approach is proposed on the basis of HDP.
2262 VOLUME 3, 2015

FIGURE 2. ADP architecture.
A. ADP STRUCTURE
As shown in Fig. 2, this ADP is structured to estimate
the cost function J(t) in the Bellman equation of dynamic
programming:
J(t) =
∞
k=t+1
αk−t−1
r(t), (6)
where α is a discount factor bounded within (0, 1) and r(t) is
the external reinforcement signal. In addition, x(t) is the state
of the plant and u(t) is the control signal of the plant at
time t.
Through the use of the critic network of ADP, the J is cal-
culated as an approximator of the optimal value function J∗.
It is implemented by minimizing the following error measure
over time:
Ec(t) =
1
2
× e2
c(t), (7)
ec(t) = αJ(t) − (J(t − 1) − r(t)). (8)
Meanwhile, the action network of ADP is to backpropagate
the error between the desired objective Uc and the approxi-
mator J∗. Since we have defined ‘‘0’’ as the reinforcement
signal for ‘‘success’’ in our proposed approach, Uc is set to 0
all the time without loss of generality. Then, by minimizing
the following performance error measure, the weights in the
action network can be updated.
Ea(t) =
1
2
× e2
a(t), (9)
ea(t) = J(t) − Uc(t). (10)
B. QoS PREDICTION BASED ON ADP USING ANFIS
Our proposed approach aims to predict the missing QoS value
of web service mentioned in Section 2. To deal with the
nonlinear and dynamic properties of QoS data in context-
aware environment, the ADP using FNN is proposed to
enhance the adaptability and flexibility of Internet. Unlike
the traditional actor-critic design in ADP normally using
feedforward networks with one hidden layer to implement the
critic network and the action network, our proposed approach
employs FNN to construct those two networks.
The online training and optimization algorithm in ADP
with ANFIS is shown in Algorithm 1. In traditional ADP,
x(t) is the state of the plant and u(t) is the control signal of
Algorithm 1 The ADP Using ANFIS
1 initialize t = 0, x(0), and u(0);
2 apply x(t) to the action network and obtain u(t);
3 apply u(t) to the system model and obtain x(t + 1);
4 apply x(t + 1) and u(t) to the critic network, and obtain
J(t);
5 update weights of the critic network till the requirement
on Ec(t) is satisfied;
6 update weights of the action network till the requirement
on Ea(t) is satisfied;
7 t = t + 1, continue from 2.
the plant at time t. In our proposed approach, x(t) stands for
a vector of remaining QoS data used to predict a blank entry
at time t. And u(t) is the predictive result of the blank entry
at time t.
C. REDESCRIPTION OF ADP USING ANFIS
To analyze the convergence of the ADP using ANFIS,
we simplify the structure of ANFIS as illustrated
in Figs. 3 and 4. Layer 1 and layer 2 in ANFIS are converted
to the input layer of NN. The output layer of NN consists of
the layer 3, layer 4 and layer 5 in ANFIS.
In the simple ANFIS, pk and qk are the input and output of
the hidden layer. The subscripts ‘‘a’’ and ‘‘c’’ mean the action
network and critic network, respectively. In addition,
c(t) and σ(t) stand for the expectation and variance of
Guassian function, respectively. And φ(·) represents a func-
tion defined as:
φ(x) = exp −
1
2
x . (11)
FIGURE 3. Schematic diagram of ANFIS in critic network.
According to Fig. 3 and Fig. 4, the inputs of the critic
network include m states x(t) = (x1(t), · · · , xm(t))T and
n actions u(t) = (u1(t), · · · , un(t))T . And wc(t) is the param-
eter matrix of fuzzy rules in ANFIS. Applying the computing
rule of exponential function to the combination of layer 1
and layer 2 in ANFIS, the pc(t) can be written as follows.
VOLUME 3, 2015 2263

FIGURE 4. Schematic diagram of ANFIS in action network.
Let Nfc be the numbers of fuzzy rules in the critic network.
The output J(t) can be expressed as:
J(t) =
Nfc
k=1
wck (t)qck (t)
Nfc
k=1
qck (t)
, (12)
qck (t) = φ(pck (t)), (13)
pck (t) =
m
i=1
xi(t) − cci,k (t)
σci,k (t)
2
+
n
j=1
uj(t) − ccj+m,k (t)
σcj+m,k (t)
2
, k = 1, · · · , Nfc
(14)
Unlike the traditional action network, the inputs include
m states x(t) = (x1(t), · · · , xm(t))T . Let Nfa be the numbers
of fuzzy rules in the action network. The output u(t) can be
expressed as:
uj(t) =
Nfa
k=1
wajk (t)qak (t)
Nfa
k=1
qak (t)
j = 1, · · · , n (15)
qak (t) = φ(pak (t)), (16)
pak (t) =
m
i=1
xi(t) − cai,k (t)
σai,k (t)
2
, k = 1, · · · , Nfa (17)
D. CONVERGENCE ANALYSIS
In this section, we refer to a UUB result in [33] and provide
a convergence bound for our proposed approach to show that
the estimation errors of the network weights in this approach
are UUB.
According to (13) and (16), φc(t) = (qc1 (t), · · · , qcNfc
(t))T
and φa(t) = (qa1 (t), · · · , qaNfa
(t))T are the hidden layer
neuron activation function vectors of the critic network and
the action network, respectively. The output layer
weights wa(t) and wc(t) are initialized randomly and updated
during learning. Then, with (12) and (15), J(t) and u(t) can
be written with matrix forms:
J(t) = wT
c (t)φc(t), (18)
u(t) = wT
a (t)φa(t). (19)
The weight of critic network is updated as:
wT
c (t + 1) = wc(t) − lc
∂Ec(t)
∂J(t)
∂J(t)
∂wc(t)
= wc(t) − αlcφc(t)
×[αwT
c (t)φc(t) + r(t) − wT
c (t − 1)φc(t − 1)]T
(20)
where lc > 0 is the learning rate.
The weight of action network is updated as:
wT
a (t + 1) = wa(t) − la
∂Ea(t)
∂J(t)
∂J(t)
∂u(t)
∂u(t)
∂wa(t)
= wa(t) − laφa(t)[wT
c (t)C(t)][wT
c (t)φc(t)]T
(21)
where la > 0 is the learning rate, and
C(t) =

 −φck(t)
uj(t) − ccj+m,k (t)
σ2
ccj+m,k
(t)
Nfc
k=1


n
j=1
. (22)
Let w∗ be the optimal weight of network. Then we denote
the weight estimation error as w(t)
def
= w(t) − w∗. In the
following analysis, · represents the 2-norm.
Deﬁnition 1: The solution w(t) is said to be uniformly
ultimately bounded (UUB) within a bound ε > 0, if for any
δ > 0 and t0 > 0, there exists a positive number N = N(δ, ε),
independent of t0, so that w(t) ε for all t N +t0 when
w(t0) < δ.
Then, we will adopt a Lyapunov stability analysis frame-
work to prove the UUB property of our proposed approach.
First, we present some assumptions as follows.
Assumption 1: Let w∗
c and w∗
a be the optimal weights for
the critic and action network, respectively. They are bounded.
Then,
w∗
c wcm, w∗
a wam, (23)
where w∗
c and w∗
a are two positive constants.
Theorem 1: Let Assumption 1 hold. Take the reinforce-
ment signal as r(t) ∈ [0, 1]. Let the critic and action
network settings be (18) and (19), respectively. Let the weight
update for the critic and action network be (20) and (21),
respectively. Then the errors of network weights,
wc(t) and wa(t), are UUB, provided the following conditions
hold:
√
2
3
< α <1, 0 < lc <
1
α2 φc(t) 2
, 0 < la <
1
φa(t) 2
,
(24)
Proof: See Appendix A.
2264 VOLUME 3, 2015

Theorem 2: Let Assumption 1 hold. Then, the errors of
network weights, wc(t) and wa(t), are UUB, provided the
following conditions hold:
√
2
3
< α < 1, 0 < lc <
1
α2Nfc
, 0 < la <
1
Nfa
. (25)
where Nfc and Nfa are the numbers of fuzzy rules in the critic
and action network, respectively.
Proof: See Appendix B.
While using our approach, Theorem 2 gives a simple
sufficient condition for selecting learning rates in the pro-
posed ADP architecture to maintain stability of the weight
updates.
V. EXPERIMENTS
A. DATA SET DESCRIPTION
In our experiments, the data set comes from the
WS-DREAM [34], [35] including the QoS data of 5, 825
real-world web services from 73 countries. The QoS value
is observed by 339 distributed computers and each computer
invokes all the 5, 825 web services by sending null operating
requests. In this paper, we use a 339 × 5825 user-component
matrix which represents response time values for experiment.
The statistics of the web service QoS response time data set
is summarized in Table 2.
TABLE 2. The statistics of data set.
B. METRICS
We calculate mean absolute error (MAE) and mean absolute
percentage error (MAPE) to compare the prediction quality
of our proposed approach with other methods.
The MAE and MAPE are defined as follows:
MAE =
1
P
i,j
νij − νij , (26)
MAPE =
1
P
i,j
νij − νij
νij
, (27)
where νij is the actual QoS value of web service item j
invoked by service user i, νij is the prediction value
of each method, and P is the number of the predicted
QoS values.
C. PERFORMANCE COMPARISON
In this section, the above metrics are compared between
our proposed approach and the following popular prediction
methods.
TABLE 3. Performance comparison.
FIGURE 5. MAE of response time prediction.
FIGURE 6. MAPE of response time prediction.
• Arithmetic average value method (AVG). This method
is defined as
average =
N
i=1
νij
Nexist
(28)
VOLUME 3, 2015 2265

FIGURE 7. Experimental result with 70% matrix density.
where νij is the actual QoS response time value that the
user i invokes the web service j, N is the number of all
users, Nexist is the number of the remaining users in the
matrix who invoke web service j.
• User-based collaborative filtering method using Pearson
correlation coefficient (UPCC). This method predicts
the missing QoS data of users based on the collected
information from similar users [15], [17].
• Item-based collaborative filtering method using Pearson
correlation coefficient (IPCC). This method predicts the
missing QoS data of users based on the collected infor-
mation from similar service items [14].
• UIPCC. This method combines UPCC with IPCC for
missing QoS data prediction [16].
In our experiments, the number of the fuzzy rules in both
the critic and the action networks are set to Nfc = Nfa = 3.
According to Theorem 2, the discount factor is set to
α = 0.95 ∈ (
√
2
3 , 1), and the learning rates of the critic and
the action networks should satisfy lc < 1
0.952×3
≈ 0.369 and
la < 1
3 ≈ 0.333, respectively. Then we choose the learning
rate as lc = la = 0.3. In our experiment, we find that this
common fixed learning rate will result in slow convergence
rate of ADP. So we use the learning rates lc = la = 0.3 in
the initial training phase of NN. And they should decrease
to a value during the learning process. Here, the learning
rates are decreased by 0.05 every 50 times until they
reach 0.005.
In order to compare the performance of each approach,
we set different sparsity to randomly remove some entries
from the matrix. For instance, 10% means that 10% entries
are removed randomly, and the remaining entries are used to
predict the missing entries.
The experimental results are show in Table 3. From this
table, we can find that our proposed approach has better
performance (smaller MAE and MAPE values) than other
methods with different matrix densities, especially with high
matrix density. Figs. 5 and 6 show the experimental results
of response time prediction. From those figures, we conclude
that the performance of our proposed approach is stable as
the performance of other methods becomes worse with the
increase of the matrix density. An experimental result with
70% matrix density is shown in Fig. 7, which shows the
differences of the performance in reflecting the trend of
QoS data among these methods. Due to its potential scal-
ability of the adaptive critic designs and the intuitiveness
of Q-learning in ADP as well as strong adaptability and
flexibility of FNN in context-aware environment, our pro-
posed scheme has its unique features in dealing with those
QoS data. Compared with other methods, the improvement
of our proposed approach verifies that the idea involved
in the fusion of ADP and FNN for QoS prediction is
effective.
VI. CONCLUSION
QoS evaluates the web services while playing an important
role in web service selection and service recommendation
for cloud applications. In this paper, we present a novel
QoS prediction scheme through the fusion of FNN and ADP.
Considering the nonlinear and dynamic property of QoS data,
we construct FNN in the critic as well as the action network
under ADP architecture, and develop the weight updating
method to adjust the parameters of the networks with the
purpose of optimizing a set of fuzzy if-then rules. In addition,
we provide the stability analysis for our proposed approach
to guarantee the convergence of scheme. These conver-
gence results are better than the previous one, which provide
a guideline for the selection of parameters used in our
FNN-based ADP scheme. Experiments are conducted on
a large-scale real-world QoS service data set to predict
the response time in QoS. Compared with other traditional
methods, the experimental results shows the proposed
approach improves the accuracy of prediction for response
time in QoS.
2266 VOLUME 3, 2015

APPENDIX A
PROOF OF THEOREM 1
Let ζc(t) and ζa(t) be the approximation errors of the critic
and the action network outputs, respectively. Then,
ζc(t) = (wc(t) − w∗
c)T
φc(t) = wT
c (t)φc(t), (A.29)
ζa(t) = (wa(t) − w∗
a)T
φa(t) = wT
a (t)φa(t). (A.30)
Select the Lyapunov function as:
V(t) = V1(t) + V2(t) + V3(t), (A.31)
where
V1(t) =
1
lc
tr wT
c (t)wc(t) , (A.32)
V2(t) =
1
γ la
tr wT
a (t)wa(t) , (A.33)
V3(t) =
2
9
ζc(t − 1) 2
. (A.34)
where γ > 0 is a weighting factor.
The first difference of V1(t) is given by:
V1(t) = −α2
ζc(t) 2
− α2
1 − lcα2
φc(t) 2
× ζc(t) + (w∗
c)T
φc(t) +
1
α
r(t)
−
1
α
wT
c (t − 1)φc(t − 1)
2
+ α(w∗
c)T
φc(t) + r(t) − wT
c (t − 1)φc(t − 1)
2
.
(A.35)
By applying the Cauchy-Schwarz inequality for (A.35),
we have
V1(t) −α2
ζc(t) 2
− α2
1 − lcα2
φc(t) 2
× ζc(t) + (w∗
c)T
φc(t) +
1
α
r(t)
−
1
α
wT
c (t − 1)φc(t − 1)
2
+ 2 α(w∗
c)T
φc(t) + r(t) −
2
3
wT
c (t − 1)φc(t − 1)
−
1
3
(w∗
c)T
φc(t − 1)
2
+
2
9
ζc(t − 1) 2
. (A.36)
V2(t) =
1
γ
− wT
c (t)C(t) 2
− la wT
c (t)C(t) 2
φa(t) 2
× wT
c (t)φc(t) 2
− wT
c (t)C(t) 2
ζa(t) 2
+ wT
c (t)φc(t) − wT
c (t)C(t)ζa(t) 2
. (A.37)
By applying the Cauchy-Schwarz inequality for (A.37), we
have
V2(t)
1
γ
− 1 − la φa(t) 2
wT
c (t)C(t) 2
wT
c φc(t) 2
+ 4 (w∗
c)T
φc(t) 2
+ wT
c (t)C(t) 2
ζa(t) 2
+ 4 ζc(t) 2
. (A.38)
V3(t) =
2
9
ζc(t) 2
− ζc(t − 1) 2
. (A.39)
Substituting (A.36), (A.38), and (A.39) into the first differ-
ence of V(t), we have
V(t) = V1(t) + V2(t) + V3(t)
− α2
−
2
9
−
4
γ
ζc(t) 2
− α2
1 −lcα2
φc(t) 2
× ζc(t) + (w∗
c)T
φc(t) +
1
α
r(t)
−
1
α
wT
c (t − 1)φc(t − 1)
2
−
1
γ
(1 − la φa(t) 2
) wT
c (t)C(t) 2
wT
c (t)φc(t) 2
+ Z2
(t), (A.40)
where
Z2
(t) = 2 α(w∗
c)T
φc(t) + r(t) −
2
3
wT
c (t − 1)φc(t − 1)
−
1
3
(w∗
c)T
φc(t − 1)
2
+
1
γ
wT
c (t)C(t) 2
ζa(t) 2
+
4
γ
(w∗
c)T
φc(t) 2
. (A.41)
Applying the Cauchy-Schwarz inequality for (A.41) and
using Assumption 1, we have
Z2
(t) 8 α2
(w∗
c)T
φc(t) 2
+ r2
(t)
+
4
9
wT
c (t − 1)φc(t − 1) 2
+
1
9
(w∗
c)T
φc(t − 1) 2
+
2
γ
wT
c (t)C(t) 2
wT
a (t)φa(t) 2
+ (w∗
a)T
φa(t) 2
+
4
γ
(w∗
c)T
φc(t) 2
8α2
+
40
9
+
4
γ
w2
cmφ2
cm
+
4
γ
w2
cmC2
mw2
amφ2
am + 8r2
m = Z2
m, (A.42)
where wcm, wam, φcm, φam, Cm, and rm are the upper bounds
of w∗
c, w∗
a, φc(t), φa(t), C(t), and r(t), respectively.
According to (A.40), choose
√
2
3
< α < 1, 0 < lc <
1
α2 φc(t) 2
, 0 < la <
1
φa(t) 2
,
(A.43)
and select γ satisfying
γ >
4
α2 − 2
9
, (A.44)
then for any
ζc(t) >
Zm
α2 − 2
9 − 4
γ
, (A.45)
the first difference V(t) 0 holds.
Using the Lyapunov theorem, this implies that the
NN weight estimation errors wc(t) and wa(t) are UUB.
VOLUME 3, 2015 2267

APPENDIX B
PROOF OF THEOREM 2
According to the definition of function in (11), each
element of activation function vectors φc(t) and φa(t),
i.e., qc1 (t), · · · , qcNfc
(t), qa1 (t), · · · , qaNfa
(t), is bounded
within (0, 1]. Then, we have
0 < lc <
1
α2Nfc
1
α2 Nfc
k=1(qck (t))2
=
1
α2 φc(t) 2
,
(B.46)
0 < la <
1
Nfa
1
Nfa
k=1(qak (t))2
=
1
φa(t) 2
. (B.47)
According to the judging conditions (24) in Theorem 1,
the estimate errors of network weights wc(t) and wa(t)
are UUB.
REFERENCES
[1] T. H. Noor, Q. Z. Sheng, A. H. H. Ngu, and S. Dustdar, ‘‘Analysis of Web-
scale cloud services,’’ IEEE Internet Comput., vol. 18, no. 4, pp. 55–61,
Jul./Aug. 2014.
[2] V. Chang, ‘‘The business intelligence as a service in the cloud,’’ Future
Generat. Comput. Syst., vol. 37, pp. 512–534, Jul. 2014.
[3] C. Gravier, J. Subercaze, A. Najjar, F. Laforest, X. Serpaggi, and
O. Boissier, ‘‘Context awareness as a service for cloud resource optimiza-
tion,’’ IEEE Internet Comput., vol. 19, no. 1, pp. 28–34, Jan./Feb. 2015.
[4] R. Ranjan, J. Kolodziej, L. Wang, and A. Y. Zomaya, ‘‘Cross-layer cloud
resource configuration selection in the big data era,’’ IEEE Cloud Comput.,
vol. 2, no. 3, pp. 16–22, May/Jun. 2015.
[5] D. A. Menascé, ‘‘QoS issues in Web services,’’ IEEE Trans. Internet
Comput., vol. 6, no. 6, pp. 72–75, Nov./Dec. 2002.
[6] D. A. Menascé, ‘‘Composing Web services: A QoS view,’’ IEEE Trans.
Internet Comput., vol. 8, no. 6, pp. 88–90, Nov./Dec. 2004.
[7] S. Misra, P. V. Krishna, K. Kalaiselvan, V. Saritha, and M. S. Obaidat,
‘‘Learning automata-based QoS framework for cloud IaaS,’’ IEEE Trans.
Netw. Serv. Manage., vol. 11, no. 1, pp. 15–24, Mar. 2014.
[8] C. Mao, J. Chen, D. Towey, J. Chen, and X. Xie, ‘‘Search-based QoS rank-
ing prediction for Web services in cloud environments,’’ Future Generat.
Comput. Syst., vol. 50, pp. 111–126, May 2014.
[9] H. Zheng, W. Zhao, J. Yang, and A. Bouguettaya, ‘‘QoS analysis for
Web service compositions with complex structures,’’ IEEE Trans. Services
Comput., vol. 6, no. 3, pp. 373–386, Jul./Sep. 2013.
[10] X. Chen, Z. Zheng, Q. Yu, and M. R. Lyu, ‘‘Web service recommenda-
tion via exploiting location and QoS information,’’ IEEE Trans. Parallel
Distrib. Syst., vol. 25, no. 7, pp. 1913–1924, Jul. 2014.
[11] S.-Y. Hwang, C.-C. Hsu, and C.-H. Lee, ‘‘Service selection for Web
services with probabilistic QoS,’’ IEEE Trans. Services Comput., vol. 8,
no. 3, pp. 467–480, May/Jun. 2015.
[12] D. Ardagna, G. Casale, M. Ciavotta, J. F. Pérez, and W. Wang, ‘‘Quality-of-
service in cloud computing: Modeling techniques and their applications,’’
J. Internet Services Appl., vol. 5, no. 1, p. 11, 2014.
[13] A. Metzger, C.-H. Chi, Y. Engel, and A. Marconi, ‘‘Research challenges
on online service quality prediction for proactive adaptation,’’ in Proc.
Workshop Eur. Softw. Services Syst. Res.-Results Challenges (S-Cube),
Zürich, Switzerland, 2012, pp. 51–57.
[14] P. Resnick, N. Iacovou, M. Suchak, P. Bergstrom, and J. Riedl,
‘‘GroupLens: An open architecture for collaborative filtering of netnews,’’
in Proc. ACM Conf. Comput. Supported Cooperat. Work, New York, NY,
USA, 1994, pp. 175–186.
[15] J. S. Breese, D. Heckerman, and C. Kadie, ‘‘Empirical analysis of predic-
tive algorithms for collaborative filtering,’’ in Proc. 14th Conf. Uncertainty
Artif. Intell., San Francisco, CA, USA, 1998, pp. 43–52.
[16] H. Ma, I. King, and M. R. Lyu, ‘‘Effective missing data prediction for
collaborative filtering,’’ in Proc. 30th Annu. Int. ACM SIGIR Conf. Res.
Develop. Inf. Retr., New York, NY, USA, 2007, pp. 39–46.
[17] L. Shao, J. Zhang, Y. Wei, J. Zhao, B. Xie, and H. Mei, ‘‘Personalized QoS
prediction for Web services via collaborative filtering,’’ in Proc. IEEE Int.
Conf. Web Services, Salt Lake City, UT, USA, Jul. 2007, pp. 439–446.
[18] Z. Zheng, H. Ma, M. R. Lyu, and I. King, ‘‘Collaborative Web service QoS
prediction via neighborhood integrated matrix factorization,’’ IEEE Trans.
Services Comput., vol. 6, no. 3, pp. 289–299, Jul./Sep. 2013.
[19] L. Qi, J. Ni, X. Xia, H. Wang, and C. Yan, ‘‘A multi-dimensional weight-
ing method for historical records in cloud service evaluation,’’ in Proc.
IEEE 4th Int. Conf. Big Data Cloud Comput., Sydney, NSW, Australia,
Dec. 2014, pp. 265–266.
[20] J. J. Murray, C. J. Cox, G. G. Lendaris, and R. Saeks, ‘‘Adaptive dynamic
programming,’’ IEEE Trans. Syst., Man, Cybern. C, Appl. Rev., vol. 32,
no. 2, pp. 140–153, May 2002.
[21] J. Si, A. G. Barto, W. B. Powell, and D. Wunsch, Eds., Handbook of
Learning and Approximate Dynamic Programming. Hoboken, NJ, USA:
Wiley, 2004.
[22] S. N. Balakrishnan and V. Biega, ‘‘Adaptive-critic-based neural networks
for aircraft optimal control’’ J. Guid., Control, Dyn., vol. 19, no. 4,
pp. 893–898, Jul./Aug. 1996.
[23] Y. Tang, H. He, J. Wen, and J. Liu, ‘‘Power system stability control for a
wind farm based on adaptive dynamic programming,’’ IEEE Trans. Smart
Grid, vol. 6, no. 1, pp. 166–177, Jan. 2015.
[24] Q. Wei, D. Liu, G. Shi, and Y. Liu, ‘‘Multibattery optimal coordination
control for home energy management systems via distributed iterative
adaptive dynamic programming,’’ IEEE Trans. Ind. Electron., vol. 62,
no. 7, pp. 4203–4214, Jul. 2015.
[25] M. Fairbank, D. Prokhorov, and E. Alonso, ‘‘Clipping in neurocontrol by
adaptive dynamic programming,’’ IEEE Trans. Neural Netw. Learn. Syst.,
vol. 25, no. 10, pp. 1909–1920, Oct. 2014.
[26] J. Wang and R. Y. K. Fung, ‘‘Adaptive dynamic programming algorithms
for sequential appointment scheduling with patient preferences,’’ Artif.
Intell. Med., vol. 63, no. 1, pp. 33–40, Jan. 2015.
[27] F. Xia, W. Zhao, Y. Sun, and Y.-C. Tian, ‘‘Fuzzy logic control based QoS
management in wireless sensor/actuator networks,’’ Sensors, vol. 7, no. 12,
pp. 3179–3191, 2007.
[28] M. Zhang, X. Liu, R. Zhang, and H. Sun, ‘‘A Web service recommenda-
tion approach based on QoS prediction using fuzzy clustering,’’ in Proc.
IEEE 9th Int. Conf. Services Comput., Honolulu, HI, USA, Jun. 2012,
pp. 138–145.
[29] S.-I. Horikawa, T. Furuhashi, and Y. Uchikawa, ‘‘On fuzzy modeling using
fuzzy neural networks with the back-propagation algorithm,’’ IEEE Trans.
Neural Netw., vol. 3, no. 5, pp. 801–806, Sep. 1992.
[30] X. Luo, Y. Chen, J. Si, and F. Liu, ‘‘Longitudinal control of
hypersonic vehicles based on direct heuristic dynamic programming
using ANFIS,’’ in Proc. Int. Joint Conf. Neural Netw., Beijing, China, 2014,
pp. 3685–3692.
[31] J.-S. R. Jang, ‘‘ANFIS: Adaptive-network-based fuzzy inference sys-
tem,’’ IEEE Trans. Syst., Man, Cybern., vol. 23, no. 3, pp. 665–685,
May/Jun. 1993.
[32] D. V. Prokhorov and D. C. Wunsch, ‘‘Adaptive critic designs,’’ IEEE Trans.
Neural Netw., vol. 8, no. 5, pp. 997–1007, Sep. 1997.
[33] F. Liu, J. Sun, J. Si, W. Guo, and S. Mei, ‘‘A boundedness result
for the direct heuristic dynamic programming,’’ Neural Netw., vol. 32,
pp. 229–235, Aug. 2012.
[34] Z. Zheng, Y. Zhang, and M. R. Lyu, ‘‘Distributed QoS evaluation for real-
world Web services,’’ in Proc. IEEE Int. Conf. Web Services, Miami, FL,
USA, Jul. 2010, pp. 83–90.
[35] Y. Zhang, Z. Zheng, and M. R. Lyu, ‘‘Exploring latent features for memory-
based QoS prediction in cloud computing,’’ in Proc. 30th IEEE Symp. Rel.
Distrib. Syst., Madrid, Spain, Oct. 2011, pp. 1–10.
XIONG LUO received the Ph.D. degree from
Central South University, China, in 2004. From
2012 to 2013, he was with the School of Electrical,
Computer and Energy Engineering, Arizona State
University, USA, as a Visiting Scholar. He is cur-
rently an Associate Professor with the Department
of Computer Science and Technology, School of
Computer and Communication Engineering, Uni-
versity of Science and Technology Beijing, China.
His current research interests include machine
learning, cloud computing, cyber-physical systems, and computational
intelligence.
2268 VOLUME 3, 2015

YIXUAN LV is currently pursuing the master’s
degree with the Department of Computer Science
and Technology, School of Computer and Commu-
nication Engineering, University of Science and
Technology Beijing, China. His research inter-
ests include cyber-physical systems and adaptive
dynamic programming.
RUIXING LI is currently pursuing the master’s
degree with the Department of Computer Science
and Technology, School of Computer and Commu-
nication Engineering, University of Science and
Technology Beijing, China. Her research interests
include cloud computing and computational
intelligence.
YI CHEN received the master’s degree from the
University of Science and Technology Beijing,
China, in 2014. Her research interests include
fuzzy learning and cyber-physical systems.
VOLUME 3, 2015 2269

Web Service QoS Prediction Based on Adaptive Dynamic Programming Using Fuzzy Neural Networks for Cloud Services

Recommended

More Related Content

What's hot (19)

Similar to Web Service QoS Prediction Based on Adaptive Dynamic Programming Using Fuzzy Neural Networks for Cloud Services (20)

More from redpel dot com (20)

Recently uploaded (20)

Web Service QoS Prediction Based on Adaptive Dynamic Programming Using Fuzzy Neural Networks for Cloud Services