![International Journal Of Computational Engineering Research (ijceronline.com) Vol. 3Issue. 3
181
||Issn||2250-3005|| (Online) ||March||2013|| ||www.ijceronline.com||
Mining Association Rules From Time Series Data Using Hybrid
Approaches
Hima Suresh1,
Dr. Kumudha Raimond2
1 PG Scholar, Karunya University,
2 Professor, Karunya University
Abstract:
Due to the frequent appearance of time series data in various fields, it has always been an essential and
interesting research field. A time series analysis involves the methods for analyzing time series data, in order to
mine meaningful and other relevant characteristics of the data. In most cases, time series data are quantitative
values, so to come up with an intellectually appealing data mining algorithm to deal with quantitative type of
data presents a biggest challenge to researchers in this field. In this paper, an extended Fuzzy Frequent Pattern
(FP) growth approach is proposed and analyzed against the existing approach called Fuzzy Apriori (FA).
Keywords: Conditional Pattern Base, Fuzzy Apriori, Frequent Pattern, Fuzzy Logic, Genetic Algorithm ,
Neural Network and Frequent Pattern tree.
1. Introduction
Time series is defined as a series of well defined data values of a variable at successive time interval.
The applications of time series includes FP analysis, predictions etc. FP analysis in time series data has become
one of the most vital parts of data mining tasks and has attracted extreme interest among the researchers.Many
approaches are available to examine time series data such as the Genetic Algorithm (GA), statistical methods
etc. But the outputs of these approaches are difficult to understand/interpret the quantitative results. Apriori is
the classic algorithm of association rules which determine the number of frequent items proposed by Agrawal et.
al [6]. But uncertain time series are difficult to be dealt with this approach. So the fuzzy sets are used, in order to
handle uncertain time series data. Fuzzy sets are the sets containing degrees of membership. Fuzzy set theory
was introduced by L.A.Zadeh et. al [1] and this approach was followed by Hong et. al [2], [3], [4] and proposed
several fuzzy algorithms to extract quantitative data and to mine meaningful association rules. The fuzzy mining
algorithm integrates the concept of Apriori algorithm as well as fuzzy concepts.
However, in the Apriori algorithm, candidate generation suffers from significant costs such as: it may
need to generate a huge number of candidate sets as well as it needs to scan the database repeatedly and to check
a large set of candidates using pattern matching. This would be a tedious workload to go over each transaction
in the database to determine the support value of candidate item sets. The approach to mine complete set of
frequent item sets without candidate generation is one of the important motivations and also to generate efficient
and scalable method for mining both long and short frequent patterns. An approach based on FP-Growth to find
fuzzy association rules is proposed by Papadimitriou et. al [5]. In this approach each value in the transactions
are transferred into two corresponding fuzzy regions. A support value is set and those fuzzy regions in the
transaction which doesn’t meet or exceed the predefined support would be removed. Here only the frequent
fuzzy 1-itemsets obtained from each transaction are used for extracting. The expression of the fuzzy items with
more fuzzy regions is successive. No fuzzy operation is used to combine the regions together. However, this
made the rules mined to be difficult to interpret. The proposed extended Fuzzy FP growth approach overcomes
this limitation to a certain extend. In this paper, an extended FP growth approach has been proposed and
evaluated against the existing approach FA. It is an interesting method used for mining frequent item sets. The
remainder of this paper is organized as follow: Section 2 describes the related works. Section 3 presents the
hybrid approach using FA technique. Section 4 presents fuzzy FP-growth approach. Section 5 describes the
experimental results. Finally, conclusion is discussed in Section 6.
2. Related Works
In this section some related concepts and algorithms are discussed such as: Apriori algorithm, FP-
growth algorithm and Fuzzy set theory.
A. Apriori algorithm
The classic algorithm for mining frequent item sets and for learning association rule over the
transactional database was proposed as Apriori algorithm by Agrawal et. al [6]. The algorithm involves two
steps. Firstly, it find out the frequent item sets which exceeds or satisfies the minimum support. Secondly, the](https://image.slidesharecdn.com/ad03301810188-130511020215-phpapp02/85/Ad03301810188-1-320.jpg)
![Mining Association Rules From Time Series Data…
182
||Issn||2250-3005|| (Online) ||March||2013|| ||www.ijceronline.com||
rules are generated which satisfies the minimum confidence value. In this algorithmic method, the database is
scanned repeatedly in order to generate all frequent item sets. It would be costly to go over each transaction and
to count the support for each candidate item set. Also this approach generates huge number of candidate sets.
B. Frequent Pattern –Growth algorithm
One of the interesting methods to extract frequent item sets is FP-Growth algorithmic method and this
is proposed by Han et. al [3]. The proposed algorithm does not require the candidate generations. The extraction
of frequent patterns using Apriori is not efficient since it requires the repeated scanning of database and may
need to generate large candidate item sets. In order to overcome this, an extended FP-Growth algorithm is used.
The Approach involves several steps: First it scans the database to obtain all items with their corresponding
support count. The items which exceed or satisfy the predefined minimum support are selected as the frequent
1-itemsets. Next step is to arrange the frequent 1-itemsets according to the decreasing order of their support
count. Finally, the database is scanned again to construct the FP tree. All transactions are processed bottom up
one by one and after processing all the transactions, a complete FP tree is constructed. Once the tree is
constructed, FP-Growth procedure is executed to obtain all frequent item sets. FP growth generates the frequent
patterns directly from the FP tree instead of candidate generation (C.W. Lin et.al [8]). For each frequent item, a
conditional FP tree would be generated and the corresponding frequent item sets are derived from the same.
C. Fuzzy set theory
The word fuzzy means vagueness. Fuzzy sets have been introduced by Lotif A. Zadeh [1]. It is actually
an extension of the classic set theory which defines the set membership as possibility distribution.
The rule for the fuzzy set theory can be expressed as:
] X
[0, 1] (1)
Here “x” number of possibilities may occur.
A logic based on the truth values such as “true” and “false” would be sometimes inadequate while
describing the human reasoning. Fuzzy logic uses the entire interval between 0 (false) and 1(true) for describing
human reasoning.
3. Fuzzy Apriori Approach
The FA approach integrates the concept of Fuzzy as well as Apriori algorithm. The architecture for the
FA approach implemented by C.H.Chen.et al [9] is shown in fig 1.
Fig 1: Architectural diagram of FA approach
The architecture describes the procedures for generating rules using FA algorithm. First, the time series
data has been taken as input and from the given time series data, the subsequences of any length according to the
sliding window size are generated. After the subsequence generation, each data point of the subsequences is
transformed into fuzzy set. The total count or scalar cardinality of the corresponding fuzzy set based on the
membership function is obtained. Frequent candidate sets are generated based on this support value which
exceeds or satisfies the predefined support value. Pruning processes has been applied to filter away irrelevant
large candidate sets. Finally the association rules are generated.
4. Fuzzy Frequent Pattern Growth Approach
The proposed approach uses the concept of fuzzy set and FP-growth approach to generate the frequent
item sets without any candidate generation. The architecture for the proposed fuzzy FP-growth approach for
time series data is shown in fig 2.
Fig 2: Architectural diagram of Fuzzy FP-growth approach
Generate sub
sequence
Input time series Generate fuzzy
sets
Calculates scalar
cardinality
Removal of
redundant items
Generates
association rules
Fuzzy set
generation
Scan the DB to
generate 1-itemset
Construct the FP
tree
Generate conditional
Pattern base
Generate fuzzy
association rules](https://image.slidesharecdn.com/ad03301810188-130511020215-phpapp02/85/Ad03301810188-2-320.jpg)
![Mining Association Rules From Time Series Data…
183
||Issn||2250-3005|| (Online) ||March||2013|| ||www.ijceronline.com||
The approach uses the transformed fuzzy set for generating frequent patterns. It involves different
phases. First step is to construct FP-tree from a database and from the resulting FP-tree, frequent patterns are
generated. FP tree construction involves several steps such as: Scanning the databases to generate 1-itemset then
rescanning to establish FP tree. Next generating conditional pattern base and corresponding conditional FP tree
respectively. Finally fuzzy association rules are generated as proposed by C.W. Lin et. al [8].
5. Experimental Analysis
The Hybrid approach using both fuzzy concept and apriori algorithm is an efficient method for
handling time series data to find linguistic association rules proposed by C.H. Chen et. al [9], A.M Palacious et.
al [10]. Home price time series data over the years from 1999 to 2012 period has been used for analyzing the
performance of the existing FA approach against the proposed Fuzzy FP growth approach [16]. The home price
time series used for analyzing these two hybrid approaches are shown in table1.
Table 1: Home price time series data used as input
Years Home price Years Home price
1999 127 2006 124
2000 129 2007 118
2001 132 2008 121
2002 130 2009 120
2003 126 2010 115
2004 132 2011 113
2005 129 2012 119
A. FA Method Analysis
The time series data is first transformed into subsequences according to the predefined window size,
which could be of any length. Here in this example, window size is set as 5. So the total subsequence
transactions will be 10 (T1, T2….T10) according to the formula (total length of time series – window size +
1). The transactions of subsequences generated are shown in table 2.
Table 2: Transaction of subsequences
TID Transaction TID Transaction
T1 127,129,132,130,126 T6 132,129,124,118,121
T2 129,132,130,126,132 T7 129,124,118,121,120
T3 132,130,126,132,129 T8 124,118,121,120,115
T4 130,126,132,129,124 T9 118,121,120,115,113
T5 126,132,129,124,118 T10 121,120,115,113,119
After transforming the time series data into subsequences, the data points in each subsequence are
converted to fuzzy values according to predefined membership function. The membership values used are Low,
Middle and High and is used for the transformation of fuzzy sets from the given time series. The representation
of membership function is given in fig 3.
113 118 122 128 132
Fig 3: Membership function
The fuzzy values of each subsequence are shown in table 3. Each data point (A1, A2 …An) in the
transaction each subsequence will take the value of low (L), middle (M) and high (H).](https://image.slidesharecdn.com/ad03301810188-130511020215-phpapp02/85/Ad03301810188-3-320.jpg)
![Mining Association Rules From Time Series Data…
184
||Issn||2250-3005|| (Online) ||March||2013|| ||www.ijceronline.com||
Table 3: Fuzzy set transformed from time series
The total count or scalar cardinality has been calculated from the fuzzy set generated and is calculated
as its count value. After the calculation, fuzzy items generated are collected as the 1-itemset [13], [14], [15]. The
total count is then checked against the support value, which is predefined. Here in this example, support value is
set as 30%. The item set whose values are greater than or equal to 30% has been taken and are the following:
A1.Middle, A2.Middle, A3.Low, A3.Middle, A4.Low, A4.Middle, A5.Low, and A5.Middle. The 1-itemsets are
represented in table 4:
Table 4: The 1-itemset generated with total count
1-Itemset Total 1-Itemset Total 1-Itemset Total
A1.Middle 0.467 A2.Middle 0.4 A3.Low 0.334
A3.Middle 0.333 A4.Low 0.434 A4.Middle 0.333
A5.Low 0.501 A5.Middle 0.366 - -
After the generation of 1-itemsets, corresponding 2-itemsets are obtained. The fuzzy items with the
same attribute Ai are not put in to 2-itemset collection. The resulting 2-itemsets generated are shown in the
table 5.
Table 5: The resulting 2-itemset satisfy the support
2-itemset Total
A1.Middle ∩ A5.Low 3
A3.Low ∩ A4.Low 3.01
A4.Low ∩ A5.Low 3.68
The association rules thus generated from the 2-itemsets with corresponding confidence values are
shown in table 6:
Table 6: The Association rules with confidence values
Final Rule Confidence
A1.Middle,A5.Low 0.64
A3.Low, A4.Low 0.9
A4.Low, A5.Low 0.84
T
A1
L
A1
M
A1
H
A2
L
A2
M
A2
H
A3
L
A3
M
A3
H
A4
L
A4
M
A4
H
A5
L
A5
M
A5
H
T1 0 1 0 0
0.6
7
0.3
3
0 0 1 0 0 0 0 1 0
T2 0
0.6
7
0.3
3
0 0 1 0 0 0 0 1 0 0 0 1
T3 0 0 1 0 0 0 0 1 0 0 0 1 0
0.6
7
0.
33
T4 0 0 0 0 1 0 0 0 1 0
0.6
7
0.3
3
0 1 0
T5 0 1 0 0 0 1 0
0.6
7
0.3
3
0 1 0 1 0 0
T6 0 0 1 0
0.6
7
0.3
3
0 1 0 1 0 0
0.6
7
0.3
3
0
T7 0
0.6
7
0.3
3
0 1 0 1 0 0
0.6
7
0.3
3
0
0.6
7
0.3
3
0
T8 0 1 0 1 0 0
0.6
7
0.3
3
0
0.6
7
0.3
3
0 1 0 0
T9 1 0 0
0.6
7
0.3
3
0
0.6
7
0.3
3
0 1 0 0 1 0 0
T10
0.6
7
0.3
3
0
0.6
7
0.3
3
0 1 0 0 1 0 0
0.6
7
0.3
3
0
Total
1.6
7
4.6
7
2.6
6
2.3
4
4
2.6
6
3.3
4
3.3
3
2.3
3
4.3
4
3.3
3
1.3
3
5.0
1
3.6
6
1.
33](https://image.slidesharecdn.com/ad03301810188-130511020215-phpapp02/85/Ad03301810188-4-320.jpg)
![Mining Association Rules From Time Series Data…
186
||Issn||2250-3005|| (Online) ||March||2013|| ||www.ijceronline.com||
The fuzzy region with the maximum count among the three possible regions for each item is selected.
Take A1 as an example. Its count is 1.67 for low, 4.67 for middle and 2.66 for high. Since the count for middle
is the maximum among three counts, the region middle is thus used to represent item A1 in the later mining
process. This step would be repeated for the other items. The count of any region selected is checked against
with the predefined minimum support value. Here support value assumed as 30%. That means (10*30%=3.0).
The table 10 shows the set of frequent fuzzy region. Here every region satisfies the support 30%.
Table10: Frequent Fuzzy regions and count
Frequent fuzzy regions Count
A1. Middle 4.67
A2.Middle 4
A3.Low 3.34
A4.Low 4.34
A5.Low 5.01
After finding the regions with higher counts, the header table is arranged based on the decreasing order of the
support count. It is shown in table 11:
Table 11: The header table of the regions
Frequent fuzzy regions Count
A5.Low 5.01
A1.Middle 4.67
A4.Low 4.34
A2.Middle 4
A3.Low 3.34
The fuzzy regions which are not in the header table are removed from each transaction [11]. The
remaining fuzzy regions in each transaction are then sorted according to the membership values in a descending
order. This is shown in table 12.
Table 12: The transactions transformed with the fuzzy regions
Tid Frequent fuzzy region
T1 (1/A1Middle)(0.67/A2Middle)
T2 (0.67/A1Middle)
T3 0
T4 (1/A2Middle)
T5 (1/A1Middle)(1/A5Low)
T6 (0.67/A2Middle)(1/A4Low)(0.67/A5Low)
T7 (0.67/A1Middle)(1/A2Middle)(0.67/A4Low)(0.67/A5Low)
T8 (1/A1Middle)(0.67/A4Low)(1/A5Low)
T9 (0.33/A2Middle)(1/A4Low)(1/A5Low)
T10 (0.33/A1Middle)(0.33/A2Middle)(1/A4Low)(0.67/A5Low)
After this step, the transactions with only the sorted frequent fuzzy regions selected which are shown
in table 13:
Table 13: The transactions with sorted the fuzzy regions
TID Frequent fuzzy region
T1 (1/A1Middle)(0.67/A2Middle)
T2 (0.67/A1Middle)
T3 0
T4 (1/A2Middle)
T5 (1/A1Middle)(1/A5Low)
T6 (1/A4Low)(0.67/A2Middle)(0.67/A5Low)
T7 (1/A2Middle)(1/A3Low)(0.67/A1Middle)(0.67/A4Low)(0.67/A5Low)
T8 (1/A1Middle)(1/A5Low)(0.67/A3Low)(0.67/A4Low)
T9 (1/A4Low)(1/A5Low)(0.33/A2Middle)(0.67/A3Low)
T10 (1/A4Low)(1/A3Low)(0.67/A5Low)(0.33/A1Middle)(0.33/A2Middle)](https://image.slidesharecdn.com/ad03301810188-130511020215-phpapp02/85/Ad03301810188-6-320.jpg)
![Mining Association Rules From Time Series Data…
187
||Issn||2250-3005|| (Online) ||March||2013|| ||www.ijceronline.com||
All transactions up to 10 are inserted in the fuzzy FP tree [12]. In the FP tree representation, header table is
shown in the left side and it contains the frequent Fuzzy Regions (FR) and their supporting count. The final
fuzzy FP tree is shown in figure 4.
Fig 4: The Complete FP tree
While processing the items from bottom of the header table, first item would be A3L: the traversal from
which A3L exist are:
1) (A4L:3)(A3L:1)
2) (A4L:3)(A5L:1)(A2M:0.33)(A3L:0.67)
3) (A1M:3.67)(A5L:2)(A3L:0.67)
4) (A2M:2)(A3L:1)
Table 14: The frequent item sets with A3L
1-Item 2-Item 3-Item
A5L:5.01 A4L, A3L:1.67 A1M, A5L, A3L:0.67
A1M:4.67 A5L, A3L:1.34 A4L, A5L, A3L:0.67
A4L:4.34 A2M, A3L:1.33 A5L, A2M, A3L:0.33
A2M:4 A1M, A3L:0.67 -
A3L:3.34 - -
Here in this example taking the intersection (minimum) of items to obtain 2-itemset and 3-itemsets.
For eg: A4L:3 and A3L:1 means1 (taking the smaller value or minimum operator) and then if the same
transaction occurs again, it would be incremented with the corresponding value. Likewise all regions in the
header table should be processed and frequent item sets are generated. On processing the items bottom up from
the header table, corresponding item sets are obtained. Same item sets from all transactions are taken together
and their count is evaluated using the intersection (minimum operator). From the corresponding item sets of all
transactions, those exceed or meet the support count are chosen as the rules. The rules obtained are shown in
table 15 and are similar to the rules generated by FA approach.
Table 15: The final rules generated
Final Rules Count
A1.Middle,A5.Low 3
A3.Low, A4.Low 3
A4.Low, A5.Low 3.01
The two hybrid approaches are implemented in VB.NET at a personal computer with Intel Core i3,
2.10 GHZ and 4GB RAM. The data set used is the home price time series data over the period of 1999 to 2012.
The sliding window size is predefined as 5. The experimental result shows the FA approach require more
number of scans for each transaction compared to the proposed approach. In the case of FA approach, for 20
support value it requires 1001 total number of scans. Likewise for 25 support value, it requires a total scan of
539 and for 30, it requires 275 total scans. Whereas FP approach requires only 11 times total number of scans](https://image.slidesharecdn.com/ad03301810188-130511020215-phpapp02/85/Ad03301810188-7-320.jpg)
![Mining Association Rules From Time Series Data…
189
||Issn||2250-3005|| (Online) ||March||2013|| ||www.ijceronline.com||
References
[1] L.A. Zadeh, Fuzzy Sets. Information and Control, 8:338- 353, 1965.
[2] T.P. Hong, C.S. Kuo, S.C. Chi, Mining association rules from quantitative data. Intelligent Data
Analysis, no.3 (5), pages338- 353, 1965.
[3] T.P. Hong, C.S. Kuo, S.C. Chi, Trade-off between Time Complexity and Number of Rules for Fuzzy
Mining from Quantitative Data, International Journal of Uncertainty: Fuzziness and Knowledge-based
Systems, 9 (5), pages338- 353, 2001.
[4] Y.C. Lee, T.P. Hong, W.Y. Lin, Mining Fuzzy Association Rules with Multiple Minimum Supports
using Maximum Constraints , Number of Rules. Lecture Notes in Computer Science. 3214: pages1283-
1290, 2004.
[5] S. Papadimitriou, S. Mavroudi, The Frequent Fuzzy Pattern Tree, Microsoft Corporation. Example
Database Food Mart Of Microsoft Analysis Services: In the 19th WSEAS international conference on
computers, 2005.
[6] R. Agrawal, R, Srikant , Fast Algorithm for Mining Association Rules. The International Conference
on Very Large Data Bases, pages487- 499, 1994.
[7] S.M. Chen, J.R. Hwang, Temperature Prediction using Fuzzy Time Series. IEEE Transactions on
Systems, Man, and Cybernetics B: Cybernetics, 30 (2), pages263- 275, 2000.
[8] C.H. Lin, T.P. Hong, W.Y. Lu, Linguistic Data Mining with Fuzzy FP-trees, Expert systems with
applications, 37, pages4560- 4567, 2010.
[9] C.H. Chen, T.P. Hong, V.S. Tseng, Fuzzy data mining for time-series data, Applied Soft Computing,
12: pages536- 542, 2012.
[10] A.M. Palacious , M.J. Gacto, J. Alcala-Fdez, Mining Fuzzy Association Rules from Low Quality Data.
Soft Computing- A Fusion of Foundations, Methodologies and Applications,Vol 16:pages883- 901,
2012.
[11] J. Han, J. Pie, Y. Yin, Mining frequent patterns without candidate generation: In the ACM SIGMOD
International Conference on Management of Data, pages1- 12, 2000.
[12] K. Suriya Prabha, R Lawrance, Mining Fuzzy Frequent Item set using Compact Frequent Pattern Tree
Algorithm. International Conference On Computing and Control Engineering, April 2012.
[13] T.P. Hong, J.B. Chen, Finding Relevant Attributes and Membership Functions. Fuzzy Sets and
Systems- A Fusion of Foundations, Vol 103, pages389- 404, 1999.
[14] T.P. Hong, C.S. Kuo, S.L. Wang, A Fuzzy Apriori Tid Mining Algorithm with Reduced Computational
Time, Applied Soft Computing, Vol 5, pages1-10, 2004.
[15] T. Hua, S.Y. Sung, H.Xiong, Q. Fud, Discovery of Maximum Length Frequent Item sets, Information
Science: An International Journal, Vol (178), Issue 1, 2008.
[16] H. Suresh, K. Raimond, Analysis of Time Series Rule Extraction Techniques, IOSR Journal of
Computer Engineering (IOSR-JCE), Vol (8), Issue 5, pages22-27, 2013.](https://image.slidesharecdn.com/ad03301810188-130511020215-phpapp02/85/Ad03301810188-9-320.jpg)
Ad03301810188
- 1. International Journal Of Computational Engineering Research (ijceronline.com) Vol. 3Issue. 3 181 ||Issn||2250-3005|| (Online) ||March||2013|| ||www.ijceronline.com|| Mining Association Rules From Time Series Data Using Hybrid Approaches Hima Suresh1, Dr. Kumudha Raimond2 1 PG Scholar, Karunya University, 2 Professor, Karunya University Abstract: Due to the frequent appearance of time series data in various fields, it has always been an essential and interesting research field. A time series analysis involves the methods for analyzing time series data, in order to mine meaningful and other relevant characteristics of the data. In most cases, time series data are quantitative values, so to come up with an intellectually appealing data mining algorithm to deal with quantitative type of data presents a biggest challenge to researchers in this field. In this paper, an extended Fuzzy Frequent Pattern (FP) growth approach is proposed and analyzed against the existing approach called Fuzzy Apriori (FA). Keywords: Conditional Pattern Base, Fuzzy Apriori, Frequent Pattern, Fuzzy Logic, Genetic Algorithm , Neural Network and Frequent Pattern tree. 1. Introduction Time series is defined as a series of well defined data values of a variable at successive time interval. The applications of time series includes FP analysis, predictions etc. FP analysis in time series data has become one of the most vital parts of data mining tasks and has attracted extreme interest among the researchers.Many approaches are available to examine time series data such as the Genetic Algorithm (GA), statistical methods etc. But the outputs of these approaches are difficult to understand/interpret the quantitative results. Apriori is the classic algorithm of association rules which determine the number of frequent items proposed by Agrawal et. al [6]. But uncertain time series are difficult to be dealt with this approach. So the fuzzy sets are used, in order to handle uncertain time series data. Fuzzy sets are the sets containing degrees of membership. Fuzzy set theory was introduced by L.A.Zadeh et. al [1] and this approach was followed by Hong et. al [2], [3], [4] and proposed several fuzzy algorithms to extract quantitative data and to mine meaningful association rules. The fuzzy mining algorithm integrates the concept of Apriori algorithm as well as fuzzy concepts. However, in the Apriori algorithm, candidate generation suffers from significant costs such as: it may need to generate a huge number of candidate sets as well as it needs to scan the database repeatedly and to check a large set of candidates using pattern matching. This would be a tedious workload to go over each transaction in the database to determine the support value of candidate item sets. The approach to mine complete set of frequent item sets without candidate generation is one of the important motivations and also to generate efficient and scalable method for mining both long and short frequent patterns. An approach based on FP-Growth to find fuzzy association rules is proposed by Papadimitriou et. al [5]. In this approach each value in the transactions are transferred into two corresponding fuzzy regions. A support value is set and those fuzzy regions in the transaction which doesn’t meet or exceed the predefined support would be removed. Here only the frequent fuzzy 1-itemsets obtained from each transaction are used for extracting. The expression of the fuzzy items with more fuzzy regions is successive. No fuzzy operation is used to combine the regions together. However, this made the rules mined to be difficult to interpret. The proposed extended Fuzzy FP growth approach overcomes this limitation to a certain extend. In this paper, an extended FP growth approach has been proposed and evaluated against the existing approach FA. It is an interesting method used for mining frequent item sets. The remainder of this paper is organized as follow: Section 2 describes the related works. Section 3 presents the hybrid approach using FA technique. Section 4 presents fuzzy FP-growth approach. Section 5 describes the experimental results. Finally, conclusion is discussed in Section 6. 2. Related Works In this section some related concepts and algorithms are discussed such as: Apriori algorithm, FP- growth algorithm and Fuzzy set theory. A. Apriori algorithm The classic algorithm for mining frequent item sets and for learning association rule over the transactional database was proposed as Apriori algorithm by Agrawal et. al [6]. The algorithm involves two steps. Firstly, it find out the frequent item sets which exceeds or satisfies the minimum support. Secondly, the
- 2. Mining Association Rules From Time Series Data… 182 ||Issn||2250-3005|| (Online) ||March||2013|| ||www.ijceronline.com|| rules are generated which satisfies the minimum confidence value. In this algorithmic method, the database is scanned repeatedly in order to generate all frequent item sets. It would be costly to go over each transaction and to count the support for each candidate item set. Also this approach generates huge number of candidate sets. B. Frequent Pattern –Growth algorithm One of the interesting methods to extract frequent item sets is FP-Growth algorithmic method and this is proposed by Han et. al [3]. The proposed algorithm does not require the candidate generations. The extraction of frequent patterns using Apriori is not efficient since it requires the repeated scanning of database and may need to generate large candidate item sets. In order to overcome this, an extended FP-Growth algorithm is used. The Approach involves several steps: First it scans the database to obtain all items with their corresponding support count. The items which exceed or satisfy the predefined minimum support are selected as the frequent 1-itemsets. Next step is to arrange the frequent 1-itemsets according to the decreasing order of their support count. Finally, the database is scanned again to construct the FP tree. All transactions are processed bottom up one by one and after processing all the transactions, a complete FP tree is constructed. Once the tree is constructed, FP-Growth procedure is executed to obtain all frequent item sets. FP growth generates the frequent patterns directly from the FP tree instead of candidate generation (C.W. Lin et.al [8]). For each frequent item, a conditional FP tree would be generated and the corresponding frequent item sets are derived from the same. C. Fuzzy set theory The word fuzzy means vagueness. Fuzzy sets have been introduced by Lotif A. Zadeh [1]. It is actually an extension of the classic set theory which defines the set membership as possibility distribution. The rule for the fuzzy set theory can be expressed as: ] X [0, 1] (1) Here “x” number of possibilities may occur. A logic based on the truth values such as “true” and “false” would be sometimes inadequate while describing the human reasoning. Fuzzy logic uses the entire interval between 0 (false) and 1(true) for describing human reasoning. 3. Fuzzy Apriori Approach The FA approach integrates the concept of Fuzzy as well as Apriori algorithm. The architecture for the FA approach implemented by C.H.Chen.et al [9] is shown in fig 1. Fig 1: Architectural diagram of FA approach The architecture describes the procedures for generating rules using FA algorithm. First, the time series data has been taken as input and from the given time series data, the subsequences of any length according to the sliding window size are generated. After the subsequence generation, each data point of the subsequences is transformed into fuzzy set. The total count or scalar cardinality of the corresponding fuzzy set based on the membership function is obtained. Frequent candidate sets are generated based on this support value which exceeds or satisfies the predefined support value. Pruning processes has been applied to filter away irrelevant large candidate sets. Finally the association rules are generated. 4. Fuzzy Frequent Pattern Growth Approach The proposed approach uses the concept of fuzzy set and FP-growth approach to generate the frequent item sets without any candidate generation. The architecture for the proposed fuzzy FP-growth approach for time series data is shown in fig 2. Fig 2: Architectural diagram of Fuzzy FP-growth approach Generate sub sequence Input time series Generate fuzzy sets Calculates scalar cardinality Removal of redundant items Generates association rules Fuzzy set generation Scan the DB to generate 1-itemset Construct the FP tree Generate conditional Pattern base Generate fuzzy association rules
- 3. Mining Association Rules From Time Series Data… 183 ||Issn||2250-3005|| (Online) ||March||2013|| ||www.ijceronline.com|| The approach uses the transformed fuzzy set for generating frequent patterns. It involves different phases. First step is to construct FP-tree from a database and from the resulting FP-tree, frequent patterns are generated. FP tree construction involves several steps such as: Scanning the databases to generate 1-itemset then rescanning to establish FP tree. Next generating conditional pattern base and corresponding conditional FP tree respectively. Finally fuzzy association rules are generated as proposed by C.W. Lin et. al [8]. 5. Experimental Analysis The Hybrid approach using both fuzzy concept and apriori algorithm is an efficient method for handling time series data to find linguistic association rules proposed by C.H. Chen et. al [9], A.M Palacious et. al [10]. Home price time series data over the years from 1999 to 2012 period has been used for analyzing the performance of the existing FA approach against the proposed Fuzzy FP growth approach [16]. The home price time series used for analyzing these two hybrid approaches are shown in table1. Table 1: Home price time series data used as input Years Home price Years Home price 1999 127 2006 124 2000 129 2007 118 2001 132 2008 121 2002 130 2009 120 2003 126 2010 115 2004 132 2011 113 2005 129 2012 119 A. FA Method Analysis The time series data is first transformed into subsequences according to the predefined window size, which could be of any length. Here in this example, window size is set as 5. So the total subsequence transactions will be 10 (T1, T2….T10) according to the formula (total length of time series – window size + 1). The transactions of subsequences generated are shown in table 2. Table 2: Transaction of subsequences TID Transaction TID Transaction T1 127,129,132,130,126 T6 132,129,124,118,121 T2 129,132,130,126,132 T7 129,124,118,121,120 T3 132,130,126,132,129 T8 124,118,121,120,115 T4 130,126,132,129,124 T9 118,121,120,115,113 T5 126,132,129,124,118 T10 121,120,115,113,119 After transforming the time series data into subsequences, the data points in each subsequence are converted to fuzzy values according to predefined membership function. The membership values used are Low, Middle and High and is used for the transformation of fuzzy sets from the given time series. The representation of membership function is given in fig 3. 113 118 122 128 132 Fig 3: Membership function The fuzzy values of each subsequence are shown in table 3. Each data point (A1, A2 …An) in the transaction each subsequence will take the value of low (L), middle (M) and high (H).
- 4. Mining Association Rules From Time Series Data… 184 ||Issn||2250-3005|| (Online) ||March||2013|| ||www.ijceronline.com|| Table 3: Fuzzy set transformed from time series The total count or scalar cardinality has been calculated from the fuzzy set generated and is calculated as its count value. After the calculation, fuzzy items generated are collected as the 1-itemset [13], [14], [15]. The total count is then checked against the support value, which is predefined. Here in this example, support value is set as 30%. The item set whose values are greater than or equal to 30% has been taken and are the following: A1.Middle, A2.Middle, A3.Low, A3.Middle, A4.Low, A4.Middle, A5.Low, and A5.Middle. The 1-itemsets are represented in table 4: Table 4: The 1-itemset generated with total count 1-Itemset Total 1-Itemset Total 1-Itemset Total A1.Middle 0.467 A2.Middle 0.4 A3.Low 0.334 A3.Middle 0.333 A4.Low 0.434 A4.Middle 0.333 A5.Low 0.501 A5.Middle 0.366 - - After the generation of 1-itemsets, corresponding 2-itemsets are obtained. The fuzzy items with the same attribute Ai are not put in to 2-itemset collection. The resulting 2-itemsets generated are shown in the table 5. Table 5: The resulting 2-itemset satisfy the support 2-itemset Total A1.Middle ∩ A5.Low 3 A3.Low ∩ A4.Low 3.01 A4.Low ∩ A5.Low 3.68 The association rules thus generated from the 2-itemsets with corresponding confidence values are shown in table 6: Table 6: The Association rules with confidence values Final Rule Confidence A1.Middle,A5.Low 0.64 A3.Low, A4.Low 0.9 A4.Low, A5.Low 0.84 T A1 L A1 M A1 H A2 L A2 M A2 H A3 L A3 M A3 H A4 L A4 M A4 H A5 L A5 M A5 H T1 0 1 0 0 0.6 7 0.3 3 0 0 1 0 0 0 0 1 0 T2 0 0.6 7 0.3 3 0 0 1 0 0 0 0 1 0 0 0 1 T3 0 0 1 0 0 0 0 1 0 0 0 1 0 0.6 7 0. 33 T4 0 0 0 0 1 0 0 0 1 0 0.6 7 0.3 3 0 1 0 T5 0 1 0 0 0 1 0 0.6 7 0.3 3 0 1 0 1 0 0 T6 0 0 1 0 0.6 7 0.3 3 0 1 0 1 0 0 0.6 7 0.3 3 0 T7 0 0.6 7 0.3 3 0 1 0 1 0 0 0.6 7 0.3 3 0 0.6 7 0.3 3 0 T8 0 1 0 1 0 0 0.6 7 0.3 3 0 0.6 7 0.3 3 0 1 0 0 T9 1 0 0 0.6 7 0.3 3 0 0.6 7 0.3 3 0 1 0 0 1 0 0 T10 0.6 7 0.3 3 0 0.6 7 0.3 3 0 1 0 0 1 0 0 0.6 7 0.3 3 0 Total 1.6 7 4.6 7 2.6 6 2.3 4 4 2.6 6 3.3 4 3.3 3 2.3 3 4.3 4 3.3 3 1.3 3 5.0 1 3.6 6 1. 33
- 5. Mining Association Rules From Time Series Data… 185 ||Issn||2250-3005|| (Online) ||March||2013|| ||www.ijceronline.com|| The confidence value of the above association rules are calculated using the formula given below: Confidence = )a.item(Support b).itemUa.emSupport(it (2) The confidence value of rule is then compared with the predefined confidence threshold. The rule obtained here means that “if the value of a data point is low at the third time units then the value of a data point after fourth time units will also be low with a high probability”. Though the rule generated is useful to understand the corresponding time series data, the amount of effort required to generate the rules are more as candidate generation is more costly in time. In order to avoid the limitations of FA, the Fuzzy FP-Growth approach has been implemented and evaluated against FA approach. B. FP-Growth Method Analysis The Fuzzy FP–Growth approach is analyzed using the same home price time series data. The case study results of fuzzy FP growth approach are explained in table 7: Table 7: Transactions and corresponding items TID Items T1 (A1:127)(A2:129)(A3:132)(A4:130)(A5:126) T2 (A1:129)(A2:132)(A3:130)(A4:126)(A5:132) T3 (A1:132)(A2:130)(A3:126)(A4:132)(A5:129) T4 (A1:130)(A2:126)(A3:132)(A4:129)(A5:124) T5 (A1:126)(A2:132)(A3:129)(A4:124)(A5:118) T6 (A1:132)(A2:129)(A3:124)(A4:118)(A5:121) T7 (A1:129)(A2:124)(A3:118)(A4:121)(A5:120) T8 (A1:124)(A2:118)(A3:121)(A4:120)(A5:115) T9 (A1:118)(A2:121)(A3:120)(A4:115)(A5:113) T10 (A1:121)(A2:120)(A3:115)(A4:113)(A5:119) Each value of the transaction items are transformed in to fuzzy sets using this membership function. The fuzzy set transformed from the data is represented in table 8: Table 8: Fuzzy sets generated from home price time series data Tid Items T1 (1/A1Middle)(0.67/A2Middle+0.33/A2High)(1/A3High)(1/A5Middle) T2 (0.67/A1Middle+0.33/A1High)(1/A2High) (1/A4Middle)(1/A5High) T3 (1/A1High)(1/A3Middle)(1/A4High) (0.67/A5Middle+0.33/A5High) T4 (1/A2Middle)(1/A3High)(0.67/A4Middle+0.33/A4High) (1/A5Middle) T5 (1/A1Middle)(1/A2High)(0.67/A3Middle+0.33/A3High)(1/A4Middle)(1/A5Low) T6 (1/A1High)(0.67/A2Middle+0.33/A2High) (1/A3Middle)(1/A4Low) (0.67/A5Low +0.33/A5Middle) T7 (0.67/A1Middle+0.33/A1High)(1/A2Middle)(1/A3Low)(0.67/A4Low+0.33/A4Middle) (0.67/A5Low+ 0.33/A5Middle) T8 (1/A1Middle)(1/A2Low)(0.67/A3Low+0.33/A3Middle)(0.67/A4Low+0.33/A4Middle) (1/A5Low) T9 (1/A1Low)(0.67/A2Low+0.33/A2Middle)(0.67/A3Low+0.33/A3Middle)(1/A4Low) (1/A5Low) T10 (0.67/A1Low+0.33/A1Middle)(0.67/A2Low+0.33/A2Middle)(1/A3Low)(1/A4Low) (0.67/A5Low+0.33/A5Middle) The counts of all fuzzy regions are calculated. For example, in the case of A1.Low the total count will be 1+0.67=1.67. Likewise the steps are repeated for other regions. The corresponding results are shown in table 9. Table 9: The scalar cardinality of each fuzzy region Item Count Item Count A1.Low 1.67 A3.High 2.33 A1.Middle 4.67 A4.Low 4.34 A1.High 2.66 A4.Middle 3.33 A2.Low 2.34 A4.High 1.33 A2.Middle 2.66 A5.Low 5.01 A2.High 4 A5.Middle 3.66 A3.Low 3.34 A5.High 1.33 A3.Middle 3.33 - -
- 6. Mining Association Rules From Time Series Data… 186 ||Issn||2250-3005|| (Online) ||March||2013|| ||www.ijceronline.com|| The fuzzy region with the maximum count among the three possible regions for each item is selected. Take A1 as an example. Its count is 1.67 for low, 4.67 for middle and 2.66 for high. Since the count for middle is the maximum among three counts, the region middle is thus used to represent item A1 in the later mining process. This step would be repeated for the other items. The count of any region selected is checked against with the predefined minimum support value. Here support value assumed as 30%. That means (10*30%=3.0). The table 10 shows the set of frequent fuzzy region. Here every region satisfies the support 30%. Table10: Frequent Fuzzy regions and count Frequent fuzzy regions Count A1. Middle 4.67 A2.Middle 4 A3.Low 3.34 A4.Low 4.34 A5.Low 5.01 After finding the regions with higher counts, the header table is arranged based on the decreasing order of the support count. It is shown in table 11: Table 11: The header table of the regions Frequent fuzzy regions Count A5.Low 5.01 A1.Middle 4.67 A4.Low 4.34 A2.Middle 4 A3.Low 3.34 The fuzzy regions which are not in the header table are removed from each transaction [11]. The remaining fuzzy regions in each transaction are then sorted according to the membership values in a descending order. This is shown in table 12. Table 12: The transactions transformed with the fuzzy regions Tid Frequent fuzzy region T1 (1/A1Middle)(0.67/A2Middle) T2 (0.67/A1Middle) T3 0 T4 (1/A2Middle) T5 (1/A1Middle)(1/A5Low) T6 (0.67/A2Middle)(1/A4Low)(0.67/A5Low) T7 (0.67/A1Middle)(1/A2Middle)(0.67/A4Low)(0.67/A5Low) T8 (1/A1Middle)(0.67/A4Low)(1/A5Low) T9 (0.33/A2Middle)(1/A4Low)(1/A5Low) T10 (0.33/A1Middle)(0.33/A2Middle)(1/A4Low)(0.67/A5Low) After this step, the transactions with only the sorted frequent fuzzy regions selected which are shown in table 13: Table 13: The transactions with sorted the fuzzy regions TID Frequent fuzzy region T1 (1/A1Middle)(0.67/A2Middle) T2 (0.67/A1Middle) T3 0 T4 (1/A2Middle) T5 (1/A1Middle)(1/A5Low) T6 (1/A4Low)(0.67/A2Middle)(0.67/A5Low) T7 (1/A2Middle)(1/A3Low)(0.67/A1Middle)(0.67/A4Low)(0.67/A5Low) T8 (1/A1Middle)(1/A5Low)(0.67/A3Low)(0.67/A4Low) T9 (1/A4Low)(1/A5Low)(0.33/A2Middle)(0.67/A3Low) T10 (1/A4Low)(1/A3Low)(0.67/A5Low)(0.33/A1Middle)(0.33/A2Middle)
- 7. Mining Association Rules From Time Series Data… 187 ||Issn||2250-3005|| (Online) ||March||2013|| ||www.ijceronline.com|| All transactions up to 10 are inserted in the fuzzy FP tree [12]. In the FP tree representation, header table is shown in the left side and it contains the frequent Fuzzy Regions (FR) and their supporting count. The final fuzzy FP tree is shown in figure 4. Fig 4: The Complete FP tree While processing the items from bottom of the header table, first item would be A3L: the traversal from which A3L exist are: 1) (A4L:3)(A3L:1) 2) (A4L:3)(A5L:1)(A2M:0.33)(A3L:0.67) 3) (A1M:3.67)(A5L:2)(A3L:0.67) 4) (A2M:2)(A3L:1) Table 14: The frequent item sets with A3L 1-Item 2-Item 3-Item A5L:5.01 A4L, A3L:1.67 A1M, A5L, A3L:0.67 A1M:4.67 A5L, A3L:1.34 A4L, A5L, A3L:0.67 A4L:4.34 A2M, A3L:1.33 A5L, A2M, A3L:0.33 A2M:4 A1M, A3L:0.67 - A3L:3.34 - - Here in this example taking the intersection (minimum) of items to obtain 2-itemset and 3-itemsets. For eg: A4L:3 and A3L:1 means1 (taking the smaller value or minimum operator) and then if the same transaction occurs again, it would be incremented with the corresponding value. Likewise all regions in the header table should be processed and frequent item sets are generated. On processing the items bottom up from the header table, corresponding item sets are obtained. Same item sets from all transactions are taken together and their count is evaluated using the intersection (minimum operator). From the corresponding item sets of all transactions, those exceed or meet the support count are chosen as the rules. The rules obtained are shown in table 15 and are similar to the rules generated by FA approach. Table 15: The final rules generated Final Rules Count A1.Middle,A5.Low 3 A3.Low, A4.Low 3 A4.Low, A5.Low 3.01 The two hybrid approaches are implemented in VB.NET at a personal computer with Intel Core i3, 2.10 GHZ and 4GB RAM. The data set used is the home price time series data over the period of 1999 to 2012. The sliding window size is predefined as 5. The experimental result shows the FA approach require more number of scans for each transaction compared to the proposed approach. In the case of FA approach, for 20 support value it requires 1001 total number of scans. Likewise for 25 support value, it requires a total scan of 539 and for 30, it requires 275 total scans. Whereas FP approach requires only 11 times total number of scans
- 8. Mining Association Rules From Time Series Data… 188 ||Issn||2250-3005|| (Online) ||March||2013|| ||www.ijceronline.com|| for each transaction. This is one of the major advantages of the proposed approach over FA approach. The graphical representation of number of scans required for both the approaches shown in the fig 5. Fig 5: The graphical representation of number of scans The execution time required for the proposed approach is less than FA approach. This is another benefit. The graph showing the execution time required for both the approaches are shown in fig 6. Fig 6: The graphical representation showing execution time 6. Conclusion The paper discusses about two different hybrid approaches. The main aim of this analysis is to explore highly efficient method for generating non redundant and relevant rules. This technical report provides association rule mining on the existing hybrid approach and the proposed one. In this work, an efficient FP- Growth approach has been used to mine the complete set of frequent item sets without candidate generation. This approach helps to overcome the problems of FA, which requires a need to repeated scanning of the database and checking large set of candidates by pattern matching. The analytical study of this framework shows that it is efficient and scalable for extracting both small and long frequent patterns. The Fuzzy FP-Growth approach transforms the problem of finding long frequent patterns to search for shorter ones recursively and then concatenate the corresponding suffix. It uses the least frequent items as a suffix and there by offering good selectivity. The approach reduces the search costs to a great extend. FP growth approach provides less execution time compared to FA approach. 11 Support Support
- 9. Mining Association Rules From Time Series Data… 189 ||Issn||2250-3005|| (Online) ||March||2013|| ||www.ijceronline.com|| References [1] L.A. Zadeh, Fuzzy Sets. Information and Control, 8:338- 353, 1965. [2] T.P. Hong, C.S. Kuo, S.C. Chi, Mining association rules from quantitative data. Intelligent Data Analysis, no.3 (5), pages338- 353, 1965. [3] T.P. Hong, C.S. Kuo, S.C. Chi, Trade-off between Time Complexity and Number of Rules for Fuzzy Mining from Quantitative Data, International Journal of Uncertainty: Fuzziness and Knowledge-based Systems, 9 (5), pages338- 353, 2001. [4] Y.C. Lee, T.P. Hong, W.Y. Lin, Mining Fuzzy Association Rules with Multiple Minimum Supports using Maximum Constraints , Number of Rules. Lecture Notes in Computer Science. 3214: pages1283- 1290, 2004. [5] S. Papadimitriou, S. Mavroudi, The Frequent Fuzzy Pattern Tree, Microsoft Corporation. Example Database Food Mart Of Microsoft Analysis Services: In the 19th WSEAS international conference on computers, 2005. [6] R. Agrawal, R, Srikant , Fast Algorithm for Mining Association Rules. The International Conference on Very Large Data Bases, pages487- 499, 1994. [7] S.M. Chen, J.R. Hwang, Temperature Prediction using Fuzzy Time Series. IEEE Transactions on Systems, Man, and Cybernetics B: Cybernetics, 30 (2), pages263- 275, 2000. [8] C.H. Lin, T.P. Hong, W.Y. Lu, Linguistic Data Mining with Fuzzy FP-trees, Expert systems with applications, 37, pages4560- 4567, 2010. [9] C.H. Chen, T.P. Hong, V.S. Tseng, Fuzzy data mining for time-series data, Applied Soft Computing, 12: pages536- 542, 2012. [10] A.M. Palacious , M.J. Gacto, J. Alcala-Fdez, Mining Fuzzy Association Rules from Low Quality Data. Soft Computing- A Fusion of Foundations, Methodologies and Applications,Vol 16:pages883- 901, 2012. [11] J. Han, J. Pie, Y. Yin, Mining frequent patterns without candidate generation: In the ACM SIGMOD International Conference on Management of Data, pages1- 12, 2000. [12] K. Suriya Prabha, R Lawrance, Mining Fuzzy Frequent Item set using Compact Frequent Pattern Tree Algorithm. International Conference On Computing and Control Engineering, April 2012. [13] T.P. Hong, J.B. Chen, Finding Relevant Attributes and Membership Functions. Fuzzy Sets and Systems- A Fusion of Foundations, Vol 103, pages389- 404, 1999. [14] T.P. Hong, C.S. Kuo, S.L. Wang, A Fuzzy Apriori Tid Mining Algorithm with Reduced Computational Time, Applied Soft Computing, Vol 5, pages1-10, 2004. [15] T. Hua, S.Y. Sung, H.Xiong, Q. Fud, Discovery of Maximum Length Frequent Item sets, Information Science: An International Journal, Vol (178), Issue 1, 2008. [16] H. Suresh, K. Raimond, Analysis of Time Series Rule Extraction Techniques, IOSR Journal of Computer Engineering (IOSR-JCE), Vol (8), Issue 5, pages22-27, 2013.