Clustering streaming data in distributed environments based on belief propagation techniques
View/
Keywords
Big data ; Streaming ; Affinity propagation ; Clustering ; Big Data AnalyticsAbstract
This study tries to examine a recent problem of computer science and specifically in data
mining field, the online clustering of distributed streaming. This problem belongs to big data
and analytics area, which means that we can’t apply the traditional techniques, software or
databases to capture, process and analyze this data with low-latency and we need massive
parallelism. Moreover, when these big data techniques applied to streaming data more
challenges are emerged. On the other hand, the analyzing of streaming data will give a lot of
advantages like real time insights of the data that will help to respond in emerging situations.
The sequential and distributed fashion of the data produced from a variety of devices
combined with the volume of them and constraints such as communication and storage make
a major challenge the clustering of streaming data. In our study we address the problem of
distributed clustering using two level of clustering approach, in first level, batch of data arrives
in many distributed nodes in each time slot and the nodes performs clustering in these data
extracting the most significant representatives of the batch (exemplars) which will be
forwarded to the central node which in turn performs the second level of clustering in order
to identify global patterns in the data arrived from every node The algorithm that we will try
to implement uses belief propagation techniques in a distributed environment. The exemplars
will feed back to the nodes with the appropriately modified weight which reflect their global
significance. We adopt belief propagation techniques in both levels to perform streaming
clustering.