Tackling Variety in Big Data Implementation
June 1, 2017
Big Data Platform to parse & analyze various data sources
Big Data Management at a high level consists of managing Velocity, Volume and Variety of data. While most integrated platforms tackle volume & velocity, very few pay attention to managing variety of data.
We have tackled the problem of 'Variety' by integrating handling of various data sources like twitter streams, Databases, block chain data etc. using a single integrated platform which uses Open Source Technologies as the backbone.
Below are some of the use cases tackled by the integrated solution.
1. CRM Analytics based on various databases ( Oracle, SQL, Mariadb)
We analyzed campaign effectiveness, lead, opportunity & sales order data from ERP systems to produce intelligent reports.
Data Source: Maria Db
Data Source: Oracle Db
2. Real Time Twitter feed & Facebook sentiment analysis
Using publicly available Twitter & Facebook feeds, real time analysis of feed data is possible.
Architecture for Social Media Analytics
Below is interesting sentiment analysis of twitter & Facebook feeds of various politicians during the UP elections.
Data Source: Twitter & Facebook
3. Block Chain Data
Block chain has many applications, the most famous one being bit coin. As a case study, we have parsed publicly available bit coin data using a block chain parser integrated with HDFS.
This data can be further analyzed to detect fraudulent patterns.
Data Source: Publicly available bitcoin data
Note: This blog is just for informative reading and not a product that we sell.
Pawan is the Principal Solution Consultant at Open eTechnologies. He helps enterprises implement various ERP solutions based on their business processes.