Tackling Variety in Big Data Implementation

June 1, 2017

Big Data Platform to parse & analyze various data sources

Big Data Management at a high level consists of managing Velocity, Volume and Variety of data. While most integrated platforms tackle volume & velocity, very few pay attention to managing variety of data.

We have tackled the problem of 'Variety' by integrating handling of various data sources like twitter streams, Databases, block chain data etc. using a single integrated platform which uses Open Source Technologies as the backbone.

Below are some of the use cases tackled by the integrated solution.

 1. CRM Analytics based on various databases ( Oracle, SQL, Mariadb)

We analyzed campaign effectiveness, lead, opportunity & sales order data from ERP systems to produce intelligent reports.


Data Source: Maria Db



Data Source: Oracle Db

 2. Real Time Twitter feed & Facebook sentiment analysis

Using publicly available Twitter & Facebook feeds, real time analysis of feed data is possible.

Architecture for Social Media Analytics

Below is interesting sentiment analysis of twitter & Facebook feeds of various politicians during the UP elections.

Data Source: Twitter & Facebook

3. Block Chain Data

Block chain has many applications, the most famous one being bit coin. As a case study, we have parsed publicly available bit coin data using a block chain parser integrated with HDFS.

This data can be further analyzed to detect fraudulent patterns.

Data Source: Publicly available bitcoin data

Note: This blog is just for informative reading and not a product that we sell.

Pawan Mehta
Pawan Mehta

Pawan is the Principal Solution Consultant at Open eTechnologies. He helps enterprises implement open source ERP solutions like ERPNext and is also the proprietor of Open eTechnologies.

No comments yet

No comments yet. Start a new discussion.

Add Comment