Jul 1, 2012

Challenges of Big Data


Volume:
According to a study we produce around 2.5 quintillion bytes of data each day. We can no longer use the existing database management tools on such rapidly growing data. We will experience difficulties to capture, store, search, share, analyze and visualize data. Managing large volume and rapidly increasing database is a challenging issue and cannot be anymore mitigated with faster processors. The volume of data is exceeding the processing capacity. Moreover, the algorithms necessary to process the data are much more complex than usual ones making analysis on big data very complicated.
Velocity:
Some scenarios require analysis of data to be presented immediately. For example: fraudulent credit card transaction should be flagged before a transaction even occurs. The larger the data set to be processed the longer it will take to analyze. How we can capture the most important data as it happens? How quickly does the data move across the enterprise when you need to make a decision and deliver that to the right people in real-time?
Variety:
Big data when it comes in an unstructured format, such as text or video. Machines or algorithms expect homogenous data and so require data to be prepared before analysis. To apply appropriate filters that can reduce the size of raw data and at the same time ensures the availability of necessary data. Even after initial data cleaning some errors and incompleteness may exist in the data which must be handled during data analysis.

7 comments:

benslin kard said...

Data warehousing to analyze big sets of information for enterprise purposes, smaller companies wishing to review just one subject often utilize "data marts."

Negi said...

Thanks for great information you write it very clean. I am very lucky to get this tips from you


Customs compliance consulting

Website said...

Nice posting,and you have incredible work in this blog to collect the information.And you provide the information is amazing and this information is really helpful for us.

hotels price scraper

Jamyla Taylor said...

Got much information on your post, will surely share it, these will help others gain knowledge on these side. Thanks for sharing. Cheers!
data mining/web mining

John Hill said...

Thanks for updating with your authentic views in which you talking about Data Mining and please keep sharing some more information.

RML Warehousing

sonia mehata said...

virtual accountant is the best way to opt for an assistant without any issues. The virtual accounting services helps in keeping a track of the accounts. Visit our website to know more about us. marhabareception.com

Shalini said...

Really Good blog post about challenges of big data .provided a helpful information.I hope that you will post more updates like this.
Digital marketing company in Chennai