Big data is a blanket term for any collection of data sets so large and complex that it becomes difficult to process using on-hand database management tools or traditional data processing applications. The challenges include capture, curation, storage, search, sharing, transfer, analysis and visualization. The trend to larger data sets is due to the additional information derivable from analysis of a single large set of related data, as compared to separate smaller sets with the same total amount of data, allowing correlations to be found to “spot business trends, determine quality of research, prevent diseases, link legal citations, combat crime, and determine real-time roadway traffic conditions.”
Infographic source: Visual.ly
How One AI-Driven Media Platform Cut EBS Costs for AWS ASGs by 48%
The basic idea behind the phrase ‘Big Data‘ is that everything we do in our lives is (or will soon) leave a digital trace (or data), which we (and others) can use and analyze. The advances in capturing and analyzing big data allow us to decode human DNA in minutes, find cures for cancer, accurately predict human behavior, foil terrorist attacks, pinpoint marketing efforts, prevent diseases and so much more. Big data refers to our ability to collect and analyze the vast amounts of data we are now generating in the world. Big Data’s significance is huge, and can be used to boost your analytics and up your game.
These are the five ways to leverage Big Data:
- Big Data can unlock significant value by making information transparent. 25 percent of the effort in some knowledge worker workgroups consists of searching for data and then transferring them to another (sometimes virtual) location. This effort represents a significant source of inefficiency.
- As more transactional data is stored and created in digital form, Big Data can collect more accurate and detailed performance info on everything, expose variability and boost performance.
- Big Data allows ever-narrower segmentation of customers and therefore much more precisely tailored products or services.
- Sophisticated analytics can substantially improve decision-making, minimise risks, and unearth valuable insights that would otherwise remain hidden.
- Big Data can be used to develop the next generation of products and services.
More on Big Data pros at IveyBusinessJournal.
Funded Big Data Companies to Watch in 2014
These startups have managed to differentiate themselves in an aggressive market by offering unique technologies, products, and services:
CloudPhysics – CloudPhysics provisions detailed troubleshooting of IT databanks to minimize operational costs and human intervention in IT services. Currently, they receive 100+ billion samples of IT data from their global users, which they utilize to solve configuration- or performance-related problems in enterprise IT outfits, backed by $12.5 million in capital funding raised funds from Angel Investors and the Mayfield Fund.
MemSQL – This company received its latest round of Series B, venture-capital funds in January 2014, which amounted to $35 million.
Altiscale – his service provider ensures that their expert engineers are available to deploy and manage the Hadoop infrastructure from start to finish—so that the customers can singularly focus on their data and applications. Altisca received a $12 million capital funding in Series A from General Catalyst and Sequoia Capital, and other private sponsors.
Couchbase – is an open-source vendor, building its business model around a community of developers and users of Couchbase database technology. The company’s flagship product – Couchbase Server is marketed in two editions – Enterprise and Community. Couchbase, to date, has raised $56 million in venture capital funding from Mayfield Fund, North Bridge Venture Partners, Ignition Partners, and others. The possibilities and application of Big Data is virtually infinite.
Big Data with IoT
Large data volumes from IoT will drive radical changes within today’s datacenters and will require new Big Data strategies within enterprises. Due to a skills shortage and the need to constantly procure infrastructure to keep up with the amounts of incoming data, enterprises will start moving away from the DIY models towards PaaS, managed, and orchestrated solutions. The value of IoT is in the data. The quicker enterprises can start analyzing their data the more business value they can derive. Vendors are stepping up to the plate to remove the complexity and risks of data ingestion and data management so that customers can focus on analytics.
With the advent of the “Internet of things,” improved analytics and increased connectivity through new technology and software bring significant opportunities for companies. Big Data is a major gamechanger, IBM predicts that the large enterprises will all hire Chief Data Officers (CDO) in 2014 to manage the enterprise data centers. To strengthen its view, IBM cites Gartner statistics that show some 100-plus CDOs engaged in large enterprises today.
Big Data from a mobile perspective
As mobile devices have severe technology limitations on writing data, their popularity as big data analytics tools on the go is short-lived. Also, mobile devices are vulnerable to security attacks. Datanami believes that mobile devices will continue to dominate but in a new avatar, meaning mobile technologies will have to adapt to stringent, security and privacy needs of organizational data. Business intelligence (BI) will be embedded on smart systems. Sales forces may manage their customer relationships through embedded, smart apps with built-in analytics to make decisions. Progressively, smart software in mobile and enterprise systems will make decisions and make data scientists redundant.
IDC’s predictions for the big data market in 2014
The market for big data will reach $16.1 billion in 2014, growing 6 times faster than the overall IT market. IDC includes in this figure Infrastructure (servers, storage, etc., the largest and fastest growing segment at 45% of the market), services (29%) and software (24%).
IDC predicts that “cloud infrastructure will be the fastest-growing sub-segment of the big data market, with a 2013-2017 CAGR of close to 50%.”
Both IDC and IIA predict that the approved method of combating talent shortage in big data market will be re-skilling in-house analysts and data scientists.
Another application, security, will become the next big front for big data, IDC predicts, as security infrastructure will increasingly take on big data-like attributes. Big data will be used to correlate log data and identify malicious activity in real time, allowing companies to react quickly, rather than after the event.
Read more at Forbes.
Gartner predicts that by 2016, 25 percent of large global companies will have adopted big data analytics for at least one security or fraud detection use case, up from 8 percent today, and will achieve a positive return on investment within the first six months of implementation.
Presentations
Big Data Trends from David Feinleib
Big Data and Advanced Analytics from McKinsey on Marketing & Sales
