While it’s too early to say that Big Data is all grown up, it is mature enough to have spawned a number of new and very interesting offspring. As Gartner analyst Betsy Burton explained in late 2015 when she removed Big Data from the firm’s Hype Cycle, “Big Data has quickly moved over the peak of inflated expectations and has become prevalent in our lives across many hype cycles.”
Big Data is now a fundamental basis of several emerging technologies including the IoT, self-driving vehicles, artificial intelligence (AI), machine learning, deep learning, and augmented (AR) and virtual reality (VR). It has moved beyond elemental data into more sophisticated areas such as image recognition and correlation, and natural language querying systems such as AI-based personal assistants.
The Big Data category is evolving so rapidly it’s difficult to say where it will be at year’s end but strong trends are evident. 10Fold has a dedicated Big Data team that has been driving and closely tracking its evolution, and below is a short list of some of the important trends we see for 2017.
Data Democratization
Delivering ease of use and understandable analytics to people who are not data engineers or scientists is evidence of the industry’s maturity, a key to its growth, and increases ROI via simplification. Improvements in data processing and cloud apps and services, including BDaaS and STaaS, have delivered simple and sometimes free tools that make Big Data results easier to access. The cloud is now the main means of implementing most Big Data initiatives, allowing users to specify the needed storage and compute by spinning up databases for apps and data warehouses in mere minutes, at minimal cost, and without the all the previous physical hassles of configuration. This year and the coming decade will see more from the next level of data democratization, and one that is born of Big Data itself, with VR- and AR-based data interaction capabilities providing an immersive and further simplified experience.
IoT, Big Data – and Blockchain?
IoT perfectly exemplifies Big Data, delivering constant generation of unstructured data from a variety of sources. IoT is hot, but it also expands the attack surface among a variety of new vectors. Interestingly, media and analysts alike see blockchain technology growing beyond its financial origins to impact Big Data and as a potential remedy for IoT’s security issues. Blockchain’s relevance comes from its distributed ledger capabilities that hasten communications, its encryption, and from its unalterable nature. If these capabilities can be successfully applied to IoT and across other distributed Big Data systems, then not only will they speed and improve performance, but will greatly reduce risks.
AI Continues Learning
According to IDC’s 2017 predictions, “by 2019, 40 percent of digital transformation initiatives and 100 percent of IoT initiatives will be supported by AI capabilities.” AI provides timely analytics from Big Data and is especially useful with unstructured data by rapidly sifting through and identifying which data are most relevant for specific use cases. AI has morphed into a variety of new applications including machine learning, deep learning, neural networks, cognitive computing, image recognition, speech recognition and natural language processing just to name a few.
Feeding Big Data’s analytic output back into the system so the database learns from itself creates an iterative process that is the main tenet of machine learning, with AI hastening that process. Cognitive solutions that leverage AI are particularly useful by providing explanations, recommendations, and informing future actions or outcomes via their predictive nature.
While the predictive nature of these solutions positively impacts a variety of industries, it is especially useful in the most critical area to us all—healthcare. Using AI and other learning technologies to harness Big Data sources such as genomic sequencing, imaging analytics, medical devices (IoT), and data from medical records can deliver decision support capabilities enabling: health risk predictions; prevention of hospital readmissions; and faster decisions for improved patient outcomes. As proof of its importance, industry giants including Microsoft, SAP, Dell Services, IBM, Google and others have invested heavily in healthcare with the goal of applying machine learning strategies to complex problems such as cancer research.
Better Than Humans and Accelerating
Recently published results from experiments at Google’s Brain and DeepMind artificial intelligence research groups, OpenAI, MIT and UC Berkeley indicate AI software can design machine-learning systems with better results than those designed by humans. This has powerful implications such as: reducing market demand/stress for AI engineers that are in low supply; accelerating the pace at which machine-learning software is deployed; and reducing the amount of required data consumed for a system to perform (learn) a task well—with the last two further accelerating the pace of machine-learning’s evolution.
The pace of innovation enabled by Big Data and its various intelligent and self-learning spawn is so rapid and widespread that its outcomes may be impossible for mere humans to predict, though perhaps AI and the learning systems themselves will have an answer soon. One thing is for sure, at this pace we won’t have to wait long for the results.