With the new year comes new challenges and opportunities for the many businesses around the globe.While there are plenty of predictions and trends surrounding the biggest buzzwords and most visible technologies of the digital transformation space - augmented reality, cloud computing, Internet of Things (IoT), virtual reality, and so on - sometimes it’s easy to lose track of all the amazing advancements happening behind the scenes in data analytics and data management, helping our businesses make our growing datasets more accessible, manageable and actionable for better operational decisions.
With research giants like Forester projecting the global big data market to be worth AU $140 billion in less than ten years from now, it’s clear the rapid advancements and adoption of artificial intelligence (AI), business intelligence and cloud computing technologies will continue to re-define the way we do data analytics in 2019 and beyond.
It can be difficult to keep up with all the new and emerging technologies and services in the big data space, so Xello have picked their top 5 incoming data analytics and data management disruptors they expect to make big waves in 2019 - so you know which innovations to prepare and watch out for next.
Augmented analytics is a new approach that transforms data workflows using machine learning (ML) automation, artificial intelligence (AI), and natural-language generation.
Representing the “third major wave” for data and analytics capabilities according to Gartner, this new analytics paradigm aims to remove bias from the preparation, modelling, interpretation and analysis of our data. It will also enhance impartial contextual awareness for acting on insights, and help data scientists deploy better automated algorithms to explore more hypotheses. Finally, it compliments rather than replaces the rising adoption of enterprise and self-service BI.
By 2020, augmented analytics and the many benefits it provides is predicted to be the dominant driver of new purchases of analytics and business intelligence (BI) toolsets and ML platforms, and flagged by several industry heavyweights to eventually be embedded and integrated in our enterprise applications in the next two years. The key differentiation in augmented analytics is its use of advanced ML automation and other technologies to better identify hidden patterns in our business data, without the risk of unintentional human bias inserted into the algorithms we use to generate insights.
Augmented analytics works by applying a range of models and ensemble learning to data in parallel to find key insights and actionable findings, proving faster than manual exploration. It also helps companies discover any overlooked correlations in their data.
For example, a US health insurance company previously tracked cost metrics based on the sickness of their patients. They fed their data to augmented analytics algorithms and models running in parallel, which explored all possible combinations of the many different factors driving cost. The end result found children under 12 were the largest cost driver, as its contract allows ambulances to charge per person in the vehicle and children under 12 typically have one parent accompanying them. Augmented analytics allowed them to look at their data in a completely new way and explore relationships they might not have discovered via manual exploration.
With increasing amounts of big data for our data scientists and business analysts to prepare, analyse and organise, exploring all possibilities is an almost impossible task to accomplish on manual techniques and processes alone. Augmented analytics will ultimately ensure organisations don’t miss key insights - and given it’s also predicted more than 40% of data science tasks will be automated by 2020, the increasing availability of augmented analytics will help make our data insights more broadly available for not just our data experts, but our key decision-makers and the rest of the business.
Enterprise Data Catalogs
Enterprise data catalogs are a metadata management tool and information governance model that helps businesses find and manage all data stored in their many systems - ERP, e-commerce, finance, etc - and addresses user-enabled data sprawl (undocumented and ungoverned data) by improving overall accessibility, governance, transparency and security. With data risk and governance and data value realisation the second and third most common challenges companies face in their analytics initiatives, data catalogs are more vital than ever.
A data catalog is a key part of the modern organisation’s information architecture, acting as a central place to contain, organise and perform discovery of data assets - and the demand for it is growing rapidly. With an inventory of all data that allows up-to-date curation of all assets, it gives valuable context that helps data experts and decision-makers find and understand their categorised datasets faster and easier for the purpose of extracting better business insights.
Gartner predicts over 80% of data lakes won’t include effective metadata management capabilities through 2019, leading to avoidable inefficiencies - and that the companies that do organise and implement data catalogs will realise twice the business benefits than those that don’t. And while they aren’t exactly new, there has been a resurgence in demand in the past two years due to analytics leaders struggling to inventory distributed data assets to conform to new compliance requirements and regulatory standards, such as Europe’s General Data Protection Regulation (GDPR) and Australia’s own Notifiable Data Breach (NBD), which are now in effect.
Enterprise data catalogs are increasingly being used to document and provide context to the meaning and value of our data loaded into data lakes. Information stored in data lakes is often done so in its raw, unstructured forms or its native format that limits its reliability and usability.
Without an overarching metadata management tool to document the intended usage of each dataset, vasts amounts of information is becoming unreliable and unusable - which is why data catalogs are the perfect solution for the increasing amounts of big data collected and stored.
Read more: What is a Data Lake?
Edge computing and analytics at the edge
Edge computing is a computing paradigm that places information processing at the device level where data is generated - near the edge of your network - rather than within the cloud or a centralised data-processing center, reducing conventional connectivity and latency dependencies. It helps improve data compression and transfer at the connectivity part of the technology stack, and in conjunction with Internet of Things (IoT) will help make more IoT applications possible.
Like many of the disruptors on this list, edge computing has existed for some time but is rapidly growing in adoption. The number of enterprises making it a core part of their strategy has doubled in a year - and more than half of all enterprise data will be processed outside traditional data centers and clouds by 2022, up from 10 per cent this year according to Gartner.
McKinsey also estimates that of the $500 billion in growth expected for IoT through 2020, around 25 per cent will be attributed to edge technology. Much of the development and focus on edge computing is tied to Internet of Things, due to early B2B applications of IoT needing to deliver distributed capabilities and real-time actions into the embedded IoT world faster.
While there is no standard formula for implementing edge computing, there are several existing use cases which have demonstrated the value it adds to organisations.
Blockchain: Distributed ledger technology needs decentralised computing models to function, as they need to be able to be processed and housed locally. Because every node is a compute unit in a blockchain, it’s not centralised, which means it’s edge.
Connected office: Smart speakers and home assistant devices such as Amazon Alexa, Google Assistant and Microsoft Cortana all perform voice-demanded tasks with a lag-time of a few seconds, but edge computing can make this happen in real-time.
Predictive maintenance: Closely linked to industrial sectors using Internet of Things, edge computing can help detect when machines and equipment are in danger of breaking and log requests for repairs before they do so, and these type of real-time alerts are best done when the data and processing is done close to that machine - which is what edge computing allows.
Retail: International chains such as Nordstrom use augmented reality (AR) technology to attract customers in-store with more immersive environments, which is only possible with the lower latency edge computing offers.
With edge computing also comes edge analytics, which is enabling businesses to perform real-time data analysis at the exact point where the data is being collected - much faster than existing traditional capture and relay processes for data analysis. This has created extensive new value around the burgeoning IoT field, and with cloud-based business intelligence and data analytics services continuing to mature this year, closing the gap between the way data is generated and where companies can use it to act in a more insightful way for further competitive advantage is fast becoming hard to ignore any longer.
High levels of latency and WAN costs will ultimately be addressed with the edge, and Gartner expects a gradual increase in the embedding of AI, compute, sensor and storage capabilities in edge devices through 2028, which will see everything from smartphones to industrial devices shifting data intelligence capabilities toward the edge.
Read also: Australian IoT and edge computing use cases
Natural language querying (NLQ)
Natural language querying (NLQ) is fast becoming major features of the most popular data analytics and business intelligence tools on the market. It’s also something companies deploying BI systems to create visualised dashboards and reports on top of their data warehouses to improve their overall data analytics pipeline can no longer afford to ignore.
Gartner expects approximately 50% of all business intelligence software queries will incorporate NLQ by 2019. Natural-language generation and artificial intelligence are also expected to be standard features of 90% of modern business intelligence platforms, including those powered by cloud platforms like Microsoft Azure, and we can already see the prevalence of AI in the many data analytics services available on these platforms.
Tools like Azure Machine Learning and Power BI now boast ever-evolving, in-built Q&A features which allows users type a data-specific question using natural language, such as “what were total units sold last year,” which then generates fast and useful visualisations in response.
It’s evident from its increasing availability and implementation in the most powerful analytics tools on the market that NLQ will be key to achieving greater efficiency and insights in our data exploration efforts in 2019 and beyond.
Predictive analytics is a type of advanced analytics which offers organisations deeper exploration of their historical data through a suite of techniques such as data mining, modelling, statistics, machine learning (ML), deep learning (DL) and artificial intelligence (AI) and associated services to make predictions on the future.
While it’s far from new, the rapid rise in cloud platforms like Azure - housing some of the most accessible, powerful and scalable enterprise predictive analytics toolsets - will see investment in machine learning grow to be worth more than $12 billion by 2020. Predictive analytics is also helping companies get around the lack of data scientists and analysts trained in Python or R, skill-sets necessary to analyse historical datasets and reliably forecast trends thanks to its automated nature, and with more than 40% of data science tasks to be automated by 2020, we fully expect Australian companies to dive head-first into what the technology offers this year.
Because this form of analysis is heavily automated, end users can see better what will most likely happen and can better understand what they can do to affect that outcome. Instead of manual, time-consuming analytics, organisations can act on other identified meaningful patterns and trends faster and more reliably forecast patterns - it’s not just about predicting the future.
As an example, Xello helped a franchise business in the hospitality industry predict anticipated sales forecasting based on both historical data and additional sources such as weather data by storing the later in a Modern Data Warehouse, using predictive modelling to combine both datasets for a more reliable forecasting capabilities than previously possible. Combined with a strong foundation for data management (data warehouse) and BI software for visualisation (Power BI), predictive analytics is invaluable.
Read more: Xello’s predictive analytics use case
How to prepare for the next data analytics trends
Big data, machine learning, predictive analytics and so on is often just a buzzword for many businesses. The organisations that are succeeding in adopting and leveraging these exciting new trends aren't just approaching them as technology enablers, but proactively preparing and examining how such advanced analytics capabilities could benefit their overall business strategy and help them extract more value from their big data.
If you need help building a new analytics and data management strategy or are interested in learning how these advanced capabilities could benefit your business and drive greater data-driven insights, take a look at Xello's Data & AI Solutions and download our free white papers for more information on how we can help.