Great Expectations: The Drivers of Change Making 'Data First'​ an IT and Business Imperative

In our increasingly all connected, sensing and intelligent world, data is driving everything - aptly heralded as the new ‘currency’ of change across both business and society.

Indeed, as individuals our data becomes our digital DNA, a virtual footprint of everything we believe and do. Going back to our title from ‘Great Expectations’, data really is in ‘every line I have ever read’. But data in of itself does not deliver value at all - value only arises from the informed acting upon the insights that quality data can bring. How can we quickly realise that value, especially shared value, and move from data and information, to knowledge and wisdom?

To do this, requires data quality at scale alongside visibility and accessibility, so the right data is available to the right person/entity, in the right place and at the right time. It also demands consistent attention to data ethics including issues of ‘baked-in’ biases - including human ones! If we use big data to teach Artificial Intelligence to imitate human preferences, such as by feeding in historical data and biased human decisions, it will not just replicate them, but augment, exacerbate, and scale them too (Eaves 2020).

With rising system complexity and environmental fragmentation, from on-premises to increasing cloud, hybrid and multi-cloud variations, how can organisations embed ‘data first’ and by design? And why does this matter so much, for business and broader societal impact too? In this first of a two-part series, I explore the core drivers of change and then in Part 2, move on to unpack just how to enable ‘insights on demand’, moving from resilience to growth by investing in data, whilst saving on storage.

When we look at drivers of the ‘data-first’ imperative, the ‘3 Vs’: Volume, Veracity and Velocity of data are key, underpinned by the context of the 4th V: Volatility. From the pandemic, to Brexit, to international geo-politics, this backdrop is not only increasing volatility and with it financial pressures and a state of continual uncertainty; but it has also brought to the fore the vital role of emergent technology integration, especially across digital – and of big data and its highly efficient near-real time evaluation too.

Data and analytics have proved indispensable during this pandemic, organisations must invest in data and analytics for resilience and business growth' (Gartner 2020)

The rapid operational and societal changes we have seen have led to a global and cross-sector acceleration of digital transformation, even in markets that have been slower to adapt previously, such as insurance (Verisk 2020). And the same applies to the acceleration and extending of consumer behaviours, for example around the growth of alternative payment methods, notably the ‘opening-up’ of digital commerce to older consumers and the rise of communities including social commerce. Early research suggests the permanency of (some) of these payment habits and other changes alongside it, notably the increasing influence of the ‘next generation’ of consumers, the truly digitally native Gen Z.

Taken in combination, I believe that COVID19 could justifiably be described as the digital accelerant of the last decade.

With this acceleration of digital innovation comes more data! With 90% of the world’s data created in the last two years alone, it is perhaps unsurprising that an anticipated 463 exabytes of data will be generated each day by humans as of 2025. Especially when we look at the latest data creation and consumption statistics! (TechJury 2020) – figures that are likely to be conservative given the evolving events of 2020 and its legacy of change.

- 1.7MB of data is created every second by every person during 2020.

- 2.5 quintillion bytes of data are produced by humans every day.

- By the end of 2020, 44 zettabytes will make up the entire digital universe.

And in terms of social media:

- 95 million photos and videos are shared every day on Instagram.

- Every day, 306.4 billion emails are sent, and 5 million Tweets are made.

We are creating and consuming a ‘data deluge’. Yet 90% of data that is archived is never used again. The time is now to look at exactly what you’re using and how, with alignment across Business and IT areas critical. Key challenges include firstly, the need to navigate this myriad of data arising from omni-channel and often silo-ed sources, and extracting meaning from the ‘noise’. Next, we have the increasing need to do more with less – to be more agile and more productive, all whilst typically needing to optimise cost efficiencies alongside the challenge of traditional IT operational models and technologies. Compounding this further, is the rising risk of cyberattack. The IBM X-Force Threat Intelligence Index 2020 found:

- 8.5 billion records were breached in 2019.

- Operational Technology attacks surging 2,000% year-over-year.

- Ransomware attacks alone rose 67% year-on-year in the last quarter.

And finally, we are moving beyond a multi market context of ‘great expectations’ to new expectations – as needs and preferences have rapidly changed. From the demand for ever more personalised customer experiences, to Gen Z wanting and expecting more from how they buy, and who they buy from. And from employees and customers alike wanting and expecting more around convenience and speed, to rising expectations in areas including energy efficiency to support carbon neutrality, and the repurposing of data for social good to close digital equity gaps (Aspirational Futures 2020). In combination, this again highlights the need for more flexible approaches to business and IT that creates shared value for all.