Big Data is simply the collection, processing, and analysis of huge data sets from traditional or digital means which deliver business intelligence for companies to enhance their services or create a new competitive advantage.
Raw data which provides at best basic information is generated at an alarming rate of multiplication since 2000. This is fuelled by the emergence of social media and the proliferation of mobile devices leading to mobile computing.
The crucial challenge for long-term survival and relevance for every business is to convert information to insight and finally to intelligence as quickly and efficiently as possible.
In order to explain Big Data to many companies, we need to study its attributes and how it affects the future relationship between business and its information collected, analyzed and processed.
If we take all the information generated by human beings since the beginning of time till 2018, the same amount of data is being generated every minute right now. This results in most dataset becoming too large to store and analyze using traditional database technology.
The speed of data generation forces some companies to analyze data on the fly without storing into any system. Data that enters the system become input to other bigger and more complex data analysis.
Collecting and analyzing Big Data is futile unless we can generate massive value in converting information to intelligence. It is not about the speed or volume of the input or processing, it is about the size of the value of the data output.
The diversity and credibility of data may cause companies to have less control of the reliability of the content. Data technology now enables information to be mined meaningfully even if there are typo-errors or colloquial speeches.
In the past, we are geared to analyze structured data like relational databases and financial data. Now, 80% of the data we generate is non-structured like text, images, videos, or voice. With the emergence of social media, most of these unstructured data are messages, conversation, and other media recordings.
Data Source Layer:
Every input to a system can become a source layer for Big Data. It can be as simple as emails and audit trails to as complex as customer service interaction like calls, frequency, duration, locations, etc. The key questions for any business to ask are if new sources of monitoring are needed, or what else do they wish to find out from the existing layer source. The more relationships and co-relationships you see from the data trends, the more important selection of source layers are.
Data Storage Layer:
This is where your Big Data will reside after it is compiled from your diverse sources. As the volume of data generated and stored by companies start to escalate, complex but accessible systems and tools– such as Apache Hadoop DFS (distributed file system), and Google File System – have already been developed to help with this task.
Data Processing/Analysis Layer:
When you want to utilize the data you have stored to filter out something useful, you will need processing and analysis capabilities. A common method is to use a MapReduce tool. Essentially, the tool is used to select the elements of the data that you want to and put it into a format from which insights can be shown.
Data Out Layer:
This is where the insights gleaned from the analysis are submitted to the corporate users who will see new connections and leverage from them. Clear and concise communication (particularly if your decision makers do not have a background in statistics) is essential, and this output can take the form of reports, charts, figures and key recommendations.
To harness the power of Big Data for the future, we will require the new skills of the future. Executives at different levels need to sharpen the following skills in order to stay relevant and competitive.
The ability to determine what data to collect and how to point out new relationships between different . It is about identifying patterns and links to create connection between data.
The ability to generate new ways of collecting, and interpreting data from current data system performance. It is the art of seeing the behind the analytics.
Mathematics and Statistics:
The ability to simply crunch traditional numbers will not be sufficient for Big Data. One needs to be able to create and perform scenario planning models. It will also include company, competitors and industry practices.
Information Technology Skills:
The age of programming may return because of Big Data. New and smarter algorithms will have to be written from systems like Hadoop, Python, Pig to deliver more robust business applications.
In the past, it is sufficient to possess the ability to identify business opportunities. Now we need to be able to identify future business and information needs and objectives to constantly create opportunities for the company.
The ability to deliver clarity from complexity in different communication modes (presentation, white paper, social media conversation) is key to providing relevant insights to different stakeholders for call-to-action.
In conclusion, Big Data will be a big opportunity for companies who understand its value and application. Just like an army needs intelligence about its enemies to win the war, companies need business intelligence to defeat the speed of business irrelevance to survive.
Andrew Chow is a successful social media and public relations strategist, entrepreneur and speaker based in Singapore. He is also the best-selling author of a highly popular series of books: Social Media 247, Public Relations 247 and Personal Branding 247. Andrew has spoken in over 15 countries within 5 years and addressed more than 20,000 people on Digital Marketing, Personal Branding, Enneagram, Public Relations and Branding. Andrew’s career of 30 years; has seen him work with an array of clients including AXA Insurance, Abbot Medical Optics, Singtel and Sony Pictures, M1, Starhub, and Sennheiser. Andrew had more than 300 interviews and features about him or his business since 2005 from more than 40 local and regional media. He is listed as the Top 10 Most Influential Speaker in Singapore in 2013 by the Singapore Business Review. He won the Spirit of Enterprise in 2008 and the Successful Entrepreneur in 2010. Before he served as the President of the Asia Professional Speakers – Singapore (APSS), he also won the coveted Spirit of Service Award from the Industry. He loves travelling and held his solo Photo Art Exhibition for 3 days in Singapore to raise funds for a charity – Teen Challenge. Andrew is known by the moniker @ideasandrew in all his social media platforms.