Could Big Data Have Prevented SA Riots?

Data analysis is crucial for any organisation that needs deeper insights into the challenges facing them, and informed and effective solutions to those problems. Had the South African government done so, measures could have been taken to prevent the country’s recent descent into violence, chaos, and destruction – not to mention loss of life.

Social media platforms like Facebook, Twitter, TikTok, and Instagram were awash with information as rioters used them to coordinate the looting and violent attacks that devastated business premises, infrastructure, and the collective psyche of South Africa as the world watched in disbelieving and appalling horror.

What is Big Data?

To understand the concept of big data, remember the three Vs!

Big data is information that arrives in ever-increasing and ever-vaster Volumes.
It is information that manifests in ever greater Variety.
It arrives faster all the time, so with increasing Velocity.

Traditional processing software and systems simply can’t cope with it. Big Data sets are simply too large, too complex, and too ‘new’. They are often generated by completely new data sources and come in structured, semi-structured, and completely unstructured formats.

Using predictive modelling, machine learning and advanced analytics, Big Data can be the motherlode for developing and mining vital new information and insights.

Could big data analysis have mitigated the #SAUnrest?

Governments across the world have realised that crucial insights can be gained from analysing Big Data, especially from social media platforms. This information can be leveraged in effective, impactful, and inventive ways to the benefit and safety of their citizens.

It can improve transparency, access to vital information, and boost efficiency in public management. Big data analytics can help governments understand trends and interpret developments and use those insights to improve and inform their decision-making.

South African academics believe had this been done by the government, the destructive impact of recent violent and criminal activities could have been significantly mitigated – in the sense of being forewarned and, therefore, be forearmed.

The South African government seems to lack the infrastructure and the technical skills to tap into and use the great benefits of big data. Ironically, SA did have a highly skilled corps of Secret Service and National Intelligence personnel. However, these skills were largely lost and pushed out during the Zuma presidency, leaving South Africa vulnerable.

And yet, the University of Johannesburg has already developed a platform that analyses and measures the feelings and the mood of a country by tapping into and analysing big data from social media. It is part of an ongoing project, the Gross National Happiness Index for 11 nations, including South Africa, New Zealand, and Australia.

The project measures the mood of a nation as revealed on the Twitter API. The team analyses and categorises the emotions underlying tweets, namely trust, anger, anticipation, surprise, fear, disgust, joy, or sadness. This index clearly showed negative and unsettled sentiment across South Africa, with emotions like fear and anger predominating.

So, yes, analysis of Big Data from social media could have helped if the South Africa government had taken cognisance of the mood of the nation. Informed and effective measures could have been deployed to mitigate the devastating effects of the violent insurrection that South Africa fell prey to.

This is just one example of the many potential applications for Big Data Analysis.

Opportunities in Big Data for IT professionals

Most organisations have now emphatically climbed onto the Big Data bandwagon. Swathes of Big Data are being collected, organised, and interpreted across most sectors of the economy. With this development have come lucrative and challenging jobs to drive this juggernaut of technological expansion:

Data Engineers

Data engineers are required to employ their computer science and engineering skillset to manipulate, aggregate, and analyse massive data sets for their organisations. This would include:

Developing and translating computer algorithms into prototype code
Creating technical systems to boost data accessibility
Writing reports, designing dashboards and other tools for end-users

Data Analyst

Data analysts typically correlate useful information by designing and implementing surveys, often on a large scale. They need to select people to participate in the surveys. They then have to compile and interpret all the data and then submit their findings both in traditional formats, like graphics and charts, as well as in digital form.

Data Scientist

Job opportunities abound for talented IT data scientists who can mine and interpret the complexities of big data as they develop models of statistical data to inform and drive recommendations and systems-related action plans.

Database Manager

With their skills in project management and abilities to multitask, database managers are required to perform diagnostics and maintain and repair sophisticated databases. They are also required to evaluate data sources, assess requests for data, boost data feeds, and design/install storage hardware.

Technical Recruiter

The technical recruiter’s speciality is sourcing and screening qualified talented big data candidates according to their organisation’s staffing requirements and specific job vacancies. They will then also support and walk the selected candidate through their application, interview, and onboarding processes.

Data Architect

The data architect uses his/her in-depth knowledge of computer languages to maintain and organise the data in company repositories and relational databases. Their duties also include developing strategies for the data architecture of each subject area of their data model.

Security Engineer

The importance of the security engineer’s role in IT disaster planning, mitigation and reduction are crucial. They reduce risk exposure for organisations by establishing firewalls, spotting and dealing with intrusions, and focusing on security aspects. In addition, they design, implement and test protocols for new/updated hardware and software. Crucially, they implement defence systems for the computer networks of their organisations.

How to expand your Big Data Analysis skills

Convinced? Here are some ways in which you can boost your Big Data analysis skills:

1. Acquire a new skill by doing an online course

There are many excellent courses, lectures, and tutorials available online to enable you to learn new skills. Here are just a few:

2. Invest in some data science books

There is a wealth of excellent literature available to help any data scientist on their path to mastering their subject. Here are just a few:

For beginners:

Naked Statistics by Charles Wheelan
Think Stats: Probability and Statistics for Programmers by Allen B. Downey
Data Science from Scratch by Joel Grus

If you’re at the intermediate level:

Python for Data Analysis by Wes McKinney
Python Data Science Handbook by Jake VanderPlas

Advanced level:

Deep Learning with Python by Francois Chollet, who is the creator of Keras
Deep Learning by Ian Goodfellow, Yoshua Bengio and Aaron Courville

3. Blogs

Blogs are an excellent way of learning new skills or consolidating those you already have in 10 – 15 minutes. Look at sites like:

Medium, specifically the Analytics Vidhya community
Towards Data Science

4. Practice your technical skills as much as you can

Doing this consistently will ensure you ace that intimidating technical interview! Here are some resources that will help you practise the different programming languages:

5. Start a side project

There are few better ways of honing your skills than this! Sites like Kaggle can be a great help to you here.

Whether you want to pursue a whole new career path or simply expand your skillset, learning about Big Data is one of the smartest moves you could make right now.