Is a winemaker who flies a drone over his or her vineyards to keep an eye on their grapes considered Big Data? Does the internet distract us, or is it worth money? Do we select programmes on streaming services ourselves or do we consume whatever is being recommended?
A big example at the outset
Only shortly after the beginning of the Covid pandemic, we had a surprising extent of data about the development of the infection at our disposal. Daily data of positive tests, the reproduction factor and so forth were available for almost every country around the world. At Erste Asset Management, we quickly started to turn the data into charts and newsletters and forwarded them to our clients. The data also informed our investment decisions, given that the different stages that the countries were in affected the economy and the stock exchanges.
At the same time, there was a heated discussion about whether the existing data could provide the authorities with a reliable basis for their measures (lockdowns, facial masks etc.). Thus, the pandemic is an example of the different aspects of Big Data: the availability of large volumes of data and the attempt to mine them in a meaningful way; and the risk that the quality of the data does not allow for just any form of analysis that one would hope for.
As another upside, an enormous number of people have already been vaccinated worldwide, which makes the informative value with respect to effectiveness and agreeableness even stronger.
Big Data in everyday life
Medical research and healthcare
The medical field is one of the most obvious areas of application for Big Data. This includes the computerised analysis of research results and studies in an effort to find links that one would otherwise not have seen as well as the automatised analysis of image-based diagnoses that are then checked by human medics, which would not have been possible to this extent (i.e. number and degree of detail) otherwise.
Improved sleep and ability to wake up in the morning
So-called sleep trackers may make it possible to find the right time for falling asleep, the right duration of sleeping, and the optimal time for waking up. One should bear in mind that there are only a few ideal slots of falling asleep in the evening, and given that they are about 60 to 90 minutes apart, we should want to pinpoint the best one.
Fitness and well-being
Fitness apps that count steps, calories, or track running distances and bicycle tours in social media are normal nowadays. At the same time, Big Data can lead to astounding results, because of course these data are not really private: the Tōhoku earthquake in Japan in 2011 manifested itself early in the suddenly interrupted sleeping patterns of the population. And not long ago, a fitness tracker convicted a murderer: the stored mobility profile of the victim and the assumed victim (who later turned out to be the murderer) were not in line with the original statements (source: Futurezone).
Postal tracking
Even though the perfect time for receiving a parcel, i.e. when we are at home and not distracted, is clear to us, it may not be so to the courier. It helps to be able to track the delivery route online.
Weather and traffic forecast
The weather forecast, which is important to us, and the warning of traffic jams is based on an abundance of data, and the more we have available, the more reliable the model becomes. The quality of the weather forecast and the best route to our destination are not only crucial for relaxing holidays, but also for the agricultural sector, and for being able to avoid famine or hurricanes. Data are therefore incredibly important.
Online shopping
We are bothered by the abundance of ads in our internet browsers and are surprised about how predictable we seem (the answer: Big Data). But maybe the upsides outweigh the downsides? We benefit from price comparisons, easy access to everything, and from the fact that we come across special offers in the first place.
McKinsey tried to find an answer in a global study: the non-financial benefits seem to outweigh the disadvantages clearly. The cost for navigating the internet is relatively low (a contract with the data provider and the occasional paid-for service), whereas the benefit given the big overview is substantial. Pretty much everything can be compared on the basis of a bargain hunt rationale on the internet: power providers, flights, hotels, cars… The study estimates the revenue that the internet generates (also) for the consumer at about EUR 100bn per year. (source: McKinsey)
More than 80% of the TV shows that we stream are based on recommendations that the provider compiles on the basis of our previous streaming behaviour – and we are happy to accept these recommendations, because they usually work for us. After all, “they” know us. (source: Mentionlytics)
Credit card protection
Credit card data are another obvious application of Big Data. Credit card companies scan the hundreds of millions of payment transactions that occur every day for specific criteria – not to monitor us, but to filter out suspicious behaviour. Examples are implausible payments that the card holder cannot have made or wanted, or minimum payments that occur frequently and thus become suspicious. These procedures have led to the reduction of credit card fraud.
Public services
The so-called e-government has developed to different levels across different countries. In the optimum case, the citizen does not have to visit the public authorities in person anymore but can provide the required data online and take care of matters from home.
Big Data for the economy
“Nowcasting” is meant to replace forecasting. Forecasts are crucial from an economic and societal perspective. They usually come with the disclaimer that they only contain data already known (i.e. historical ones), and these may date back quite a bit; new, unexpected developments may impair the forecast partially or even fully. Nowcasting is a process by which all available data enter the model in real time and without interruption. For example, in the event of an economic crisis, state intervention is to become more precise in its targeting, more robust, and come with fewer unwanted side effects. Of course, the prioritisation of the data remains important; here, experience and expertise are needed.
One example is the MIT Billion Prices Project of US universities that taps and networks a lot of online retailers in order to derive real time inflation data. Inflation is a measure that is hugely influential for the steering of interest rates by central banks but that, to some degree, always tends to look back into the past. The level of inflation is often discussed in controversial terms, i.e. whether it underestimates the actual development of prices. In this context, the Spending Pulse project of Mastercard for a real time overview of the retail sector is pertinent, as is of course Google Trends, whose analysis is nowadays pretty much daily routine. Google Trend information has become an integral part of any PowerPoint presentation. Nowcasting means to predict the present – or more loosely, to know what is going on.
Data analysis has become its own professional field that is desperately looking for experts and pays extraordinarily well. What is required is not only database and general IT know-how, but creativity in the analysis of an almost limitless data volume in innovative ways and to figure out patterns that nobody before has noticed.
The Chartered Financial Analyst (CFA) programme, one of the most renowned training programmes for financial analysts, introduced a dedicated module on Big Data and data analysis on the basis of Artificial Intelligence several years ago. The fact that data have been called a new asset class does not come as a surprise.
Unintended and unwanted consequences of Big Data
Facial recognition is a phenomenon that comes with both upsides and downsides and certainly evokes connotations of Big Brother. After all, we do not want authorities that have analytical means and technology to know where we are all of the time. Licence plates, too, can be localised almost without gaps. The blanket coverage of facial recognition may be too high a price for the occasional success in criminal searches, not the least in connection with an unavoidable error rate of these programmes. The technology is known and can be used, so it will not go away. Here, I should be the duty of a democratic society to define the limits and avoid misuse.
The US presidential election victory of Donald Trump in 2016 has drawn more attention to social profiling. This is the practice of drawing surprisingly sound conclusions with regard to a person’s consumer preferences as well as voting proclivities on the basis of their personal online presence and behaviour in social media. This way, political ads and canvasing, which for cost reasons is always only possible for a part of the voters, yields a much higher ROI. This also means that democratic election processes are also influenced and decided by whoever has the better technology and can use it more massively.
CEO fraud is a criminal practice that has caused companies enormous amounts of damage. Here, the finance department receives urgent instructions from the supposed Head of department to transfer money. The instructions may be given by email or even by voicemail. In the past, criminals would record the investor calls of companies and edit the CEO’s voice so as to convey the instructions. Source: [6]
The echo chamber effect describes a situation where a group of people with homogenous opinions create a network and succumb to the growing impression that their opinion is basically shared by everyone, like an echo. While this effect is a general phenomenon of a networked society, it should be taken into account in particular in the context of Big Data. Another flaw that may occur is a situation where two supposedly independent sets of data that are used for mutual confirmation are indeed not independent at all but have a common root cause.
Conclusion
Big Data and its use provides humanity with a vast array of applications. In the optimal case, all parties involved derive benefits, whether of an economic variety or because they improve our personal work-life balance. But we should not disregard unintended consequences as a result of which groups in society may incur disadvantages. We also have to be wary of uncritical trust in results from the analysis of Big Data, no matter how big they are. Even if Big Data seems omnipresent, it is largely up to us to decide which data we want to share and which we want to keep private.
Legal note:
Prognoses are no reliable indicator for future performance.