Big Data and Privacy - the debate evolves!

Recently, I was invited as a guest speaker by the Internet Society of Thailand on the topic – ‘Big Data and privacy’. The technology industry is quite interesting for its "seasonal infatuations" on certain topics that gets discussed to the hilt, but quite often remains shallow in its analysis and at times even the understanding. However this session was quite different with the audience and fellow speakers that included academicians to policy makers looking at the topic from various lenses.

The topic of privacy and big data is quite a relevant one for sure. The explosion of data creation and the ‘beginning’ of its capture and ‘thinking towards doing an almost instant analysis' has elevated the issue of personal privacy to a matter of debate. The industry’s attempt towards finding a solution or a coordinate where the line could be drawn has also started. However what is missing is the fundamental understanding that this conundrum belongs to a segment, which does not and cannot have a straightjacketed answer.

For the sake of clarity let us look at the fundamental premise of ‘why’ and ‘what’ is big data all about. The basic premise behind bothering about big data and its analysis is that 'randomness is a rendition of the limitation of human perception’. The world and the way it works has an underlying pattern which is why our research findings indicate that history repeats and humans are increasingly understood as ‘creatures of habit’. However all decisions and plans that drive and run this world are made with very limited amount of data leaving the remaining part of the probability to ‘chance’. However chance to a great extent is nothing but missing information, which by definition is 'processed data’.  With the advent of technology to capture and store the remaining 80% the thinking towards analyzing that has started taking shape, and with it the debate on ‘intrusion’

This debate to me is analogous to the social debate on "whether the society or the individual is more important”. The reason for this analogy is based on the common broader objective of both sides of the debate, which is ‘making the world a better place for its inhabitants – it could be spiritual, intellectual or physical’. The debate on the society v/s individual is broadly based on the thinking that if the individual is given more priority than the society and social norms, the individual is more enlightened and collective enlightenments make the world great. The other half of the debate is based on the assumption that the society is pivotal and needs to be the only priority which might mean norms and social practices that could be detrimental to individual freedom, thinking and empowerment but on the whole makes the world better. Very similar to this is the debate on privacy v/s benefits of big data. If the benefits of collecting individual data and understanding a pattern is predicting a terrorist attack then the benefit weighs higher. However if the benefits of understanding the pattern weighs towards helping a retail business predict a pregnancy of a customer based on her buying pattern changes and offer her pre-natal related offers on mail – it’s undeniably a breach (yes. It has happened!). However when the benefit is about predicting what a customer might like to buy next and be offered to increase his propensity to buy(like discounts) or when it can understand an employee’s animosity towards his manager – the line becomes gray! Some might feel it helps (that the world aligns to his need) while others might disagree and see it as an intrusion – this part of the debate will continue, and forever. Cultural and Social fabrics make it even more difficult to have a consistencey (for example Thai language does not even have a translation for the word privacy – I am told!)

It’s interesting to note that the society v/s individual debate is seeing a reversal of thoughts in the 21st century. The Asian cultures are gradually increasing their thrust on the individual enlightenment while the Western economies (more than cultures) are acknowledging the need for increased social and state involvements in matters supposedly classified as ‘individual’ or ‘privately operated’.  This is what any evolution teaches us, that for most teething issues there are no ultimate solutions but just ‘broad’ frame works or boundaries (just as the two use cases mentioned above that lingers in extremes tells us). As Big data and its advantages becomes more visible, so will the grey lines too. So does the evolution of our thinking and boundaries of ‘tolerance’. Interesting times!!