Data mining is the process of analysing big data sets derived from various organisations platforms and using the information to inform decision-making, strategic planning and fine-tune business operations and processes, increasing efficiency and effectiveness. At a recent function I attended in Sydney, it was estimated that $67 billion of value was added to the Australian economy in 2013 alone thanks to data mining, with at least $48 billion worth of room to grow. With these sorts of numbers it is easy to see why data-driven innovation is transforming the western world, impacting heavily on our economy, society and the organisations within.

A recent report from the market intelligence firm IDC estimates that in 2009 stored information totaled 0.8 zetabytes or 800 billion gigabytes. “It is predicted that by 2020, 35 zetabytes of information will be stored globally”. So what are some of the benefits of analysing this huge amount information?

The Benefits of Data Mining

Data mining has a seemingly endless array of applications and has become a very important tool for organisations to uncover patterns and valuable information to improve function and product/service offerings for the consumer. Every sector is taking advantage of this innovation, using it to predict weather patterns, track the location of public transport using GPS for apps like tramTRACKER, use real time information to optimise harvesting and water catchment, detect issues with machinery and supply chain to reduce waste as well as improve delivery of educational content by tailoring lessons to individual skills and learning styles.

A key example for me however, is its value in the healthcare setting with:

• Health insurers being able to detect fraudulent and abusive activity more readily by tracking unusual or inappropriate claims;
• Healthcare providers being able to identify the most effective course of action for specific patient groups by comparing various treatments and their outcomes;
• Development of best-practice guidelines and standards of care by comparing millions of cases from around the world end-to-end;
• More comprehensive and affordable health services for consumers by tracking high-risk patients and designing appropriate interventions to increase efficiency of service provision, reducing hospital admissions and claims; and
• Information to guide patient interactions and improve patient satisfaction by determining preferences, usage patterns, and needs.

These are just some examples that highlight the enormous benefit data mining can have on society. Having said that, there are also some examples that raise concerns on the ethical use of data and consumer privacy; one of which I came across recently called Monkey Parking.Monkey Parking is an app with the premise of connecting drivers looking for a parking space with those who are about to leave their space. The idea is to help save time and reduce congestion with drivers no longer having to circle the block until a space appears. Where the controversy arises however, is that the people leaving the park are taking bids for the spot, essentially selling a public space that they do not own.

This is creating what San Francisco City Attorney Dennis Herrera describes as a “predatory private market for public parking spaces” exacerbating the fast-growing divide between rich and poor. Monkey Parking’s Italian based creators are justifying this apparent breach of law by insisting they are only selling information, and are not the arbitrator of who and who doesn’t get parking spaces. Although fighting the decision, Monkey Parking has recently been forced to shutdown.

So is this form of free-market economy just or is it what some people view as a tech company profiting off selling consumer data like that of Google, Amazon and Facebook?

Data Mining and Consumer Privacy

Many San Franciscans dislike the idea of apps like Parking Monkey and believe companies like Google owe “their users countless billions for selling off their private data for profit and making products that “innovate” the surveillance state”.

Like San Franciscans, people all over the world are becoming aware that there is little we can hide in our online lives and whether we want to or not, we are disseminating personal information in many of our daily activities thanks to our transactions, IP address, GPS and tracking cookies. This has led to a new profession called data broking where information is bought from companies like Monkey Parking and Google and used to build massive databases with detailed records on hundreds of millions of consumers that is then sold to various organisations. Information on your age, race, sex, location, education level, economic situation, buying habits, medical and psychological health concerns, etc. is all available for organisations to data mine and they are well within the boundary of the law.

The key concern for consumers, as raised by Minister for Communications, Malcolm Turnbull at a recent function is, who really owns this data? Is it the consumer who undertakes various online activities? Or the organisations that own the platforms we are using? Do organisations have the right to sell data we have knowingly or unknowingly given them? Or is there a legitimate need for these organisations to track our online lives? Is it ethical to create behavioral profiles of people without their permission? Or do the advantages outweigh the disadvantages? Are these organisations infringing on our privacy for profit? Or will selling this data benefit society as a whole?

Do we need a Code of Ethics for Big Data?

Like many others, I strongly agree with data mining as a process, recognising the vast benefits to society when used appropriately. I also understand consumer concerns however, in that, under current laws, there is room for organisations to misuse and invade our privacy for profit. All these questions and more have understandably led to the call for a code of ethics in regard to data collection and handling and I wholeheartedly agree. I believe a key component is the need for transparency, with businesses making it clear how consumer data is going to be used and who the information will be shared with. The aim being to prevent misuse and discrimination, enabling users to decide who can track their information and for what purpose, increasing user safety while maintaining benefit to society. Ultimately, I hope consumers feel protected and can see that their data is being used to benefit society in extraordinary ways.

If you enjoyed this post then you may enjoy our most recent whitepaper on how to effectively partner.