Good and short introduction for anyone interested in the subject of big data and its potential negative side effects, I think the core message can be summarized in this quote:
Big Data processes codify the past. They do not invent the future. Doing that requires moral imagination, and that's something only humans can provide.
Data science is nothing new, but I don’t think many people are familiar with how its being applied, chances are that you have already been indirectly affected by it, positively or negatively. So how is it being used then? When people try to answer a seemingly simply question with a system, questions like which school is the best? Can I trust this person enough to give a loan? Will this person end up in jail again? Through computers and enough data available we believe we can answer these questions easily, though in reality many of these systems end up as a Weapon of Math Destruction (WMD).
Most of the time people’s own biases and racisms are unintentionally encoded into these systems, for example race might not be an explicit variable but zip codes can make for a near identical substitute. In the end these systems end up working great for many people and can go horribly wrong for others, do you live in a bad neighbourhood but have no criminal record? Chances are you’ll be judged more harshly than someone from a different background. In the race for improved school ratings tuition fees is not a variable which has contributed to the astronomical increase in fees for students.
The book starts with defining what a WMD is and continues by going through various examples in great detail. it kind of stops a bit abruptly after this without much further elaboration. Most of the examples throughout the book are very US-centric, where regulation is usually more reactive rather than pre-emptive, compared to Europe, as far as I know these systems are not as widely adopted or even usable in Europe. I’d be very interested to know more of such systems being used here as it’s not something that’s generally widely reported on.
Bottom line, great short read that can open up your eyes to something that feels obvious in retrospect, which is one of the better results after finishing a book.