Weapons of Math Destruction by Cathy O’Neil - Book Summary

This is a very important book that seeks to call to account algorithm-based automated computer systems which are increasingly making decisions about a whole range of aspects of our lives. With a Harvard Doctorate in algebraic number theory, Cathy O’Neill has an insider’s of view of WMDs, having both lectured on the subject and subsequently worked for a Hedge Fund on Wall Street at the time of the 2008 crash. 

The central theme is that those designing these systems which Cathy O’Neill wittily dubs Weapons of Math Destruction (WMDs) often set out with the best of intentions but that they are often very flawed. WMDs often promise efficiency, fairness and freedom from human prejudice, but in practice sacrifice fairness, justice and equality on the altars on efficiency and profit. 

The book has a simple structure working through how WMDs are making (flawed) decisions about: 

• Banking 
• University Entrance 
• Online Advertising 
• Policing and Justice 
• Job Recruitment 
• Hours of Employment and Shift Patterns 
• Credit Ratings • Getting Insurance 
• Political Elections and Social Media Advertising Campaigns 

Her fundamental criticisms of many of these WMDs are 

1. they are opaque, 
2. they don't take into account feedback to improve the mathematical model, and 
3. they rarely use real data; instead they use proxies. 

The scariest example (of many in the book) was that insurance companies use credit scores as a proxy for careful driving - thus in Florida adults with clean driving records and poor credit scores paid an average of $1,552 more than the same drivers with excellent credit scores and a drunk driving conviction (p.165) Indeed, the whole chapter on credit ratings and how various organisations are using big data to evaluate our credit-worthiness is quite frightening. (Did you know that Facebook patented a type of credit rating based on our social networks? (p.155 – See ‘Could a Bank Deny Your Loan Based on Your Facebook Friends?’ Atlantic Magazine 25/09/2015). 

The losers of so many of these WMDs is that the poor and minorities who are caught in a trap from which they cannot escape, be that policing algorithms which mean that they are more likely to be stop-and-searched; or it recidivism algorithms which give them longer jail sentences, or banking and insurance algorithms which charge them more for credit or insurance cover. 

The book is ultimately a call for greater regulation of WMDs so that there is greater transparency about their assumptions and methods. This is a must read for anyone who believes in a fair civil society.