Big Data Ethics: racially biased training data versus machine learning

Writing in Slate, Cathy “Weapons of Math Destruction” O’Neill, a skeptical data-scientist, describes the ways that Big Data intersects with ethical considerations. O’Neill recounts an exercise to improve service to homeless families in New York City, in which data-analysis was used to identify risk-factors for long-term homelessness. The problem, O’Neill describes, was that many of the factors in the existing data on homelessness were entangled with things like race (and its proxies, like ZIP codes, which map extensively to race in heavily segregated cities like New York). Using data that reflects racism in the system to train a machine-learning algorithm whose conclusions can’t be readily understood runs the risk of embedding that racism in a new set of policies, these ones scrubbed clean of the appearance of bias with the…


Link to Full Article: Big Data Ethics: racially biased training data versus machine learning