Artificial intelligence systems are powerful tools for businesses and governments to process data and respond to changing situations, whether on the stock market or on a battlefield. But there are still some things AI isn’t ready for.
We are scholars of computer science working to understand and improve the ways in which algorithms interact with society. AI systems perform best when the goal is clear and there is high-quality data, like when they are asked to distinguish between different faces after learning from many pictures of correctly identified people.
Sometimes AI systems do so well that users and observers are surprised at how perceptive the technology is. However, sometimes success is difficult to measure or defined incorrectly, or the training data does not match the task at hand. In these cases, AI algorithms tend to fail in unpredictable and spectacular ways, though it’s not always immediately obvious that something has even gone wrong. As a result, it’s important to be wary of the hype and excitement about what AI can do, and not assume the solution it finds is always correct.
When algorithms are at work, there should be a human safety net to prevent harming people. Our research demonstrated that in some situations algorithms can recognize problems in how they’re operating, and ask for human help. Specifically, we show, asking for human help can help alleviate algorithmic bias in some settings.
How sure is the algorithm?
Artificial intelligence systems are being used in criminal sentencing, facial-based personality profiling, resume screening, health care enrollment and other difficult tasks where people’s lives and well-being are at stake. U.S. government agencies are beginning to ramp up their exploration and use of AI systems, in response to a recent executive order from President Donald Trump.
It’s important to remember, though, that AI can cement misconceptions in how a task is addressed, or magnify existing inequalities. This can happen even when no one told the algorithm explicitly to treat anyone differently.
For instance, many companies have algorithms that try to determine features about a person by their face – say to guess their gender. The systems developed by U.S. companies tend to do significantly better at categorizing white men than they do women and darker-skinned people; they do worst at dark-skinned women. Systems developed in China, however, tend to do worse on white faces.
The difference is not because one group has faces that are easier to classify than others. Rather, both algorithms are typically trained on a large collection of data that’s not as diverse as the overall human population. If the data set is dominated by a particular type of face – white men in the U.S., and Chinese faces in China – then the algorithm will probably do better at analyzing those faces than others.
No matter how the difference arises, the result is that algorithms can be biased by being more accurate on one group than on another.