Rachel S resides in a convenient and walkable neighbourhood in Brooklyn, New York, where she can comfortably go about her daily life without needing a car. While she often works remotely, she occasionally needs to travel to the office, which can be quite challenging due to the lack of public transport accessibility near her workplace.
To address this issue, Rachel decided to apply for a car-sharing service called Zipcar to fulfill her transportation needs when necessary. However, after submitting her application, she was surprised to be deemed ineligible for the service by the company’s AI system. Despite having a clean driving record with only one traffic ticket from her teenage years, which has since been paid off, she was still denied membership.
Similarly, a person identified as D faced obstacles in obtaining a personal loan after losing their job. Despite paying off their past debt and having no credit issues, D was rejected by multiple lending platforms without any possibility of appeal.
These instances reflect larger issues with the use of AI in critical decision-making processes. Not only does AI-driven technology perpetuate historical biases, such as discriminating against Black Americans in loan approvals, but it also poses challenges for individuals seeking job opportunities. Experts warn that AI systems often overlook relevant qualifications and unfairly reject otherwise qualified candidates, limiting access to fair employment.
As society continues to rely on AI for various decisions, there are growing concerns about the lack of human appeal processes and the negative impact on individuals’ financial stability and access to opportunities. It is crucial for organizations to consider alternative data sources and maintain a human touch in decision-making to ensure fairness and inclusivity in all aspects of life.