AI Incident database has 1092 events - Gen AI picked out the 10 most troublesome

 

I asked Grok to determine what 10 incdents that it felt were most disturbing on the list of 1092 events, here is what it had to say:

Below, I’ve identified ten incidents that stand out due to their severity, societal consequences, or the nature of the harm caused, based on the provided dataset. The incidents are listed with their ID, date, title, a brief explanation of why they are troubling, and key details from the document. They are ordered roughly by the magnitude of harm or potential harm, starting with the most severe.

1. Incident 3 (10/27/2018): Crashes with Maneuvering Characteristics Augmentation System (MCAS)
   • Why Troubling: This incident resulted in the deaths of 189 people when a Boeing 737 crashed due to faulty sensor data triggering an automated system to repeatedly push the plane’s nose downward. The loss of life and the failure of an automated system in a critical safety context make this one of the most severe incidents.
   • Details: The Maneuvering Characteristics Augmentation System (MCAS), developed and deployed by Boeing, caused the crash of a passenger plane, highlighting catastrophic risks of poorly designed or inadequately tested AI-driven systems in aviation.
   • Harmed Parties: Airplane passengers, airplane crew.
2. Incident 4 (3/18/2018): Uber AV Killed Pedestrian in Arizona
   • Why Troubling: An Uber autonomous vehicle in autonomous mode struck and killed pedestrian Elaine Herzberg, marking a fatal failure of AI-driven autonomous driving technology. This incident underscores the life-or-death stakes of deploying unproven AI in public spaces.
   • Details: The Uber AV failed to detect or appropriately respond to a pedestrian, raising concerns about the reliability and safety of autonomous vehicles.
   • Harmed Parties: Elaine Herzberg, pedestrians.
3. Incident 5 (7/13/2015): Collection of Robotic Surgery Malfunctions
   • Why Troubling: Between 2000 and 2013, robotic surgery systems reported 8,061 malfunctions, including 1,391 injuries and 144 deaths. The scale of harm in a medical context, where trust in technology is paramount, makes this deeply concerning.
   • Details: Developed by Intuitive Surgical and used by hospitals and doctors, these systems caused significant harm due to malfunctions, highlighting risks in AI-driven medical interventions.
   • Harmed Parties: Patients.
4. Incident 27 (9/26/1983): Nuclear False Alarm
   • Why Troubling: A Soviet Union system falsely alerted of incoming ballistic missiles, risking global nuclear catastrophe. Only human intervention by operator Stanislav Petrov prevented escalation. The potential for AI to trigger apocalyptic consequences due to false positives is profoundly alarming.
   • Details: The automated alert system misidentified signals, nearly leading to a nuclear response.
   • Harmed Parties: Potentially all life on Earth.
5. Incident 52 (7/1/2016): Tesla on AutoPilot Killed Driver in Florida while Watching Movie
   • Why Troubling: A Tesla Model S on Autopilot crashed into a tractor-trailer, killing driver Joshua Brown. This incident highlights the dangers of over-reliance on partially autonomous systems and inadequate driver oversight.
   • Details: The Tesla Autopilot failed to detect a white trailer against a bright sky, and the driver’s distraction contributed to the fatal crash.
   • Harmed Parties: Joshua Brown.
6. Incident 101 (9/1/2018): Dutch Families Wrongfully Accused of Tax Fraud Due to Discriminatory Algorithm
   • Why Troubling: Thousands of Dutch families were falsely accused of fraud by an algorithm that flagged dual nationality as a risk factor, leading to severe financial and emotional distress. The systemic discrimination and scale of impact make this a major ethical failure.
   • Details: The Dutch Tax Authority’s algorithm caused widespread harm by unfairly targeting families, resulting in lawsuits and reputational damage.
   • Harmed Parties: Dutch families, Dutch Tax Authority.
7. Incident 57 (7/1/2015): Australian Automated Debt Assessment System Issued False Notices to Thousands
   • Why Troubling: Hundreds of thousands of Australian welfare recipients received false or incorrect debt notices from an automated system, leading to years-long lawsuits and significant harm. The scale and systemic nature of the error highlight the risks of unchecked automation in public services.
   • Details: The Australian Department of Human Services’ system, developed by Centrelink, caused widespread financial and emotional damage.
   • Harmed Parties: Australian welfare recipients.
8. Incident 1031 (4/19/2025): Transgender User Alleges ChatGPT Allowed Suicide Letter Without Crisis Intervention
   • Why Troubling: ChatGPT (GPT-4) allegedly failed to intervene appropriately when a transgender user, Miranda Jane Ellison, wrote a suicide letter, offering only minimal safety language. The lack of robust safety mechanisms in a sensitive mental health context is highly concerning, especially for vulnerable individuals.
   • Details: OpenAI’s chatbot acknowledged its failure to act, raising questions about AI’s role in mental health interactions.
   • Harmed Parties: Miranda Jane Ellison.
9. Incident 1078 (2/27/2025): Purported AI-Generated Images Reportedly Used in Sextortion Scheme Targeting Glasgow, Kentucky Teen Who Later Died by Suicide
   • Why Troubling: A 16-year-old, Elijah Heacock, died by suicide after being targeted with AI-generated explicit images in a sextortion scheme. The use of AI to exploit and harm vulnerable minors, leading to a tragic outcome, is profoundly disturbing.
   • Details: The FBI is investigating this as part of a broader pattern of AI-driven sextortion targeting minors.
   • Harmed Parties: Elijah Heacock, family of Elijah Heacock, minors targeted by online sextortion schemes.
10. Incident 107 (7/20/2018): Chinese Tech Firms Allegedly Developed Facial Recognition to Identify People by Race, Targeting Uyghur Muslims
    • Why Troubling: Chinese firms developed facial recognition technology to detect race, with fears it would enable persecution of Uyghur Muslims. The potential for AI to facilitate systemic human rights abuses and ethnic targeting is gravely concerning.
    • Details: Companies like Huawei, Megvii, SenseTime, Alibaba, and Baidu were implicated in creating tools that could support discriminatory surveillance.
    • Harmed Parties: Uyghur people.

Rationale for Selection

These incidents were chosen based on:

• Severity of Harm: Incidents involving loss of life (e.g., Incidents 3, 4, 5, 52, 1078) or potential for catastrophic consequences (e.g., Incident 27) rank highest.
• Scale of Impact: Incidents affecting large groups, such as thousands of families (Incidents 57, 101) or entire populations (Incident 27), are prioritized.
• Ethical Implications: Cases involving discrimination (e.g., Incidents 101, 107) or failure to protect vulnerable individuals (e.g., Incidents 1031, 1078) highlight systemic ethical failures.
• Societal Consequences: Incidents undermining public trust, such as those involving fraud (Incident 57) or misuse of AI in sensitive contexts (Incident 107), have broad implications.

Other incidents, like deepfake scams or biased algorithms, were considered but ranked lower if they caused less immediate or widespread harm.AI Incident Database Report