These are documented, verified incidents in which the absence of adequate AI governance caused direct harm to individuals, organizations, or the public – and produced legal, regulatory, or financial consequences.
Air Canada Chatbot Ruling (2024)
In February 2024, the British Columbia Civil Resolution Tribunal ruled that Air Canada was liable for misinformation provided by its AI chatbot to a passenger seeking a bereavement fare. The chatbot incorrectly told passenger Jake Moffatt that he could book a full-price ticket and request the bereavement discount retroactively – which was false. When Air Canada refused the discount, it argued the chatbot was “a separate legal entity responsible for its own actions.”
Tribunal member Christopher Rivers dismissed that argument in terms that have since been cited across the global business community: “It should be obvious to Air Canada that it is responsible for all the information on its website. It makes no difference whether the information comes from a static page or a chatbot.” The ruling establishes a clear legal principle in Canadian law: organizations are accountable for what their AI systems say and do, without exception.
iTutorGroup EEOC Settlement (2023)
In August 2023, the U.S. Equal Employment Opportunity Commission settled its first AI-related discrimination lawsuit with iTutorGroup, a tutoring company. The company’s recruiting software had been programmed to automatically reject applications from women over 55 and men over 60. The settlement required iTutorGroup to pay USD 365,000 to more than 200 applicants who were automatically screened out due to age, and to submit proposed anti-discrimination procedures applicable to AI-assisted hiring. This case is widely cited as the first EEOC AI discrimination settlement in U.S. history.
Dutch SyRI Welfare Surveillance Case (2020)
A Dutch court ruled in February 2020 that the Netherlands’ SyRI system – an algorithmic welfare fraud detection tool – was unlawful because it violated the right to privacy under the European Convention on Human Rights. The system targeted low-income neighborhoods, combining data from multiple government sources to generate risk scores for individuals. The court found the legislation “insufficiently transparent and verifiable” and that it did not strike “a fair balance between the interest in fraud detection and the human right to privacy.” The Dutch government did not appeal and immediately halted the program. It became a foundational case in establishing that algorithmic welfare systems require genuine transparency and proportionality.
Australian Robodebt Scheme (Royal Commission Report, 2023)
The Australian government’s Robodebt scheme used an automated income-averaging algorithm to generate welfare debt notices, sending more than 20,000 notices per week at its peak. The Royal Commission into the Robodebt Scheme – which issued its final report on July 7, 2023 – found the scheme “a crude and cruel mechanism, neither fair nor legal.” The government ultimately settled a class action for more than AUD 720 million, reimbursing approximately 430,000 debts. The Royal Commission made 57 recommendations, including establishing a dedicated oversight body for automated government decision-making. Royal Commissioner Catherine Holmes found that “people were traumatised on the off chance they might owe money.”
Apple Card Credit Limit Disparity Investigation (2019–2021)
Following complaints from users – including tech entrepreneur David Heinemeier Hansson, who publicly reported that his wife received a credit limit 20 times lower than his despite having a higher credit score – the New York Department of Financial Services launched an investigation into Apple Card and its banking partner Goldman Sachs. The March 2021 DFS report found no evidence of intentional discrimination, but identified significant concerns about the opacity of the algorithm and the inability of affected consumers to obtain meaningful explanations for decisions. Harvard Business School noted the case as illustrating a “dark side to fintech” – that even unintentional algorithmic discrimination triggers regulatory scrutiny and reputational damage.
COMPAS Recidivism Algorithm (ProPublica, 2016)
The ProPublica investigation “Machine Bias” (2016) documented that the COMPAS algorithm – used by courts across the United States to inform parole and sentencing decisions – produced risk scores that were almost twice as likely to falsely flag Black defendants as future criminals, and almost twice as likely to falsely classify white defendants as low risk. The study found the score was “remarkably unreliable in forecasting violent crime: only 20 percent of the people predicted to commit violent crimes actually went on to do so.” The case became the defining public example of algorithmic bias in criminal justice and is now cited in virtually every major AI governance framework as a cautionary example.
Clearview AI Fines Across Europe
The facial recognition company Clearview AI – which built a database of more than 10 billion faces by collecting images from public websites without consent – has been fined by multiple European regulators for violating GDPR. The French CNIL fined Clearview EUR 20 million in October 2022 and ordered the deletion of all data on French residents within two months. The Dutch Data Protection Authority followed with a EUR 30.5 million fine in September 2024. Multiple other European jurisdictions issued similar orders. A UK appellate tribunal ruled in 2025 that UK GDPR applies to Clearview’s operations because its facial recognition activities constitute “monitoring of behaviour” of UK residents. The Clearview cases establish a clear international precedent: collecting and processing biometric data at scale without legal basis is not a business model – it is an enforcement target.
Italian ChatGPT Temporary Ban (2023)
In March 2023, Italy’s data protection authority – the Garante – temporarily blocked ChatGPT after OpenAI experienced a data breach involving user conversations and payment information. The Garante cited insufficient transparency about how personal data was collected, the lack of a legal basis for mass processing of Italian residents’ data to train the model, and the absence of age verification mechanisms. OpenAI implemented remediation measures and ChatGPT was restored to Italian users in April 2023. The episode demonstrated that even voluntary platforms used globally can be suspended by national regulators acting on privacy law – with no advance notice to business users.
NYC Bias Audit Enforcement (Local Law 144)
New York City’s Local Law 144 requires any employer using an automated employment decision tool (AEDT) in New York City hiring or promotion to conduct an independent bias audit annually, publish the results publicly, and notify candidates that such a tool is being used. Enforcement began July 5, 2023, with civil penalties of USD 500 to USD 1,500 per day per violation. A December 2025 audit by the New York State Comptroller found that enforcement challenges remain due to the difficulty of identifying non-compliant employers, but that the law has materially changed hiring practices in New York and set a precedent for U.S. state and local AI regulation.
Arup Deepfake Fraud – $25 Million (2024)
In January 2024, a finance employee at Arup – the London-based engineering firm – was deceived into making 15 wire transfers totalling HKD 200 million (approximately USD 25.6 million) to fraudulent accounts. The employee had participated in a video conference call in which every other participant – including individuals posing as the CFO and several colleagues – were AI-generated deepfakes. The fraud was only discovered when the employee followed up with Arup’s actual head office. As of early 2025, no arrests have been made and the funds remain unrecovered. The World Economic Forum cited Arup’s Chief Information Officer Rob Greig: “It’s freely available to someone with very little technical skill to copy a voice, image or even a video.” The incident has become the defining case for AI-enabled fraud risk in corporate cybersecurity.
