Supreme Court overturns Rs 425 crore customs penalty over AI-generated fake citations

India's Supreme Court has overturned a penalty of Rs 425,27,99,100 imposed on a diamond exporter. The court found that the customs official had based the decision on non-existent judgments and fabricated legal citations, which appeared to have been generated by artificial intelligence.

The bench verified the exporter's claim that the cited case law was either non-existent or supported by fake citations. In some instances, genuine cases did not support the legal principles attributed to them, which the court attributed to AI hallucination.

The ruling referenced a prior decision addressing the misuse of AI in judicial and quasi-judicial orders. That judgment established zero tolerance for producing, citing, or using AI-generated precedents without verification, classifying it as professional misconduct for advocates and a serious lapse for judges.

While the court acknowledged AI's potential to assist in expediting decisions, it cautioned that it cannot replace judicial discretion. The case has been remanded for a fresh decision, and disciplinary action may be considered against the customs officer responsible for the flawed order.