ChatGPT Health in Australia Creates Regulation Concerns

A 60-year-old man with no prior history of mental illness was admitted to a hospital emergency department after claiming his neighbour was poisoning him, before developing worsening hallucinations and attempting to flee the facility within 24 hours. Doctors later determined he had been ingesting sodium bromide daily, an industrial chemical used in cleaning and water treatment, after purchasing it online.

The man reportedly began using sodium bromide as a salt substitute after receiving information from ChatGPT while trying to reduce dietary salt. Medical staff diagnosed bromism, a condition caused by bromide accumulation that can lead to hallucinations, confusion and impaired coordination.

The case has raised concerns among health experts amid the rollout of ChatGPT Health in Australia. Alex Ruani, a doctoral researcher in health misinformation at University College London, warned that users may struggle to distinguish between general information and medical advice when AI responses appear confident and personalised.

Ruani said there are no published, peer-reviewed studies specifically assessing the safety of ChatGPT Health. OpenAI said the tool was developed with input from hundreds of physicians and includes privacy protections, but it is not regulated as a medical device.