Melbourne psychiatrist refuses new patients who don’t consent to AI note-taking

View image in fullscreen

Registration form informs patients that if they do not wish AI to be used, they will need their referring doctor to refer them to a different service provider

• Get our breaking news email, free app or daily news podcast

A Melbourne psychiatrist has refused new patients unless they agree to allow her to use an AI scribe to transcribe the conversations in their sessions.

AI-driven note taking tools are becoming popular within the medical industry – with two in five general practitioners now using such scribes, according to the Royal Australian College of General Practitioners (RACGP).

But there have also been concerns about the security of the data and how it might be used by the AI companies, along with the accuracy of the transcriptions.

In a registration form seen by Guardian Australia, psychiatrist Dr Hemlata Ranga outlined the requirement to potential patients.

“I consent for use of AI transcription (such as Heidi health AI/ Microsoft) software to assist with notes taking during the appointments, for preparation of clinical letters, reports, and other clinical documents to assist in my clinical care,” the form says.

The patient is told if they do not wish AI to be used they will “need to arrange [their] regular doctor/ referring doctor to refer [them] to a different service provider”.

The uptake of AI scribes has doubled in 12 months, the RACGP says, and comes amid increased demand for mental health services.

But one patient, who asked to remain anonymous, who was referred to Ranga told Guardian Australia they objected to AI being used in their sessions.

“The way [the form] was worded – so defensively – it made me think that maybe there are a number of other people who have said something and that it’s gotten to the point where they just thought: ‘I just don’t want to deal with their objections any more. Let’s just make them sign something that says that we have to use it’.”

The patient said finding a psychiatrist was not like going to another chemist, especially when Ranga came recommended to them, and has not sought another psychiatrist.

“She’s not easily replaceable, and the process of having to go back to your doctor, make an appointment and often wait on another wait list to get back in … all just because you don’t want to use AI … it’s really weird.”

Ranga, whose practice is in the Melbourne Clinic in Richmond, did not respond to a request for comment.

A spokesperson for the Melbourne Clinic said it does not direct its accredited psychiatrists on their use of AI, but understands psychiatrists were increasingly incorporating the use of AI to help manage administrative load.

“In those cases, psychiatrists will disclose the use of AI to all patients and request consent prior to its use. If consent is not granted, they will not use AI.”

The spokesperson said practitioners at the clinic are ultimately independent from the clinic.

Heidi AI – one of the most popular AI scribes – has been used in 115m sessions in the past 18 months.

The Melbourne co-founder of Heidi AI, Dr Tom Kelly, said in March that data was processed in the country the patient was in and was not used to train the AI or sold to others. The company uses third-party testing and auditing to keep the data secure, and aims to ensure high standard in transcripts – but said doctors still need to check their notes.

Risks and regulations

The head of policy for Digital Rights Watch, Tom Sulston, said AI transcription was imperfect and was often trained on male, white, heterosexual, English-speaking users, meaning mistakes were more common for other demographics.

He said there was also a “very real risk” of medical data being compromised or leaked, and patients that were wary of AI scribes may self-censor.

“That’s particularly important with healthcare that may carry stigma in wider society – sexual and mental health, for instance,” he said.

“AI is not implemented to improve healthcare outcomes or patient experience, but to reduce administration costs for a clinic.”

“In this scenario, patients are not the customer, they’re the product.”

According to the RACGP consumer groups have raised concerns that consent conversations with patients over use of AI tools are either lacking, or there’s a power imbalance between patients and doctors.

Sulston said no one should be denied healthcare because they don’t want to be forced to give their private data to AI, and called for stronger regulation, given the tools are now exempt from Therapeutic Goods Administration regulations because they do not diagnose patients.

“Regulators and legislators need to step-up and provide Australians with a legally enshrined right to refuse AI systems without facing repercussions to our health,” he said.

The patient said people should be able to opt out of scribes and suggested they were not confident the notes made would be accurate.

• Health
• AI (artificial intelligence)
• Mental health
• news