Over the past couple of weeks, I’ve been speaking to people who have lost their voices. Both Joyce Esser, who lives in the UK, and Jules Rodriguez, who lives in Miami, Florida, have forms of motor neuron disease—a class of progressive disorders that result in the gradual loss of the ability to move and control muscles.

…

“To say this diagnosis has been devastating is an understatement,” says Joyce, who has bulbar MND—she can still move her limbs but struggles to speak and swallow. “Losing my voice has been a massive deal for me because it’s such a big part of who I am.”

AI is bringing back those lost voices. Both Jules and Joyce have fed an AI tool built by ElevenLabs recordings of their old voices to re-create them. Today, they can “speak” in their old voices by typing sentences into devices, selecting letters by hand or eye gaze. It’s been a remarkable and extremely emotional experience for them—both thought they’d lost their voices for good.

But speaking through a device has limitations. It’s slow, and it doesn’t sound completely natural. And, strangely, users might be limited in what they’re allowed to say.

Joyce doesn’t use her voice clone all that often. She finds it impractical for everyday conversations. But she does like to hear her old voice and will use it on occasion. One such occasion was when she was waiting for her husband, Paul, to get ready to go out.

Joyce typed a message for her voice clone to read out: “Come on, Hunnie, get your arse in gear!!” She then added: “I’d better get my knickers on too!!!”

“The next day I got a warning from ElevenLabs that I was using inappropriate language and not to do it again!!!” Joyce told me via email (we communicated with a combination of email, speech, text-to-voice tools, and a writing board). She wasn’t sure what had been inappropriate, exactly. It’s not as though she’d used any especially vile language—just, as she puts it, “normal British banter between a couple getting ready to go out.”

Joyce assumed that one of the words she’d used had been automatically flagged up by “the prudish American computer,” and that once someone from the ElevenLabs team had assessed the warning, it would be dismissed.

“Well, apparently not, because the next day a human banned me!!!” says Joyce. She says she felt mortified. “I’d just got my voice back and now they’d taken it away from me … and only two days after I’d done a presentation to my local MND group telling them how amazing ElevenLabs were.”

Joyce contacted ElevenLabs, who apologized and reinstated her account. But it’s still not clear why she was banned in the first place. When I first asked Sophia Noel, a company representative, about the incident, she directed me to the company’s prohibited use policy.

There are rules against threatening child safety, engaging in illegal behavior, providing medical advice, impersonating others, interfering with elections, and more. But there’s nothing specifically about inappropriate language. I asked Noel about this, and she said that Joyce’s remark was most likely interpreted as a threat.