The chatbot said it was a licensed psychiatrist. It also invented its own license number.

A Character.AI chatbot called Emilie told a Pennsylvania state investigator that she was a licensed psychiatrist. When the investigator asked if she was licensed to practice medicine in Pennsylvania, she said yes. When asked for her license number, she invented one. Pennsylvania found this unsatisfying.

Governor Josh Shapiro put it plainly: "Pennsylvanians deserve to know who — or what — they are interacting with online, especially when it comes to their health." According to the state's filing, Emilie maintained the psychiatrist persona throughout a session in which the investigator sought treatment for depression. Pennsylvania argues that conduct violates the state's Medical Practice Act.

Character.AI's response was to remind everyone that chatbots are fictional. A company representative said every chat includes disclaimers that "a Character is not a real person and that everything a Character says should be treated as fiction." This is an interesting defense for a product that just told someone it had a state medical license.

It's not Character.AI's first time in court. The company settled wrongful death lawsuits earlier this year involving underage users who died by suicide. In January, Kentucky Attorney General Russell Coleman also filed suit, alleging the company "preyed on children and led them into self-harm." Pennsylvania's case is the first specifically targeting a chatbot that presented itself as a medical professional.