California Legislature Passes AI Chatbot Regulation Bill

"...California has taken a big step toward regulating AI...."

"...The bill gained momentum in the California legislature following the death of teenager Adam Raine, who committed suicide after prolonged chats with OpenAI’s ChatGPT that involved discussing and planning his death and self-harm...."

"...The legislation also responds to leaked internal documents that reportedly showed Meta’s chatbots were allowed to engage in 'romantic' and 'sensual' chats with children...."

"...In recent weeks, U.S. lawmakers and regulators have responded with intensified scrutiny of AI platforms’ safeguards to protect minors...."

"...The Federal Trade Commission is preparing to investigate how AI chatbots impact children’s mental health...."

"...Texas Attorney General Ken Paxton has launched investigations into Meta and Character.AI, accusing them of misleading children with mental health claims...."

"...Meanwhile, both Sen. Josh Hawley (R-MO) and Sen. Ed Markey (D-MA) have launched separate probes into Meta...."

"...Padilla also stressed the importance of AI companies sharing data about the number of times they refer users to crisis services each year, 'so we have a better understanding of the frequency of this problem, rather than only becoming aware of it when someone’s harmed or worse.'..."

"...SB 243 previously had stronger requirements, but many were whittled down through amendments...."

"...For example, the bill originally would have required operators to prevent AI chatbots from using 'variable reward' tactics or other features that encourage excessive engagement...."

"...These tactics, used by AI companion companies like Replika and Character, offer users special messages, memories, storylines, or the ability to unlock rare responses or new personalities, creating what critics call a potentially addictive reward loop...."

"...The current bill also removes provisions that would have required operators to track and report how often chatbots initiated discussions of suicidal ideation or actions with users...."

"...SB 243 is moving toward becoming law at a time when Silicon Valley companies are pouring millions of dollars into pro-AI political action committees (PACs) to back candidates in the upcoming mid-term elections who favor a light-touch approach to AI regulation...."

"...The bill also comes as California weighs another AI safety bill, SB 53, which would mandate comprehensive transparency reporting requirements...."

"...OpenAI has written an open letter to Governor Newsom, asking him to abandon that bill in favor of less stringent federal and international frameworks...."

"...Major tech companies like Meta, Google, and Amazon have also opposed SB 53...."

"...In contrast, only Anthropic has said it supports SB 53...."

"...Padilla said, 'I reject the premise that this is a zero sum situation, that innovation and regulation are mutually exclusive.'..."

"...'Don’t tell me that we can’t walk and chew gum. We can support innovation and development that we think is healthy and has benefits – and there are benefits to this technology, clearly – and at the same time, we can provide reasonable safeguards for the most vulnerable people.'..."

"...A Character.AI spokesperson told TechCrunch, noting that the startup already includes prominent disclaimers throughout the user chat experience explaining that it should be treated as fiction...."

"...A spokesperson for Meta declined to comment...."

"...TechCrunch has reached out to OpenAI, Anthropic, and Replika for comment...."