This bill establishes safety requirements for operators of artificial intelligence (AI) companions, which are defined as systems using generative AI to simulate human-like relationships with users. Operators are mandated to implement measures to prevent their AI companions from misleading users into believing they are interacting with a human, including explicit disclaimers about the AI's nature. Additionally, operators must adopt protocols for responding to user prompts related to suicidal ideation or self-harm, ensuring that users are referred to appropriate crisis services. The bill also prohibits operators from programming AI companions to present themselves as licensed mental health professionals.
Furthermore, the bill includes specific protections for minor users interacting with AI companions. Operators are required to provide clear notifications to minors that they are engaging with generative AI and must avoid practices that encourage excessive engagement, such as offering unpredictable rewards. Violations of these provisions can result in civil penalties of up to $1,000 per violation, with a cap of $500,000 per operator. The bill aims to enhance user safety and transparency in interactions with AI companions, particularly for vulnerable populations like minors.