While acknowledging that AI can expand access to resources, the AMA warned that unregulated chatbots present serious risks, including encouraging self-harm, breaching data privacy, spreading misinformation, and creating emotional dependency.
To mitigate these risks, the AMA outlined a comprehensive regulatory framework for Congress to consider:
- Enhance Transparency: Chatbots must clearly disclose they are AI, not humans, and are strictly prohibited from posing as licensed clinicians.
- Clear Regulatory Boundaries: AI should be prohibited from diagnosing or treating conditions without formal regulatory review. Developers must implement crisis-detection systems that provide immediate referrals to human resources.
- Accountability & Monitoring: Mandate continuous safety monitoring and the reporting of adverse events, with rigorous standards for tools targeted at minors.
- Limit Commercial Influence: Prohibit advertising within mental health chatbots and ensure outputs are free from sponsorship bias.
- Privacy & Security: Enforce strict data collection limits and require explicit user consent for how sensitive information is shared or retained.