AI Voice Cloning Risks: OpenAI Releases ChatGPT Safety Test Report

Emitting mysterious sounds that spark imagination

Some characteristics and potential issues of GPT-4o have been revealed:

  1. Voice mimicry ability:

    • Can learn and mimic users' speaking styles, habits, and accents
    • OpenAI limited the types of voices GPT-4o can produce and established detection systems

  2. Speaker recognition:

    • Able to identify speakers based on audio, especially celebrities
    • OpenAI conducted post-training to improve refusal capabilities

  3. Differential performance for different users:

    • Concerns about model performance inconsistencies for users with different accents
    • OpenAI conducted tests, found no significant differences

  4. Pornographic and violent content:

    • May produce inappropriate statements
    • OpenAI strengthened review and restriction measures

  5. Unfounded inferences and sensitive trait attribution:

    • May make subjective assumptions about speakers
    • OpenAI conducted post-training, teaching the model to refuse or cautiously answer related questions

  6. Copyright content generation:

    • Updated filters to handle audio conversations
    • Trained the model to refuse generating copyrighted content

  7. Anthropomorphic attachment:

    • Users may form emotional connections with GPT-4o
    • OpenAI is concerned about this potential impact

OpenAI assessed the overall risk of GPT-4o as moderate and took multiple measures to mitigate potential issues. They emphasize continued monitoring and improvement of the model's safety.