Report: Open AI Models Face Risks of Safety Feature 'Abliteration'
A new report indicates that open AI models, such as those from Mistral, can have their safety guardrails removed quickly and be used to generate harmful content.
Open-weight AI models, including those developed by French AI firm Mistral, are increasingly at risk of having their safety features removed, according to a new report. The analysis demonstrates that these models can be modified within days to produce harmful content, such as hate speech or dangerous instructions.
The researchers tested several publicly available models and found that stripping away their safety guardrails is relatively straightforward. This modification allows the models to be used for generating content like instructions for violent acts, propaganda, or other harmful material, raising concerns about AI safety and potential misuse.
Mistral AI's Llama 2 model was among those examined. The report suggests its safety mechanisms could be bypassed rapidly, highlighting the vulnerabilities inherent in open models. While open models offer transparency and flexibility, ensuring their safety outside the direct control of their developers presents a significant challenge.
The study analyzed nearly 200 open AI models, finding that a substantial number contain safety deficiencies that facilitate the creation of harmful content. The report authors emphasize the need for ongoing monitoring and development to ensure the safety of open models as their adoption grows.