Microsoft Azure AI Content Safety introduces advanced methods for harmful content detection

Today, Microsoft made significant announcements regarding generative AI safety features. Among these, a new tool within the Azure AI Content Safety framework has been introduced, aimed at detecting and rectifying AI-generated hallucinated content. Additionally, Microsoft revealed that its Azure OpenAI Service will begin implementing invisible watermarks in images generated through the DALL-E 3 model.

Furthermore, Microsoft is continuing its focus on AI safety with the public preview of its innovative Multimodal API within the Azure AI Content Safety service. This API is engineered to identify harmful or inappropriate content, whether it originates from human creators or AI tools.

According to a blog post by Microsoft:

The multimodal API accommodates both text and image inputs. It is designed for multi-class and multi-severity detection, allowing it to classify content across various categories and assign a severity score to each. For every category, the system provides a severity level ranging from 0, 2, 4, to 6. A higher number signifies greater severity of the content.

This newly launched Multimodal API can identify harmful content in both text and images, including emojis, that may possess unsafe or inappropriate subject matter. This encompasses explicit content, hate speech, violence, self-harm, and sexual content. Microsoft emphasizes that the API can detect such content, even when it appears in a combination of text and images, which might otherwise be benign when analyzed individually.

In its statement, Microsoft highlighted:

By achieving these goals, the multimodal detection feature promotes a safer and more respectful user experience, enabling creative yet responsible content generation.

Additionally, Microsoft noted that the new Multimodal API is capable of rapidly detecting harmful content, ensuring it is not disseminated to users of various applications or services.

Source