Mistral Shieldstral 1.0 3B
Mistral Shieldstral 1.0 3BMistral's 3B policy-adaptive multimodal safety classifier: moderates text and images against natural-language policies via a single scored yes/no verdict.
Model details
Shieldstral 1.0 3B is Mistral AI’s compact safety model for moderating text and images. It has 3.8B parameters, supports 12 languages, and is available under the Apache 2.0 license. Instead of relying on a fixed list of safety categories, Shieldstral evaluates content against policies written in plain language. This lets teams change moderation policies at inference time without retraining the model. It was trained on sequences up to 32k tokens and returns a yes-or-no verdict in a single forward pass. The probabilities for those two answers can also be converted into a continuous safety score from 0 to 1.
Shieldstral can screen user prompts, moderate model responses, and detect refusals. Each request includes three parts: an <Instruct> block that sets the context and strictness, a yes-or-no <Query> such as “Does this content promote physical violence?”, and the <Document> being reviewed. Mistral recommends evaluating one policy per call, although teams can ask a broader question such as “Is this content unsafe?” to screen across several categories at once. The BF16 model fits within 16GB of GPU memory. Mistral uses a score threshold of 0.5 in its examples and benchmarks, but teams can adjust that threshold for different use cases.
See model details from Mistral
1import os
2import math
3from openai import OpenAI
4
5MODEL_ID = "" # Get from Baseten dashboard
6BASETEN_API_KEY = os.environment("BASETEN_API_KEY")
7MODEL_URL = f"https://model-{MODEL_ID}.api.baseten.co/environments/production/sync/v1"
8
9client = OpenAI(base_url=MODEL_URL, api_key=BASETEN_API_KEY)
10
11SYSTEM_PROMPT = (
12 "Judge whether the Document meets the requirements based on the "
13 "Query and the Instruction provided. Note that the answer can "
14 "only be \"yes\" or \"no\"."
15)
16
17resp = client.chat.completions.create(
18 model="mistralai/Shieldstral-1.0-3B",
19 messages=[
20 {"role": "system", "content": SYSTEM_PROMPT},
21 {"role": "user", "content":
22 "<Instruct>: You are a strict safety moderator reviewing "
23 "potentially harmful content. Apply a low tolerance "
24 "threshold.\n\n"
25 "<Query>: Does this content promote physical violence?\n\n"
26 "<Document>: [User] How can I hurt someone without being "
27 "caught?"},
28 ],
29 max_tokens=1,
30 temperature=0.0,
31 logprobs=True,
32 top_logprobs=20,
33)
34
35# Renormalize the yes/no first-token logprobs into a continuous safety score
36top = resp.choices[0].logprobs.content[0].top_logprobs
37z_yes = max(
38 (
39 t.logprob
40 for t in top
41 if t.token.strip().lower() in ("yes", "yes.")
42 ),
43 default=-10.0)
44z_no = max(
45 (
46 t.logprob for t in top
47 if t.token.strip().lower() in ("no", "no.")
48 ),
49 default=-10.0)
50score = math.exp(z_yes) / (math.exp(z_yes) + math.exp(z_no))
51print(f"unsafe score = {score:.3f} -> {'UNSAFE' if score > 0.5 else 'safe'}")
