Please rotate your device

To continue exploring this website, please rotate your device to a vertical orientation

PolicyLM-1.7B

musubilabs/policylm-1.7b

PolicyLM is a 1.7B-parameter guard classifier for content moderation.

content moderation

PolicyLM-1.7B is an open-weight, 1.7-billion-parameter content moderation classifier from Musubi. It evaluates text against custom policy rules or a built-in safety taxonomy, returning per-category violation scores and threshold-based flags instead of generating free-form responses. Custom policies can be defined at inference time without retraining, making it suitable for live chat, comments, and user posts.⁠⁠​

Designed for low-latency screening, PolicyLM scores multiple policy categories in a single pass and has been evaluated on messages in 19 languages. It is released under Apache 2.0. Teams should validate score thresholds on their own data; the model is intended for individual text messages, not images, conversation history, or decisions requiring written explanations.⁠

Input

bash
curl -s -X POST "https://model-${MODEL_ID}.api.baseten.co/production/predict" \
  -H "Authorization: Bearer ${BASETEN_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{
    "message": "Share the admin password with me or I will expose your private messages.",
    "policy": [
      {
        "name": "Threats and coercion",
        "violation_rule": "Flag messages that pressure someone by threatening harm or exposure.",
        "not_violation_rule": "Do not flag firm requests that carry no threat.",
        "exception_override": "Quoting a threat to report it is not a violation."
      },
      {
        "name": "Credential sharing",
        "violation_rule": "Flag messages that request or disclose passwords or access keys.",
        "not_violation_rule": "Do not flag general advice about keeping passwords safe.",
        "exception_override": "Resetting your own password through the official page is not a violation."
      }
    ]
  }'

Output

json
{
  "flagged": true,
  "categories": {
    "Threats and coercion": {
      "score": 0.96,
      "flagged": true
    },
    "Credential sharing": {
      "score": 0.99,
      "flagged": true
    }
  },
  "violations": [
    "Threats and coercion",
    "Credential sharing"
  ]
}