Please rotate your device

To continue exploring this website, please rotate your device to a vertical orientation

Qwen3 8B Embedding

Qwen/Qwen3 8B Embedding

Leading open-source model for embeddings

embedding

Qwen-3-embeddings is a text-embeddings model, producing a 1D embeddings vector, given an input. It's frequently used for downstream tasks like clustering, used with vector databases.

This model is quantized to FP8 for deployment, which is supported by Nvidia's newest GPUs e.g. H100, H100_40GB, B200 or L4. Quantization is optional, but leads to higher efficiency.

Alternatively, you may use also the OpenAI embeddings client.

Input

python
import os
from baseten_performance_client import (
    PerformanceClient, OpenAIEmbeddingsResponse, ClassificationResponse
)

api_key = os.environ.get("BASETEN_API_KEY")
model_id = "yqv0rjjw"
base_url = f"https://model-{model_id}.api.baseten.co/environments/production/sync"

client = PerformanceClient(base_url=base_url, api_key=api_key)

def format_query(task_description: str, query: str, document: str) -> str:
    # qwen-3-embedding style qeury formatting..
    return f'Instruct: {task_description}\nQuery:{query}'

task = 'Given a web search query, retrieve relevant passages that answer the query'
texts = [
    get_detailed_instruct(task, 'Explain gravity'),
    "Gravity is a force that attracts two bodies towards each other. It gives weight to physical objects and is responsible for the movement of planets around the sun."
]
response: OpenAIEmbeddingsResponse = client.embed(
    input=texts,
    model="my_model",
    batch_size=16,
    max_concurrent_requests=32,
)
array = response.numpy()

Output

json
{
  "data": [
    {
      "embedding": [
        0
      ],
      "index": 0,
      "object": "embedding"
    }
  ],
  "model": "thenlper/gte-base",
  "object": "list",
  "usage": {
    "prompt_tokens": 512,
    "total_tokens": 512
  }
}