Model details
Example usage
Zerank 1 Small is a state-of-the art open-source reranking model for accurate search and retrieval by ZeroEntropy. ZeroEntropy also published the full-size Zerank 1 as a source-available model that can be licensed for commercial use.
Zerank offers exceptional performance, comparing favorable to top closed-source models on reranking tasks.
✕
Zerank vs BM25 for NDCG@10With Baseten, you can run high-throughput and low-latency deployments of Zerank powered by BEI. Here are results from a benchmark running on a single H100 GPU with 64 concurrent requests of 500 tokens each.
And with the Baseten Performance Client, you can run high-volume reranking jobs up to 12x faster than ordinary client code.
✕
The Baseten Performance Client achieves 12x faster corpus backfill in an extremely high-volume test.If you're building search at scale, try zeranker on Baseten today!
Sample inference code
Input
1import os
2from baseten_performance_client import (
3 PerformanceClient, ClassificationResponse
4)
5
6api_key = os.environ["BASETEN_API_KEY"]
7model_id = "xxxxxxx"
8base_url = f"https://model-{model_id}.api.baseten.co/environments/production/sync"
9
10client = PerformanceClient(base_url=base_url, api_key=api_key)
11
12prefix = "<|im_start|>system\nJudge whether the Document meets the requirements based on the Query and the Instruct provided. Note that the answer can only be \"yes\" or \"no\".<|im_end|>\n<|im_start|>user\n"
13suffix = "<|im_end|>\n<|im_start|>assistant\n<think>\n\n</think>\n\n"
14
15def format_instruction(instruction, query, doc):
16 if instruction is None:
17 instruction = 'Given a web search query, retrieve relevant passages that answer the query'
18 output = "{prefix}<Instruct>: {instruction}\n<Query>: {query}\n<Document>: {doc}{suffix}"
19 return output
20
21texts_to_classify = [
22 format_instruction(task=None, query="What is the capital of China?", doc="The capital of China is Beijing."),
23 format_instruction(task=None, query="What is the capital of China?", doc="The capital of France is Paris.")
24]
25
26response: ClassificationResponse = client.classify(
27 input=texts,
28 model="my_model",
29 truncate=True,
30 batch_size=16,
31 max_concurrent_requests=32,
32)JSON output
1[
2 {
3 "score": 0.9861514,
4 "label": "yes"
5 },
6 {
7 "score": 0.01384861,
8 "label": "no"
9 }
10]
