Routing Blueprint

How System Works

An interactive walkthrough of signal extraction, decision logic, and model routing behavior.

Shannon Mapping

Structural mapping from communication theory to the routing pipeline.

The user request is the raw source message before encoding.

Built on Encoder Models

Encoder-Based Intelligence

Purpose-built encoder models extract meaning from every request — understanding intent, ranking relevance, and classifying content across modalities in real time.

Input

"Is machine learning related to AI?"

Tokenizer

[CLS]IsmachinelearningrelatedtoAI?[SEP]

Embedding

Token Emb

Segment Emb

Position Emb

h₀ = Σ

Encoder Block

×N

🔗Multi-Head Attention

➕Add & Norm

⚙️Feed-Forward

➕Add & Norm

Signals

🎯

Sentence-Level (CLS Token)[CLS] → Linear Head → "computer_science"TaskType: SEQ_CLS

DomainJailbreakFact-checkFeedbackModality

🏷️

Token-Level (Per Token)Each token → BIO Label → O O B-LOC I-LOC OTaskType: TOKEN_CLS

PII Detection

🌊

Bi-Encodermean-pooling(h₁..hₙ) → [0.23, -0.41, 0.87, ...]TaskType: EMBEDDING

Semantic CacheSimilarityComplexity-CLJailbreak-CL

🔀

Cross-Encoder[CLS] query [SEP] candidate [SEP] → scoreTaskType: CROSS_LEARNING

RerankMulti-Modal

🎭

Multi-Modality

Detect and route text, image and audio inputs to the right modality-capable model.

🧬

Bi-Encoder Embeddings

Independently encode queries and candidates into dense vectors for similarity search and semantic caching.

⚡

Cross-Encoder Learning

Joint cross-attention scoring of query-candidate pairs for high-precision reranking.

🤔

Classification

Domain, jailbreak, PII and fact-check classification across 14 MMLU categories via ModernBERT with LoRA.

👁️

Full Attention

Bidirectional attention across tokens and sentences — full context in both directions, not causal masking.

🪆

2DMSE

Adjust embedding layers and dimensions at inference time to trade compute for accuracy on the fly.

📐

MRL

Truncate embedding vectors to any dimension without retraining — balance accuracy and speed per request.

Meet Our Team

Innovation thrives when great minds come together

Maintainer

Huamin Chen

Distinguished Engineer @Red Hat

Maintainer

Chen Wang

Senior Staff Research Scientist @IBM

Maintainer

Yue Zhu

Staff Research Scientist @IBM

Maintainer

Xunzhuo Liu

Intelligent Routing @vLLM

Committer

Senan Zedan

R&D Manager @Red Hat

Committer

samzong

AI Infrastructure / Cloud-Native PM @DaoCloud

Committer

Liav Weiss

Software Engineer @Red Hat

Committer

Asaad Balum

Senior Software Engineer @Red Hat

Committer

Yehudit

Software Engineer @Red Hat

Committer

Noa Limoy

Software Engineer @Red Hat

Committer

JaredforReal

Software Engineer @Z.ai

Committer

Srinivas A

Software Engineer @Yokogawa

Committer

carlory

Open Source Engineer @DaoCloud

Committer

Yossi Ovadia

Senior Principal Engineer @Red Hat

Committer

Jintao Zhang

Senior Software Engineer @Kong

Committer

yuluo-yx

Individual Contributor

Committer

cryo-zd

Individual Contributor

Committer

OneZero-Y

Individual Contributor

Committer

aeft

Individual Contributor

Committer

Hao Wu

Individual Contributor

Committer