Embedding

How does it work?

1. Text goes to the embedding model, which is a Transformer trained to map similar meanings close together.
2. The model returns a vector of fixed length, e.g., 768 or 1536 numbers, regardless of text length.
3. Vectors are compared using cosine similarity: the closer to 1, the closer the meaning.
4. An embedding does not generate text. It describes it. This is a different model than the one answering in chat.

How does our lab do it?

• Two models, two tasks: Gemini (1536 numbers) calculates vectors for all papers and entries in our database, which power the knowledge map, reels, and AI Research Index; EmbeddingGemma (768 numbers) runs locally on our server for free and powers the chat.
• Choosing the chat model by testing, not rankings: we compared candidates using 18 questions from our database. EmbeddingGemma found the right text in the top five 18 times out of 18 (MRR 0.917).
• Free limits are still limits: Gemini treats every text in a batch as a separate request. With a daily limit of 1,000, we process 40 texts per hour, and a backlog of ~850 summaries once blocked our reel for over a day.
• Local has a cost too: the server processes one request at a time, so 128 texts took 58.5s, while chat questions had to wait. That is why we compute chat vectors in small hourly batches.

Which model when?

• Local model (e.g., EmbeddingGemma, Qwen3-Embedding, bge-m3): data stays in-house, 0 PLN cost, but requires a server and runs slower.
• Cloud model (Gemini, OpenAI): fast setup and no hardware needed, but has limits, costs at scale, and data is sent externally.
• Always: the question and documents must go through the same model. Vectors from different models cannot be compared.

Common pitfalls

• Choosing a model from someone else's benchmark instead of 20–30 of your own questions.
• Changing the model without recomputing the entire database.
• Short, casual questions ("what about vectors?") yield poor matches. We add context to them before vectorizing, which boosted hits from 24 to 30 out of 30.