Vector database

How does it work?

1. The model converts each text into a vector, which is a list of several hundred or thousand numbers. Texts with similar meanings get similar numbers.
2. Vectors go into the database along with the text and source address.
3. The user's query is converted into a vector using the same model.
4. The database returns texts whose vectors are closest. This is usually measured by cosine similarity: 1 means identical meaning, 0 means no relation.
5. With millions of vectors, the database doesn't compare every single one. It uses an index (e.g. HNSW) that quickly finds "approximate nearest neighbors".

How does our lab do it?

• We don't buy a database. It's regular PostgreSQL with the pgvector extension on our server, costing 0 PLN per month.
• Vectors are generated by two models: Gemini (1536 numbers per text, with daily free tier limits) and local EmbeddingGemma (768 numbers, run on our hardware) for chat.
• The chat responds only when the closest text exceeds the threshold of 0.764. We determined this threshold by measuring 30 on-topic questions and 6 off-topic ones: all on-topic were above the threshold, all off-topic were below, and the margin between them was 0.019.
• Most interesting result: the question "what are vector databases?" got a similarity of 0.733–0.756, which is the same as "who won the match?" (0.745). The threshold worked well. Simply put, among ~20k texts in the database, only 3 titles discussed vectors. A vector database cannot find an answer that isn't in it. This page was created partly to give the chat somewhere to get it from.

When is it worth it, when not?

• Worth it: company document chat, semantic search, duplicate detection, "similar content" recommendations.
• Not worth it: when you are looking for exact values (invoice number, product code), where traditional search wins. Also with a small number of documents that fit entirely within the model's context window.

Common pitfalls

• Changing the model = recalculating everything. Vectors from two different models are incomparable.
• Arbitrary threshold. Without measuring on real questions, the chat either refuses everything or answers everything.
• Overly long chunks. A single vector for a long article dilutes the meaning. We chunk down to ~1.2k characters.
• Missing content looks like a threshold error. First check what is in the database, then tune the threshold.