Large Language Model (LLM)

How does it work?

1. Pre-training: the model reads trillions of tokens of text and learns to predict the next word. This is how it acquires language and world knowledge.
2. Fine-tuning: the model learns to be a helpful assistant, including via human feedback (RLHF). This also leads to the tendency to agree, known as sycophancy.
3. Response: the model receives your question and writes a response token by token, each time choosing the likely continuation.

Underneath almost every LLM is the Transformer architecture from the paper "Attention Is All You Need" (2017).

How does our lab do it?

• Lab work relies on the free Gemini tier (Flash Lite model): source evaluation, writing, translations. Cost of lab work models: 0 PLN per month.
• The website chat uses a paid model via OpenRouter with zero data retention (the provider does not store conversations). Free Gemini returned overload errors in about 3 out of 10 tries, and human chat must always respond.
• We dropped local LLMs. Three models at once on our server (15 GB RAM, no GPU) pushed it into swap memory and everything slowed down. Locally we only compute vectors (embeddings), because a small model is enough for that.
• The model writes, code checks. Numbers in reports are compared with data by a program, and publication is approved by a human.

Limitations

• Hallucinations: the model can confidently state falsehoods.
• Sycophancy: the model tends to agree with the user.
• Training cutoff knowledge: without search (RAG), the model does not know what happened later.
• Context window: the model sees only a limited amount of text at once.