Open a streaming app and the rows of titles look effortless. Behind them sits a ranking problem with a catalogue far too big to browse and a user who will give up within a minute or two if nothing looks good. Recommendation systems exist to solve exactly that: given what we know about this person and this catalogue, what should we show first? Understanding how recommendation systems work is useful well beyond streaming – the same ideas drive "customers also bought" boxes, job boards, news feeds and the product grid of a mid-sized online shop.
The core idea: predict what someone will engage with
Every recommender, however complex, predicts some kind of engagement – a click, a purchase, a play that lasts more than a few minutes, a rating – and then ranks items by that prediction. The data it learns from comes in two kinds:
- Explicit feedback: star ratings, thumbs up or down, reviews. Clear, but sparse; most people never rate anything.
- Implicit feedback: views, clicks, add-to-carts, purchases, watch time. Plentiful but noisy; a click isn't the same as liking something.
Most production systems run mainly on implicit feedback, because that's what users generate in volume.
Content-based filtering: "more like this"
The simplest approach recommends items similar to ones a user already liked, based on item attributes. If you bought a trail running shoe, show other trail shoes; if you watched a Scandinavian crime drama, show more of those.
It needs no other users' data, it works for brand-new items as soon as they have attributes, and it is easy to explain. The weakness is a narrow bubble: content-based systems rarely surprise anyone, and they're only as good as your product data. Messy categories and missing attributes produce weak recommendations.
Collaborative filtering: learning from the crowd
Collaborative filtering ignores item attributes and looks at behaviour. The idea: people who agreed in the past will tend to agree in the future.
- User-based: find users with similar histories to yours and recommend what they liked that you haven't seen.
- Item-based: find items that tend to be consumed by the same people. "Customers who bought this also bought..." is the classic example, and it scales well because item-to-item similarities change slowly and can be precomputed.
The more powerful version is matrix factorisation. Picture a huge table with users as rows, items as columns and interactions in the cells – almost all empty. Matrix factorisation learns a short vector for every user and every item so that the dot product of a user vector and an item vector approximates their interaction. Those vectors end up encoding hidden tastes nobody labelled: a dimension that roughly means "dark and serious" or "budget-conscious", discovered purely from behaviour. This approach became famous through the Netflix Prize competition and remains a strong baseline.
Embeddings and two-stage recommendation engines
Modern recommendation engines generalise that idea with embeddings: dense vectors for users, items and context, learned by neural networks that can also take in text, images, time of day and device. Items whose embeddings are close are "similar" in the sense that matters for engagement.
At scale, systems typically split the work into two stages:
- Candidate generation (retrieval). From millions of items, quickly pull a few hundred plausible ones – via nearest-neighbour search over embeddings, co-purchase lists, popular items in the user's region, and so on.
- Ranking. A heavier model, often gradient boosting or a neural network, scores each candidate using rich features (price, freshness, past interactions with the brand, predicted margin) and orders them.
A final layer applies business rules: remove out-of-stock items, avoid showing five near-identical products, respect content restrictions, mix in some fresh items.
The cold-start problem
Collaborative methods need history. A new user has none; a new product has none. This is the cold-start problem, and every real recommender needs an answer to it:
- New users: start with popular or trending items, ask a couple of onboarding questions, or use context (location, referral source, the page they landed on).
- New items: rely on content-based similarity until interaction data builds up, and deliberately give new items some exposure so they can collect it.
Hybrid systems that blend content-based and collaborative signals are the norm precisely because each covers the other's blind spots.
Measuring recommendations properly
Offline metrics help you iterate: precision and recall at k (how many of the top k recommendations were relevant), NDCG (which rewards putting the best items higher), and coverage (how much of the catalogue ever gets recommended). But offline metrics on historical data are biased – users could only click on what the old system showed them.
The real test is an A/B test on business metrics: conversion rate, revenue per session, retention, return rate. Watch for side effects too. A recommender that pushes cheap accessories may lift click-through while lowering average order value.
Building product recommendations for an e-commerce store
You don't need a research team to get value. A sensible progression for a shop:
| Stage | What to build | Data needed |
|---|---|---|
| 1 | Best-sellers per category, recently viewed | Orders, page views |
| 2 | "Frequently bought together" via co-purchase counts | Order lines |
| 3 | Item-based collaborative filtering on views and purchases | Several months of behaviour |
| 4 | Embeddings + a learned ranking model | Sizeable traffic, tracking discipline |
Many stores get most of the achievable lift from stages 1–3. Move to stage 4 when traffic is large enough that small ranking improvements are worth real money. Mraviko builds this kind of recommendation feature into the shops and platforms we develop, and we nearly always start at stage 2.
A few practical rules regardless of stage:
- Track events cleanly (view, add-to-cart, purchase) with consistent product IDs – it's the foundation of everything.
- Exclude items the customer just bought, unless they're consumables.
- Keep a share of recommendations for exploration so the system doesn't only reinforce what's already popular.
- Log what was shown, not just what was clicked, so future models can learn from it.
Closing
How recommendation systems work comes down to a few durable ideas: similar items, similar people, learned embeddings and a ranking step that respects business reality. Get the event data right, start with simple collaborative filtering, and measure with live experiments. The sophistication can come later – the habit of measuring has to come first.
