Recap: Recommender Systems

DS701 Session 22 — Wed Nov 18, 2026

Today’s plan

  • 5 min — knowledge-check review
  • 20 min — highlights and Q&A (answer or pass — answering always earns credit)
  • 60 min — in-class activity (small groups)
  • wrap-up and cold-call check-ins

Knowledge-Check Review

KC 1: Predicting a missing rating

In item–item collaborative filtering, how is a missing rating \(\hat r_{ui}\) predicted? Name the two ingredients (what is compared to what, and how the pieces are combined), and say why the ratings matrix being 99.98% empty is the central difficulty rather than a detail.

KC 2: “Something like an SVD”

The lecture said we want “something like an SVD” of the ratings matrix but cannot use the SVD algorithm directly, and instead minimises \(\Vert (R - UV^T)_S \Vert^2 + \lambda(\Vert U\Vert^2 + \Vert V\Vert^2)\) by alternating least squares. Explain (a) why the SVD cannot be applied as-is, (b) what the subscript \(S\) does to the objective, and (c) why holding \(U\) fixed turns the problem into one you already know how to solve.

KC 3: The latent space, again

In Session 14 you took the SVD of a small ratings matrix with planted taste factors and saw users and items land in a shared latent space. Connect that to today: what does a row of \(U\) and a row of \(V\) mean* in the recommender’s factorization, how is a rating formed from them, and what can this shared space do that item–item CF cannot? Then name one situation in which neither CF nor MF has anything to say.*

Highlights

The user–item matrix — and the hole in it

  • Rows are items, columns are users, entries are ratings \(r_{ui}\) — the same object as the bipartite graph of known ratings.
  • Amazon movies: 4.6M reviews, ~250K movies, ~330K users → 99.98% empty. \[\text{sparsity} = \frac{\#\text{ reviews}}{\#\text{ users}\times\#\text{ items}}\]
  • And it is skewed: the average movie has 34 reviews, but most have far fewer (power law) — for most rows there is almost nothing to go on.
  • Goal: predict the blanks; score with RMSE on held-out known ratings.

Collaborative filtering: neighbours in rating space

Item–item: to guess \(r_{ui}\), look at items like \(i\) that \(u\) has rated, and average.

\[ \hat r_{ui} = \frac{\sum_{j \in n_k(i,u)} s_{ij}\, r_{uj}}{\sum_{j \in n_k(i,u)} s_{ij}} \qquad \frac{0.2\cdot 2 + 0.3\cdot 3}{0.2 + 0.3} = 2.6 \]

  • \(n_k(i,u)\): the \(k\) nearest neighbours of \(i\) among the items \(u\) rated — it is \(k\)-NN.
  • \(s_{ij}\): correlation of the two items’ rating columns, over the users who rated both.
  • User–user is the mirror image. Rule of thumb: compare along the short side (many users, few items → user–user).

Correct for bias first

Some users rate everything high; some items are simply loved. Left in, that noise masquerades as similarity.

\[ b_{ui} = \mu + \alpha_u + \beta_i \qquad\text{(global mean, user offset, item offset)} \]

Compute similarity on the residuals \(r_{ui} - b_{ui}\), and add the bias back when predicting:

\[ \hat r_{ui} = b_{ui} + \frac{\sum_{j \in n_k(i,u)} s_{ij}\,(r_{uj} - b_{uj})}{\sum_{j \in n_k(i,u)} s_{ij}} \]

Two practicalities: estimate \(\mu, \alpha, \beta\) by alternating regularised least squares over the observed entries (fix two, solve the third); and keep only positively similar neighbours, or the denominator can collapse.

CF’s report card: no training, easy to update, explainable (“because you liked Blade Runner”) — but accuracy is limited and neighbour search does not scale.

Matrix factorization: low rank = latent tastes

CF uses item similarity or user similarity. MF puts both in one latent space:

  • \(R \approx UV^T\) — one \(k\)-vector per user, \(\mathbf u_u\), and per item, \(\mathbf v_i\); predicted rating \(= \mathbf u_u^\top \mathbf v_i\). (The lecture’s figure stacks items in \(U\) and users in \(V\); the Netflix-prize slides write \(\mathbf u_u^\top \mathbf v_i\) — same thing, either way the rating is the inner product.)
  • This is Session 14’s SVD picture — rank \(k\) latent factors — but the matrix is partially observed, so we fit only where we can see: \[ \min_{U,V}\ \Vert (R - UV^T)_S\Vert^2 + \lambda\big(\Vert U\Vert^2 + \Vert V\Vert^2\big) \]
  • \(\lambda\): ridge penalty — with \(k=20\) the training RMSE was tiny; the penalty is what keeps held-out error honest.

ALS in one slide

Jointly convex: fix one factor and the objective is convex in the other.

  1. Hold \(U\) fixed → solve for \(V\)
  2. Hold \(V\) fixed → solve for \(U\)
  3. Not converged? Go to 1.

Each half-step is a ridge regression per row, restricted to that row’s observed entries:

\[ \mathbf u_u = \big(V_{J_u}^\top V_{J_u} + \lambda I\big)^{-1} V_{J_u}^\top \mathbf r_{u, J_u} \]

\(J_u\) = the items user \(u\) rated, \(V_{J_u}\) = their factor rows. Every step lowers the objective → converges.

MF in practice (lifted from Recommender Systems II):

  • \(U\) and \(V\) are embedding tables: a lookup from an ID to a learned vector; the dot product is the interaction. Modern deep recommenders (DLRM) keep exactly this core and add MLPs around it.
  • Netflix Prize: plain MF → + biases (\(\mu + \alpha_u + \beta_i + \mathbf u^\top\mathbf v\)) → + who rated what → + time-varying biases; RMSE 0.951 → 0.856. Billions of parameters ⇒ gradient descent, not ALS.

Evaluate on ratings you hid

  • The lecture’s ALS run reported RMSE on the visible entries of the densest block — optimistic twice over.
  • Honest protocol: hold out a random slice of the known ratings, fit on the rest, predict the slice, RMSE there. Training RMSE keeps falling with \(k\); held-out RMSE tells you when you are memorising.
  • Today: 70% missing, 100 ratings held out, and you will read all three numbers off — global mean, item–item CF, MF.

The standing limitation: cold start

Everything on the previous slides needs a history.

  • New user, zero ratings: CF has no \(\bar r_u\) and no neighbours; MF has no row in \(V\). Both can only say the global mean — for every item.
  • New item, zero ratings: the mirror image — no column, no similarity, no factor.
  • Real systems are hybrids: content and popularity priors bridge the gap until collaborative signal arrives.

Today’s stretch: sign up twenty new users, reveal their ratings one at a time, and watch each method’s RMSE climb — then find the cheapest thing that beats the global mean at \(n = 0\).

In-Class Activity

Activity: collaborative filtering and matrix factorization by hand — then cold start

Goal: on a synthetic streaming service with planted genres and 70% missing ratings, build item–item CF (adjusted-cosine similarity, \(k\)-NN prediction), implement ALS for matrix factorization and check the learned factors recover the genres; then experience the cold-start problem — a new user with 0, 1, 3, 10 ratings, a new item, and one hybrid fix.

  • Work in groups of 2–3. Open your section’s notebook — the service differs.
  • Parts marked (autograded) are submitted to Gradescope; the rest is participation.
  • Staff will circulate — be ready to explain any part of your work.
  • Dependencies: numpy, pandas, matplotlib, scikit-learn.

Section A1

Section B1

The activity notebook goes live on the day of the lecture. Colab is optional — you can also open it on GitHub and run it locally.

Going deeper

Full lecture notes: Recommender Systems