---
title: Proof of TSS Decomposition
---

## Explainer Video on Proof

<iframe id="kaltura_player" src='https://cdnapisec.kaltura.com/p/2159741/embedPlaykitJs/uiconf_id/54272102?iframeembed=true&amp;entry_id=1_q9rbodgt&amp;config%5Bprovider%5D=%7B%22widgetId%22%3A%221_si7os6ia%22%7D&amp;config%5Bplayback%5D=%7B%22startTime%22%3A0%7D'  style="width: 608px;height: 402px;border: 0;" allowfullscreen webkitallowfullscreen mozAllowFullScreen allow="autoplay *; fullscreen *; encrypted-media *" sandbox="allow-downloads allow-forms allow-same-origin allow-scripts allow-top-navigation allow-pointer-lock allow-popups allow-modals allow-orientation-lock allow-popups-to-escape-sandbox allow-presentation allow-top-navigation-by-user-activation" title="Lecture Appendix: Why TSS = RSS + TSS -- An explainer"></iframe>

> Only accessible with you BU login.

## Proof that TSS = RSS + ESS

**Given:**

- TSS = $\sum_{i=1}^n (y_i - \bar{y})^2$ (Total Sum of Squares)
- RSS = $\sum_{i=1}^n (y_i - \hat{y}_i)^2$ (Residual Sum of Squares)
- ESS = $\sum_{i=1}^n (\hat{y}_i - \bar{y})^2$ (Explained Sum of Squares)

**Proof:**

Start by decomposing the deviation from the mean:
$$y_i - \bar{y} = (y_i - \hat{y}_i) + (\hat{y}_i - \bar{y})$$

Square both sides:
$$
\begin{align}
(y_i - \bar{y})^2 &= [(y_i - \hat{y}_i) + (\hat{y}_i - \bar{y})]^2\\
&= (y_i - \hat{y}_i)^2 + 2(y_i - \hat{y}_i)(\hat{y}_i - \bar{y}) + (\hat{y}_i - \bar{y})^2
\end{align}
$$

Sum over all $i$:
$$
\sum_{i=1}^n (y_i - \bar{y})^2 = \sum_{i=1}^n (y_i - \hat{y}_i)^2 + 2\sum_{i=1}^n (y_i - \hat{y}_i)(\hat{y}_i - \bar{y}) + \sum_{i=1}^n (\hat{y}_i - \bar{y})^2
$$

Which gives us:
$$
\text{TSS} = \text{RSS} + 2\sum_{i=1}^n (y_i - \hat{y}_i)(\hat{y}_i - \bar{y}) + \text{ESS}
$$

**The key step:** We need to show that the cross-term equals zero:
$$
\sum_{i=1}^n (y_i - \hat{y}_i)(\hat{y}_i - \bar{y}) = 0
$$

**Why is this zero?**

For OLS regression **with an intercept**, the residuals $(y_i - \hat{y}_i)$ satisfy two orthogonality conditions:

1. $\sum_{i=1}^n (y_i - \hat{y}_i) = 0$ (residuals sum to zero)
2. $\sum_{i=1}^n (y_i - \hat{y}_i)\hat{y}_i = 0$ (residuals are orthogonal to predictions)

Both are the normal equations $X^T\mathbf{e} = \mathbf{0}$ read row by row: the row
for the column of 1s says the residuals sum to zero (condition 1), and because
$\hat{\mathbf{y}} = X\hat{\boldsymbol\beta}$ is a combination of the columns of $X$,
$\mathbf{e}$ is orthogonal to it too (condition 2). The proofs below spell this out.

Using condition (2):
$$
\sum_{i=1}^n (y_i - \hat{y}_i)(\hat{y}_i - \bar{y}) = \sum_{i=1}^n (y_i - \hat{y}_i)\hat{y}_i - \bar{y}\sum_{i=1}^n (y_i - \hat{y}_i) = 0 - \bar{y} \cdot 0 = 0
$$

Therefore: **TSS = RSS + ESS** ✓

**Geometric Intuition:**

This decomposition is a Pythagorean theorem in high-dimensional space! The vectors $(y_i - \bar{y})$, $(y_i - \hat{y}_i)$, and $(\hat{y}_i - \bar{y})$ form a right triangle because the residuals are orthogonal to the fitted values.

#### Why residuals sum to exactly zero (with intercept)

Condition 1 is satisfied **exactly** (not just approximately), but this is **only true when the model includes an intercept term**.

When you include an intercept $\beta_0$, the normal equations $X^TX\boldsymbol\beta = X^T\mathbf{y}$ include a row corresponding to the column of 1s in the design matrix.

#### Proof

Consider a linear model with an intercept. The design matrix has a column of 1s:

$$
X = \begin{bmatrix}
1 & x_{11} & x_{12} & \cdots & x_{1p} \\
1 & x_{21} & x_{22} & \cdots & x_{2p} \\
\vdots & \vdots & \vdots & \ddots & \vdots \\
1 & x_{n1} & x_{n2} & \cdots & x_{np}
\end{bmatrix}, \quad
\boldsymbol\beta = \begin{bmatrix}
\beta_0 \\
\beta_1 \\
\vdots \\
\beta_p
\end{bmatrix}, \quad
\mathbf{y} = \begin{bmatrix}
y_1 \\
y_2 \\
\vdots \\
y_n
\end{bmatrix}
$$

#### The Normal Equations

The OLS solution satisfies the normal equations:
$$
X^T X \boldsymbol\beta = X^T \mathbf{y}
$$

#### Looking at $X^T$

The transpose of $X$ is:
$$
X^T = \begin{bmatrix}
1 & 1 & 1 & \cdots & 1 \\
x_{11} & x_{21} & x_{31} & \cdots & x_{n1} \\
x_{12} & x_{22} & x_{32} & \cdots & x_{n2} \\
\vdots & \vdots & \vdots & \ddots & \vdots \\
x_{1p} & x_{2p} & x_{3p} & \cdots & x_{np}
\end{bmatrix}
$$

**The first row is all 1s!**

#### The First Normal Equation

The normal equations give us a system of $(p+1)$ equations. The **first equation** comes from multiplying the first row of $X^T$ with both sides:

Left side: First row of $X^T$ times $X\boldsymbol\beta$
$$
[1, 1, 1, \ldots, 1] \cdot X\boldsymbol\beta = \sum_{i=1}^n (X\boldsymbol\beta)_i = \sum_{i=1}^n \hat{y}_i
$$

Right side: First row of $X^T$ times $\mathbf{y}$
$$
[1, 1, 1, \ldots, 1] \cdot \mathbf{y} = \sum_{i=1}^n y_i
$$

#### The Result

Therefore, the first normal equation is:
$$
\boxed{\sum_{i=1}^n \hat{y}_i = \sum_{i=1}^n y_i}
$$

This is **exact**, not approximate. It's a direct consequence of having the column of 1s in $X$.

#### Interpretation

Dividing by $n$:
$$
\bar{\hat{y}} = \bar{y}
$$

The mean of the fitted values equals the mean of the observed values. This is why, with an intercept, the regression line always passes through the point $(\bar{x}, \bar{y})$.

**End of proof**


#### Without an intercept

If you force the regression through the origin (no $\beta_0$ term), then:

- Condition 1 is **NOT** satisfied - residuals don't sum to zero
- The TSS = RSS + ESS decomposition **doesn't hold**
- $R^2$ can even be negative!


#### Key takeaway

For standard OLS with intercept:
- $\sum (y_i - \hat{y}_i) = 0$ ✓ (exactly)
- $\sum (y_i - \hat{y}_i)\hat{y}_i = 0$ ✓ (exactly, from normal equations)

Both are mathematical properties, not approximations!