Recap: What is Data Science?

DS701 Session 2 — Wed Sep 9, 2026

Today’s plan

  • ~30 min — ⚡ Spark pitches ⚡ (before this deck)
  • 5 min — knowledge-check review
  • 20 min — highlights and Q&A (answer or pass — answering always earns credit)
  • 45 min — in-class activity (small groups): question types, then hands-on Pandas
  • wrap-up and cold-call check-ins

Knowledge-Check Review

KC 1: The taxonomy

List the six question types from the lecture’s taxonomy (Leek & Peng), in order from least to most demanding, and for any three of them say in a few words what that type of question asks.

KC 2: The churn model

A team builds a churn model with excellent held-out accuracy and notices that customers who received a discount rarely churn. They recommend launching the discount to everyone. Which question type was asked, which was answered, and why does the model’s accuracy not settle the matter?

KC 3: Your own question

Pick one question you would like to answer with data — from your own background, a Spark pitch, or anything you are curious about. State the question in one sentence, classify it using the taxonomy, and name the row of the lecture’s techniques map (“You want to… / Technique”) you would reach for first, with one sentence on why that row fits.

Highlights

Data science in one line

Data science is the discipline of drawing defensible conclusions and making decisions from data, combining statistical reasoning, computation, and domain understanding.

  • Statistical reasoning — what the data can and cannot tell you.
  • Computation — the data rarely fits in your head, increasingly not on your laptop.
  • Domain understanding — otherwise you confidently answer the wrong question.

Drop one and the result has a name: stats without computation is small; computation without stats is confident nonsense; both without domain understanding is a solution in search of a question.

First, classify the question

Before choosing a technique, classify what kind of question you are asking (after Leek & Peng, Science 2015):

Question type Asks Example
Descriptive What does the data say, summarized? Average ride time by hour
Exploratory What patterns are worth a closer look? Riders cluster into commuter/leisure groups
Inferential Does the pattern hold beyond this sample? Do commuters in general ride longer in rain?
Predictive What will happen for a new case? Will this user churn next month?
Causal What happens if we intervene? Does the discount cause retention?
Mechanistic Exactly how does the effect operate? Through which pathway does the drug act?

Type errors are the common failure

  • Most real analyses fail not on the math but by answering a different type than was asked.
  • The classic: a causal question (“should we launch the discount?”) answered with a predictive model (“churners tend to lack discounts”).
  • Each type has different standards of evidence — held-out accuracy for prediction, controlled confounding for causation. Being excellent at one says nothing about the other.
  • The same data can support a descriptive claim, hint at an exploratory one, and be silent on the causal one. Say which you are making.
  • The prediction/causation gap gets its own two sessions (S16, S18). Keep the distinction in mind from today — the first half of the activity is a drill on it.

The lifecycle is a loop

Every project cycles through roughly six stages — rarely in a straight line:

  1. Frame the question — with the stakeholder, in their terms; classify it; decide what “good enough” means before modeling.
  2. Acquire the data — the least glamorous and most decisive stage.
  3. Clean and explore — plots before models, always. Expect half the effort here.
  4. Model — the simplest technique that answers the question type.
  5. Validate — held-out evaluation, sanity checks, error analysis. The stage that separates analysis from wishful thinking.
  6. Communicate and deploy — if nothing changes, the project did not happen.

Stage 3 is what the second half of today’s activity practices: read, inspect, filter, group, plot — in Pandas.

The techniques map: this course in one table

You want to… Technique Sessions
Group similar things, no labels k-means, hierarchical, GMM 4–7
Say how alike two things are distance & similarity measures 3
Predict a category trees & random forests, k-NN, logistic regression 9–13
Predict a number linear regression + regularization 12–13
Compress / de-noise / visualize high-dimensional data SVD, PCA, t-SNE 14–15
Ask “did X cause Y?” causal graphs, effect estimation 16, 18
Learn from raw signal (images, text) neural networks 19–20
Work with language NLP pipeline, embeddings 21
Recommend items to users matrix-factorization recommenders 22
Reason about connected data graph analysis, centrality, communities 23–24

Each row answers a type of question. In the last session of the semester you fill in a blank copy from memory, with one sentence per row on why — a rehearsal for every data science interview you will sit.

In-Class Activity

Activity: Question types, then Pandas

Goal: (core) classify ten short scenarios by question type and spot a type error; (stretch) load a small real dataset (Titanic passengers) in Pandas and walk the clean-and-explore stage — read, inspect, filter, groupby, plot — then say which question type your groupby result actually answers.

  • Work in groups of 2–3. ~45 minutes today.
  • Parts marked (autograded) are submitted to Gradescope; the rest is participation.
  • Staff will circulate — be ready to explain any part of your work.
  • Dependencies: pandas, numpy, matplotlib only.
  • Pandas is new to some of you: the Pandas tutorial is the reference; the notebook links the sections you need.

Section A1:

Section B1:

The activity notebook goes live on the day of the lecture. Colab is optional — you can also open it on GitHub and run it locally.

Going deeper

Full lecture notes: What is Data Science?