Why there is no such thing as a “quick look” at the data

Nice blog post from Rafa Irizarry on why Interactive Data Analysis (IDA) is important, instead of mindlessly applying workflows.

Some points I agree with:

  • IDA is necessary to discover outliers, to get a “feel” for the data, to check if applied analyses are appropriate
  • “Data generators” who produce the raw data are usually not trained data analysts

Some reservations I have about the post:

  • I think that knocking on mindlessly-applied workflows is a bit of a crowd-pleasing, “preaching to the choir” statement. If you ask people directly, no one would sign on to the statement “We should use workflows without thinking about whether they are appropriate” (even if in practice that is what many of us are doing, myself included)
  • Standardized workflows are useful for reproducibility. Outliers that screw up data analyses are like bugs in computer code. And as anybody who’s tried to get IT help knows, one of the first things we’re asked to do is to reproduce the bug.

What I especially like is his call for IDA to be a bigger part of existing workflows. That is to say, when designing a data analysis pipeline, one should think about how to incorporate diagnostic checks and interactive analysis steps along the way, as a sort of heuristic debugging process. My hunch is that most people already do this, but the challenge is to formalize it as part of the process. That’s definitely something I’ll think about as I go about analyzing my own data.

The necessity of IDA also explains why there’s no such thing as taking “a quick look” at the data to see if there’s something interesting there (also sometimes overheard: “just run it through your pipeline”). I work mostly with genomic data, and most of my time is spent on interacting with the data, determining if a particular question is even appropriate to ask for a particular set of data. “Quick and dirty” is usually more dirty than quick, when all is said and done….


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s