Seen from the air

2017-05-16 14.20.30

Earlier this week I was flying through Munich airport and the plane took the scenic route over Munich and its surroundings. The sun was out and the rapeseed fields were in bloom, making patches of bright and pretty yellow all over the landscape. I saw this from the window and wondered what it could be – was it the world’s longest swimming pool?

A quick search on Google Maps after I got home gave me the answer: it’s the Regattastrecke Oberschleißheim, an artificial rowing course built for the 1972 Olympics. I really enjoyed watching the contrasting colors in the landscape from the air, especially the different blues and greens of the various water bodies, and the snowy mountain ranges that we flew over before getting to Munich. The patterns in the formal gardens at the Nymphenberg Palace could also be seen, but I wasn’t fast enough with my camera.

If only the weather was always so nice when I am in the air!

Why there is no such thing as a “quick look” at the data

Nice blog post from Rafa Irizarry on why Interactive Data Analysis (IDA) is important, instead of mindlessly applying workflows.

Some points I agree with:

  • IDA is necessary to discover outliers, to get a “feel” for the data, to check if applied analyses are appropriate
  • “Data generators” who produce the raw data are usually not trained data analysts

Some reservations I have about the post:

  • I think that knocking on mindlessly-applied workflows is a bit of a crowd-pleasing, “preaching to the choir” statement. If you ask people directly, no one would sign on to the statement “We should use workflows without thinking about whether they are appropriate” (even if in practice that is what many of us are doing, myself included)
  • Standardized workflows are useful for reproducibility. Outliers that screw up data analyses are like bugs in computer code. And as anybody who’s tried to get IT help knows, one of the first things we’re asked to do is to reproduce the bug.

What I especially like is his call for IDA to be a bigger part of existing workflows. That is to say, when designing a data analysis pipeline, one should think about how to incorporate diagnostic checks and interactive analysis steps along the way, as a sort of heuristic debugging process. My hunch is that most people already do this, but the challenge is to formalize it as part of the process. That’s definitely something I’ll think about as I go about analyzing my own data.

The necessity of IDA also explains why there’s no such thing as taking “a quick look” at the data to see if there’s something interesting there (also sometimes overheard: “just run it through your pipeline”). I work mostly with genomic data, and most of my time is spent on interacting with the data, determining if a particular question is even appropriate to ask for a particular set of data. “Quick and dirty” is usually more dirty than quick, when all is said and done….

The R tidyverse

Most regular R users will have felt the influence of Hadley Wickham, whether through the widely-used ggplot2 package that implements the “grammar of graphics”, devtools, plyr, … the list goes on. I was astounded when I first realized that the same person was responsible for all these really useful things.

Most software packages aim at providing tools to make particular tasks easier in a certain language. In comparison, many of the tools that he has developed are in effect streamlining the grammar of the language itself. Once you use ggplot2 and see how intuitive it is to deal with statistical graphics in that way, then the base R plot commands feel impossibly clunky. Similarly, his paper on tidy data and the accompanying tidyr and plyr packages articulate basic ideas about data should be organized in tables. These are ideas that sound very simple, and most of us have probably had some similar thoughts cross our minds as we struggled to reshape raw data into analyzable form, but I certainly would not have been able to formulate the concepts so clearly or implement solutions to change our relationship to data wrangling.

The various packages have seemed to evolve towards a common style and design philosophy, and late last year most of them have been bundled together in a ‘super-package’ called tidyverse. It makes installation much easier, because now you can make sure all these inter-dependent packages are up-to-date with a single command, and probably makes development easier for him and his team. It also goes together with a book titled R for Data Science that he and a coauthor have just released, which is also available online. Noted here for future reference!

Pixel-painting in your browser

Stumbled across a fun website called make 8-bit art –  it’s a pixel painting application like MS Paint, except in your browser! Two basic tools – pencil and fill – and 256 colors to choose from, and best of all, it’s open source.

More fun awaits when you look at the page source (in Firefox: Ctrl-U). I’m not going to give away what’s there, except that it’s also pretty artful.

The site is run by an artist and developer who goes by jenmoneydollars, but despite the moniker it appears to be free….

His oldest, closest friend

I happened upon an interesting phrase in a story, “Signal” by John Lanchester, from the New Yorker (3 Apr 2017):

“Michael wasn’t my oldest friend and he wasn’t my closest friend, but he was older than any of the ones who were closer and closer than any of the ones who were older, ….”

This is an odd way to describe a friendship, but it is precise. However, the more I thought about it, the more dissatisfied I was.

Continue reading