Jupyter in the lab

As a new and small team, it's easy for us to experiment with different ways of working. We don't have a lot data or code locked away in legacy systems, and we don't have the inertia of established workflows.

One of our experiments in "how to work" has been surprisingly successful, and that is using Jupyter notebooks in the wet lab.

We initially resisted the idea, even though we were already big fans of Jupyter. Instead, we tried to make purpose-built wet lab tools work for us, but we kept feeling like the problems they were solving were less important than the problems they weren't solving.

What finally pushed us over the edge was our experience developing a protocol to produce a protein from a new construct. In those kinds of situations, you usually have a basic framework for a protocol, but you need to work out the specifics in the lab — which depend on whatever is "new" about what you're doing.

We think of those specifics, which include things like time, temperature, and concentration as well as more complicated biological variables, as parameters of an abstract protocol. However, our tools didn't allow us to express the problem this way, and every time we adjusted a "parameter" or some assumption of our protocol, we needed to manually rederive the protocol steps.

Translating this protocol into a Jupyter notebook allowed us express the protocol design and steps as functions of parameters, which more closely matched our conceptual model. This immediately solved our practical pain points, and it also made scaling up production trivial, since the protocol was paramterized over all inputs.

We've put together a notebook to illustrate some of the concepts behind our Jupyter workflows. You can play around with a live version using Binder or just view it using nbviewer . The source is in a GitLab repository.

We've since transitioned all of our lab protocols and notebooks (and some of our analyses) to Jupyter notebooks, and we've discovered that our Jupyter-based workflows increase documentaiton, reduce data duplication, and ease analysis.

So far, we haven't had any trouble working with partners or collaborators, because a PDF of a well annotated notebook is as good or better than many alternatives.

We hope over time to share more details of our workflows because we think they could be useful to other labs. Of course, this approach won't be for everyone; it's technology that our team was already comfortable with, and our switching costs were low.

We'd love to hear what you think — please drop us a line!

—Jeff and Colin