Skip to main content

Reproducibility of science and open source

I'm all for open access.
I'm all for open source.
I'm all for reproducible science.
I'm all for replicable studies.

So I like that data are shared.
I like that protocols are shared.
I also really appreciate it when code is shared - but only when it is appropriate.

Times that I think are appropriate to share code are, for instance, when there's an entirely new method introduced - then I think it is important to release the code/script as source code, package, program etc so that it enables other scientists to reproduce your work or use it in their own analyses.

However, I've noticed that, often times, shared code/script are nothing more than just the authors' workflows - in which case I don't want to see it. Everyone has a different workflow and I don't want to have to get into the heads of other people to figure out exactly what I'm looking at and what the code is doing - because commonly associated with shared workflow are uncommented code. This is not only useless but a hinderance because I have to take head-ouchy inducing time to figure out what every line of code is doing...

#####  Tangential rant starts here #####
The primary source of headache is R script written in a way that combines multiple operations in a single line - e.g. combining subsetting data with feeding it through some function. People, R is object oriented programming, there is no point in doing everything in a single line. Break it up. Make objects. Subset data and call them something else. Then perform functions on subsetted data objects...
#####  Tangential rant ends  #####

Another thing I don't want to see is a data dump, organised into a million folders with no annotations or explanations, with only an uncommented R script associated with the data in each of the million folders.

#####  Tangential rant2 starts here #####
I also found that if you contact authors for clarification on such data dump, their first response is typically, "What are you planning to do with my data?" - Dude, you published it already...it's fair game, man.
#####  Tangential rant2 ends  #####

What I think are most useful for reproducible science are:

  1. The prepared data - For a quick reanalysis, prepped data - e.g. missing data removed, taxa matched with phylogeny - is essential for replication.
  2. The raw data - So that you can prepare the data in the same way the authors did but using your own methods. Or better yet, you think you have a better way to process the data - i.e. you disagree with the authors' choice of data preparation. Maybe you have additional data the original authors didn't have and you want to augment the raw data.
  3. The protocol - I value this more than raw script or source code. I think it is very important to share a step-by-step instruction on how the analyses were conducted and what parameter values were used for inputs. People can figure out how to code up that protocol into their own workflows. After all, some labs might prefer to code things in Python or Matlab - like some wet labs might buy gels while others make their own, or prefer Corning tubes over Falcon tubes, or have totally different cell incubation systems - who knows...
  4. The full table/list of summary statistics - So that you can compare your results and see how well they match up.

Perhaps people will disagree but these are my opinions regarding the role of open source and sharing with respect to reproducible science.

Comments

Popular posts from this blog

R for beginners and intermediate users 3: plotting with colours

For my third post on my R tutorials for beginners and intermediate users, I shall finally touch on the subject matter that prompted me to start these tutorials - plotting with group structures in colour.

If you are familiar with R, then you may have noticed that assigning group structure is not all that straightforward. You can have a dataset that may have a column specifically for group structure such as this:

B0 B1 B2 Family
Acrocanthosaurus 0.308 -0.00329 3.28E-05 Allosauroidea
Allosaurus 0.302 -0.00285 2.04E-05 Allosauroidea
Archaeopteryx 0.142 -0.000871 2.98E-06 Aves
Bambiraptor 0.182 -0.00161 1.10E-05 Dromaeosauridae
Baryonychid 0.189 -0.00238 2.20E-05 Basal_Tetanurae
Carcharodontosaurus 0.369 -0.00502 5.82E-05 Allosauroidea
Carnotaurus 0.312 -0.00324 2.94E-05 Neoceratosauria
Ceratosaurus 0.377 -0.00522 6.07E-05 Neoceratosauria
Citipati 0.278 -0.00119 5.08E-06 Oviraptorosauria
Coelophysi…

Top 10 scientifically important theropod dinosaurs of all time (off the top of my head)

I thought I'd do a fun post for once. And since list based articles are the norm for fun on the internet, I thought I'd do one on dinosaurs, but given that I know most about theropods, I've decided to restrict my list to theropods (...maybe in a future post, I'll do other clades).

My ranking is based mostly on scientific importance so it may not reflect awesomeness, and it is obviously subjective as to how I rank importance to science. For instance, interesting discoveries or unique palaeobiology are ranked relatively low compared to wealth of information and data or completely revolutionising our understanding of the evolution of theropods.

So here are my top 10 scientifically important theropod dinosaurs of all time (off the top of my head)

10. Megalosaurus

Being the first dinosaur to be named, Megalosaurus automatically deserves a spot on this list, but given the fragmentary nature of known fossil specimens, and being mostly useless as a meaningful source for biologi…

The difference between Lion and Tiger skulls

A quick divergence from my usual dinosaurs, and I shall talk about big cats today. This is because to my greatest delight, I had discovered today a wonderful book. It is called The Felidæ of Rancho La Brea (Merriam and Stock 1932, Carnegie Institution of Washington publication, no. 422). As the title suggests it goes into details of felids from the Rancho La Brea, in particular Smilodon californicus (probably synonymous with S. fatalis), but also the American Cave Lion, Panthera atrox. The book is full of detailed descriptions, numerous measurements and beautiful figures. However, what really got me excited was, in their description and comparative anatomy of P. atrox, Merriam and Stock (1932) provide identification criteria for the Lion and Tiger, a translation of the one devised by the French palaeontologist Marcelin Boule in 1906. I have forever been looking for a set of rules for identifying lions and tigers and ultimately had to come up with a set of my own with a lot of help fro…