Reproducible Research and Geospatial Badges at AGILE 2019 conference in Limassol

Last week o2r team member Daniel went to Asia (or not?) to help a European conference with the transformation towards reproducible research. How?

The 2019 edition of the annual conference of the Association of Geographic Information Laboratories in Europe’s (AGILE) took place in Limassol, Cyprus. It was excellently organised at the Cyprus University of Technology and consisted of a pre-conference day of workshops and three days of talks and posters across the full breadth of GI Science.

On the first day, Daniel contributed to the organisation of the third workshop in the “Reproducible Research @ AGILE” series of workshops. Adjusting the scope of the workshop after the first two iterations, the participants learned first about the basics of reproducibility before being split up into a “beginners” and “advanced” group. The former continued with practical experiences in reproducing a tailored small manuscript with data and code, while the latter took on real world papers in a reproduction sprint. Starting only with a DOI, the participants skimmed real articles for practical instructions and shared how far they got after only 30 minutes. The results were mixed, as it could be expected, but the lessons that could be drawn were already very educational and could be connected directly with concrete steps towards preproducibility.

#agileconf2019 starting today in ☀️ Limassol with workshops. @f_ostermann is kicking things off at our #rragile19 workshop on reproducible research! #openscience #reproducibleresearch pic.twitter.com/SecPmEFl6z

— Daniel Nüst (@nordholmen) 17. Juni 2019

After lunch, the groups joined again for getting to know the AGILE Reproducible Paper Guidelines. The guidelines were developed in online collaboration and a recent expert meeting at TU Delft (see report). They require authors to be transparent about the underlying building blocks of their work by adding a Data and Software Availability section. Beyond this minimal requirement of transparency, the guidelines intent to nudge authors towards higher degrees of reproducibility with concrete steps and recommendations for both data and software. The steps are illustrated by examples from the GI Science domain. Leave your feedback about the guidelines OSF! The ensuing discussion about the challenges, opportunities, and ethics of reproducible research made clear the participants were serious on their way to becoming experts in RR. They continued on this path in the final session, in which both groups took on the role of an author and applied practices of Open Science and reproducible research. Find all workshop material online at https://osf.io/d9kcr/.

Besides the workshop, the RR@AGILE team advertised and sought feedback on the guidelines in many small discussions and with a dedicated poster. The feedback will be incorporated into a first release of the guidelines in the coming weeks, just in time for the call for papers for the next AGILE conference in Chania, Crete! The RR@AGILE team is proud that the AGILE council and next year’s organising team support a transformation towards reproducible research publications and looks forward to working with authors, reviewers and organisers to making the move a success.

On the second day of the conference, Daniel presented the short paper “Guerrilla Badges for Reproducible Geospatial Data Science”. The paper is based on the work of a project seminar at the Institute for Geoinformatics from 2017, which explains the long list of co-authors. The article demonstrates how and what kind of novel badges can be created based on executable research compendia (ERC) and how they can be distributed on the web. The full postprint of the peer-reviewed article is available on EarthArXiv and it contains links to the related software projects.

Just presented short paper at #agileconf2019 - Thanks the #SDI session participants & fun questions. Slides: https://t.co/N1OMj3iNJv Paper (postprint w/ DOI pending): https://t.co/gvkalcchHt Reproduction package: https://t.co/SC4i99pk5l #badges pic.twitter.com/Uyv8jyIAUS

— Daniel Nüst (@nordholmen) 18. Juni 2019

As always, AGILE was a delightful conference with many engaging discussions which may have started more collaborations to foster reproducible research. Daniel also continued the text analysis of all AGILE papers for this year’s conference.

Obviously this needs advertising, as no one spotted the error in the "trends" analysis: Increase in "reproducibility" keywords can be largely awarded to mine and @pjkedron's papers. Still lots of data, algorithms, and processing going on - clear need for reproducibility! pic.twitter.com/W4FMhtm3as

— Daniel Nüst (@nordholmen) 27. Juni 2019

A small rise in “reproducibility” terms can be traced to a couple of articles on the topic. Yet the stronger trend prevails: AGILE papers talk about data, processing, and algorithms - so the transformation for more transparency and reproducibility continues to be relevant.

Find the full analysis online on RPubs and see the source code on GitHub.