Reproducible Publications at AGILE Conferences

The AGILE council generously supports an initiative for development of new guidelines for AGILE conference submissions to improve reproducibility.

The planned duration of the initiative is January 2019 to June 2019. Funding is made available up to €6000.

Initiative members

Description

Reproducible research (RR) is a crucial topic for any research domain using computational processes, including GIScience. Recent research has shown, that AGILE conference publications leave room for improvement regarding their computational reproducibility. While code & data availability seemed to be mentioned a bit more at the 2018 conference presentations, authors and reviewers lack clear guidelines to make workflow transparency the norm. This need is underpinned by the aforementioned publication and experiences from two pre-conference workshops.

The initiative will prepare new author guidelines and reviewer guidelines suitable for all submission types (full, short, poster). The guidelines help authors to prepare and reviewers to assess high-quality scholarly manuscripts by giving concrete recommendations for data and code repositories (e.g., OSF, GitLab), transparent workflows (e.g. Binder notebooks), citing software & data, and storing data (e.g. VGI).

The funding is used for a face-to-face meeting to be held in spring 2019. The initiative’s members convene with invited experts on reproducibility, open science, and peer review, and with representatives of the AGILE 2020 organisation and programme committee. The group composition ensures familiarity with AGILE and RR-expertise in GIScience. The meeting will be prepared by online collaborations starting in 2018.

The first draft of the guidelines will be presented in an one-hour online webinar during the workshop. All AGILE members are cordially invited to the webinar, which will be recorded and includes time for questions. The immediate feedback and comments from a 3-week consultation period after the webinar will be included in the final draft, which will be submitted to the AGILE council for consideration.

At the 2019 AGILE conference, the initiative members submit an application for a full day workshop for (a) teaching authors hands-on to apply the new guidelines to their accepted AGILE conference submissions and act as multipliers in their labs, and (b) reaching out to the AGILE community to introduce the guidelines and seek further feedback. The guidelines may also be presented as a poster to maximize outreach.

After the conference, the initiative members compile a report for publication on the AGILE website providing the outcomes and an overview of spent money.

Motivation and expected benefits

Increased awareness of the role of data and software in research and subsequently improved transparency and reproducibility of computations leads to better research. Besides normative reasons (“RR and Open Science are the right things to do”) a sensible combination of giving credit, public acknowledgement, and concrete requirements by conference organisers is the only way to encourage researchers to sustainably change their habits. AGILE is already in the process of reconsidering the publication format and process, changing towards an Open Access model and potentially a new submission management platform. This initiative’s timeline leaves enough room to align with such plans and make two promising transitions at the same time, benefiting from each other in personpower and expertise.

If the new guidelines are put in place, the quality of AGILE conference submissions is expected to improve, the usefulness of AGILE papers for readers is expected to grow considerably, the number of citations of published articles is expected to increase, and the relevance of the conference in the field is expected to be solidified. When reproducibility becomes the norm outside of AGILE conference, senior and early career researchers in AGILE member labs will be well prepared. The AGILE conference series will remain a highly popular venue for impactful manuscripts. The activities establish RR and Open Science within the AGILE community and are a starting point for further actions (e.g. reproducibility chair, updates for syllabi at member labs, awards, badges etc.) to make AGILE a leading reproducible conference.

Workshop

Location & date: TBA

Invited experts: TBA (feel free to self-nominate; contact daniel.nuest@uni-muenster.de)

Creative Commons Licence
Except where otherwise noted content is licensed under a Creative Commons Attribution 4.0 International License.
Privacy policy / Disclaimer This site (https://o2r.info/reproducible-agile) does not collect or store any personal data nor uses it cookies. Embedded Twitter widgets on this site do not collect information used for personalization. Although we check the content carefully, we cannot accept responsibility for the content of external links. The linked sites’ carriers are responsible for their sites’ content.