T7 Workshop Format

About T7 Workshop Format Detailed Schedule Abstracts Mailing List Register Now

The following are instructions for all attendees of the T7 Workshop on Provenance for Transparent Research–not just speakers. If you attend you are a participant (even if anonymous).

This workshop is going to be a little different

It is easy to read through a call for abstracts and expect an upcoming workshop to be like all the others. No matter how a workshop is described, we generally assume that it will comprise a series of presentations, with time for just a few questions between each talk.

This workshop aims to be different. Here is how:

Presentations will be brief. Presenters will be given five minutes to speak. The experience will be somewhere between a series of lightning talks and a panel discussion. The keynote will be the single exception. Please see the T7 Workshop Agenda.
Time will be reserved for discussion. A ten-minute discussion will follow each 5-minute presentation. The workshop will conclude with a longer discussion for a final group assessment of the material shared during the workshop. Please see the T7 Workshop Agenda.
Presenters will explicitly identify their insights. Speakers will summarize the insights and arguments they share during their presentations. These summaries will take the form of short assertions that can be assessed by the workshop participants and included in a workshop report with those assessments.
Giving a voice to non-speakers is the goal of the workshop. Polls and other Zoom features will enable all workshop participants to react to and provide feedback on the assertions made during the presentations and discussions–even if they never enable their microphones. Identifying oneself with feedback provided this way will be optional.
The objective is a shared assessment of the topics discussed. We will collect and include the insights and feedback provided throughout the workshop in a workshop report. Our hope is that the report will help drive future research and development leading to greater transparency in computatationally-enabled research.
We are not seeking consensus. Diverse research communities stand to gain from better approaches to transparency, provenance management, and research quality. We do not expect all of the communities represented at the workshop to have the same needs. Claims that yield bimodal distributions in the levels of agreement are welcomed–and possibly the most useful outcome of this workshop. We will highlight these divergent viewpoints and priorities in the workshop report.

Here is what we are looking for from everyone

The following are examples of the kinds of contributions we hope to collect, discuss, and assess during the workshop. We suggest that presenters include a small number of similarly phrased statements on otherwise blank slides in their talks.

Please remember that all workshop attendees will be invited to submit their own statements for discussion and assessment, as well as to indicate the extent to which they concur with those of others.

Examples of observations

Correct metadata makes data transparent. Correct provenance makes results transparent.
A completely invalid computation can be perfectly repeatable.
Timeliness of research results is important. The Greenland ice sheet is not going to wait for us to get our climate models right.

Examples of definitions

Transparency is that which enables the quality of research to be evaluated without repeating it.
Traceability is what enables one to distinguish the computational steps and data leading to one product but not to another in the same study.

Examples of “insights”

Computational reproducibility does not imply trustworthiness.
Successfully repeating another researcher’s computation demonstrates not validity but transparency; it shows that the description of the computation provided is complete and correct.
The point of code reuse in research is not efficiency. The point is transparency and trust.

Examples of priorities

Research communities need ways to extend provenance vocabularies to represent domain-specific concepts important to them.
It should be easy to plug effective, correct provenance management capabilities into new frameworks without tool developers becoming experts in the provenance research field.

Does your experience suggest alternatives to any of the above? Great. Let us hear your insights at the T7 workshop!

What if your experience and insights differ from those of others?

The goal of the T7 workshop is neither to discover nor to debate universal truths. Instead, we aim to bring to light benefits of research transparency and how provenance management approaches can help in the context of particular research communities. We fully expect the expectations and insights shared by researchers in the social sciences, for example, to be very different from those shared by genomics researchers.

We will assume that every insight we evaluate is implicitly qualified with “In my research domain…” or “In domains I am familiar with…”

Instructions for presenters

Organize presentations around the insights you wish to share. Participants do want to hear about your latest results. However, they are even more interested in how your insights can help them and their research communities. Please use the content that would normally form the core of your presentation as supporting material for the definitions, priorities and other insights you are sharing.
Phrase your insights to invite assessment. We want to find out what workshop participants think about the insights you share with them. We will employ simple polling approaches to do this in real time. Please consider how best to phrase your statements so that they can easily be assessed, e.g. on a five-level Likert-type scale.
Be ready to hear alternative viewpoints. The meeting format will enable all attendees to suggest alternatives to the insights that you share. For example, if a speaker asserts that “We must ensure that researchers provide complete metadata for all data sets”, an audience member may counter (anonymously) with the claim “Incomplete metadata is preferable to incorrect metadata”. The audience would then assess the value of both claims, possibly after a discussion that considers the assumptions, implications and limitations of each point of view.

More Information

See the description of the T7 Workshop on Provenance for Transparent Research for an overview of the workshop topics and format, and the T7 Workshop Agenda for a detailed schedule.

We encourage everyone to subscribe to the T7 mailing list to receive further updates about the workshop format, agenda, and schedule by email.

Please contact Tim McPhillips with any questions.