Blog

APAeval Challenge @ RNA Society 2021 Meeting

Call for Participation: Community effort for evaluation and development of methods for polyadenylation site detection/quantification from RNA-seq data

Click here to register

The organizers of the RNA Society 2021 meeting would like to solicit participation in APAeval: a communal effort to evaluate and/or develop methods for detecting polyadenylation sites, quantifying their usage, and estimating differential usage of alternative polyadenylation (APA) sites from RNA sequencing data (RNA-seq).

Organizers

Dr. Mihaela Zavolan, University of Basel

Dr. Yoseph Barash, University of Pennsylvania

Background:

Many labs within the RNA community rely on methods for detecting and quantifying PA sites and their differential usage across samples. However, no comprehensive evaluation of such methods has been carried out so far. We propose to organize a community effort to evaluate existing methods during the 2021 RNA Society Meeting, with a view of providing a benchmark for further development of such methods. 

The 2021 RNA Society Meeting will be held virtually, spanning 10 days. Any interested RNA Society member is welcomed to join this effort by contributing to one of the teams listed below.

Forms of Participation:

  1. Evaluation committee: We expect this committee to be composed of trainees (students/postdocs) interested in genomics and computational analysis, mentored and guided by two members of the organizing committee for computational challenges – Dr. Mihaela Zavolan and Dr. Yoseph Barash. This committee will assemble prior to the meeting and will be responsible for:
    1. Defining suitable datasets – these datasets could come from the literature, but could also be produced specifically for this grand challenge. They should include RNA-seq data for the inference, and orthogonal data sets, generated with other approaches, for validating predictions
    2. Defining the evaluation metrics. 
    3. Assembling the set methods to be benchmarked, by contacting method developers to solicit participation (see below).
    4. Contribute to the summarization of the results into (a) presentation at the RNA Society meeting 2021 (b) manuscript draft authored jointly by the contributors to this challenge (see below).
  2. Evaluation teams: During the meeting, participants will be responsible for:
    1. Setting up a common code repository
    2. Perform evaluation tasks on the methods included in this challenge
    3. Contribute to summary of results into (a) presentation at the RNA Society meeting 2021 (b) manuscript draft authored jointly by the contributors to this challenge (see below).
  3. Method teams: Method developers who wish their method to be included in this challenge will coordinate with the organizing committee to make sure that the input/output of their method follows a common format to allow easy comparison, and that their method is applied as intended. 

Expected outcome:

  1. Presentation at the end of the RNA Society Meeting. 
  2. Publication of a manuscript describing the results. The organizers will coordinate the effort of submitting the manuscript to the RNA journal. All participants in this effort will be acknowledged as contributing authors. 
  3. Up to ten most active contributors in the effort will be awarded a travel fellowship at the RNA Society meeting 2022. 

Timeline:

March 1st – Evaluation committee is assembled

March 15th – Framework of the evaluation is established (tests and metrics)

April 1st – Data sets identified or their generation is kick-started

April 1st – Method developers contacted

May 1st – Data sets identified or generated, set of methods to be benchmarked is defined

May 7th – Evaluation teams are identified based on registrations

May 15th – Resources for the challenge are set up

May 21st-29th – Benchmarking challenge

May 29th – Presentation at the RNA Society 2021 Meeting