Chronic pain is a major problem facing individuals, families, healthcare providers, and society as a whole. Technology that can assess the behaviour of a person with chronic pain and associated levels of pain and related states could deliver personalised therapies and support in long-term self-management of the condition particularly aiming to improve engagement in valued everyday physical activities. This vision is based on the multidisciplinary approach to chronic pain management advocated by clinicians and grounded in findings in pain research of the importance of addressing pain as a multifaceted experience (with physical, psychological, and social factors and outcomes). The EmoPain challenge provides an opportunity for the FG community to collectively contribute to solving the fundamental problem of automatic detection of pain behaviours and pain levels, based on real data collected from people with chronic pain performing movements that are identical to those that make up daily physical functioning.

The EmoPain challenge, to be held in conjunction with FG2020 is the first international challenge addressing pain and related behaviour detection. It is based on the EmoPain dataset which contains both face and multimodal movement data from real participants with chronic pain performing physical activity. Previous related benchmark datasets have been aimed at detection from facial cues (The UNBC-McMaster Shoulder Pain Expression Archive Database Lucey et al. 2011) or experimental pain (Biovid Heat Pain Database Walter et al. 2013) whereas body movement is a critical modality to consider in assessing pain experience. This is also the first time a controlled challenge in pain behaviour is held such that competitors only have access to the training and validation partition, while test partition is held back for performance comparison to ensure a level playing field. Participating teams will be required to send working programs, together with clear documentation of any input and output parameters and the organisers will run thee on the test data. Details of the challenge tasks and guidelines are provided below.

The organisers will write a baseline paper detailing the Challenge’s objectives, rules and baseline methods for comparison. This will include baseline experimental results on the test partition and would be available on the challenge website from 24th January 2020. To save space, participants are kindly requested to cite the baseline paper instead of reproducing the method descriptions and results contained in the paper.

Challenge Description

The EmoPain 2020 Challenge consists of three main tasks focussed on pain recognition from facial expressions and body movements, as well as the recognition of pain-related body movements. Participants are expected to complete at least one of them. All tasks are based on the EmoPain dataset and are described as follows:

Table 1. Facial expression features description

Figure 1. Joint angle illustration

Figure 2. sEMG data from 4 sensors on the back

Dataset Partition

The face dataset comprising 36 participants is partitioned randomly into

The movement challenge dataset comprising 30 participants is partitioned randomly into

Performance Metrics


The application for the dataset has finished for this year.

Participating teams will be expected to attempt at least one or more of the challenge tasks and send their trained models (in the form of a working code) and a clear description of any input and output parameters to the committee before the stipulated deadline. Links to trained models can be shared with the organizers via cloud platforms, e.g., Google Drive, Dropbox or One-drive.

Participating teams can submit up to three different models for each task. The organisers will make reasonable attempts to run submitted code within five working days, and the teams can make repeated submissions to fix bugs in their code before the submission deadline. Note that repeated submissions to bug-fix code do not count towards the number of models a participating team can submit.

At the end of the competition, the test results for all participants will be published on the Challenge’s website. Participants will be ranked based on the performance metrics described above, and the winner(s) will be selected as the best performing submission. Prizes will be presented to the winning teams during the workshop program at the FG2020 conference.

Paper Submission

Each participating team would be required to submit a paper to the workshop, describing their proposed approach for tackling the challenge tasks as well as the results obtained. The organisers reserve the right to re-evaluate the findings, but will not participate in the challenge themselves. Participants are encouraged to compete in the three tasks. Submission should follow the FG 2020 stipulated guidelines for short papers, i.e., 4 pages + 1 page for reference. The review process will be double-blind. More information on paper submission timelines is provided below. Please submit your paper through EasyChair.

In addition to the paper describing the dataset (Aung et al 2016) mentioned in the EULA, please refer to the baseline paper describing the features extraction methods used in the three tasks (as described above) and the baselines for this challenge. This paper should be cited in the challenge camera ready paper.

The results of the challenge will be presented at the EmoPain 2020 workshop to be held in conjunction with the Automatic Face and Gesture Recognition 2020 conference in Buenos Aires, Argentina.

Baseline Results

Here we provide the results we have using some baseline systems.

Table 2. Baseline results for pain intensity estimation from facial expressions

Table 3. Baseline results for pain intensity estimation from movements

Table 4. Baseline results for movement behaviour classification

Important Dates


General Chairs

  Prof Nadia Berthouze      Dr. Amanda Williams      Dr. Michel Valstar        Dr. Hongying Meng 
           UCL                       UCL          University of Nottingham  Brunel University London
      Dr. Min Aung           Dr. Nicholas Lane           
University of East Anglia   University of Oxford              

Data Chairs

      Dr. Joy Egede        Dr. Olugbade Temitayo       Chongyang Wang             Siyang Song 
 University of Nottingham            UCL                     UCL           University of Nottingham

Keynote Speakers

Stay tuned.


Date: 20 Nov, 2020 (London Time)

12:00 Opening of the EmoPain Challenge

12:12 “LSTM-DNN based Approach for Pain Intensity and Protective Behaviour Prediction” Yi Li, Shreya Ghosh, Jyoti Joshi Dhall and Sharon Oviatt Monash University

12:24 “ALANet:Autoencoder-LSTM for pain and protective behaviour detection” Xinhui Yuan and Marwa Mahmoud Changchun University of Science and Technology

12:36 “Multimodality Pain and related Behaviors Recognition based on Attention Learning” Van Thong Huynh, Soo-Hyung Kim, Guee-Sang Lee and Hyung-Jeong Yang Chonnam National University

12:48 “Three-level Training of Multi-Head Architecture for Pain Detection” Saandeep Aathreya Sidhapur Lakshminarayan, Saurabh Hinduja and Shaun Canavan University of South Florida

13:00 Break

13:15 “Multimodal Multilevel Fusion for Sequential Protective Behavior Detection and Pain Estimation” Md Taufeeq Uddin and Shaun Canavan University of South Florida

13:27 “Automatic Recognition of Low-Back Chronic Pain Level and Protective Movement Behaviour using Physical and Muscle Activity Information” Fasih Haider, Pierre Albert and Saturnino Luz University of Edinburgh

13:39 “A Curriculum Learning Approach for Pain Intensity Recognition from Facial Expressions” Adria Mallol-Ragolta, Shuo Liu, Nicholas Cummins and Björn Schuller University of Augsburg

13:51 Break

14:15 Announcement of Winner and Award Ceremony

14:35 Closing