CALL FOR PAPERS
Perspectivist Approaches to Disagreement in NLP (NLPerspectives)
Until recently, the dominant paradigm in natural language processing (and other areas of artificial intelligence) has been to resolve observed label disagreement into a single “ground truth” or “gold standard” via aggregation, adjudication, or statistical means. However, in recent years, the field has increasingly focused on subjective tasks, such as abuse detection or quality estimation, in which multiple points of view may be equally valid, and a unique ‘ground truth’ label may not exist. At the same time, as concerns have been raised about bias and fairness in AI, it has become increasingly apparent that an approach which assumes a single “ground truth” can erase minority voices.
Strong perspectivism in NLP (Basile et al., 2021) pursues the spirit of recent initiatives such as Data Statements (Bender and Friedman, 2018), extending their scope to the full NLP pipeline, including the aspects related to modelling, evaluation and explanation.
The “Perspectivist Approaches to Disagreement in NLP” (NLPerspectives) workshop will explore current and ongoing work on the collection and labelling of non-aggregated datasets, and approaches to modelling and including these perspectives, as well as evaluation and applications of multi-perspective Machine Learning models. We also welcome opinion pieces and literature reviews, e.g. in the context of fairness and inclusion.
A key outcome of the workshop will be to build on the work begun at https://pdai.info/ to create a repository of perspectivist datasets with non-aggregated labels for use by researchers in perspectivist NLP modelling.
Authors are therefore invited to share their LRs. In particular, when submitting a paper from the START page, authors will be asked to provide essential information about resources (in a broad sense, i.e. also technologies, standards, evaluation kits, etc.) that have been used for the work described in the paper or are a new result of your research.
Moreover, ELRA encourages all LREC authors to share the described LRs (data, tools, services, etc.) to enable their reuse and replicability of experiments (including evaluation ones).
We accept two types of submissions, long papers and short papers (e.g., demonstration papers, dataset and resource papers, short focused contributions), all following the LREC template: https://lrec2022.lrec-conf.org/en/submission2022/authors-kit/
Long papers may consist of up to 8 pages of content, plus unlimited references. Short papers may consist of up to 4 pages of content; shorter versions are also welcome. Final versions will be given one additional page of content so that reviewers' comments can be taken into account. Submissions should be sent in electronic forms, using the Softconf START conference management system. The submission site is https://www.softconf.com/lrec2022/NLPerspectives/.
We invite original research papers from a wide range of topics, including but not limited to:
Non-aggregated data collection and annotation frameworks
Descriptions of corpora collected under the perspectivist paradigm
Multi-perspective Modelling and Machine Learning
Evaluation of multi-perspective models/ models of disagreement
Multi-perspective disagreement as applied to NLP evaluation
Fairness and inclusive modelling
Applications of multi-perspective modelling
Computing with (dis)agreement
Perspectivist Natural Language Generation
Foundational aspects of perspectivism
Opinion pieces and reviews on perspectivist approaches to NLP
Submissions are open to all, and are to be submitted anonymously. All papers will be refereed through a double-blind peer review process by at least three reviewers with final acceptance decisions made by the workshop organizers.
The workshop will take place at LREC, and is scheduled to last for half a day on June 20th 2022.
Friday April 15, 2022: Paper submission
Friday May 6, 2022: Notification of acceptance
Friday May 20, 2022: Camera-ready papers due
Monday June 20, 2022: Workshop
Gavin Abercrombie, Heriot-Watt University
Valerio Basile, University of Turin
Verena Rieser, Heriot-Watt University
Sara Tonelli, Fondazione Bruno Kessler
Alexandra Uma, Queen Mary University of London
Contact us at firstname.lastname@example.org if you have any questions.
Founded in 1821, Heriot-Watt is a leader in ideas and solutions. With campuses and students across the entire globe we span the world, delivering innovation and educational excellence in business, engineering, design and the physical, social and life sciences. This email is generated from the Heriot-Watt University Group, which includes:
The contents (including any attachments) are confidential. If you are not the intended recipient of this e-mail, any disclosure, copying, distribution or use of its contents is strictly prohibited, and you should please notify the sender immediately and then delete it (including any attachments) from your system.