DialAM-2024 Shared Task - 2nd Call for Participation

RR
Ramon Ruiz-Dolz (Staff)
Mon, Mar 25, 2024 10:20 AM

Dear colleagues,

We would like to invite the community to participate in DialAM-2024: The First Shared Task on Dialogical Argument Mining, part of the 11th Workshop in Argument Mining (https://argmining-org.github.io/2024/index.html#shared_task)

*** Registration ***

https://dialam.arg.tech

*** Slack Channel ***

https://join.slack.com/t/dialam-2024/shared_invite/zt-2d9z2wvlj-9AVjILUMwFT_1iycO09NGA

*** Important Dates ***

17th April - Test data release start of evaluation
1st May - Evaluation end
7th May - Results announcement
20th May - Paper submission due
1st July - Camera-ready version due
15th of August - ArgMining 2024 workshop (ACL)

*** Overview ***

With the DialAM-2024 task, we propose the first shared task in dialogue argument mining where argumentation and dialogue information is modelled together in a domain-independent framework. The Inference Anchoring Theory (IAT) framework, makes possible to obtain homogeneous annotations of dialogue argumentation including relevant information and structural data from speech and argumentation, regardless of the domain, and allowing a more complete analysis of argumentation in dialogues together with a consistent cross-domain evaluation of the resulting argument mining systems. The DialAM-2024 consists of two sub-tasks: the identification of propositional (argumentative) relations, and the identification of illocutionary (speech act) relations. For both tasks all the information belonging to argumentation and dialogue will be available for the development of the submitted systems. We invite the community to participate in the DialAM-2024 task and explore how the use of additional information from the dialogue can be integrated into the argument retrieval process, in an attempt to take a step forward from sequence modelling approaches, where much of the relevant information to argumentation remains implicit behind the natural language.

*** Tasks ***

Task A: Identification of Propositional Relations

In the first task, the goal is to detect argumentative relations existing between the propositions identified and segmented in the argumentative dialogue. Such relations are: Inference, Conflict, and Rephrase.

Task B: Identification of Illocutionary Relations

In the second task, the goal is to detect illocutonary relations existing between locutions uttered in the dialogue and and the argumentative propositions associated with them such as: Asserting, Agreeing, Arguing, or Disagreeing among others.

*** Data ***

We will use the QT30 corpus [1], the largest available corpus in dialogical argumentation in English. QT30 is a collection of 30 episodes of Question Time aired between June 2020 and November 2021, with a total of more than 29 hours of transcribed broadcast material and comprises of 19,842 locutions by more than 400 participants: one moderator, 125 panel members (7 of them appearing more than once), and 300+ audience members. The QT30 dataset contains 10,818 propositional relations divided into Default Inferences, Default Conflicts, and Default Rephrases, and 32,303 illocutionary relations divided into Asserting, Agreeing, Arguing, Disagreeing, Restating, Questioning, and Default Illocuting.

[1] Hautli-Janisz, A., Kikteva, Z., Siskou, W., Gorska, K., Becker, R., & Reed, C. (2022, June). Qt30: A corpus of argument and conflict in broadcast debate. In Proceedings of the 13th Language Resources and Evaluation Conference (pp. 3291-3300). European Language Resources Association (ELRA).

*** Organisers ***

Ramon Ruiz-Dolz (rruizdolz001@dundee.ac.uk), ARG-tech, University of Dundee
John Lawrence, ARG-tech, University of Dundee
Ella Schad, ARG-tech, University of Dundee
Chris Reed, ARG-tech, University of Dundee

The University of Dundee is a registered Scottish Charity, No: SC015096

Dear colleagues, We would like to invite the community to participate in DialAM-2024: The First Shared Task on Dialogical Argument Mining, part of the 11th Workshop in Argument Mining (https://argmining-org.github.io/2024/index.html#shared_task) *** Registration *** https://dialam.arg.tech *** Slack Channel *** https://join.slack.com/t/dialam-2024/shared_invite/zt-2d9z2wvlj-9AVjILUMwFT_1iycO09NGA *** Important Dates *** 17th April - Test data release start of evaluation 1st May - Evaluation end 7th May - Results announcement 20th May - Paper submission due 1st July - Camera-ready version due 15th of August - ArgMining 2024 workshop (ACL) *** Overview *** With the DialAM-2024 task, we propose the first shared task in dialogue argument mining where argumentation and dialogue information is modelled together in a domain-independent framework. The Inference Anchoring Theory (IAT) framework, makes possible to obtain homogeneous annotations of dialogue argumentation including relevant information and structural data from speech and argumentation, regardless of the domain, and allowing a more complete analysis of argumentation in dialogues together with a consistent cross-domain evaluation of the resulting argument mining systems. The DialAM-2024 consists of two sub-tasks: the identification of propositional (argumentative) relations, and the identification of illocutionary (speech act) relations. For both tasks all the information belonging to argumentation and dialogue will be available for the development of the submitted systems. We invite the community to participate in the DialAM-2024 task and explore how the use of additional information from the dialogue can be integrated into the argument retrieval process, in an attempt to take a step forward from sequence modelling approaches, where much of the relevant information to argumentation remains implicit behind the natural language. *** Tasks *** Task A: Identification of Propositional Relations In the first task, the goal is to detect argumentative relations existing between the propositions identified and segmented in the argumentative dialogue. Such relations are: Inference, Conflict, and Rephrase. Task B: Identification of Illocutionary Relations In the second task, the goal is to detect illocutonary relations existing between locutions uttered in the dialogue and and the argumentative propositions associated with them such as: Asserting, Agreeing, Arguing, or Disagreeing among others. *** Data *** We will use the QT30 corpus [1], the largest available corpus in dialogical argumentation in English. QT30 is a collection of 30 episodes of Question Time aired between June 2020 and November 2021, with a total of more than 29 hours of transcribed broadcast material and comprises of 19,842 locutions by more than 400 participants: one moderator, 125 panel members (7 of them appearing more than once), and 300+ audience members. The QT30 dataset contains 10,818 propositional relations divided into Default Inferences, Default Conflicts, and Default Rephrases, and 32,303 illocutionary relations divided into Asserting, Agreeing, Arguing, Disagreeing, Restating, Questioning, and Default Illocuting. [1] Hautli-Janisz, A., Kikteva, Z., Siskou, W., Gorska, K., Becker, R., & Reed, C. (2022, June). Qt30: A corpus of argument and conflict in broadcast debate. In Proceedings of the 13th Language Resources and Evaluation Conference (pp. 3291-3300). European Language Resources Association (ELRA). *** Organisers *** Ramon Ruiz-Dolz (rruizdolz001@dundee.ac.uk), ARG-tech, University of Dundee John Lawrence, ARG-tech, University of Dundee Ella Schad, ARG-tech, University of Dundee Chris Reed, ARG-tech, University of Dundee The University of Dundee is a registered Scottish Charity, No: SC015096