The Twelfth Dialog System Technology Challenge (DSTC12)
---====
Call for Track Proposals
---====
Following on the success of the DSTC shared tasks since 2013, the DSTC
organizing committees would like to invite track proposals for the 12th
Dialog System Technology Challenge (DSTC12) which will be held in 2024-2025.
TOPICS OF INTEREST
The theme of DSTC12 is “LLM-Empowered Dialog Systems”. The challenge will
consist of 3-5 tracks on the dialog research problems including, but not
restricted to, the following topics:
Adapting LLMs for Spoken Dialogue Systems
Proposals that involve the application of current SOTA LLMs to various
spoken dialog systems which require more conversational and proactive
responses versus more verbose responses that are prevalent in current LLMs.
RAG for Dialog systems
Proposals for projects focusing on Retrieval Augmented Generation (RAG),
for conversational interfaces. Being able to integrate external knowledge
sources or tools into LLMs.
Guardrails for LLM dialog systems
Proposals are invited to explore guardrails, which ensure harmless
responses in Language Model (LLM)-based conversational interfaces while
still being helpful. This also involves preventing these LLMs to produce
hallucinated responses.
LLM evaluation
Proposals that involve automatic and manual evaluation of different aspects
of LLMs, such as instruction-following, safety, factuality, general
conversational abilities, reasoning, planning, etc.
Integration of LLMs with Multi-Modal Inputs
Proposals that demonstrate innovative ways to combine text, speech, images,
and other modalities with the linguistic prowess of LLMs to achieve more
coherent and contextually relevant dialog systems.
Multi-lingual dialog processing
Proposals that explore novel approaches to multi-lingual dialog systems,
including but not limited to the development of robust cross-lingual
embeddings, transfer learning strategies for low-resource languages, and
the integration of cultural and contextual nuances into dialog models, and
handling code-switching scenarios.
As in prior editions of DSTC, we also welcome task proposals on end-to-end
dialog systems, dialog state tracking, dialog policy learning, spoken
language understanding, natural language generation for dialog systems,
multi-lingual dialog processing, multi-modal dialog processing,
knowledge-grounded dialog processing, dialog evaluation, any other
dialog-related tasks.
PROPOSAL CONTENT
Track proposals should be up to three (3) pages of main content, including
the following:
-
A title and a brief description of the proposed tasks.
-
A description of the main novelties or innovations of the proposal. If
the track is an extension of a track at a previous DSTC, explain the
novelty of the extended track.
-
A brief description of the datasets, and the status of their collection
(from planned to already collected).
-
A brief description of the evaluation methods.
-
The names, short biographies, affiliations, and full contact information
of the track organizers. Biographies can be included in the appendix.
-
An ethics section: discuss the risks and any societal benefits related to
the task and data of the proposed track.
Track organizers are also encouraged to provide examples of input and
output data in the appendix. The ethics section, references, and appendix
do not count towards the 3-page limit.
SUBMISSION
Track proposals should be submitted by filling out the submission form
https://forms.gle/G3YoPtsqVAZozXhS9. PDF format is preferred for your
proposal document.
Note: the submission form contains many questions, so please read the form
and share it with your team well before the deadline. Here
https://docs.google.com/forms/d/1lVVONlBDXJVQ1Cba5sngKhOameizKY4UTow0WWVwMus/prefill
is the list of all the questions.
PRESENTATION
The organizers of every submitted proposal are invited to present their
track details at the DSTC12. Presentation is not a requirement, and we will
try to accommodate remote presentation (e.g., via Zoom or recorded video).
IMPORTANT DATES
ORGANIZING COMMITTEE
General Chair: Behnam Hedayatnia (Amazon)
Workshop Chairs: Yun-Nung (Vivian) Chen (NTU), Chen Zhang (NUS)
Publication Chair: Raghav Gupta (Google)
Publicity Chair: Michel Galley (Microsoft)
CONTACT
Please feel free to post any question or comment to the DSTC mailing list
list@dstc.community.
To join the mailing list: visit
https://groups.google.com/a/dstc.community/forum/#!forum/list/join
https://groups.google.com/a/dstc.community/forum/#!forum/list/join.
The Twelfth Dialog System Technology Challenge (DSTC12)
======================================================================
Call for Track Proposals
======================================================================
Following on the success of the DSTC shared tasks since 2013, the DSTC
organizing committees would like to invite track proposals for the 12th
Dialog System Technology Challenge (DSTC12) which will be held in 2024-2025.
TOPICS OF INTEREST
------------------
The theme of DSTC12 is “LLM-Empowered Dialog Systems”. The challenge will
consist of 3-5 tracks on the dialog research problems including, but not
restricted to, the following topics:
*Adapting LLMs for Spoken Dialogue Systems*
Proposals that involve the application of current SOTA LLMs to various
spoken dialog systems which require more conversational and proactive
responses versus more verbose responses that are prevalent in current LLMs.
*RAG for Dialog systems*
Proposals for projects focusing on Retrieval Augmented Generation (RAG),
for conversational interfaces. Being able to integrate external knowledge
sources or tools into LLMs.
*Guardrails for LLM dialog systems*
Proposals are invited to explore guardrails, which ensure harmless
responses in Language Model (LLM)-based conversational interfaces while
still being helpful. This also involves preventing these LLMs to produce
hallucinated responses.
*LLM evaluation*
Proposals that involve automatic and manual evaluation of different aspects
of LLMs, such as instruction-following, safety, factuality, general
conversational abilities, reasoning, planning, etc.
*Integration of LLMs with Multi-Modal Inputs*
Proposals that demonstrate innovative ways to combine text, speech, images,
and other modalities with the linguistic prowess of LLMs to achieve more
coherent and contextually relevant dialog systems.
*Multi-lingual dialog processing*
Proposals that explore novel approaches to multi-lingual dialog systems,
including but not limited to the development of robust cross-lingual
embeddings, transfer learning strategies for low-resource languages, and
the integration of cultural and contextual nuances into dialog models, and
handling code-switching scenarios.
As in prior editions of DSTC, we also welcome task proposals on end-to-end
dialog systems, dialog state tracking, dialog policy learning, spoken
language understanding, natural language generation for dialog systems,
multi-lingual dialog processing, multi-modal dialog processing,
knowledge-grounded dialog processing, dialog evaluation, any other
dialog-related tasks.
PROPOSAL CONTENT
----------------
Track proposals should be up to three (3) pages of main content, including
the following:
* A title and a brief description of the proposed tasks.
* A description of the main novelties or innovations of the proposal. If
the track is an extension of a track at a previous DSTC, explain the
novelty of the extended track.
* A brief description of the datasets, and the status of their collection
(from planned to already collected).
* A brief description of the evaluation methods.
* The names, short biographies, affiliations, and full contact information
of the track organizers. Biographies can be included in the appendix.
* An ethics section: discuss the risks and any societal benefits related to
the task and data of the proposed track.
Track organizers are also encouraged to provide examples of input and
output data in the appendix. The ethics section, references, and appendix
do not count towards the 3-page limit.
SUBMISSION
----------
Track proposals should be submitted by filling out the submission form
<https://forms.gle/G3YoPtsqVAZozXhS9>. PDF format is preferred for your
proposal document.
Note: the submission form contains many questions, so please read the form
and share it with your team well before the deadline. Here
<https://docs.google.com/forms/d/1lVVONlBDXJVQ1Cba5sngKhOameizKY4UTow0WWVwMus/prefill>
is the list of all the questions.
PRESENTATION
------------
The organizers of every submitted proposal are invited to present their
track details at the DSTC12. Presentation is not a requirement, and we will
try to accommodate remote presentation (e.g., via Zoom or recorded video).
IMPORTANT DATES
---------------
* Track Proposal Submission Deadline: 5/4/2024
* Track Acceptance Notification: 6/7/2024
ORGANIZING COMMITTEE
--------------------
General Chair: Behnam Hedayatnia (Amazon)
Workshop Chairs: Yun-Nung (Vivian) Chen (NTU), Chen Zhang (NUS)
Publication Chair: Raghav Gupta (Google)
Publicity Chair: Michel Galley (Microsoft)
CONTACT
-------
Please feel free to post any question or comment to the DSTC mailing list
list@dstc.community.
To join the mailing list: visit
https://groups.google.com/a/dstc.community/forum/#!forum/list/join
<https://groups.google.com/a/dstc.community/forum/#!forum/list/join>.