Second Shared Task on Automatic Minuting at INLG 2023

TG
Tirthankar Ghosal
Tue, Mar 21, 2023 6:56 AM

*** Second Shared Task on Automatic Minuting ***
*** AutoMin 2023 as INLG 2023 Generation Challenge ***

Automatic summarization of meetings into meeting minutes (aka
`minuting') would be an amazing tool. Superficially, large language
models like BART, GPT*, and friends may seem to have solved it already,
but serious concerns still remain regarding completeness and factual
correctness.

We would like to invite you to take part in AutoMin 2023, the second
instance of the shared on automatic minuting, i.e., summarization of
English and Czech meeting transcripts into meeting minutes and their
evaluation. See the details on task options below.

AutoMin 2023 is a registered INLG Challenge this year.

*** Website ***

https://ufal.github.io/automin-2023/

*** Key Tasks (one or both) ***

  • Task A: The main task consists of automatically creating minutes
    from multiparty meeting transcripts. The generated minute will be
    evaluated both via automatic and manual metrics.

  • Task D: Given a meeting transcript, a candidate minutes, and a set of
    one or more reference minutes, assign a score indicating the quality
    of the candidate. Task D will not be evaluated by a single criterion.
    All submissions to Task D will be evaluated in terms of Pearson
    correlation against all manual and all automatic evaluation scores
    available in AutoMin 2023, including other submissions to Task D.

*** Registration ***

Registration and participation are free of charge.

All participants need to register to get access to the test data.
Use this form and provide us with your GitHub user name (one member
per team is fine):

https://forms.office.com/e/jdvsdtV5Tm

After registration, we will give you GitHub access to the test data.

Please get in touch in case you would be willing to contribute to the
manual evaluation
of AutoMin submissions.

*** Important Dates (EXTENDED) ***

May 1, 2023      System Submission Deadline
May 15, 2023    Report Submission Deadline
July 25, 2023    Camera Ready Submission; in line with INLG
September, 2023  AutoMin @ INLG 2023

*** Publication ***

All teams are required to submit a brief technical report describing
their method. Please submit using the INLG paper template (details
here: https://inlg2023.github.io/calls.html).

All reports must be a minimum of 2 pages and a maximum of 4
pages excluding references. Reports must be written in English.
The proceedings will be published in ACL Anthology.

*** Organizers ***

Ondřej Bojar, UFAL, Charles University, Czech Republic
Tirthankar Ghosal, UFAL, Charles University, Czech Republic
Marie Hledikova, UFAL, Charles University, Czech Republic
Anja Nedoluzhko, UFAL, Charles University, Czech Republic

*** Contact ***

For inquiries, contact automin@ufal.mff.cuni.cz.

+++++++++++++++++++++++++++++++++++

Tirthankar Ghosal

https://member.acm.org/~tghosal

++++++++++++++++++++++++++++++++++++

*** Second Shared Task on Automatic Minuting *** *** AutoMin 2023 as INLG 2023 Generation Challenge *** Automatic summarization of meetings into meeting minutes (aka `minuting') would be an amazing tool. Superficially, large language models like BART, GPT*, and friends may seem to have solved it already, but serious concerns still remain regarding completeness and factual correctness. We would like to invite you to take part in AutoMin 2023, the second instance of the shared on automatic minuting, i.e., summarization of English and Czech meeting transcripts into meeting minutes and their evaluation. See the details on task options below. AutoMin 2023 is a registered INLG Challenge this year. *** Website *** https://ufal.github.io/automin-2023/ *** Key Tasks (one or both) *** - Task A: The main task consists of automatically creating minutes from multiparty meeting transcripts. The generated minute will be evaluated both via automatic and manual metrics. - Task D: Given a meeting transcript, a candidate minutes, and a set of one or more reference minutes, assign a score indicating the quality of the candidate. Task D will not be evaluated by a single criterion. All submissions to Task D will be evaluated in terms of Pearson correlation against all manual and all automatic evaluation scores available in AutoMin 2023, including other submissions to Task D. *** Registration *** Registration and participation are free of charge. All participants need to register to get access to the test data. Use this form and provide us with your GitHub user name (one member per team is fine): https://forms.office.com/e/jdvsdtV5Tm After registration, we will give you GitHub access to the test data. Please get in touch in case you would be willing to contribute to the manual evaluation of AutoMin submissions. *** Important Dates (EXTENDED) *** May 1, 2023 System Submission Deadline May 15, 2023 Report Submission Deadline July 25, 2023 Camera Ready Submission; in line with INLG September, 2023 AutoMin @ INLG 2023 *** Publication *** All teams are required to submit a brief technical report describing their method. Please submit using the INLG paper template (details here: https://inlg2023.github.io/calls.html). All reports must be a minimum of 2 pages and a maximum of 4 pages excluding references. Reports must be written in English. The proceedings will be published in ACL Anthology. *** Organizers *** Ondřej Bojar, UFAL, Charles University, Czech Republic Tirthankar Ghosal, UFAL, Charles University, Czech Republic Marie Hledikova, UFAL, Charles University, Czech Republic Anja Nedoluzhko, UFAL, Charles University, Czech Republic *** Contact *** For inquiries, contact <automin@ufal.mff.cuni.cz>. -- +++++++++++++++++++++++++++++++++++ Tirthankar Ghosal https://member.acm.org/~tghosal ++++++++++++++++++++++++++++++++++++