5th Werewolf AI Competition (INLG 2023 Gerenation Challenge)
Call For Participation
Summary:
Werewolf is a social, hidden identity game that requires debate
between players and coalition building. The goal of our AIWerewolf
contest is to build an AI agent that is able to play this game against
other AI.
We provide three divisions, the Natural Langauge Division, the
Protocol Division, and the Infrastructure Division. Their regsitration
deadlines are as follows:
Natural Language Division: June 30th, 2023 (Anywhere on Earth)
Protocol Division: July 21st, 2023 (Anywhere on Earth)
Infrastructure Division: August 18th, 2023 (Anywhere on Earth)
Presetation of Results: September 11th-15th (in INLG 2023)
Our shared task is one of the Generation Challenges (GenChal) of INLG
2023 (16th International Natural Language Generation Conference),
which will be held in Prague from September 11th to 15th, but it is
not mandatry for our shared task participants to attend the INLG 2023
conference.
Please refer to the details of each division in this CFP and the websites below.
Natural Language Division: https://github.com/aiwolfdial/RandomTalkAgent
Protocol Division: https://github.com/aiwolf/CompetitionProtocolDivision
Infrastructure Division:
https://github.com/aiwolf/CompetitionInfrastructureDivision
The AIWerewolf Project: http://aiwolf.org/5th-international-aiwolf-contest
INLG 2023: https://inlg2023.github.io/
SIGDial 2023, which is jointly held with INLG 2023 this year:
https://2023.sigdial.org/
Why AI Werewolf?
Recent achievements of generation models, e.g. ChatGPT, are gathering
greater attentions. However, such a huge language model would not be
sufficiently able to handle coherent responses, longer contexts,
common grounds, and logics.
AIWolfDial 2023 is an international open contest for automatic players
of the conversation game "Mafia". It requires players not just to
communicate but to infer, persuade, deceive other players via coherent
logical conversations, while having the role-playing non-task-oriented
chats as well. We believe that this contest reveals current issues in
the recent huge language models, showing directions of next
breakthrough in the NLP area.
From the viewpoint of Game AI area, players must hide information, in
contrast to perfect information games such as chess or Reversi. Each
player acquires secret information from other players' conversations
and behavior and acts by hiding information to accomplish their
objectives. Players are required persuasion for earning confidence,
and speculation for detecting fabrications.
Natural Language Division
In this division, participants must build an artificial intelligence
agent that can play the werewolf game as humans do, using natural
language. Participants in this division will be evaluated by a panel
of judges, who will grade the subjective quality of the dialog
generated by the agent. Agents must communicate in Japanese or
English.
Registration
A team should send a mail to aiwolf [at] kanolab.net (replace at by
@), describing your team name, a contact e-mail address, names and
affiliations of its members (please mark a contact person when a team
consists of multiple members). Registration is free.
Agent Testing
A participant is required to implement an AI werewolf agent that
connects to our AIWolf server at the specified time. Before that, we
will provide an AIWolf server where participants can try connecting
with dummy agents to check their system's behavior. Make sure that
your system works correctly on the testing system before the actual
competition run.
Preliminary Run
Participants should run five agents connected to our server, running
50 games. Submit your game logs to the organisers via e-mail. If there
are too many participants for the final run, organisers will select
final run teams from these logs. In either case, these logs will be
used in the final evaluation.
Final Run
Each participant team is required to connect its agent to our server
at a specified time to play games with other participants. If your
team cannot remotely attend the final run, you should prepare your
system so that the organisers can somehow trigger your remote system
to connect our server and run, for example, via a web browser.
System Evaluation
Participants should submit a system design description document to the
organizers. This document and logs of the games might be used for
research purposes and included and published in an overview paper
without any further permission. Participants are encouraged to submit
a paper to the INLG workshop. Reviewers will perform subjective
evaluations on the game logs.
Implementation of Natural Language Division Agents
Agents must communicate in English or Japanese. We might translate
English-Japanese by machine translation internally when the number of
participants for either of the languages is small. Participants are
required to implement an AI werewolf agent that connects to our AIWolf
server via the network. That is, there is no limitation on the client
side regarding hardware or software other than the network I/O
specifications.
Please refer to our sample agent code with runtime environment (java),
distributed from our Github repository:
https://github.com/aiwolfdial/RandomTalkAgent
which also describes how to connect to our game server, etc.
Resources
The Mafiascum Dataset provides is a collection of over 700 games of
Mafia: https://bitbucket.org/bopjesvla/thesis/src/master/
Protocol Division
In the protocol division, the AIWolf agents which communicate with
each other via the AIWolf protocol, compete for win rate.
Agents are evaluated by their average win rate in 15-player games and
5-player games.
This division is divided into a qualifying competition and a final one.
In the qualifying competition, all participants play against each
other, and 15 agents with the highest win rate can advance to the
final competition as finalists.
In the final competition, finalists play against each other as many
times as possible until the difference in win rate becomes
signigicant.
For more information, please visit the web site of this division.
https://github.com/aiwolf/CompetitionProtocolDivision
Infrastructure Division
In the 5th International AI Werewolf Competition, we open an
infrastructure division to evaluate the contribution to the
development of AI Werewolf such as sample agents, libraries for making
agents, alternative game servers, cool game viewers, etc.
Creations submited to this division will be evaluated by the
participants and organizers of the AI Werewolf competition, and a vote
will choose the best contribution of this year.
For more information, please visit the web site of this division.
https://github.com/aiwolf/CompetitionInfrastructureDivision
Organizers
Yoshinobu Kano, Shizuoka University, Japan
Claus Aranha, Tsukuba University
Takashi Otsuki, Yamagata University, Japan
Fujio Toriumi, The University of Tokyo, Japan
Hirotaka Osawa, University of Tsukuba, Japan
Daisuke Katagami, Tokyo Polytechnic University, Japan
Michimasa Inaba, The University of Electro-Communications, Japan
Kei Harada, The University of Electro-Communications, Japan
On behalf of the AIWolf project organizers,
Yoshinobu Kano
Associate Professor, Shizuoka University
kano@inf.shizuoka.ac.jp
5th Werewolf AI Competition (INLG 2023 Gerenation Challenge)
Call For Participation
# Summary:
Werewolf is a social, hidden identity game that requires debate
between players and coalition building. The goal of our AIWerewolf
contest is to build an AI agent that is able to play this game against
other AI.
We provide three divisions, the Natural Langauge Division, the
Protocol Division, and the Infrastructure Division. Their regsitration
deadlines are as follows:
Natural Language Division: June 30th, 2023 (Anywhere on Earth)
Protocol Division: July 21st, 2023 (Anywhere on Earth)
Infrastructure Division: August 18th, 2023 (Anywhere on Earth)
Presetation of Results: September 11th-15th (in INLG 2023)
Our shared task is one of the Generation Challenges (GenChal) of INLG
2023 (16th International Natural Language Generation Conference),
which will be held in Prague from September 11th to 15th, but it is
not mandatry for our shared task participants to attend the INLG 2023
conference.
Please refer to the details of each division in this CFP and the websites below.
Natural Language Division: https://github.com/aiwolfdial/RandomTalkAgent
Protocol Division: https://github.com/aiwolf/CompetitionProtocolDivision
Infrastructure Division:
https://github.com/aiwolf/CompetitionInfrastructureDivision
The AIWerewolf Project: http://aiwolf.org/5th-international-aiwolf-contest
INLG 2023: https://inlg2023.github.io/
SIGDial 2023, which is jointly held with INLG 2023 this year:
https://2023.sigdial.org/
# Why AI Werewolf?
Recent achievements of generation models, e.g. ChatGPT, are gathering
greater attentions. However, such a huge language model would not be
sufficiently able to handle coherent responses, longer contexts,
common grounds, and logics.
AIWolfDial 2023 is an international open contest for automatic players
of the conversation game "Mafia". It requires players not just to
communicate but to infer, persuade, deceive other players via coherent
logical conversations, while having the role-playing non-task-oriented
chats as well. We believe that this contest reveals current issues in
the recent huge language models, showing directions of next
breakthrough in the NLP area.
From the viewpoint of Game AI area, players must hide information, in
contrast to perfect information games such as chess or Reversi. Each
player acquires secret information from other players' conversations
and behavior and acts by hiding information to accomplish their
objectives. Players are required persuasion for earning confidence,
and speculation for detecting fabrications.
# Natural Language Division
In this division, participants must build an artificial intelligence
agent that can play the werewolf game as humans do, using natural
language. Participants in this division will be evaluated by a panel
of judges, who will grade the subjective quality of the dialog
generated by the agent. Agents must communicate in Japanese or
English.
## Registration
A team should send a mail to aiwolf [at] kanolab.net (replace at by
@), describing your team name, a contact e-mail address, names and
affiliations of its members (please mark a contact person when a team
consists of multiple members). Registration is free.
## Agent Testing
A participant is required to implement an AI werewolf agent that
connects to our AIWolf server at the specified time. Before that, we
will provide an AIWolf server where participants can try connecting
with dummy agents to check their system's behavior. Make sure that
your system works correctly on the testing system before the actual
competition run.
## Preliminary Run
Participants should run five agents connected to our server, running
50 games. Submit your game logs to the organisers via e-mail. If there
are too many participants for the final run, organisers will select
final run teams from these logs. In either case, these logs will be
used in the final evaluation.
## Final Run
Each participant team is required to connect its agent to our server
at a specified time to play games with other participants. If your
team cannot remotely attend the final run, you should prepare your
system so that the organisers can somehow trigger your remote system
to connect our server and run, for example, via a web browser.
## System Evaluation
Participants should submit a system design description document to the
organizers. This document and logs of the games might be used for
research purposes and included and published in an overview paper
without any further permission. Participants are encouraged to submit
a paper to the INLG workshop. Reviewers will perform subjective
evaluations on the game logs.
## Implementation of Natural Language Division Agents
Agents must communicate in English or Japanese. We might translate
English-Japanese by machine translation internally when the number of
participants for either of the languages is small. Participants are
required to implement an AI werewolf agent that connects to our AIWolf
server via the network. That is, there is no limitation on the client
side regarding hardware or software other than the network I/O
specifications.
Please refer to our sample agent code with runtime environment (java),
distributed from our Github repository:
https://github.com/aiwolfdial/RandomTalkAgent
which also describes how to connect to our game server, etc.
## Resources
The Mafiascum Dataset provides is a collection of over 700 games of
Mafia: https://bitbucket.org/bopjesvla/thesis/src/master/
# Protocol Division
In the protocol division, the AIWolf agents which communicate with
each other via the AIWolf protocol, compete for win rate.
Agents are evaluated by their average win rate in 15-player games and
5-player games.
This division is divided into a qualifying competition and a final one.
In the qualifying competition, all participants play against each
other, and 15 agents with the highest win rate can advance to the
final competition as finalists.
In the final competition, finalists play against each other as many
times as possible until the difference in win rate becomes
signigicant.
For more information, please visit the web site of this division.
https://github.com/aiwolf/CompetitionProtocolDivision
# Infrastructure Division
In the 5th International AI Werewolf Competition, we open an
infrastructure division to evaluate the contribution to the
development of AI Werewolf such as sample agents, libraries for making
agents, alternative game servers, cool game viewers, etc.
Creations submited to this division will be evaluated by the
participants and organizers of the AI Werewolf competition, and a vote
will choose the best contribution of this year.
For more information, please visit the web site of this division.
https://github.com/aiwolf/CompetitionInfrastructureDivision
# Organizers
Yoshinobu Kano, Shizuoka University, Japan
Claus Aranha, Tsukuba University
Takashi Otsuki, Yamagata University, Japan
Fujio Toriumi, The University of Tokyo, Japan
Hirotaka Osawa, University of Tsukuba, Japan
Daisuke Katagami, Tokyo Polytechnic University, Japan
Michimasa Inaba, The University of Electro-Communications, Japan
Kei Harada, The University of Electro-Communications, Japan
On behalf of the AIWolf project organizers,
Yoshinobu Kano
Associate Professor, Shizuoka University
kano@inf.shizuoka.ac.jp