WIP

site/closed-track.md

@@ -2,12 +2,11 @@
 
 There is a _Closed Track_ competition for each of the two subtasks. Please note the following:
 
-* The closed track competitions are the main competitions: in order to submit a paper that describes your approach, you have to submit to the closed track of 
-  one or both of Subtask 1 and Subtask 2.
-* **Note:** If you have submitted to the closed track of one or both subtasks, you can also include information about your open track approach in your paper submission.
+* We strongly encourage submitting to the closed track
 * **IMPORTANT**: In the closed tracks, participants agree to use **only** the annotated data provided within this task to develop their model. More specifically:
   * the use of additional data labelled for sexism or misogyn is not allowed
   * the use of pretrained models or embeddings trained on data labelled for sexism or misogyny is not allowed
+  * the use of large language models, where the exact training data and, if applicable, fine-tuning steps are not known or reproducible is not allowed
   * the use of other models, ontologies, knowledge bases or similar that contains specific knowledge about sexism / misogyny is not allowed
   * pretrained models like BERT or embeddings are allowed as long as they have not been specifically pre-trained or fine-tuned on sexism/misogyny-specific data other than the data shared for this competition
 * If in doubt if your approach is compatible with the closed track requirements, please ask in the competition forum or send an email to the 

site/index.md

@@ -1,5 +1,9 @@
 ## GermEval2024 Shared Task: GerMS-Detect -- Sexism Detection in German Online News Fora
 
+
+UPDATE 2024-04-30: please not that we have updated the rules for [open track](open-track.html) and [closed track](closed-track.html) 
+and corresponding paper submissions as well es the [terms and conditions](terms.html)
+
 This shared task is about the detection of sexism/misogyny in comments
 posted in (mostly) German language to the comment section of an Austrian
 online newspaper. 

site/open-track.md

@@ -3,9 +3,12 @@
 
 There is an _Open Track_ competition for each of the two subtasks. Please note the following:
 
-* In the open tracks, participants are encouraged to use whatever approach they prefer.
+* NOTE: we strongly encourage participants to also consider submitting to the [closed track](closed-track.md) 
+* In the open tracks, participants are encouraged to use whatever approach they prefer
+* Please note that while open track solutions can make use of models or approaches where it is hard or impossible to 
+  fully reproduce all components and results, we welcome submissions where as much of the solution is open source and 
+  reproducible as possible. 
 * Additional labelled data or models or embeddings trained on labelled data are allowed.
   * HOWEVER, additional labelled data, embeddings or models must be publically available as open source or with a creative-commons license. 
-* IMPORTANT: Participants submitting in open tracks are only invited to submit a paper for the Shared Task at KONVENS 2024 describing their system, if they also made a submission in a closed track during the Competition Phase. 
-* Due to reproducibility issues, e.g. when including results from commercial or closed-source models, we do not accept papers which solely present approaches for the open tracks.
-* We do look forward, however, to find out how the results in the open tracks will compare to the closed track results.
+
+

site/terms.md

@@ -9,9 +9,9 @@ To participate please follow these steps:
 
 By your participation, you agree to the terms and conditions in their entirety, without amendment or provision. By participating in the competition, you consent to the public release of your scores and submissions at the GermEval-2024 workshop and in the associated proceedings. Participation is understood as any direct or indirect contributions to this site or the shared task organizers, such as, but not limited to: results of automatic scoring programs; manual, qualitative and quantitative assessments of the data submitted; task and system papers submitted.
 
-**Individual and team participation**: Participants can participate as individuals or as part of one team. Teams and individual participants must create exactly one account to participate in the Codabench competition and both teams and individuals must register a team name. Team name and user name must be supplied Team composition may not be changed once the Test Phase starts. Your system is named according to the team name provided at the time of submission, or to a suitable shorthand as determined by the task organizers.
+**Individual and team participation**: Participants can participate as individuals or as part of one team. Teams and individual participants must create exactly one account to participate in the Codabench competition and both teams and individuals must register a team name. Team name and user name must be supplied Team composition may not be changed once the Competition Phase starts. Your system is named according to the team name provided at the time of submission, or to a suitable shorthand as determined by the task organizers.
 
-**Scoring of submissions**: Submissions are evaluated with automatic and manual quantitative judgements, qualitative judgements, and any other metrics as the task organizers see fit. You accept that the ultimate decision of metric choice and score value is that of the task organizers. Organizers are under no obligation to release scores. Official scores may be withheld if organizers judge the submission incomplete, erroneous, deceptive, or violating the letter or spirit of the competition's rules. Inclusion of a submission's scores is not an endorsement of a team or individual's submission. If multiple submission files are uploaded during the Test Phase, the last submission file per group will be understood as the team's or participant's definitive submission and ranked as such in the task description paper. 
+**Scoring of submissions**: Submissions are evaluated with automatic and manual quantitative judgements, qualitative judgements, and any other metrics as the task organizers see fit. You accept that the ultimate decision of metric choice and score value is that of the task organizers. Organizers are under no obligation to release scores. Official scores may be withheld if organizers judge the submission incomplete, erroneous, deceptive, or violating the letter or spirit of the competition's rules. Inclusion of a submission's scores is not an endorsement of a team or individual's submission. If multiple submission files are uploaded during the Competition Phase, the last submission file per group will be understood as the team's or participant's definitive submission and ranked as such in the task description paper. 
 
 Important: submission will only be considered for the final ranking if the team name specified for the submission matches a team that has been registered for participation 
 and the codabench user who made the submission also matches the codabench user name that has been specified in the registration for participation. 
@@ -20,10 +20,9 @@ and the codabench user who made the submission also matches the codabench user n
 
 **Specific conditions for closed and open tracks**: Participants agree to follow the specific conditions for [closed tracks](closed-track.html) and [open tracks](open-track.html), which specify the type of data allowed for pretraining the model.
 
-**Submission of system description papers**: Participants having made at least one submission for a closed track during the Test Phase will be able to submit a paper describing their system. Participants having made only submissions for open tracks will not be able to submit a paper describing their system (see the specific conditions for closed and open tracks). For the closed track paper submission the open-source code of their solution has to be accessible to the public. We also stronly encourage providing open source solutions 
-for the open track submissions as far as possible. 
+**Submission of system description papers**: Participants having made at least one submission the Competition Phase will be able to submit a paper describing their system. For the closed track paper submission the open-source code of their solution has to be accessible to the public. We also stronly encourage providing open source solutions for the open track submissions as far as possible. 
 
-**Final Competition Ranking**: the final rankings for the closed tracks of subtask 1 and subtask 2 are determined after the paper review and camera ready submission have ended. 
+**Final Competition Ranking**: the final rankings for subtask 1 and subtask 2 are determined after the paper review and camera ready submission have ended. 
 Only submissions for which a paper describing the solution has been submitted and an open source solution is available will be considered in the ranking. 
 
 **Acknowledgements**: This shared task was created by OFAI with funding from the FFG project EKIP.