add valueeval24 results

touche-webis-de · Oct 10, 2024 · 2e49a96 · 2e49a96
1 parent f9e0525
commit 2e49a96
Showing 1 changed file with 66 additions and 1 deletion.
diff --git a/clef24/touche24-web/human-value-detection.html b/clef24/touche24-web/human-value-detection.html
@@ -78,7 +78,7 @@ <h1 class="uk-margin-remove-top uk-margin-remove-bottom">Human Value Detection 2
 <li><span data-uk-icon="chevron-down"></span><a class="uk-margin-small-right" href="#data">Data</a></li>
 <li><span data-uk-icon="chevron-down"></span><a class="uk-margin-small-right" href="#submission">Submission</a></li>
 <li><span data-uk-icon="chevron-down"></span><a class="uk-margin-small-right" href="#evaluation">Evaluation</a></li>
-<!-- <li><span data-uk-icon="chevron-down"></span><a class="uk-margin-small-right" href="#results">Results</a></li> -->
+<li><span data-uk-icon="chevron-down"></span><a class="uk-margin-small-right" href="#results">Results</a></li>
 <li><span data-uk-icon="chevron-down"></span><a class="uk-margin-small-right" href="#related-work">Related Work</a></li>
 <li><span data-uk-icon="chevron-down"></span><a class="uk-margin-small-right" href="#task-committee">Task Committee</a></li>
 </ul>
@@ -427,6 +427,71 @@ <h2 id="submission">Submission</h2>
 <h2 id="evaluation">Evaluation</h2>
 <p>For both sub-tasks, the submission system will evaluate runs automatically using F1-score, Precision, Recall, and ROC-curves (for each value and averaged). Runs on the sub-task leaderboards are ranked according to averaged F1-score. Use our <a href="https://github.com/touche-webis-de/touche-code/tree/main/clef24/human-value-detection/evaluator">evaluator</a> to produce a detailed report (as HTML) for your runs, including ROC curves and tables for identifying the sentences your approach got the most wrong. [<a href="https://github.com/touche-webis-de/touche-code/tree/main/clef24/human-value-detection/evaluator">evaluator</a>]</p>
 
+<h2 id="results">Results</h2>
+<div class="uk-overflow-auto">
+<table class="uk-table uk-table-divider uk-table-small uk-table-hover sortable">
+<caption>Submitted run of each team for sub-task 1.</caption>
+<thead>
+<tr><th class="header"><span>Team</span></th><th>Run</th><th colspan="20">F1-score</th></tr>
+<tr><th></th><th></th><th class="header">All</th><th class="header">Self-direction: thought</th><th class="header">Self-direction: action</th><th class="header">Stimulation</th><th class="header">Hedonism</th><th class="header">Achievement</th><th class="header">Power: dominance</th><th class="header">Power: resources</th><th class="header">Face</th><th class="header">Security: personal</th><th class="header">Security: societal</th><th class="header">Tradition</th><th class="header">Conformity: rules</th><th class="header">Conformity: interpersonal</th><th class="header">Humility</th><th class="header">Benevolence: caring</th><th class="header">Benevolence: dependability</th><th class="header">Universalism: concern</th><th class="header">Universalism: nature</th><th class="header">Universalism: tolerance</th></tr>
+</thead>
+<tbody>
+<tr><td>Hierocles of Alexandria</td><td>XLM-RoBERTa-xl tkns 38 train+val</td><td>0.39</td><td>0.15</td><td>0.27</td><td>0.30</td><td>0.37</td><td>0.45</td><td>0.42</td><td>0.49</td><td>0.31</td><td>0.42</td><td>0.49</td><td>0.46</td><td>0.51</td><td>0.24</td><td>0.00</td><td>0.34</td><td>0.33</td><td>0.47</td><td>0.63</td><td>0.27</td></tr>
+<tr><td>Hierocles of Alexandria</td><td>XLM-RoBERTa-xl tkns 19 train+val</td><td>0.38</td><td>0.15</td><td>0.27</td><td>0.31</td><td>0.36</td><td>0.43</td><td>0.41</td><td>0.51</td><td>0.32</td><td>0.44</td><td>0.49</td><td>0.48</td><td>0.51</td><td>0.23</td><td>0.00</td><td>0.34</td><td>0.35</td><td>0.50</td><td>0.63</td><td>0.24</td></tr>
+<tr><td>Hierocles of Alexandria</td><td>RoBERTa-l tkns weighted-19 train</td><td>0.37</td><td>0.19</td><td>0.23</td><td>0.31</td><td>0.32</td><td>0.40</td><td>0.41</td><td>0.45</td><td>0.31</td><td>0.43</td><td>0.48</td><td>0.51</td><td>0.48</td><td>0.26</td><td>0.11</td><td>0.34</td><td>0.33</td><td>0.48</td><td>0.60</td><td>0.27</td></tr>
+<tr><td>Hierocles of Alexandria</td><td>RoBERTa-l tkns 19 train+val</td><td>0.37</td><td>0.16</td><td>0.28</td><td>0.33</td><td>0.35</td><td>0.43</td><td>0.38</td><td>0.48</td><td>0.29</td><td>0.44</td><td>0.48</td><td>0.51</td><td>0.49</td><td>0.27</td><td>0.05</td><td>0.34</td><td>0.27</td><td>0.48</td><td>0.61</td><td>0.27</td></tr>
+<tr><td>Hierocles of Alexandria</td><td>DeBERTa-v2-xxl tkns 19 train</td><td>0.37</td><td>0.15</td><td>0.26</td><td>0.32</td><td>0.32</td><td>0.44</td><td>0.40</td><td>0.45</td><td>0.32</td><td>0.41</td><td>0.47</td><td>0.49</td><td>0.50</td><td>0.24</td><td>0.05</td><td>0.34</td><td>0.33</td><td>0.48</td><td>0.62</td><td>0.27</td></tr>
+<tr><td>Hierocles of Alexandria</td><td>RoBERTa-l tkns 38 train+val</td><td>0.37</td><td>0.12</td><td>0.24</td><td>0.32</td><td>0.36</td><td>0.42</td><td>0.39</td><td>0.46</td><td>0.28</td><td>0.43</td><td>0.47</td><td>0.49</td><td>0.49</td><td>0.22</td><td>0.00</td><td>0.34</td><td>0.32</td><td>0.47</td><td>0.61</td><td>0.27</td></tr>
+<tr><td>Hierocles of Alexandria</td><td>XLM-RoBERTa-l 19 train+val</td><td>0.36</td><td>0.15</td><td>0.29</td><td>0.35</td><td>0.35</td><td>0.44</td><td>0.39</td><td>0.47</td><td>0.29</td><td>0.40</td><td>0.48</td><td>0.49</td><td>0.50</td><td>0.20</td><td>0.08</td><td>0.33</td><td>0.32</td><td>0.47</td><td>0.60</td><td>0.24</td></tr>
+<tr><td>Hierocles of Alexandria</td><td>XLM-RoBERTa-l tkns 19 train+val</td><td>0.35</td><td>0.14</td><td>0.25</td><td>0.30</td><td>0.29</td><td>0.41</td><td>0.40</td><td>0.46</td><td>0.25</td><td>0.40</td><td>0.48</td><td>0.48</td><td>0.48</td><td>0.20</td><td>0.05</td><td>0.34</td><td>0.30</td><td>0.46</td><td>0.59</td><td>0.25</td></tr>
+<tr><td>Arthur Schopenhauer</td><td></td><td>0.35</td><td>0.12</td><td>0.24</td><td>0.33</td><td>0.35</td><td>0.40</td><td>0.37</td><td>0.47</td><td>0.24</td><td>0.38</td><td>0.46</td><td>0.49</td><td>0.50</td><td>0.19</td><td>0.00</td><td>0.32</td><td>0.31</td><td>0.46</td><td>0.60</td><td>0.27</td></tr>
+<tr><td>Hierocles of Alexandria</td><td>XLM-RoBERTa-l tkns weighted-19 train</td><td>0.34</td><td>0.13</td><td>0.20</td><td>0.29</td><td>0.28</td><td>0.37</td><td>0.37</td><td>0.45</td><td>0.22</td><td>0.33</td><td>0.46</td><td>0.46</td><td>0.49</td><td>0.21</td><td>0.04</td><td>0.32</td><td>0.32</td><td>0.47</td><td>0.63</td><td>0.21</td></tr>
+<tr><td>Philo of Alexandria</td><td></td><td>0.29</td><td>0.08</td><td>0.22</td><td>0.27</td><td>0.31</td><td>0.35</td><td>0.31</td><td>0.34</td><td>0.17</td><td>0.33</td><td>0.40</td><td>0.47</td><td>0.42</td><td>0.09</td><td>0.00</td><td>0.21</td><td>0.28</td><td>0.40</td><td>0.58</td><td>0.21</td></tr>
+<tr><td>SCaLAR NITK (Peter Abelard)</td><td></td><td>0.28</td><td>0.05</td><td>0.17</td><td>0.27</td><td>0.27</td><td>0.38</td><td>0.34</td><td>0.38</td><td>0.15</td><td>0.34</td><td>0.40</td><td>0.41</td><td>0.43</td><td>0.07</td><td>0.00</td><td>0.23</td><td>0.26</td><td>0.37</td><td>0.57</td><td>0.16</td></tr>
+<tr><td>Edward Said</td><td></td><td>0.28</td><td>0.05</td><td>0.17</td><td>0.11</td><td>0.15</td><td>0.25</td><td>0.31</td><td>0.34</td><td>0.16</td><td>0.32</td><td>0.41</td><td>0.45</td><td>0.44</td><td>0.06</td><td>0.05</td><td>0.10</td><td>0.23</td><td>0.41</td><td>0.58</td><td>0.27</td></tr>
+<tr><td>Erich Fromm</td><td></td><td>0.25</td><td>0.15</td><td>0.10</td><td>0.10</td><td>0.18</td><td>0.25</td><td>0.18</td><td>0.09</td><td>0.24</td><td>0.21</td><td>0.30</td><td>0.46</td><td>0.33</td><td>0.09</td><td>0.15</td><td>0.26</td><td>0.15</td><td>0.41</td><td>0.55</td><td>0.20</td></tr>
+<tr><td>Lawrence Kohlberg</td><td></td><td>0.25</td><td>0.08</td><td>0.11</td><td>0.19</td><td>0.23</td><td>0.31</td><td>0.22</td><td>0.31</td><td>0.11</td><td>0.28</td><td>0.37</td><td>0.34</td><td>0.42</td><td>0.09</td><td>0.00</td><td>0.21</td><td>0.23</td><td>0.34</td><td>0.54</td><td>0.18</td></tr>
+<tr><td>Aristotle</td><td>BERT baseline</td><td>0.24</td><td>0.00</td><td>0.13</td><td>0.24</td><td>0.16</td><td>0.32</td><td>0.27</td><td>0.35</td><td>0.08</td><td>0.24</td><td>0.40</td><td>0.46</td><td>0.42</td><td>0.00</td><td>0.00</td><td>0.18</td><td>0.22</td><td>0.37</td><td>0.55</td><td>0.02</td></tr>
+<tr><td>John Shelby Spong</td><td></td><td>0.07</td><td>0.00</td><td>0.00</td><td>0.02</td><td>0.00</td><td>0.16</td><td>0.05</td><td>0.11</td><td>0.00</td><td>0.01</td><td>0.28</td><td>0.00</td><td>0.15</td><td>0.00</td><td>0.00</td><td>0.00</td><td>0.00</td><td>0.13</td><td>0.27</td><td>0.00</td></tr>
+<tr><td>Alain Badiou</td><td></td><td>0.07</td><td>0.00</td><td>0.00</td><td>0.02</td><td>0.00</td><td>0.16</td><td>0.05</td><td>0.11</td><td>0.00</td><td>0.01</td><td>0.28</td><td>0.00</td><td>0.15</td><td>0.00</td><td>0.00</td><td>0.00</td><td>0.00</td><td>0.13</td><td>0.27</td><td>0.00</td></tr>
+<tr><td>Aristotle</td><td>Random baseline</td><td>0.06</td><td>0.02</td><td>0.07</td><td>0.05</td><td>0.02</td><td>0.11</td><td>0.08</td><td>0.10</td><td>0.03</td><td>0.04</td><td>0.14</td><td>0.03</td><td>0.11</td><td>0.03</td><td>0.00</td><td>0.05</td><td>0.04</td><td>0.09</td><td>0.04</td><td>0.02</td></tr>
+</tbody>
+</table>
+</div>
+
+<div class="uk-overflow-auto">
+<table class="uk-table uk-table-divider uk-table-small uk-table-hover sortable">
+<caption>Submitted run of each team for sub-task 2.</caption>
+<thead>
+<tr><th class="header"><span>Team</span></th><th>Run</th><th colspan="20">F1-score</th></tr>
+<tr><th></th><th></th><th class="header">All</th><th class="header">Self-direction: thought</th><th class="header">Self-direction: action</th><th class="header">Stimulation</th><th class="header">Hedonism</th><th class="header">Achievement</th><th class="header">Power: dominance</th><th class="header">Power: resources</th><th class="header">Face</th><th class="header">Security: personal</th><th class="header">Security: societal</th><th class="header">Tradition</th><th class="header">Conformity: rules</th><th class="header">Conformity: interpersonal</th><th class="header">Humility</th><th class="header">Benevolence: caring</th><th class="header">Benevolence: dependability</th><th class="header">Universalism: concern</th><th class="header">Universalism: nature</th><th class="header">Universalism: tolerance</th></tr>
+</thead>
+<tbody>
+<tr><td>Arthur Schopenhauer</td><td></td><td>0.83</td><td>0.77</td><td>0.83</td><td>0.85</td><td>0.88</td><td>0.87</td><td>0.73</td><td>0.84</td><td>0.80</td><td>0.82</td><td>0.84</td><td>0.78</td><td>0.80</td><td>0.79</td><td>0.74</td><td>0.91</td><td>0.89</td><td>0.86</td><td>0.85</td><td>0.81</td></tr>
+<tr><td>Edward Said</td><td></td><td>0.83</td><td>0.77</td><td>0.82</td><td>0.85</td><td>0.88</td><td>0.88</td><td>0.79</td><td>0.80</td><td>0.77</td><td>0.84</td><td>0.84</td><td>0.85</td><td>0.80</td><td>0.80</td><td>0.76</td><td>0.90</td><td>0.86</td><td>0.85</td><td>0.85</td><td>0.78</td></tr>
+<tr><td>Philo of Alexandria</td><td></td><td>0.82</td><td>0.85</td><td>0.80</td><td>0.85</td><td>0.91</td><td>0.86</td><td>0.79</td><td>0.80</td><td>0.78</td><td>0.85</td><td>0.80</td><td>0.82</td><td>0.77</td><td>0.78</td><td>0.77</td><td>0.93</td><td>0.89</td><td>0.84</td><td>0.83</td><td>0.79</td></tr>
+<tr><td>Aristotle</td><td>BERT baseline</td><td>0.81</td><td>0.83</td><td>0.79</td><td>0.86</td><td>0.88</td><td>0.84</td><td>0.77</td><td>0.80</td><td>0.74</td><td>0.84</td><td>0.81</td><td>0.78</td><td>0.78</td><td>0.79</td><td>0.87</td><td>0.89</td><td>0.86</td><td>0.85</td><td>0.81</td><td>0.78</td></tr>
+<tr><td>John Shelby Spong</td><td></td><td>0.81</td><td>0.81</td><td>0.77</td><td>0.83</td><td>0.88</td><td>0.88</td><td>0.77</td><td>0.79</td><td>0.76</td><td>0.83</td><td>0.82</td><td>0.85</td><td>0.76</td><td>0.81</td><td>0.84</td><td>0.90</td><td>0.85</td><td>0.81</td><td>0.81</td><td>0.79</td></tr>
+<tr><td>Alain Badiou</td><td></td><td>0.81</td><td>0.81</td><td>0.77</td><td>0.83</td><td>0.88</td><td>0.88</td><td>0.77</td><td>0.79</td><td>0.76</td><td>0.83</td><td>0.82</td><td>0.85</td><td>0.76</td><td>0.81</td><td>0.84</td><td>0.90</td><td>0.85</td><td>0.81</td><td>0.81</td><td>0.79</td></tr>
+<tr><td>Hierocles of Alexandria</td><td>XLM-RoBERTa-xl tkns 38 train+val</td><td>0.77</td><td>0.73</td><td>0.73</td><td>0.77</td><td>0.75</td><td>0.78</td><td>0.77</td><td>0.79</td><td>0.71</td><td>0.78</td><td>0.79</td><td>0.77</td><td>0.78</td><td>0.74</td><td>0.25</td><td>0.74</td><td>0.77</td><td>0.78</td><td>0.84</td><td>0.71</td></tr>
+<tr><td>Hierocles of Alexandria</td><td>RoBERTa-l tkns 38 train+val</td><td>0.77</td><td>0.72</td><td>0.72</td><td>0.78</td><td>0.74</td><td>0.78</td><td>0.78</td><td>0.78</td><td>0.73</td><td>0.78</td><td>0.78</td><td>0.78</td><td>0.77</td><td>0.73</td><td>0.22</td><td>0.78</td><td>0.77</td><td>0.78</td><td>0.82</td><td>0.74</td></tr>
+<tr><td>SCaLAR NITK (Peter Abelard)</td><td></td><td>0.77</td><td>0.69</td><td>0.72</td><td>0.78</td><td>0.73</td><td>0.79</td><td>0.77</td><td>0.79</td><td>0.71</td><td>0.78</td><td>0.81</td><td>0.79</td><td>0.77</td><td>0.70</td><td>0.70</td><td>0.77</td><td>0.76</td><td>0.79</td><td>0.80</td><td>0.71</td></tr>
+<tr><td>Erich Fromm</td><td></td><td>0.70</td><td>0.71</td><td>0.69</td><td>0.73</td><td>0.70</td><td>0.72</td><td>0.74</td><td>0.73</td><td>0.67</td><td>0.60</td><td>0.66</td><td>0.76</td><td>0.70</td><td>0.68</td><td>0.73</td><td>0.75</td><td>0.71</td><td>0.70</td><td>0.73</td><td>0.67</td></tr>
+<tr><td>Hierocles of Alexandria</td><td>RoBERTa-l tkns weighted-19 train</td><td>0.69</td><td>0.73</td><td>0.71</td><td>0.76</td><td>0.71</td><td>0.71</td><td>0.74</td><td>0.68</td><td>0.65</td><td>0.55</td><td>0.59</td><td>0.77</td><td>0.69</td><td>0.64</td><td>0.70</td><td>0.77</td><td>0.74</td><td>0.70</td><td>0.72</td><td>0.66</td></tr>
+<tr><td>Hierocles of Alexandria</td><td>XLM RoBERTa-l tkns weighted-19 train</td><td>0.69</td><td>0.71</td><td>0.70</td><td>0.74</td><td>0.72</td><td>0.71</td><td>0.74</td><td>0.68</td><td>0.66</td><td>0.61</td><td>0.58</td><td>0.76</td><td>0.69</td><td>0.65</td><td>0.70</td><td>0.76</td><td>0.75</td><td>0.70</td><td>0.73</td><td>0.65</td></tr>
+<tr><td>Hierocles of Alexandria</td><td>RoBERTa-l tkns 19 train+val</td><td>0.68</td><td>0.73</td><td>0.71</td><td>0.77</td><td>0.72</td><td>0.71</td><td>0.74</td><td>0.67</td><td>0.63</td><td>0.55</td><td>0.59</td><td>0.78</td><td>0.69</td><td>0.62</td><td>0.70</td><td>0.77</td><td>0.72</td><td>0.68</td><td>0.72</td><td>0.65</td></tr>
+<tr><td>Hierocles of Alexandria</td><td>XLM-RoBERTa-l tkns 19 train+val</td><td>0.68</td><td>0.72</td><td>0.72</td><td>0.77</td><td>0.74</td><td>0.72</td><td>0.74</td><td>0.67</td><td>0.64</td><td>0.57</td><td>0.55</td><td>0.77</td><td>0.68</td><td>0.64</td><td>0.70</td><td>0.77</td><td>0.76</td><td>0.69</td><td>0.73</td><td>0.63</td></tr>
+<tr><td>Hierocles of Alexandria</td><td>XLM-RoBERTa-l 19 train+val</td><td>0.68</td><td>0.72</td><td>0.72</td><td>0.77</td><td>0.73</td><td>0.72</td><td>0.74</td><td>0.67</td><td>0.64</td><td>0.58</td><td>0.58</td><td>0.77</td><td>0.68</td><td>0.63</td><td>0.70</td><td>0.75</td><td>0.73</td><td>0.69</td><td>0.72</td><td>0.64</td></tr>
+<tr><td>Hierocles of Alexandria</td><td>XLM-RoBERTa-xl tkns 19 train+va</td><td>0.68</td><td>0.71</td><td>0.71</td><td>0.76</td><td>0.72</td><td>0.72</td><td>0.74</td><td>0.66</td><td>0.64</td><td>0.56</td><td>0.56</td><td>0.79</td><td>0.68</td><td>0.62</td><td>0.70</td><td>0.75</td><td>0.74</td><td>0.69</td><td>0.72</td><td>0.66</td></tr>
+<tr><td>Hierocles of Alexandria</td><td>DeBERTa-v2-xxl tkns 19 train</td><td>0.68</td><td>0.71</td><td>0.70</td><td>0.77</td><td>0.74</td><td>0.72</td><td>0.76</td><td>0.66</td><td>0.63</td><td>0.58</td><td>0.58</td><td>0.77</td><td>0.69</td><td>0.61</td><td>0.70</td><td>0.78</td><td>0.75</td><td>0.68</td><td>0.73</td><td>0.65</td></tr>
+<tr><td>Lawrence Kohlberg</td><td></td><td>0.66</td><td>0.81</td><td>0.77</td><td>0.83</td><td>0.80</td><td>0.70</td><td>0.76</td><td>0.63</td><td>0.57</td><td>0.33</td><td>0.45</td><td>0.85</td><td>0.63</td><td>0.46</td><td>0.84</td><td>0.90</td><td>0.79</td><td>0.69</td><td>0.70</td><td>0.60</td></tr>
+<tr><td>Aristotle</td><td>Random baseline</td><td>0.52</td><td>0.51</td><td>0.47</td><td>0.54</td><td>0.52</td><td>0.53</td><td>0.55</td><td>0.53</td><td>0.52</td><td>0.52</td><td>0.50</td><td>0.54</td><td>0.53</td><td>0.49</td><td>0.45</td><td>0.53</td><td>0.57</td><td>0.52</td><td>0.49</td><td>0.56</td></tr>
+</tbody>
+</table>
+</div>
+
+
+
 
 <h2><a id="related-work"></a>Related Work</h2>
 <ul>