Adding papers

_posts/papers/2024-01-01-2024.eurali-1.8.md

@@ -0,0 +1,23 @@
+---
+title: An Evaluation of Language Models for Hyperpartisan Ideology Detection in Persian
+  Twitter
+venue: EURALI
+names: Sahar Omidi Shayegan, Isar Nejadgholi, Kellin Pelrine, Hao Yu, Sacha Lévy,
+  Zachary Yang, J. Godbout, Reihaneh Rabbany
+tags:
+- EURALI
+link: https://www.semanticscholar.org/paper/05d76a00dd42fd5e75a7222d4479d2a35608c7a3
+author: Kellin Pelrine
+categories: Publications
+
+---
+
+*{{ page.names }}*
+
+**{{ page.venue }}**
+
+{% include display-publication-links.html pub=page %}
+
+## Abstract
+
+Large Language Models (LLMs) have shown significant promise in various tasks, including identifying the political beliefs of English-speaking social media users from their posts. However, assessing LLMs for this task in non-English languages remains unexplored. In this work, we ask to what extent LLMs can predict the political ideologies of users in Persian social media. To answer this question, we first acknowledge that political parties are not well-defined among Persian users, and therefore, we simplify the task to a much simpler task of hyperpartisan ideology detection. We create a new benchmark and show the potential and limitations of both open-source and commercial LLMs in classifying the hyper-partisan ideologies of users. We compare these models with smaller fine-tuned models, both on the Persian language (ParsBERT) and translated data (RoBERTa), showing that they considerably outperform generative LLMs in this task. We further demonstrate that the performance of the generative LLMs degrades when classifying users based on their tweets instead of their bios and even when tweets are added as additional information, whereas the smaller fine-tuned models are robust and achieve similar performance for all classes. This study is a first step toward political ideology detection in Persian Twitter, with implications for future research to understand the dynamics of ideologies in Persian social media.

_posts/papers/2024-01-01-2024.personalize-1.10.md

@@ -0,0 +1,22 @@
+---
+title: Quantifying learning-style adaptation in effectiveness of LLM teaching
+venue: PERSONALIZE
+names: Ruben Weijers, Gabrielle Fidelis de Castilho, J. Godbout, Reihaneh Rabbany,
+  Kellin Pelrine
+tags:
+- PERSONALIZE
+link: https://www.semanticscholar.org/paper/8ad551a161b2af427dff42bed378828ec513aa7c
+author: Reihaneh Rabbany
+categories: Publications
+
+---
+
+*{{ page.names }}*
+
+**{{ page.venue }}**
+
+{% include display-publication-links.html pub=page %}
+
+## Abstract
+
+This preliminary study aims to investigate whether AI, when prompted based on individual learning styles, can effectively improve comprehension and learning experiences in educational settings. It involves tailoring LLMs baseline prompts and comparing the results of a control group receiving standard content and an experimental group receiving learning style-tailored content. Preliminary results suggest that GPT-4 can generate responses aligned with various learning styles, indicating the potential for enhanced engagement and comprehension. However, these results also reveal challenges, including the model’s tendency for sycophantic behavior and variability in responses. Our findings suggest that a more sophisticated prompt engineering approach is required for integrating AI into education (AIEd) to improve educational outcomes.

_posts/papers/2024-01-02-2401.01197.md

@@ -0,0 +1,22 @@
+---
+title: Uncertainty Resolution in Misinformation Detection
+venue: UNCERTAINLP
+names: Yury Orlovskiy, Camille Thibault, Anne Imouza, J. Godbout, Reihaneh Rabbany,
+  Kellin Pelrine
+tags:
+- UNCERTAINLP
+link: https://arxiv.org/abs/2401.01197
+author: Reihaneh Rabbany
+categories: Publications
+
+---
+
+*{{ page.names }}*
+
+**{{ page.venue }}**
+
+{% include display-publication-links.html pub=page %}
+
+## Abstract
+
+Misinformation poses a variety of risks, such as undermining public trust and distorting factual discourse. Large Language Models (LLMs) like GPT-4 have been shown effective in mitigating misinformation, particularly in handling statements where enough context is provided. However, they struggle to assess ambiguous or context-deficient statements accurately. This work introduces a new method to resolve uncertainty in such statements. We propose a framework to categorize missing information and publish category labels for the LIAR-New dataset, which is adaptable to cross-domain content with missing information. We then leverage this framework to generate effective user queries for missing context. Compared to baselines, our method improves the rate at which generated questions are answerable by the user by 38 percentage points and classification performance by over 10 percentage points macro F1. Thus, this approach may provide a valuable component for future misinformation mitigation pipelines.

_posts/papers/2024-01-03-2401.01990.md

@@ -0,0 +1,21 @@
+---
+title: 'GPS-SSL: Guided Positive Sampling to Inject Prior Into Self-Supervised Learning'
+venue: arXiv.org
+names: Aarash Feizi, Randall Balestriero, Adriana Romero-Soriano, Reihaneh Rabbany
+tags:
+- arXiv.org
+link: https://arxiv.org/abs/2401.01990
+author: Aarash Feizi
+categories: Publications
+
+---
+
+*{{ page.names }}*
+
+**{{ page.venue }}**
+
+{% include display-publication-links.html pub=page %}
+
+## Abstract
+
+We propose Guided Positive Sampling Self-Supervised Learning (GPS-SSL), a general method to inject a priori knowledge into Self-Supervised Learning (SSL) positive samples selection. Current SSL methods leverage Data-Augmentations (DA) for generating positive samples and incorporate prior knowledge - an incorrect, or too weak DA will drastically reduce the quality of the learned representation. GPS-SSL proposes instead to design a metric space where Euclidean distances become a meaningful proxy for semantic relationship. In that space, it is now possible to generate positive samples from nearest neighbor sampling. Any prior knowledge can now be embedded into that metric space independently from the employed DA. From its simplicity, GPS-SSL is applicable to any SSL method, e.g. SimCLR or BYOL. A key benefit of GPS-SSL is in reducing the pressure in tailoring strong DAs. For example GPS-SSL reaches 85.58% on Cifar10 with weak DA while the baseline only reaches 37.51%. We therefore move a step forward towards the goal of making SSL less reliant on DA. We also show that even when using strong DAs, GPS-SSL outperforms the baselines on under-studied domains. We evaluate GPS-SSL along with multiple baseline SSL methods on numerous downstream datasets from different domains when the models use strong or minimal data augmentations. We hope that GPS-SSL will open new avenues in studying how to inject a priori knowledge into SSL in a principled manner.

_posts/papers/2024-01-12-2401.06920.md

@@ -0,0 +1,21 @@
+---
+title: Comparing GPT-4 and Open-Source Language Models in Misinformation Mitigation
+venue: arXiv.org
+names: Tyler Vergho, J. Godbout, Reihaneh Rabbany, Kellin Pelrine
+tags:
+- arXiv.org
+link: https://arxiv.org/abs/2401.06920
+author: Reihaneh Rabbany
+categories: Publications
+
+---
+
+*{{ page.names }}*
+
+**{{ page.venue }}**
+
+{% include display-publication-links.html pub=page %}
+
+## Abstract
+
+Recent large language models (LLMs) have been shown to be effective for misinformation detection. However, the choice of LLMs for experiments varies widely, leading to uncertain conclusions. In particular, GPT-4 is known to be strong in this domain, but it is closed source, potentially expensive, and can show instability between different versions. Meanwhile, alternative LLMs have given mixed results. In this work, we show that Zephyr-7b presents a consistently viable alternative, overcoming key limitations of commonly used approaches like Llama-2 and GPT-3.5. This provides the research community with a solid open-source option and shows open-source models are gradually catching up on this task. We then highlight how GPT-3.5 exhibits unstable performance, such that this very widely used model could provide misleading results in misinformation detection. Finally, we validate new tools including approaches to structured output and the latest version of GPT-4 (Turbo), showing they do not compromise performance, thus unlocking them for future research and potentially enabling more complex pipelines for misinformation mitigation.

_posts/papers/2024-01-13-2401.08694.md

@@ -0,0 +1,22 @@
+---
+title: Combining Confidence Elicitation and Sample-based Methods for Uncertainty Quantification
+  in Misinformation Mitigation
+venue: UNCERTAINLP
+names: Mauricio Rivera, J. Godbout, Reihaneh Rabbany, Kellin Pelrine
+tags:
+- UNCERTAINLP
+link: https://arxiv.org/abs/2401.08694
+author: Reihaneh Rabbany
+categories: Publications
+
+---
+
+*{{ page.names }}*
+
+**{{ page.venue }}**
+
+{% include display-publication-links.html pub=page %}
+
+## Abstract
+
+Large Language Models have emerged as prime candidates to tackle misinformation mitigation. However, existing approaches struggle with hallucinations and overconfident predictions. We propose an uncertainty quantification framework that leverages both direct confidence elicitation and sampled-based consistency methods to provide better calibration for NLP misinformation mitigation solutions. We first investigate the calibration of sample-based consistency methods that exploit distinct features of consistency across sample sizes and stochastic levels. Next, we evaluate the performance and distributional shift of a robust numeric verbalization prompt across single vs. two-step confidence elicitation procedure. We also compare the performance of the same prompt with different versions of GPT and different numerical scales. Finally, we combine the sample-based consistency and verbalized methods to propose a hybrid framework that yields a better uncertainty estimation for GPT models. Overall, our work proposes novel uncertainty quantification methods that will improve the reliability of Large Language Models in misinformation mitigation applications.

_posts/papers/2024-02-06-2402.03651.md

@@ -0,0 +1,22 @@
+---
+title: Temporal Graph Analysis with TGX
+venue: Web Search and Data Mining
+names: Razieh Shirzadkhani, Shenyang Huang, Elahe Kooshafar, Reihaneh Rabbany, Farimah
+  Poursafaei
+tags:
+- Web Search and Data Mining
+link: https://arxiv.org/abs/2402.03651
+author: Shenyang Huang
+categories: Publications
+
+---
+
+*{{ page.names }}*
+
+**{{ page.venue }}**
+
+{% include display-publication-links.html pub=page %}
+
+## Abstract
+
+Real-world networks, with their evolving relations, are best captured as temporal graphs. However, existing software libraries are largely designed for static graphs where the dynamic nature of temporal graphs is ignored. Bridging this gap, we introduce TGX, a Python package specially designed for analysis of temporal networks that encompasses an automated pipeline for data loading, data processing, and analysis of evolving graphs. TGX provides access to eleven built-in datasets and eight external Temporal Graph Benchmark (TGB) datasets as well as any novel datasets in the .csv format. Beyond data loading, TGX facilitates data processing functionalities such as discretization of temporal graphs and node sub-sampling to accelerate working with larger datasets. For comprehensive investigation, TGX offers network analysis by providing a diverse set of measures, including average node degree and the evolving number of nodes and edges per timestamp. Additionally, the package consolidates meaningful visualization plots indicating the evolution of temporal patterns, such as Temporal Edge Appearance (TEA) and Temporal Edge Traffic (TET) plots. The TGX package is a robust tool for examining the features of temporal graphs and can be used in various areas like studying social networks, citation networks, and tracking user interactions. We plan to continuously support and update TGX based on community feedback. TGX is publicly available on: https://github.com/ComplexData-MILA/TGX.

_posts/papers/2024-03-24-10.1609-aaai.v38i20.30233.md

@@ -0,0 +1,24 @@
+---
+title: 'T-NET: Weakly Supervised Graph Learning for Combatting Human Trafficking'
+venue: AAAI Conference on Artificial Intelligence
+names: Pratheeksha Nair, Javin Liu, Catalina Vajiac, Andreas M. Olligschlaeger, Duen
+  Horng Chau, M. Cazzolato, Cara Jones, Christos Faloutsos, Reihaneh Rabbany
+tags:
+- AAAI Conference on Artificial Intelligence
+link: https://doi.org/10.1609/aaai.v38i20.30233
+author: Pratheeksha Nair
+categories: Publications
+
+---
+
+*{{ page.names }}*
+
+**{{ page.venue }}**
+
+{% include display-publication-links.html pub=page %}
+
+## Abstract
+
+Human trafficking (HT) for forced sexual exploitation, often described as modern-day slavery, is a pervasive problem that affects millions of people worldwide. Perpetrators of this crime post advertisements (ads) on behalf of their victims on adult service websites (ASW). These websites typically contain hundreds of thousands of ads including those posted by independent escorts, massage parlor agencies and spammers (fake ads). Detecting suspicious activity in these ads is difficult and developing data-driven methods is challenging due to the hard-to-label, complex and sensitive nature of the data. 
+
+In this paper, we propose T-Net, which unlike previous solutions, formulates this problem as weakly supervised classification. Since it takes several months to years to investigate a case and obtain a single definitive label, we design domain-specific signals or indicators that provide weak labels. T-Net also looks into connections between ads and models the problem as a graph learning task instead of classifying ads independently. We show that T-Net outperforms all baselines on a real-world dataset of ads by 7% average weighted F1 score. Given that this data contains personally identifiable information, we also present a realistic data generator and provide the first publicly available dataset in this domain which may be leveraged by the wider research community.

_posts/papers/2024-05-22-10.1038-s41598-024-62271-0.md

@@ -0,0 +1,21 @@
+---
+title: Static graph approximations of dynamic contact networks for epidemic forecasting
+venue: Scientific Reports
+names: Razieh Shirzadkhani, Shenyang Huang, Abby Leung, Reihaneh Rabbany
+tags:
+- Scientific Reports
+link: https://doi.org/10.1038/s41598-024-62271-0
+author: Shenyang Huang
+categories: Publications
+
+---
+
+*{{ page.names }}*
+
+**{{ page.venue }}**
+
+{% include display-publication-links.html pub=page %}
+
+## Abstract
+
+None

_posts/papers/2024-06-28-10.1145-3675805.md

@@ -0,0 +1,21 @@
+---
+title: 'Game On, Hate Off: A Study of Toxicity in Online Multiplayer Environments'
+venue: Games Res. Pract.
+names: Zachary Yang, Nicolas Grenon-Godbout, Reihaneh Rabbany
+tags:
+- Games Res. Pract.
+link: https://doi.org/10.1145/3675805
+author: Zachary Yang
+categories: Publications
+
+---
+
+*{{ page.names }}*
+
+**{{ page.venue }}**
+
+{% include display-publication-links.html pub=page %}
+
+## Abstract
+
+The advent of online spaces, particularly social media platforms and video games, has brought forth a significant challenge: the detection and mitigation of toxic and harmful speech. This issue is not only pervasive but also detrimental to the overall user experience. In this study, we leverage small language models to reliably detect toxicity, achieving an average precision of 0.95. Analyzing eight months of chat data from two Ubisoft games, we uncover patterns and trends in toxic behavior. The insights derived from our research will contribute to the development of healthier online communities and inform preventive measures against toxicity.