-
Notifications
You must be signed in to change notification settings - Fork 3
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
Showing
1 changed file
with
113 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,113 @@ | ||
| <!DOCTYPE html> | ||
| <html lang="en"> | ||
|
|
||
| <head> | ||
| <meta charset="utf-8"> | ||
| <meta http-equiv="X-UA-Compatible" content="IE=edge"> | ||
| <meta name="viewport" content="width=device-width, initial-scale=1"> | ||
| <meta name="description" content="5th Workshop on Scholarly Document Processing"> | ||
|
|
||
| <title>5th Workshop on Scholarly Document Processing</title> | ||
|
|
||
| <!-- Bootstrap core CSS --> | ||
| <link href="./dist/css/bootstrap.min.css" rel="stylesheet"> | ||
|
|
||
| <!-- Fira Sans font --> | ||
| <link href="https://fonts.googleapis.com/css?family=Fira+Sans&display=swap" rel="stylesheet"> | ||
|
|
||
| <!--[if lt IE 9]><script src="../../assets/js/ie8-responsive-file-warning.js"></script><![endif]--> | ||
|
|
||
| <!-- HTML5 shim and Respond.js IE8 support of HTML5 elements and media queries --> | ||
| <!--[if lt IE 9]> | ||
| <script src="https://oss.maxcdn.com/libs/html5shiv/3.7.0/html5shiv.js"></script> | ||
| <script src="https://oss.maxcdn.com/libs/respond.js/1.4.2/respond.min.js"></script> | ||
| <![endif]--> | ||
|
|
||
| <!-- Custom styles for this template --> | ||
| <link href="styles.css" rel="stylesheet"> | ||
|
|
||
| <!-- icons --> | ||
| <link rel="stylesheet" href="./font-awesome-4.1.0/css/font-awesome.min.css"> | ||
|
|
||
| <!-- jQuery --> | ||
| <script src="https://ajax.googleapis.com/ajax/libs/jquery/1.12.4/jquery.min.js"></script> | ||
|
|
||
| </head> | ||
|
|
||
| <body> | ||
|
|
||
| <!-- NAVBAR ================================================== --> | ||
|
|
||
| <div class="navbar-wrapper"></div> | ||
| <script src="menu.js" type="text/javascript"></script> | ||
|
|
||
| <!-- MAIN CONTENT ============================================= --> | ||
|
|
||
| <div class="container marketing navbar-spacing"> | ||
|
|
||
| <div class="row featurette"> | ||
| <div class="col-md-12"> | ||
| <h1 style="display: flex; align-items: center;"> | ||
| <img src="scivqa_icon.png" alt="Logo" style="height: 50px; margin-right: 10px;"> | ||
| SciHal: Hallucination Detection for Scientific Content | ||
| </h1> | ||
| </div> | ||
| </div> | ||
|
|
||
|
|
||
| <p> | ||
| Generative AI-enhanced academic research assistants are transforming how research is conducted. By allowing users to pose research-related questions in natural language, these systems can generate structured and concise summaries supported by relevant references. However, hallucinations — unsupported claims introduced by large language models — remain a significant obstacle to fully trusting these automatically generated scientific answers. | ||
| </p> | ||
|
|
||
| <p> | ||
| SciHal stands for "Hallucination Detection for Scientific Content". SciHal will invite participants to detect hallucinated claims in the answers to scientific questions generated by GenAI-powered research assistants. | ||
|
|
||
| </p> | ||
| <p> | ||
| The dataset comprises research-oriented questions sourced from subject matter experts, along with corresponding answers and references. These answers are produced by real-world retrieval-augmented generation (RAG) systems which indexed approximately millions of published academic abstracts. Each answer is annotated to indicate whether it includes unsupported claims that are not grounded in the provided references. Two levels of labeling will be provided: a three-class scheme (entailment, neutral, contradiction) and a fine-grained scheme encompassing 10+ categories. Teams are challenged to classify claims into the appropriate categories, with evaluation metrics focusing on the precision, recall, and F1 of detecting unsupported claims. | ||
| </p> | ||
|
|
||
| <h3>Important Dates</h3> | ||
|
|
||
| <ul> | ||
| <li>Release of training data: April 1, 2025 </li> | ||
| <li>Release of testing data: April 15, 2025</li> | ||
| <li>Deadline for system submissions: May 16, 2025 </li> | ||
| <li>Paper submission deadline: May 23, 2025 </li> | ||
| <li>Notification of acceptance: June 13, 2025 </li> | ||
| <li>Camera-ready paper due: June 20, 2025 </li> | ||
| <li>Workshop: July 31, 2025 or August 1, 2025 (TBA) </li> | ||
| </ul> | ||
|
|
||
| <h3>Organizers</h3> | ||
| <p> | ||
| <p><a href="mailto:[email protected]">Dan Li (Elsevier)</a></p> | ||
| <p><a href="mailto:[email protected]">Colin Kehan Zhang (Elsevier)</a></p> | ||
| <p><a href="mailto:[email protected]">Bogdan Palfi (Elsevier)</a></p> | ||
| <p><a href="mailto:[email protected]">Adrian Raudaschl (Elsevier)</a></p> | ||
| <p><a href="mailto:[email protected]">Anita de Waard (Elsevier)</a></p> | ||
|
|
||
| <!-- FOOTER ========================================== --> | ||
|
|
||
| <hr><br /> | ||
|
|
||
| <footer> | ||
| <div class="footer-wrapper"> | ||
| <div class="footer-left"> | ||
| <p>Contact: <a href="mailto:[email protected]">[email protected]</a></p> | ||
| <p>Sign up for updates: <a href="https://groups.google.com/g/sdproc-updates">https://groups.google.com/g/sdproc-updates</a></p> | ||
| <p>Follow us: <a href="https://twitter.com/SDPWorkshop">https://twitter.com/SDPWorkshop</a></p> | ||
| </div> | ||
| <div class="footer-right"> | ||
| <a href="#">Back to top</a> | ||
| </div> | ||
| </footer> | ||
|
|
||
| </div> | ||
|
|
||
| <!-- Bootstrap core JavaScript ================================================== --> | ||
| <script src="./dist/js/bootstrap.min.js"></script> | ||
|
|
||
| </body> | ||
|
|
||
| </html> |