Skip to content

Commit

Permalink
Merge 7a2d60b into a93b84a
Browse files Browse the repository at this point in the history
  • Loading branch information
stephenrparsons authored Feb 7, 2025
2 parents a93b84a + 7a2d60b commit 488fdc8
Show file tree
Hide file tree
Showing 25 changed files with 321 additions and 154 deletions.
2 changes: 1 addition & 1 deletion crackle-viewer/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ This project won multiple awards in the [Vesuvius Challenge 2023](https://scroll

## Overview

This tool is designed to assist researchers and enthusiasts in inspecting and labeling ink residue on virtually unrolled scrolls that were carbonized during the Vesuvius eruption nearly 2000 years ago. The primary goal is to generate a ground-truth ink dataset to make these virtually unrolled Vesuvius scrolls readable, as it's difficult to discern ink residue by the naked eye.
This tool is designed to assist researchers and enthusiasts in inspecting and labeling ink residue on virtually unwrapped scrolls that were carbonized during the Vesuvius eruption nearly 2000 years ago. The primary goal is to generate a ground-truth ink dataset to make these virtually unwrapped Vesuvius scrolls readable, as it's difficult to discern ink residue by the naked eye.

This project was developed as part of the **Vesuvius Challenge 2023** and aims to contribute towards efforts in deciphering ancient texts carbonized during the Vesuvius eruption nearly 2000 years ago.

Expand Down
383 changes: 272 additions & 111 deletions scroll-renderer/scroll_renderer/slim_uv.py

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion scrollprize.org/docs/02_data_fragments.md
Original file line number Diff line number Diff line change
Expand Up @@ -69,7 +69,7 @@ Originally held back for automated scoring in the [Kaggle](https://kaggle.com/co
<p>Volume [20230222173037](https://dl.ash2txt.org/fragments/Frag4/PHercParis1Fr39.volpkg/volumes/20230222173037/): 3.24µm, 88keV, 9209 x 24MB .tif files. Total size: 216 GB</p>

**Fragment 5 (PHerc. 1667 Cr 1 Fr 3)**
From the same original scroll as Scroll 4 (PHerc. 1667), which was partially unrolled in 1987 using the Oslo method. Find this fragment on [Chartes.it](https://www.chartes.it/index.php?r=document/view&id=1691).
From the same original scroll as Scroll 4 (PHerc. 1667), which was partially opened in 1987 using the Oslo method. Find this fragment on [Chartes.it](https://www.chartes.it/index.php?r=document/view&id=1691).
<div className="flex flex-wrap mb-4">
<div className="w-[45%] mb-2 mr-2 max-w-[300px]"><img src="/img/data/fr5-2.webp" className="w-[100%]"/><figcaption className="mt-[-6px]">Fragment 5 (PHerc. 1667 Cr 1 Fr 3)</figcaption></div>
</div>
Expand Down
8 changes: 4 additions & 4 deletions scrollprize.org/docs/05_tutorial2.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ import TabItem from '@theme/TabItem';

The micro-CT scan converts the scroll into a volumetric grid of density measurements, providing a mathematical description with integer coordinates and average material densities in small adjacent cubes.

<div className="flex w-[100%]">
<div className="flex max-w-[400px]">
<div className="w-[100%] mb-2 mr-2"><img src="/img/tutorials/rep_ash2vox.webp" className="w-[100%]"/><figcaption className="mt-0">Physical scroll to voxel representation</figcaption></div>
</div>

Expand All @@ -68,7 +68,7 @@ There have been varying levels of success using each of these types of represent

Traditional image processing techniques have mostly been intended to aid manual segmentation, either to make the papyrus easier to interpret or to help segmentation algorithms follow the surface better. Popular techniques include thresholding, noise reduction, various colourization schemes, edge detection, and skeletonization.

<div className="flex w-[100%]">
<div className="flex max-w-[400px]">
<div className="w-[100%] mb-2 mr-2"><img src="/img/tutorials/rep_trad_voxel.webp" className="w-[100%]"/><figcaption className="mt-0">Voxel to voxel representations using traditional techniques.</figcaption></div>
</div>

Expand All @@ -83,7 +83,7 @@ Most of these techniques have not been successful at improving segmentation.

The goal of point clouds is to extract a limited but useful subset of information from the raw scan volume. In the point cloud representation, calculations select voxels that describe the surface of a sheet.

<div className="flex w-[100%]">
<div className="flex max-w-[400px]">
<div className="w-[100%] mb-2 mr-2"><img src="/img/tutorials/rep_vox2pointcloud.webp" className="w-[100%]"/><figcaption className="mt-0">Transforming voxels into a point cloud representation.</figcaption></div>
</div>

Expand All @@ -98,7 +98,7 @@ The fixed grid and density values are discarded, using traditional edge/surface

Volumetric segmentation involves mapping the exact 3D shape of the papyrus sheets, and annotating which voxels are papyrus and which belong to the same papyrus wrap vs adjacent wraps. The semantic mask assists in this process by labelling papyrus vs air.

<div className="flex w-[100%]">
<div className="flex max-w-[400px]">
<div className="w-[100%] mb-2 mr-2"><img src="/img/tutorials/rep_semantic.webp" className="w-[100%]"/><figcaption className="mt-0">Binarizing papyrus and no papyrus</figcaption></div>
</div>

Expand Down
2 changes: 2 additions & 0 deletions scrollprize.org/docs/09_faq.md
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,8 @@ We have a [background page](background) for that!

### Where can I read the texts Vesuvius Challenge has uncovered?

Check out [these images](https://drive.google.com/drive/folders/1nnWR3ZMcUxJhp4SJYkV8-REwHsQT0fQb?usp=sharing) of some text we've so far revealed!

Here are the scholarly publications so far resulting from Vesuvius Challenge:

* [Revealing Text from a Still-rolled Herculaneum Papyrus Scroll (PHerc. Paris. 4)](https://www.iris.unina.it/handle/11588/959427) (2023 First Letters results)
Expand Down
2 changes: 1 addition & 1 deletion scrollprize.org/docs/12_grand_prize.md
Original file line number Diff line number Diff line change
Expand Up @@ -101,7 +101,7 @@ We will work with the Grand Prize winners to verify their results, put them in a

<div>If you have a qualifying result, submit it for consideration by sending an email to <a href="mailto:[email protected]">[email protected]</a> and provide the following:</div>

* **Images.** Submissions must be in the form of images of the virtually unrolled papyrus, showing visible and legible text.
* **Images.** Submissions must be in the form of images of the virtually unwrapped papyrus, showing visible and legible text.
* Submit a single static image for each text region. Images must be generated programmatically, as direct outputs of CT data inputs, and should not contain manual annotations of characters or text.
* Specify which scroll each image came from.
* Specify where in the scroll they were found: include information about the position of the text vertically as well as radially within the scroll. One easy way to do this is to provide images showing the 3D position of the text surface inside the scroll.
Expand Down
8 changes: 4 additions & 4 deletions scrollprize.org/docs/26_grandprize.md
Original file line number Diff line number Diff line change
Expand Up @@ -155,7 +155,7 @@ These teams each brought to the table new approaches to the subtleties of ink la

## What does the scroll say?

To date, our efforts have managed to unroll and read about 5% of the first scroll. Our eminent team of papyrologists has been hard at work and has achieved a preliminary transcription of all the revealed columns. We now know that this scroll is not a duplicate of an existing work; it contains never-before-seen text from antiquity. The papyrology team are preparing to deliver a comprehensive study as soon as they can. You all gave them a lot of work to do! Initial readings already provide glimpses into this philosophical text. From our scholars:
To date, our efforts have managed to unwrap and read about 5% of the first scroll. Our eminent team of papyrologists has been hard at work and has achieved a preliminary transcription of all the revealed columns. We now know that this scroll is not a duplicate of an existing work; it contains never-before-seen text from antiquity. The papyrology team are preparing to deliver a comprehensive study as soon as they can. You all gave them a lot of work to do! Initial readings already provide glimpses into this philosophical text. From our scholars:

<div className="ml-8 mb-4">The general subject of the text is pleasure, which, properly understood, is the highest good in Epicurean philosophy. In these two snippets from two consecutive columns of the scroll, the author is concerned with whether and how the availability of goods, such as food, can affect the pleasure which they provide.</div>

Expand Down Expand Up @@ -301,12 +301,12 @@ Machine learning models are infamous for “hallucinating”: making up text or
<figcaption className="mt-[-6px]">The models use small input/output windows. In some cases, the output is even only binary (“ink” vs “no ink”), as shown in this animation. This makes it extremely unlikely for the model to hallucinate shapes that look like letters.</figcaption>
</figure>

## How does the unrolling work?
## How does the unwrapping work?

<div>Roughly, virtual unwrapping works in <a href="tutorial">three steps:</a></div>

1. **Scanning:** creating a 3D scan of a scroll or fragment using X-ray tomography.
2. **Segmentation:** tracing the crumpled layers of the rolled papyrus in the 3D scan and then unrolling, or flattening, them.
2. **Segmentation:** tracing the crumpled layers of the rolled papyrus in the 3D scan and then unwrapping, or flattening, them.
3. **Ink Detection:** identifying the inked regions in the flattened segments using a machine learning model.

These scrolls were scanned at Diamond Light Source, a particle accelerator near Oxford, England. The facility produces a parallel beam of X-rays at high flux, allowing for fast, accurate, and high-resolution imaging. The X-ray photos are turned into a 3D volume of voxels using tomographic reconstruction algorithms, resulting in a stack of slice images.
Expand Down Expand Up @@ -408,7 +408,7 @@ With Vesuvius Challenge, we hope not only to solve the problem of reading the He

When we started the competition, most of us estimated that we had a less than 30% probability of success within the year. At that point, no letters had yet been discovered inside of a scroll. On top of that, the scrolls had barely been segmented at all. We had doubt as to whether the project could succeed, especially by the deadline we set. Was the ink signal present? Were the scans high-resolution enough? Could the techniques used to identify ink in the fragments be transferred into a scroll? None of this was known at the time. But we knew it was worth trying!

In Stage 1 of Vesuvius Challenge, we answered all of these questions, extracting 15 columns of never-before-seen text from inside a lump of carbon. We now have proven techniques for virtually unrolling the papyrus scroll and recognizing ink using machine learning.
In Stage 1 of Vesuvius Challenge, we answered all of these questions, extracting 15 columns of never-before-seen text from inside a lump of carbon. We now have proven techniques for virtually unwrapping the papyrus scroll and recognizing ink using machine learning.

### Vesuvius Challenge Stage 2

Expand Down
4 changes: 2 additions & 2 deletions scrollprize.org/docs/27_master_plan.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,11 +38,11 @@ hide_table_of_contents: true

## Vesuvius Challenge – Stage Two

* In 2023 Vesuvius Challenge [made a breakthrough](grandprize), extracting more than four passages of never-before-seen text from inside an unopened (and unopenable) carbonized scroll. We have proven techniques for virtually unrolling the papyrus scroll and recognizing the ink using machine learning. It wasn’t clear it was possible until we did it. This was stage one.
* In 2023 Vesuvius Challenge [made a breakthrough](grandprize), extracting more than four passages of never-before-seen text from inside an unopened (and unopenable) carbonized scroll. We have proven techniques for virtually unwrapping the papyrus scroll and recognizing the ink using machine learning. It wasn’t clear it was possible until we did it. This was stage one.
* Our next step is to scale these techniques up so that we can read entire scrolls, and to figure out an efficient scanning protocol to allow us to scan and read the 300 extant scrolls, mostly in Naples. Two key technical problems need to be solved: segmentation at scale, and scanning at scale.
* Segmentation at scale
* The current bottleneck is tracing the papyrus surface inside the scan of the scroll (we call this “segmentation”). Currently we use manual tracing aided by various algorithms. This is quite expensive – about \$100 per square centimeter. We spent about \$200,000 so far to trace enough material to read the fifteen partial columns of text that were revealed in 2023.
* Full scrolls are 10cm-20cm wide and up to 15 meters long. With current techniques it could cost \$1-5 million to unroll an entire scroll. Given that there are 300 scrolls that need to be read, it could cost hundreds of millions or even more to unroll all of them. Clearly impractical. Also, there are parts of the scrolls that are so compressed, current techniques cannot unroll them at all.
* Full scrolls are 10cm-20cm wide and up to 15 meters long. With current techniques it could cost \$1-5 million to unwrap an entire scroll. Given that there are 300 scrolls that need to be read, it could cost hundreds of millions or even more to unwrap all of them. Clearly impractical. Also, there are parts of the scrolls that are so compressed, current techniques cannot unwrap them at all.
* A breakthrough is needed in segmentation. So in stage two, we are going to focus on solving auto-segmentation. We believe it’s possible to bring the cost of segmenting an entire scroll to \$5000 or below. It might even be possible to fully automate it.
* Our approach to solving segmentation at scale will be to continue to leverage the community through a series of open source “progress prizes” we will award throughout the year, while hiring in full-time or part-time roles the most productive contributors to do the needed deep work. We will set as our target for 2024 to read 90% of four scrolls, and offer a \$200,000 grand prize to the first team to achieve this milestone.
* Scanning at scale
Expand Down
6 changes: 3 additions & 3 deletions scrollprize.org/docs/28_2024_prizes.md
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,7 @@ In 2023, we got from 0% to 5% of one scroll. Reading 90% of four scrolls will la
* Compute the total surface area (in cm<sup>2</sup>) of papyrus sheets present in all four complete scrolls combined.
* We may compute and specify this value ourselves as the year progresses, in which case you can skip this step.
* Compute the same measure for the papyrus sheets actually segmented in your submission. **You must segment 90% or more of the total from all scrolls** (not per-scroll).
* Segments should be flattened and shown in 2D as if the scroll were unrolled. Each scroll is ideally captured by a single segmentation (or each connected component of the scroll) rather than separate overlapping segmentations.
* Segments should be flattened and shown in 2D as if the scroll were unwrapped. Each scroll is ideally captured by a single segmentation (or each connected component of the scroll) rather than separate overlapping segmentations.
* Segments should pass geometric sanity checks; for example, no self-intersections. We will evaluate [stretch metrics](https://arxiv.org/abs/2007.15551) to measure distortion.
2. **Ink detection**
* Generate ink predictions on the segments.
Expand Down Expand Up @@ -120,7 +120,7 @@ We will judge the segmentation by the following criteria:
* **Ink detection.** Comparable or better ink detection as determined by our Papyrological Team. You can use the open source 2023 Grand Prize winning solution for this.

A submission requires:
* **Ink detection image.** Submissions must include an image of the virtually unrolled segment, showing visible and legible text.
* **Ink detection image.** Submissions must include an image of the virtually unwrapped segment, showing visible and legible text.
* Submit a single static image showing the text region. Images must be generated programmatically, as direct outputs of CT data inputs, and should not contain manual annotations of characters or text. Using annotations as training data is OK if they are not memorized by the model, for example if you use k-fold validation.
* Include a scale bar on the submission image.
* You may use open source methods (for example the 2023 Grand Prize winning submission) for ink detection.
Expand Down Expand Up @@ -164,7 +164,7 @@ The purpose of these prizes is to close the gap between our current state of the
<details>
<summary>Submission criteria and requirements</summary>

* **Image.** Submissions must be an image of the virtually unrolled segment, showing visible and legible text.
* **Image.** Submissions must be an image of the virtually unwrapped segment, showing visible and legible text.
* Submit a single static image showing the text region. Images must be generated programmatically, as direct outputs of CT data inputs, and should not contain manual annotations of characters or text.
* For the First Title Prize, please illustrate the ink predictions in spatial context of the title search, similar to what is [shown here](https://scrollprize.substack.com/p/30k-first-title-prize). You **do not** have to read the title yourself, but just have to produce an image of it that our team of papyrologists are able to read.
* Specify which scroll the image comes from. For multiple scrolls, please make multiple submissions.
Expand Down
2 changes: 1 addition & 1 deletion scrollprize.org/docs/30_2024_gp_submissions.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ slug: "2024_gp_submissions"

<div>If you have a qualifying result, submit it for consideration by sending an email to <a href="mailto:[email protected]">[email protected]</a> and provide the following:</div>

* **Images.** Submissions must be in the form of images of the virtually unrolled papyrus, showing visible and legible text.
* **Images.** Submissions must be in the form of images of the virtually unwrapped papyrus, showing visible and legible text.
* Submit a single static image for each scroll. If necessary, the image can be tiled to reasonable dimensions. Images must be generated programmatically, as direct outputs of CT data inputs, and should not contain manual annotations of characters or text.
* Specify which scroll each image came from.
* Specify where in the scroll they were found: include information about the position of the text vertically as well as radially within the scroll. One easy way to do this is to provide images showing the 3D position of the text surface inside the scroll.
Expand Down
4 changes: 2 additions & 2 deletions scrollprize.org/docs/31_jobs.md
Original file line number Diff line number Diff line change
Expand Up @@ -120,7 +120,7 @@ Send your resume, cover letter, and examples of relevant projects to jobs@scroll

## Computer Vision and Geometry Applied Researcher

The Vesuvius Challenge applied researchers advance the frontier of the most promising research directions from our community, to keep making continuous progress on the most promising research avenues in virtual unrolling and related problems.
The Vesuvius Challenge applied researchers advance the frontier of the most promising research directions from our community, to keep making continuous progress on the most promising research avenues in virtual unwrapping and related problems.

#### Overview
If the Vesuvius Challenge community performs a breadth-first search of research ideas to help us read the scrolls, this role performs depth-first search. We (and you!) are laser focused on solving the remaining hurdles to extract these hidden texts from their scroll confines.
Expand Down Expand Up @@ -184,7 +184,7 @@ This is a full-time position (30+ hours/week). Pay rate up to 40USD per hour.
- C/C++, Python

#### Impact
Not only will you unroll ancient scrolls and be the first human to lay eyes on the resulting text in two millennia, your efforts will directly contribute to automation of unrolling, which is a key ingredient to help catalyze further excavation of Herculaneum and unearth the only remaining library from antiquity before it is too late!
Not only will you unwrap ancient scrolls and be the first human to lay eyes on the resulting text in two millennia, your efforts will directly contribute to automation of unwrapping, which is a key ingredient to help catalyze further excavation of Herculaneum and unearth the only remaining library from antiquity before it is too late!

#### To Apply
Send your resume, cover letter, and examples of relevant projects to [email protected]. Include "Annotation Specialist" in the subject line.
Expand Down
Loading

0 comments on commit 488fdc8

Please sign in to comment.