Ancient languages, AI in Archeology and Digital Humanities

We adapt state-of-the-art Visual Language Models (VLMs) to recognize, analyze, and translate Ancient Egyptian texts — hieroglyphic, hieratic, and demotic. The goal is to build a research assistant for Egyptologists: a system that, given a photo of an inscription or a papyrus scan, recovers the signs, transliteration, and translation — rather than just helping look up a sign in a reference catalog.

Tests on current large language models (GPT-4o, o3, DeepSeek R1, Claude) show that even strong models confidently get hieroglyphs and translations wrong — they lack the specialized fine-tuning this domain needs. The core bottleneck is a severe shortage of annotated data to train such models on: over the past two years the lab has built and released several datasets and tools specifically to close that gap.

Tasks:

1. Data annotation for Egyptian OCR

2. Data annotation for Egyptian assistant SFT

3. Data annotation for Egyptian translation (German-speaking only)

4. Human evaluation and RLHF fine-tuning Find collaborations for real-life applications (e.g., museums).

Other research areas:

• Automatic attribution of ceramics

• Analysis of satellite images to identify potential excavation sites

• Reconstruction of frescoes from fragments

• AI-enhanced search for analogies in historical subjects

• Unite OCR-translation-analysis pipeline for a variety of ancient languages (e.g. sumerian and persian) ​​and scripts

The pipeline consists of two connected modules.

• Recognition module (OCR): locates lines/regions containing hieroglyphs in an image, identifies each sign and its orientation, assigns a Gardiner code, and reconstructs reading order.

• Machine translation module: turns the sequence of signs (or its transliteration) into coherent English or German text — not phrase by phrase, but with awareness of surrounding sentences.

To train these modules despite the shortage of real data, we rely on two additional sources: synthetic data generated with diffusion models (ControlNet plus IP-Adapter/LoRA on top of font-rendered hieroglyphs), and back-translation to expand parallel text data. All data and model outputs are checked by domain experts — Egyptologists who annotate the source material and evaluate output quality at every stage.

Datasets

All datasets are built on the Thesaurus Linguae Aegyptiae (TLA) — the largest expert-curated digital corpus of Ancient Egyptian texts — and were annotated by Egyptology students at HSE University, with final review by a PhD-level Egyptologist.

• MEH — an OCR dataset: 40 source images from 12 sources, 512 clauses, 12,744 annotated hieroglyph instances, 639 unique Gardiner codes. The material spans three writing styles — printed, handwritten, and "physical" (photographs of monuments) — ranging from the cleanest to the hardest to recognize.

  

• MMM — a dataset for classifying individual signs: 16,015 images, 513 classes, 10 style groups (from printed fonts to photographs of damaged inscriptions). It showed that models trained on one style generalize poorly to another — and helped train them to do better.

  

• MuMMy — the first multimodal dataset linking a hieroglyph image to its Gardiner code, transliteration, and translation at two levels of granularity — clause and paragraph: 87 images, 984 clause-level segments and 106 paragraph-level segments, 26,486 annotated hieroglyph instances.

  Synthetic data

  Since real annotated inscriptions are always in short supply, we trained a diffusion model to render a font-based hieroglyph in a target style (worn stone, handwritten papyrus, etc.) using ControlNet and adapters (IP-Adapter, LoRA). This lets us multiply rare signs and styles without manually annotating new photographs.

  

Adding synthetic data to the training set measurably improves quality: for one of the harder handwriting groups, recognition F1 rose from 70.4 to 86.8.

Tools for Egyptologists

• pyThoth — an annotation tool with a hieroglyph palette and a Gardiner-code renderer: it speeds up manual annotation by 2.8–5.8x compared to standard tools like Label Studio.

• Human-in-the-loop platform — a single workspace where recognition, transliteration, and translation are produced by the model, but each step can be checked and corrected on the spot without switching between programs. In a user study with Egyptologists, the system nearly halved the time spent on a text (from 9 to 5 minutes to translate a fragment) while improving output quality: recognition accuracy improved (CER dropped from 10.6% to 7.2%), and transliteration and translation quality both improved (chrF++ rose from 45.9 to 61.8, and from 41.3 to 51.8, respectively).

  

• akhet — an open-source pipeline (available on GitHub) for recognizing and translating long, connected texts, designed for augmented-reality applications.

Applications: from a museum label to AR

Beyond the AR scenario, we're testing the system as an assistant for museum and educational work — from captioning exhibits to holding a dialogue with a visitor about what an inscription says.

Results

• Individual hieroglyph classification: F1 up to 0.90 (ConvNeXt-B, pretrained on Laion2B) even when transferring across different writing styles.

• Full-text recognition (detection + classification): mAP up to 0.75 on sign detection, F1 up to 0.91 on classification, word segmentation F-score of 0.81.

• Translation of connected, multi-sentence texts (rather than isolated phrases): chrF++ up to 64; fine-tuning on multi-clause examples yields up to a 20-point BLEU improvement over training on isolated phrases alone.

• The overall accuracy of the full pipeline under real-world conditions (from photo to translated text) still falls short of reliable OCR — this remains an open research problem the lab continues to work on.

Other directions in AI for archaeology

• Automatic attribution of ceramics.

• Analysis of satellite imagery to identify potential excavation sites and trade routes.

• Reconstruction of frescoes from fragments.

• AI-assisted search for historical analogies, and analysis of myths and narratives.

• Unifying OCR → translation → analysis into a single pipeline for other ancient languages and scripts (e.g., Sumerian and Persian).

Published research already points to the potential of AI in each of these areas. The bottleneck everywhere is the same — a lack of specialized training data. That's why we lean heavily on the expertise of professional Egyptologists and historians — to define the task, annotate the data, and evaluate model output at every stage.