ACM CHI 2026: a user study of our hieroglyph translation pipeline
Imagine pointing your phone camera at a stela in a museum and instantly seeing a translation of an inscription more than three thousand years old.
We have written before about our ecosystem of tools for recognising and translating Ancient Egyptian hieroglyphs: the MEH and MuMMy datasets, the pyThoth tool and the complete translation pipeline. Since then the project has taken an important step forward: from laboratory metrics to testing with real users.
At ACM CHI 2026, one of the world's leading conferences on human-computer interaction, the team presented a prototype web application implementing our pipeline together with the results of the first user study. This time the team included not only AIRI, ISP RAS and ITMO, but also the University of Cambridge, contributing Egyptological expertise and practical requirements for the system.
From a photograph of an inscription the application successively produces Gardiner codes (the standard for sign classification), a scholarly transliteration and an English translation. Instead of one large model we built a pipeline of smaller ones, each with its own narrow task: segmenting rows and columns, isolating individual signs, matching them against the catalogue, and reconstructing the reading order. At every step a specialist can check and correct the result.
Eight professional Egyptologists and students performed identical tasks with and without the system. With the system the process took 48 minutes on average against 60, and the quality of recognition, transliteration and translation was higher than in fully manual work.
Egyptology is an extremely narrow community; there are very few actively working specialists in the world, and gathering even a handful of them for a study lasting several hours is already a difficult task. For a discipline of this size, eight participants represent a noticeable share of the available professional circle. We consider the effect itself significant: the speed-up and the reduction in manual searching through reference works appeared not at the level of statistical error but quite tangibly, repeating from participant to participant.
The approach and the methodology for generating synthetic data have already been adapted to other ancient scripts, with papers accepted at international conferences. The accumulated experience in data synthesis has also found application in an entirely different area — the recognition of Russian handwritten text. But that is another story.
The full interview about our work for Kommersant is available on their website.