Resolving High-Vowel Ambiguity (⟨ï⟩ / ⟨i⟩ / ⟨ı⟩) in OCR-Derived Old Turkic Editions: An Edition-Metadata-Driven Disambiguation Layer

Abstract

This study addresses the problem of high-vowel ambiguity in OCR-derived Old Turkic texts, in which the graphemic distinction between ⟨ï⟩, ⟨i⟩, and ⟨ı⟩ is frequently neutralized to ⟨i⟩. This issue arises from the limitations of existing normalization approaches, which fail to adequately capture edition-specific orthographic conventions or the variability across the editorial traditions. As a result, a significant portion of the original graphemic information is lost during digitization, thereby reducing the reliability of subsequent linguistic and philological analysis. To resolve this problem, the paper proposes a dedicated disambiguation layer integrated into a TEI-P5 two-layer encoding framework (orig/reg). The proposed layer operates strictly at the representation level and does not attempt to reconstruct phonology or modify the original OCR output. Instead, it combines edition-specific metadata with rule-based linguistic cues, including lexical allow-lists, morphological constraints, vowel harmony patterns, and loanword profiles. The model follows a deterministic priority structure, ensuring that each ambiguous case is resolved in a transparent, consistent, and reproducible manner. By design, the framework avoids probabilistic inference and prioritizes philological accountability over statistical generalization. The model was evaluated on a dataset of 4,485 tokens drawn from thirteen Old Turkic editions published between 1919 and 2023. Among these, 1,837 tokens exhibited unresolved high-vowel ambiguity after normalization. The proposed method successfully disambiguates approximately 88–93% of these cases, depending on the edition, while preserving unresolved forms through explicit TEI <unclear> annotation. This approach ensures that ambiguity is not obscured but remains visible and accessible for further philological evaluation and interpretation. Comparative analysis across editions further confirmed the stability of the method under varying orthographic conventions. The results demonstrate that a deterministic, edition-aware approach can significantly improve accuracy compared to baseline methods, without sacrificing transparency, reversibility, or interpretability. The study further highlights the importance of editorial traditions in shaping graphemic representation and shows that ambiguity can be effectively managed through structured metadata and constrained rule-based systems. The proposed framework offers a reproducible and extensible solution for enhancing the quality, consistency, and interoperability of OCR-derived historical corpora in Turcology and digital philology. Future work will focus on extending the framework to additional Turkic editions and integrating it with corpus-level search and annotation tools.