Illustration: AI
The «Қазақ тілі» society and OpenAI have compiled a Kazakh-language text corpus of 14 billion tokens and collected 12,000 hours of audio recordings. The society’s head, Rauan Kenzhekhanuly, briefed President Kassym-Jomart Tokayev on the joint project. The data is needed to help AI better understand and speak Kazakh (tengrinews.kz).
What happened
Tokayev met with Kenzhekhanuly, who has led the society since 2023. They discussed teaching Kazakh, expanding content in the language and using it in new technologies.
The collected texts and audio recordings are one result of the collaboration with OpenAI. The text was measured in tokens: small chunks into which a model splits text when processing it.
What this means
A corpus is a collection of texts used to work with a language. For training models, these materials provide examples of how written language works. A token can be a word or part of one, so 14 billion tokens does not mean 14 billion words.
The audio recordings serve a different purpose: they provide examples of spoken Kazakh. Models need this data to understand speech and work on pronunciation. Text and audio help with the same overall goal: handling Kazakh better.
Where Tokayev proposed using AI
The president stressed the need to make full use of artificial intelligence in teaching the state language and translating scientific literature. These were the two areas he singled out at the meeting.
The «Қазақ тілі» society was founded on September 22, 1989. Its work now includes teaching methods and the use of Kazakh in new technologies.
