Mesolitica

Mesolitica is a Malaysian artificial intelligence company that develops open-source language models and datasets for Malaysian languages. It is best known for MaLLaM (Malaysia Large Language Model), a family of models pretrained from scratch on Malaysian text, and for related speech and multimodal tools covering Bahasa Melayu and more than sixteen regional languages and dialects.^[1][2][3]

Background

Mesolitica's model work grew out of the Malaysia-AI volunteer community, whose members spent months assembling Malaysian text corpora — Malay Wikipedia, government documents, local news, forums such as Lowyat and Cari, journals and academic literature. The explicit goal was hyperlocalisation: models that understand local slang, Manglish and cultural context that English-centric systems miss. The team trained its own tokenizer to handle Malay efficiently and reported that it encoded Malay text at roughly half the token count of comparable English-centric tokenizers.^[1][2]

The first MaLLaM models were released in December 2023 in three sizes — 1.1 billion, 3 billion and 5 billion parameters — pretrained on a 349 GB dataset equivalent to about 90 billion tokens, using eighty NVIDIA A100 GPUs for around ten days at a reported cost of about US$17,000 on spot compute. An instruction-tuned version followed, and a January 2024 arXiv paper by Husein Zolkepli, Aisyah Razak, Kamarul Adha and Ariff Nazhan documented the pretraining approach, evaluation and dataset composition. Subsequent releases extended the family with later MaLLaM versions, Malaysian text-to-speech models and the Malaysian Whisper audio system.^[1][2][4][6]

Key Concepts and Technology

Unlike many national language efforts that adapt English-first models, MaLLaM was pretrained from scratch so that its representations are not shadowed by English data. The training pipeline combined roughly 200 billion tokens of Malay-specific content across 197 datasets with filtered code data and synthetic instruction datasets, including translations of common instruction corpora and material covering Jawi script, grammar tasks and school-level examinations. Models are distributed openly on Hugging Face, along with datasets, under the Mesolitica and Malaysia-AI names.^[2][3][6]

The company has since broadened into multimodality — text, vision, audio and speech — and moved its workloads to Amazon Web Services, training with AWS Trainium and Inferentia chips. AWS and Mesolitica reported an 87 per cent reduction in compute costs, a 5.5-fold throughput increase and lower latency through the AWS Malaysia region.^[3]

Applications and Impact

MaLLaM-based assistants target customer service, content generation, data analysis and public services in local languages. Health-tech company Qmed Asia uses the models to better understand local speech patterns in medical note-taking, and the Malaysian government has explored integrating MaLLaM into its operations — efforts framed by AWS as supporting the country's goals of AI sovereignty and local data governance. Mesolitica is also part of a wider field of Southeast Asian language-model projects, alongside YTL's ILMU in Malaysia and AI Singapore's SEA-LION regionally.^[3][5][7]

>See Also

• MaLLaM (Malaysian Large Language Model)
• ILMU (Malaysian Large Language Model)
• Malaysia AI Startups
• SEA-LION
• Hugging Face
• Malaysia AI Talent