Language model
MangARTI
A parallel dataset for Manado Malay (Manadonese) that I built with native-speaker volunteers, and a Llama 3 8B model fine-tuned on it to translate to and from formal Indonesian. Trained on one laptop GPU.
In short
- What it does
- Translates single sentences between formal Indonesian and Manado Malay, in both directions.
- How well
- On 300 test sentences it had never seen, about one in four translations matched my reference word for word.
- What’s next
- A model about eight times smaller scored only slightly lower, so a version that runs on ordinary hardware looks realistic.
At a glance
- Task
- Sentence translation, formal Indonesian to Manado Malay and back
- Base model
Llama-3-8b-Instruct-bnb-4bit, Llama 3 8B Instruct quantised to 4-bit- Method
- QLoRA, with Unsloth and PEFT
- Dataset
- 3,454 sentence pairs, each written in both directions: 6,908 Alpaca-format examples in JSONL, published on Hugging Face (opens in a new tab)
- Hardware
- One consumer gaming laptop with an NVIDIA RTX 5050
- Status
- 8B model published; smaller models of around one billion parameters being tested
How it started
MangARTI started as my final high school project, and it started small. The first version was a Naive Bayes classifier that sorted text into two classes: Manadonese or formal Bahasa Indonesia. For a language with almost no tooling it was a sensible first model, cheap to train and easy to inspect.
A peer challenged it with a simple question: what would anyone do with the answer? Speakers of Manadonese already know which language they are reading, and a label does nothing for someone who understands only one side of the pair. I didn’t have a good answer, so I changed the project from labelling the dialect to translating it with a fine-tuned large language model.
Why a word list wasn’t enough
The obvious shortcut for a dialect this close to formal Indonesian is a lookup table: map each Manadonese word to its formal equivalent and substitute. That doesn’t work well here, for two reasons.
First, it is a near-zero resource language. There is no large parallel corpus to learn from, and very little Manadonese in the text that general-purpose models are trained on. Second, it has no standardised orthography. People write it the way they speak it, so a single word arrives in several spellings:
- Ngana
- Ngna
- ngn
A dictionary keyed on spelling fails on the first variant it has not seen. A language model can learn from context that these forms carry the same meaning, which is why the project needed one.
Making the data with native speakers
There was no parallel corpus to download, so most of the project was building one. Everything after it depends on this step: the model can only learn the Manadonese that is in the dataset. A general model drafted the source sentences, and people did the translating.
-
Draft the source sentences
Gemini 3.1 Pro generated everyday sentences in formal Indonesian, giving the dataset breadth without asking volunteers to invent material.
-
Translate with native speakers
I coordinated a group of native-speaker volunteers who translated the sentences into Manadonese by hand, so the target side reflects how the dialect is actually used.
-
Write every pair in both directions
Each sentence pair becomes two training examples, formal to Manadonese and Manadonese to formal, so one model serves both directions.
-
Format as Alpaca JSONL
Every example carries an instruction, an input and an output, stored one per line in JSONL. The same Alpaca template is used in training and at inference, so the model is always asked in the format it learned.
The finished dataset holds 3,454 sentence pairs, which become 6,908 training examples once each is written in both directions. It is published openly on Hugging Face as Bahasa-Manado-Alpaca-Translations (opens in a new tab).
Training on one laptop GPU
All training ran locally on a consumer gaming laptop with an NVIDIA RTX 5050. It doesn’t have enough VRAM for a full fine-tune of an 8B model, so I used QLoRA.
The base model is Llama 3 8B Instruct, loaded as a 4-bit bitsandbytes quantisation
(Llama-3-8b-Instruct-bnb-4bit). QLoRA leaves those quantised weights frozen
and trains small low-rank adapters on top of them, which brings the memory needed for
training within reach of a laptop card. Training runs through Unsloth and TRL’s
supervised fine-tuning trainer, with gradient checkpointing and an 8-bit optimiser to
stretch the memory budget further. The published training script uses this
configuration:
- Adapter
- LoRA rank 16, alpha 16, no dropout
- Target modules
- Every attention and MLP projection:
q,k,v,o,gate,up,down - Sequence length
- 512 tokens
- Batch
- Batch size 1 per device, 8 gradient accumulation steps (effective batch 8)
- Optimiser
- 8-bit AdamW, learning rate 1e-4, cosine schedule, 50 warm-up steps
- Epochs
- 2
Fixing the hallucinations
Early versions hallucinated badly: translations that read fluently but didn’t say what the source said. I went back through the dataset one example at a time and revised it by hand, and that is what fixed it. With 6,908 examples, each bad one is a noticeable share of what the model sees.
Trying it
The repository includes a small web app for testing the model. It loads the 4-bit model and its adapter once, builds the same Alpaca prompt used in training, and translates one sentence at a time in either direction.
Known limits
MangARTI is trained on single sentences, and it is documented that way: paragraphs and long documents fall outside its training data and degrade the output. Casual Manadonese typed in chat, with its shifting slang, abbreviations and non-standard spellings, remains the hardest input, so the model card recommends entering one sentence at a time in relatively standard spelling. It is not meant for medical, legal or other high-stakes translation, and translations it produces should be presented as AI-generated.
MangARTI mini
Even quantised to 4-bit, the 8B model needs a decent GPU to run. MangARTI mini is the same task on models of around one billion parameters, so it could run on hardware people already own. I trained three small runs on the same MangARTI dataset and compared them with the published 8B model:
- LoRA runs
- Llama 3.2 1B Instruct and Qwen3.5 0.8B, base weights kept in 16-bit. Rank 16, alpha 16, learning rate 2e-4, batch 16, 2 epochs, loss on the response only.
- Full fine-tune
- Qwen3.5 0.8B with every weight trained, in bf16. Learning rate 2e-5, batch 2 with 4 accumulation steps, paged 8-bit AdamW, 2 epochs.
- Reference
- The published MangARTI: Llama 3 8B Instruct, 4-bit QLoRA, settings listed above.
To compare them, I wrote a separate test set of 300 sentences by hand, kept apart from the training data: 150 in each direction, split into five groups by length and by what they test. Every run translated all 300, and I scored the outputs with four numbers:
- chrF
- Overlap in character sequences with my reference, 0 to 100. It gives partial credit when a word is spelled differently, which matters for Manadonese.
- BLEU
- Overlap in whole words and word sequences, 0 to 100. Stricter than chrF.
- Exact match
- Share of outputs identical to my reference.
- Loops
- Share of outputs where the model got stuck repeating itself.
| Model | chrF | BLEU | Exact match | Loops |
|---|---|---|---|---|
| Llama 3.2 1B, LoRA | 70.89 | 44.53 | 20.67% | 0.00% |
| Qwen3.5 0.8B, LoRA | 69.61 | 41.64 | 20.00% | 0.33% |
| Qwen3.5 0.8B, full fine-tune | 70.78 | 44.40 | 22.00% | 0.00% |
| MangARTI 8B (Llama 3 8B, QLoRA) | 73.21 | 48.16 | 24.33% | 0.00% |
The 8B model scores highest on every overall number, but the best small run, Llama 3.2 1B with LoRA, is only about 2.3 chrF points behind it while being around eight times smaller. On the Qwen model, the full fine-tune came out slightly ahead of LoRA. With 300 sentences, I read all of these gaps as small.
| Model | IDtoManado chrF | IDtoManado exact | ManadotoID chrF | ManadotoID exact |
|---|---|---|---|---|
| Llama 3.2 1B, LoRA | 71.44 | 15.33% | 70.37 | 26.00% |
| Qwen3.5 0.8B, LoRA | 67.43 | 12.67% | 71.62 | 27.33% |
| Qwen3.5 0.8B, full fine-tune | 70.21 | 16.00% | 71.28 | 28.00% |
| MangARTI 8B (Llama 3 8B, QLoRA) | 69.62 | 12.00% | 76.49 | 36.67% |
Most of the 8B model’s lead comes from Manadonese into Indonesian, where it gets 36.67% of sentences exactly right against 26 to 28% for the small runs. Going the other way, into Manadonese, the small runs keep up: the Llama 1B run has the best chrF and the Qwen full fine-tune the most exact matches, both a little above the 8B. That direction is probably harder to score as well: there is often more than one acceptable way to write a Manadonese sentence, and my test set has one reference for each.
| Model | Short | Medium | Long | Pronoun | Rare |
|---|---|---|---|---|---|
| Llama 3.2 1B, LoRA | 71.60 | 75.20 | 71.84 | 69.51 | 53.81 |
| Qwen3.5 0.8B, LoRA | 72.35 | 74.99 | 68.28 | 70.34 | 54.03 |
| Qwen3.5 0.8B, full fine-tune | 73.58 | 77.41 | 69.47 | 70.74 | 51.59 |
| MangARTI 8B (Llama 3 8B, QLoRA) | 78.15 | 78.08 | 72.37 | 71.24 | 58.34 |
Rare words are the weakest group for every run, and long sentences almost never come out word for word (0 to 2% exact match). Both point back to the dataset: most of its sentences are short, and rare words, by definition, show up in only a few training examples. Loops were rare; one output out of 300, from the Qwen3.5 0.8B LoRA run, repeated itself.
Two things to keep in mind. Manado Malay and Indonesian share a lot of vocabulary, so character-overlap scores start fairly high even for weak translations, which is why I also report exact match and still read the outputs myself. The comparison also isn’t fully controlled: the 8B model was trained earlier with different settings (4-bit, a lower learning rate and a smaller batch), so the gap isn’t only about size. Still, it suggests a model of around 1B parameters is a realistic target for this task.
