Skip to content
Makata

Research, with the evidence attached.

Technical reports documenting what we investigated, why we chose a method, what we measured, and what remains uncertain.

RESEARCH REPORT 001

Completed technical report

Completed

Independent Technical Report — Not Peer Reviewed

Evaluating open-weight models on Tagalog reading comprehension

We evaluate Qwen3-1.7B-Base and Sailor2-1B on the complete Tagalog Belebele reading-comprehension test through FilBench. The report documents likelihood-based scoring, frozen revisions, verified measurements, and the limits of this descriptive comparison before Filipino-language adaptation research.

Research area
Language-model evaluation
Methods
Zero-shot · Multiple-choice likelihood · Complete 900-question test
Checkpoints
Qwen/Qwen3-1.7B-Base; sail/Sailor2-1B
Dataset
facebook/belebele · tgl_Latn · test
Read Report 001

RESEARCH REPORT 002

Completed technical report

Completed

Independent Technical Report — Not Peer Reviewed

An exploratory adaptation, with an uncertain comprehension change

We adapt Qwen3-1.7B-Base on 363 screened Aya Filipino examples. On the same 900-question Tagalog Belebele task, accuracy changes from 53.67% to 55.33% (+1.67 percentage points). The paired 95% passage-bootstrap interval spans −1.45 to +4.80 points: improvement is not statistically established. Data review and rights limitations remain unresolved; weights are not released.

Research area
Exploratory model adaptation and evaluation
Methods
Aya-only QLoRA · One epoch · final checkpoint 91 · Paired passage-cluster bootstrap
Checkpoints
Qwen/Qwen3-1.7B-Base; Makata-v0.1 experimental adapter · checkpoint 91
Dataset
Aya Filipino · 363 selected examples; facebook/belebele · tgl_Latn · test
Read Report 002

How to read this archive

Planned research has a defined intention; in-progress work has not yet met its completion criteria. Completed technical reports document finished work. Peer review is a separate status requiring verifiable external review. Both reports are independent technical reports and have not undergone formal peer review.

The learning library explains established methods. This archive presents Makata's own experimental measurements. An internal QLoRA adapter has been trained and evaluated; a reliable comprehension gain is not established, and model weights have not been released.