VLSP 2026 Legal Machine Translation
Machine Translation (MT) in the legal domain is challenging because legal documents require high accuracy and consistent use of specialized terminology. Legal texts often contain complex sentences, domain-specific terms, and expressions whose meaning depends strongly on the legal context. Inconsistent translation of the same legal term can lead to ambiguity or changes in meaning. These challenges become more difficult when using relatively small models with limited computational resources.
Challenges
- Legal terminology: Legal terms and expressions may have meanings that differ from their general-language usage and must be translated in the correct legal context.
- Terminology consistency: The same legal term or concept should be translated consistently throughout a document.
- Accuracy: Translations must preserve rights, obligations, conditions, exceptions, and all other information that affects legal meaning.
- Complex legal language: Systems must handle long sentences, clauses, cross-references, and formal expressions.
- Limited resources: Participants must balance translation quality with inference speed and memory usage.
Task setting
The domain is legal, and the supported languages are English and Vietnamese. Systems are evaluated in both directions:
- English to Vietnamese (
en-vi)
- Vietnamese to English (
vi-en)
All teams receive the same legal-domain training, development, and test data. The restricted-resource setting encourages effective domain adaptation, terminology-aware methods, and efficient training rather than reliance on larger models alone. See the Terms and Conditions for model, data, and inference requirements.
Timeline
- Public Test opens: September 25, 2026
- Private Test opens: October 9, 2026
- Competition ends: October 15, 2026
Evaluation
Submissions are evaluated using corpus-level SacreBLEU-style BLEU (13a tokenization, exponential smoothing) and chrF (character order 6, beta 2). For a phase containing both language directions, each metric is calculated separately for en-vi and vi-en, then macro-averaged so both directions contribute equally. Both metrics range from 0 to 100, and higher scores are better.
Submission guidelines
1. Archive structure
Upload one ZIP archive containing a file named exactly results.csv at the root of the archive:
submission.zip
โโโ results.csv
Do not place results.csv inside another directory.
2. CSV format
The file must be UTF-8 encoded and contain exactly these columns:
sample_id: the unchanged identifier supplied with the test source;
direction: either en-vi or vi-en;
translation: the complete translated document. The alias prediction is also accepted.
Example:
sample_id,direction,translation
Bat_dong_san/137,en-vi,"<complete Vietnamese translation>"
Bat_dong_san/137,vi-en,"<complete English translation>"
Translations may contain commas, quotation marks, and line breaks. Produce a standards-compliant CSV: quote such fields and escape an internal quotation mark by doubling it.
3. Required rows
- Public Test: 50 bilingual document pairs. Submit both directions for every
sample_id, for exactly 100 prediction rows.
- Private Test: 100 documents, each exposing only one source side. Submit the corresponding direction shown in the private test input, for exactly 100 prediction rows.
Rows may appear in any order. Every required ID/direction combination must appear exactly once.
4. Translation requirements
- Do not modify
sample_id or the provided direction.
- Translations must not be empty.
- Translate the complete document; do not omit headings, clauses, conditions, exceptions, or cross-references.
- Preserve Markdown structure where applicable, including headings, lists, tables, and emphasis.
- Use legal terminology accurately and consistently throughout each document.
- Do not include explanations, comments, prompts, or metadata in the translation field.
5. Validation and rejection
A submission is rejected if the ZIP does not contain results.csv, required columns are missing, a direction is invalid, a prediction is empty, an ID/direction is duplicated, or the submitted rows do not match the phase test set.
6. Compliance
All submissions must comply with the model, data, and inference requirements in the Terms and Conditions.
Terms and Conditions
By participating in the VLSP 2026 Legal Machine Translation shared task, a team agrees to the following conditions.
1. Model-size restriction
- The final translation model must contain fewer than 3 billion parameters.
- Teams are responsible for ensuring that every submitted system satisfies this limit.
2. Permitted models and training methods
- Pre-trained models may be used.
- Additional pre-training and fine-tuning are permitted.
- Data augmentation is permitted.
- Publicly available datasets may be used.
3. Inference restrictions
- The submitted system must run locally using the team's model.
- The system must not call external APIs or remote translation services during inference.
- All resources required for inference must be available locally in the submitted or approved execution environment.
4. Reproducibility and verification
- Teams must retain sufficient information to describe their model, parameter count, training data, training procedure, and inference configuration.
- Organizers may request supporting information or artifacts to verify compliance.
- A submission that cannot be verified or that violates these conditions may be rejected or removed from the leaderboard.
5. Data and submission integrity
- Participants must not attempt to obtain, reconstruct, or disclose hidden test references.
- Submissions must be generated by a system that complies with these terms.
- Participants must follow the submission format and phase-specific requirements described on the Evaluation page.