Competition

Evaluation

Submissions are evaluated using corpus-level SacreBLEU-style BLEU (13a tokenization, exponential smoothing) and chrF (character order 6, beta 2). For a phase containing both language directions, each metric is calculated separately for en-vi and vi-en, then macro-averaged so both directions contribute equally. Both metrics range from 0 to 100, and higher scores are better.

Submission guidelines

1. Archive structure

Upload one ZIP archive containing a file named exactly results.csv at the root of the archive:

submission.zip
└── results.csv

Do not place results.csv inside another directory.

2. CSV format

The file must be UTF-8 encoded and contain exactly these columns:

• sample_id: the unchanged identifier supplied with the test source;
• direction: either en-vi or vi-en;
• translation: the complete translated document. The alias prediction is also accepted.

Example:

sample_id,direction,translation
Bat_dong_san/137,en-vi,"<complete Vietnamese translation>"
Bat_dong_san/137,vi-en,"<complete English translation>"

Translations may contain commas, quotation marks, and line breaks. Produce a standards-compliant CSV: quote such fields and escape an internal quotation mark by doubling it.

3. Required rows

• Public Test: 50 bilingual document pairs. Submit both directions for every sample_id, for exactly 100 prediction rows.
• Private Test: 100 documents, each exposing only one source side. Submit the corresponding direction shown in the private test input, for exactly 100 prediction rows.

Rows may appear in any order. Every required ID/direction combination must appear exactly once.

4. Translation requirements

• Do not modify sample_id or the provided direction.
• Translations must not be empty.
• Translate the complete document; do not omit headings, clauses, conditions, exceptions, or cross-references.
• Preserve Markdown structure where applicable, including headings, lists, tables, and emphasis.
• Use legal terminology accurately and consistently throughout each document.
• Do not include explanations, comments, prompts, or metadata in the translation field.

5. Validation and rejection

A submission is rejected if the ZIP does not contain results.csv, required columns are missing, a direction is invalid, a prediction is empty, an ID/direction is duplicated, or the submitted rows do not match the phase test set.

6. Compliance

All submissions must comply with the model, data, and inference requirements in the Terms and Conditions.

Terms and Conditions

By participating in the VLSP 2026 Legal Machine Translation shared task, a team agrees to the following conditions.

1. Model-size restriction

• The final translation model must contain fewer than 3 billion parameters.
• Teams are responsible for ensuring that every submitted system satisfies this limit.

2. Permitted models and training methods

• Pre-trained models may be used.
• Additional pre-training and fine-tuning are permitted.
• Data augmentation is permitted.
• Publicly available datasets may be used.

3. Inference restrictions

• The submitted system must run locally using the team's model.
• The system must not call external APIs or remote translation services during inference.
• All resources required for inference must be available locally in the submitted or approved execution environment.

4. Reproducibility and verification

• Teams must retain sufficient information to describe their model, parameter count, training data, training procedure, and inference configuration.
• Organizers may request supporting information or artifacts to verify compliance.
• A submission that cannot be verified or that violates these conditions may be rejected or removed from the leaderboard.

5. Data and submission integrity

• Participants must not attempt to obtain, reconstruct, or disclose hidden test references.
• Submissions must be generated by a system that complies with these terms.
• Participants must follow the submission format and phase-specific requirements described on the Evaluation page.