Competition

VLSP 2025 challenge on Semantic Parsing

Organized by toanlnhus - Current server time: Aug. 30, 2025, 9:13 p.m. UTC

Previous

Public Test
July 10, 2025, midnight UTC

Current

Private Test
Aug. 20, 2025, midnight UTC

End

Competition Ends
Aug. 31, 2025, midnight UTC

VLSP 2025 - viAMR Challenge - Vietnamese Semantic Parsing

Shared Task Registration Form

Important dates 
  • June 23, 2025: Registration open
  • July 6, 2025: Training data release
  • July 15, 2025: Public test release
  • August 20, 2025: System submission deadline
  • August 30, 2025: Private test results release
  • September 10, 2025: Technical report submission
  • September 27, 2025: Notification of acceptance
  • October 3, 2025: Camera-ready deadline
  • October 29-30, 2025: Conference dates
Task Description

The objective of building a semantic parser for Vietnamese is to enable accurate understanding and formal representation of Vietnamese sentences by analyzing their syntactic and semantic structures. This parser aims to extract the underlying meaning of text and convert it into structured forms, such as Abstract Meaning Representation (AMR) or logical expressions. Such a tool is essential for enhancing the performance of downstream natural language processing tasks, including machine translation, information extraction, and question answering, especially in the context of Vietnamese, which currently lacks large-scale annotated datasets and semantic resources.

Evaluation

This shared task uses Smatch to evaluate the semantic parsing system. Smatch score (defined below) of two semantic graphs in terms of their matching triples (edges) by finding a variable (node) mapping that maximizes the count, M, of matching triples, then:

  • M is the number of matching triples
  • T is the total number of triples in the first semantic graph
  • G is the total number of triples in the second semantic graph
  • Precision is defined as P = M/T
  • Recall is defined as R = M/G
  • The Smatch score is the F-score: F = 2 * (P*R)/(P+R)

For example: 

a0 / watch                             (b0 / watch
         :ARG0 (a1 / boy)                       :ARG0 (b1 / girl)
         :ARG1 (a2 / tv))                         :ARG1 (b2 / boy))

In where: a0(watch)-b0(watch), a1(boy)-b1(girl)

instance (a0, watch) ∧ instance (a1, boy) ∧ instance (a2, tv) ∧ ARG0 (a0, a1) ∧ ARG1 (a0, a2)
instance(b0, watch) ∧ instance (b1, girl) ∧ instance (b2, boy) ∧ ARG0 (b0, b1) ∧ ARG1 (b0, b2)

So:  P = 3/5 , R = 3/5 , F = 0.6

Public Test

Start: July 10, 2025, midnight

Private Test

Start: Aug. 20, 2025, midnight

Competition Ends

Aug. 31, 2025, midnight

You must be logged in to participate in competitions.

Sign In