VLSP 2025 Speech Quality Assessment Challenge

Organized by vlsp_asr_organizer - Current server time: Aug. 30, 2025, 9:11 p.m. UTC

First phase

Public Test
July 1, 2025, midnight UTC

End

Competition Ends
Aug. 14, 2025, 5 a.m. UTC

Our website

https://vlsp.org.vn/vlsp2025/eval/sqa

Important dates

  • June 23, 2025: Registration open
  • July 1, 2025: Training data, public test release
  • August 14, 2025: Private test release
  • August 14, 2025: System submission deadline
  • August 14, 2025: Private test results release
  • August 30, 2025: Technical report submission
  • September 27, 2025: Notification of acceptance
  • October 3, 2025: Camera-ready deadline
  • October 29-30, 2025: Conference dates

General Description

With the advancement of information and communication technology, connecting with others via the Internet and telecommunication systems has become effortless. However, speech transmitted over these networks often degrades, diminishing its original quality and potentially leading to annoyance or misunderstandings. Consequently, Speech Quality Assessment (SQA) is crucial for evaluating the performance of communication systems, drawing interest from telephone companies and Internet service providers. In this task, participants will work with a Vietnamese dataset, where each degraded speech sample is assigned a quality score from 1 to 5. The objective is to develop a model that can predict the channel quality scores for given speech samples.

This year, in addition to the data provided by the organizers, teams can utilize external resources like pretrained models and open datasets. Before the competition starts, teams are encouraged to propose external resources. The organizers will review these suggestions and select resources based on criteria such as accuracy, popularity, and size to ensure fairness. During the competition, teams are only permitted to use the resources approved by the organizers.

Evaluation Metrics

In this task, two popular metrics are Pearson Correlation Coefficient (PCC) and Mean Square Error (MSE):

  • PCC: is a correlation coefficient that measures the linear correlation between two sets of data.
  • MSE: measures the average of the squares of the errors.

The higher PCC and the lower MSE indicate the better model. Therefore, the overall evaluation metric is calculated as (higher is better):

Final_Score = 0.7 * PCC - 0.3 * MSE

 

 

- no title specified

General rules

  • Right to cancel, modify, or disqualify. The Competition Organizer reserves the right at its sole discretion to terminate, modify, or suspend the competition.

  • By submitting results to this competition, you consent to the public release of your scores at the Competition workshop and in the associated proceedings, at the task organizers' discretion. Scores may include but are not limited to, automatic and manual quantitative judgments, qualitative judgments, and such other metrics as the task organizers see fit. You accept that the ultimate decision of metric choice and score value is that of the task organizers.

  • By joining the competition, you accepted to the terms and conditions of Agreement form of VLSP 2025 - Speech Quality Assessment, which has been sent to your email. It is noted that your participant rights will be revoked if you do not sign and send back to us before the deadline.

  • By joining the competition, you affirm and acknowledge that you agree to comply with applicable laws and regulations, and you may not infringe upon any copyrights, intellectual property, or patent of another party for the software you develop in the course of the competition, and will not breach of any applicable laws and regulations related to export control and data privacy and protection.

  • Prizes are subject to the Competition Organizer’s review and verification of the entrant’s eligibility and compliance with these rules as well as the compliance of the winning submissions with the submission requirements.

Eligibility

  • Each participant must create a AIHub account to submit their solution for the competition. Only one account per user is allowed.

  • The competition is public, but the Competition Organizer may elect to disallow participation according to its own considerations.

  • The Competition Organizer reserves the right to disqualify any entrant from the competition if, in the Competition Organizer’s sole discretion, it reasonably believes that the entrant has attempted to undermine the legitimate operation of the competition through cheating, deception, or other unfair playing practices.

Team

  • Participants are allowed to form teams. The maximum of the number of participants on the team is up to 5.

  • You may not participate in more than one team. Each team member must be a single individual operating a separate CodaLab account.

Submission

  • Maximum number of submissions in each phase:

    • Phase 1 - Public Test: 1 submissions / day / team

    • Phase 2 - Private Test: total 2 submissions / team.

  • Submissions are void if they are in whole or part illegible, incomplete, damaged, altered, counterfeit, obtained through fraudulent means, or late. The Competition Organizer reserves the right, in its sole discretion, to disqualify any entrant who makes a submission that does not adhere to all requirements.

Data

By downloading or by accessing the data provided by the Competition Organizer in any manner you agree to the following terms:

  • You will not distribute the data except for the purpose of non-commercial and academic-research.

  • You will not distribute, copy, reproduce, disclose, assign, sublicense, embed, host, transfer, sell, trade, or resell any portion of the data provided by the Competition Organizer to any third party for any purpose. 

  • The data must not be used for providing surveillance, analyses or research that isolates a group of individuals or any single individual for any unlawful or discriminatory purpose.

  • You accept full responsibility for your use of the data and shall defend and indemnify the Competition Organizer, against any and all claims arising from your use of the data.

Submission Guidelines

Submission Format

The result to upload to AI Hub is a zip file containing a single file "results.tsv" that conforms to the specifications below.

Submissions have to be made in UTF-8, lower-case, and one line for each utterance/audio.

utterance_name<TAB>SQA_score

Note: utterance_name does not contain the file extension (e.g., .wav).

For example:

0001       4.301
0002       2.975
0003       3.002

Public Test

Start: July 1, 2025, midnight

Private Test

Start: Aug. 14, 2025, 2 a.m.

Competition Ends

Aug. 14, 2025, 5 a.m.

You must be logged in to participate in competitions.

Sign In