Survey audio

The survey audio experiment presents raters with individual audio stimuli and asks them to answer custom survey questions about each one. Unlike ACR audio where raters select a score from a fixed category scale, the survey experiment supports open-ended text responses, single-choice (radio) and multiple-choice (checkbox) questions, transcript highlighting, and slider ratings.

This is useful when you need qualitative feedback or structured responses that go beyond a simple quality rating, such as asking raters to describe what they hear, identify specific artifacts, or classify the type of audio content.

Survey questions

You can configure up to 10 survey questions per experiment. Each question requires a kind (the question type), a question (the prompt), and a dimension (a short label used to identify the question in the results).

The following question types are supported:

Kind	Description
textarea	An open-ended text response field.
radio	A single-choice question where the rater selects one option.
checkbox	A multiple-choice question where the rater selects one or more options.
highlight	A transcript in which the rater highlights words by clicking or dragging.
discrete_slider	A slider that snaps to one of the labelled categories.
continuous_slider	A slider that can be placed anywhere between the labelled categories.

For radio and checkbox questions, you must also provide a list of choices. For checkbox questions, you can optionally set minSelect and maxSelect to constrain the number of selections.

Slider questions instead require a list of categories (2 to 10 {label, score} objects giving the labels shown along the slider and the score recorded at each one) and a responseType, which must be discrete for discrete_slider and continuous for continuous_slider. A discrete slider records the score of the category it snaps to; a continuous slider records a score interpolated between the two neighbouring categories. Sliders do not take choices, minSelect or maxSelect.

highlight questions take neither choices nor categories. The transcript the rater marks up is not part of the survey itself: it comes from a transcript string in the group's config.json. Groups whose transcript is missing or empty hide the highlight question.

Options

Hidden reference

When enabled, a hidden reference is included among the stimuli that raters evaluate.

Show reference

When enabled, an explicitly labeled reference audio sample is shown alongside the stimulus.

Waveform

This option controls how the waveform is rendered. Setting it to stimulus renders a waveform based on each audio sample. Setting it to reference uses the waveform of the reference for all audio samples.

Minimum play duration

This option controls the minimum duration (in milliseconds) for which the audio must be played before raters can submit their response.

Configuration via config.json

Instead of (or in addition to) setting a single survey for the whole experiment, you can define survey questions per group directly in the dataset using a config.json file. This is useful when different audio files need different questions: each group carries its own survey, which overrides the experiment-level configuration set in the app or via the API.

Place a config.json next to the audio file in each group folder:

├── source_1/
│   ├── config.json
│   └── sample.wav
├── source_2/
│   ├── config.json
│   └── sample.wav
├── ...

The survey array uses the same question format described in Survey questions:

{
    "survey": [
        {
            "kind": "textarea",
            "dimension": "Overall",
            "question": "How do you feel about the audio quality?"
        },
        {
            "kind": "radio",
            "dimension": "Content type",
            "question": "What type of audio is this?",
            "choices": [
                { "label": "Speech" },
                { "label": "Music" },
                { "label": "Environmental sound" },
                { "label": "Other" }
            ]
        }
    ]
}

A group's config.json survey takes precedence over the experiment-level survey. You can put the survey array at the root of the config as shown above, or nest it under the experiment key (surveyAudio.survey), which takes precedence over a root-level survey.

Configuration via API

Below is an example configuration to get you started when using our API to create a survey audio experiment.

config = {
    'surveyAudio': {
        'survey': [
            {
                'kind': 'textarea',
                'dimension': 'Overall',
                'question': 'How do you feel about the audio quality?',
            },
            {
                'kind': 'radio',
                'dimension': 'Content type',
                'question': 'What type of audio is this?',
                'choices': [
                    {'label': 'Speech'},
                    {'label': 'Music'},
                    {'label': 'Environmental sound'},
                    {'label': 'Other'},
                ],
            },
            {
                'kind': 'checkbox',
                'dimension': 'Artifacts',
                'question': 'Which artifacts do you notice? (Select all that apply)',
                'choices': [
                    {'label': 'Background noise'},
                    {'label': 'Clipping'},
                    {'label': 'Muffled sound'},
                    {'label': 'Echo'},
                    {'label': 'None'},
                ],
                'minSelect': 1,
            },
        ],
        'hiddenReference': True,
        'showWaveform': 'stimulus',
        'minPlayDuration': 1000,
    },
}