Skip to main content

Survey embedded

The survey embedded experiment combines the embedded experiment framework with custom survey questions. Raters view content rendered by your JavaScript code and then answer survey questions about it. Unlike the embedded rubric experiment where raters respond to statements on a fixed agreement scale, the survey experiment supports open-ended text responses, single-choice (radio) and multiple-choice (checkbox) questions, transcript highlighting, and slider ratings.

This is useful when you need qualitative or structured feedback about dynamically generated content, such as asking raters to describe their experience, identify issues, or classify content attributes.

For details on how to create datasets with JavaScript-based stimuli, see the embedded experiments documentation.

Survey embedded interface

Survey questions​

Raters answer up to 10 survey questions about each stimulus. See Surveys for the question types, visibility rules and the survey format.

This experiment also supports highlight questions, in which raters mark words in a transcript given by each group's config.json.

Options​

Audio​

If your embedded content includes audio, you can enable this option so that audio is not muted during the experiment. If the job records the browser tab, the tab's audio is recorded too, so raters must share it.

Microphone​

Enable this option if raters need a microphone, for example to talk to an avatar. Prolific shows it to raters as a study requirement, and if the job records the browser tab, the microphone is recorded with it.

Iframe size​

You can optionally set a fixed width and height (in pixels) for the iframe in which your JavaScript content is rendered. If not set, the iframe will use a default size.

Interaction duration​

Set the minimum and maximum time you expect raters to interact with each embedded stimulus. Mabyduck uses this range to estimate how long a session takes, and therefore how much crowd-sourced raters are paid. See interaction duration for details. When configuring the experiment via the API, the range is given in milliseconds.

Shuffle questions​

Shows the questions in a random order, per session or per slate. See shuffling questions.

Configuration via config.json​

Instead of (or in addition to) setting a single survey for the whole experiment, you can define survey questions per group directly in the dataset using a config.json file. This is useful when different stimuli need different questions.

Place a config.json next to the JavaScript stimuli in each group folder (see the embedded experiments documentation for the dataset layout):

├── source_1/
│ ├── config.json
│ └── content.js
├── source_2/
│ ├── config.json
│ └── content.js
├── ...

See per-group surveys for what the file holds and how it takes precedence over the experiment's survey.

Configuration via API​

Below is an example configuration to get you started when using our API to create a survey embedded experiment.

config = {
'surveyEmbed': {
'survey': {
'questions': [
{
'kind': 'textarea',
'dimension': 'Overall',
'question': 'How do you feel about the content?',
},
{
'kind': 'radio',
'dimension': 'Realism',
'question': 'How realistic does the content look?',
'choices': [
{'label': 'Very realistic'},
{'label': 'Somewhat realistic'},
{'label': 'Not realistic'},
],
},
],
},
'hasAudio': False,
'hasMicrophone': False,
'interactionDuration': {'min': 10000, 'max': 30000},
'shuffleQuestions': 'session',
},
}