The Rubric embedded experiment extends the standard embedded experiment framework by allowing raters to evaluate custom JavaScript-rendered content along multiple dimensions. Raters respond to a set of statements using an agreement scale (e.g., "Strongly disagree" to "Strongly agree").
This is useful when you want to capture nuanced feedback about different aspects of dynamically generated content, such as perceptual quality and artificiality, all within a single experiment.
For details on how to create datasets with JavaScript-based stimuli, see the embedded experiments documentation.
You can configure multiple questions, each defined by a statement and a dimension label. Raters respond to each statement independently using the same set of categories. Up to 10 questions can be configured.
For example:
| Dimension | Statement |
|---|---|
| Perceptual | The visual quality looks good. |
| Artificial | The content is generated by AI. |
By default, the rating scale uses a five-point agreement scale:
| Score | Label |
|---|---|
| 2 | Strongly agree |
| 1 | Agree |
| 0 | Neutral |
| -1 | Disagree |
| -2 | Strongly disagree |
Categories are fully customizable. You can change the number of categories (between 2 and 10), their labels, and their scores.
If your embedded content includes audio, you can enable this option so that audio is not muted during the experiment.
You can optionally set a fixed width and height (in pixels) for the iframe in which your JavaScript content is rendered. If not set, the iframe will use a default size.
Below is an example configuration to get you started when using our API to create a multi-ACR embedded experiment.
config = {
"multiAcrEmbed": {
"questions": [
{
"dimension": "Perceptual",
"statement": "The visual quality looks good.",
},
{
"dimension": "Artificial",
"statement": "The content is generated by AI.",
},
],
"categories": [
{"score": -2, "label": "Strongly disagree"},
{"score": -1, "label": "Disagree"},
{"score": 0, "label": "Neutral"},
{"score": 1, "label": "Agree"},
{"score": 2, "label": "Strongly agree"},
],
"hasAudio": False,
},
}