Randomization (roles, conditions)
Participants can be randomly assigned to treatments and roles. Randomization is flexible: researchers can either let SMARTRIQS assign treatments and roles after participants are matched in groups (Method 1, recommended) or assign treatments and roles before participants are matched in groups, by using the `Randomizer’ element in the Survey Flow. While matching is significantly faster when using Method 1, Method 2 is required if participants have to read treatment- or role-specific instructions or do treatment- or role-specific tasks before they are being matched with others. (In most experiments, however, participants can read treatment-specific instructions or complete treatment-specific tasks after they have been matched.)
Method 1: letting SMARTRIQS randomize (RECOMMENDED)
To let SMARTRIQS randomize the participant’s role, simply set participantRole =
random before the MATCH block. The matching algorithm will then randomly assign roles to participants.
Similarly, if there are more than 1 condition in the experiment, set set participantCondition =
random before the MATCH block. The matching algorithm will then randomly assign a condition to the group.
This method guarantees very short waiting times, since when participants join a group, their roles and condition are not determined yet. Therefore, whenever there are enough participants to form a group, a new group is created, and participants are assigned roles and conditions only after that. This guarantees that there will no more than n-1 participants waiting to be matched at any given time, where n = group size, regardless of the number of conditions. For example, even if there are 4 conditions in a 4-player game, there will be no more than 3 people waiting to be matched at any given time, since whenever 4 people are available, they immediately form a new group, assigned to the same condition, and then get their roles assigned within that group.
Below is a screenshot of a study with a single condition and 4 players per group, using default randomization in the MATCH block:
Note that we set participantRole =
Method 2: randomize manually in Qualtrics
Use the Randomizer element in the Survey Flow to manually assign roles / conditions to participants. Make sure to check “Evenly present elements”.
When treatments and roles are determined in Qualtrics before matching, participants have to wait until there is at least one participant in each role, which can cause significantly longer waiting times, especially when the group size is large or when there are multiple treatments. For example, if there are 4 roles in just a single condition, it is possible that even 9 people will be waiting at the same time (3-3 people in 3 roles, 0 in the 4th role). Unless recruitment is very efficient and there are a lot of new participants, longer waiting times significantly increase attrition rates.
Below is a screenshot of a study with a single condition and 4 players per group, using manual randomization in Qualtrics:
Note that instead of setting participantRole =
random, we used a Randomizer element in the Survey Flow, and assigned participants to one of the four roles randomly.
The screenshot taken from the progress monitor below demonstrates the issue with this method:
Even though there are already 6 participants, no groups have been created yet, since there is no one in role B (there are 3 D and 2 C players). Using the ‘Evenly present elements’ option in the Randomizer element might help, but since players can drop out during matching, that still does not guarantee that groups will be created once 4 players join.