Matching participants

SMARTRIQS offers a simple but very efficient matching algorithm. When a participant proceeds to the MATCH block, their data is transferred to the server. Then, a script checks if there is a suitable group for the participant (a group that has an open slot for the participant). If there is a suitable group, the participant is added to that group, and the participant has to wait until the group is full (until there are no more open slots left). If there aren’t any suitable groups, a new group is created, and the participant has to wait until others join and there are no more open slots left. When all slots are filled, participants in the group receive a message in Qualtrics that they have been successfully matched and can proceed to the next page of the survey. If recruitment is done properly and group size is relatively small (see Best Practices), matching is almost instantaneous and does not take more than 10 seconds on average.

By default, if a participant has to wait too long and there are not enough other participants available to fill the group, the experiment is terminated for that participant. However, SMARTRIQS gives researchers a lot of flexibility: researchers can change the maximum waiting time (the default is 3 minutes), so that they can allow participants to wait more (or less) to be matched with others. Researchers can also change how quickly inactive participants are dropped from the matching. By default, inactive participants are dropped after 10 seconds of continuous inactivity: they are removed from the database and their slot changes to ‘open’. This is to ensure that impatient participants (who enter matching but quit before they are matched) are removed. Finally, researchers can also decide to allow participants to complete the study, even if there aren’t enough people available. In this case, participants are matched with BOTs instead of termination: open slots are filled with BOTs when the participant’s waiting time reaches the maximum waiting time. To avoid deception, SMARTRIQS informs participants after matching whether they have been matched with other participants or BOTs.


The MATCH block

To match participants via SMARTRIQS, first you should add the MATCH block to your survey. The recommended method is to import the Generic Interactive Survey Template (“GIST”) to your Qualtrics account (see STEP II. in Getting started for importing surveys).

Once you have the MATCH block in your survey, the next step is to set up the embedded data parameters in the Survey Flow. If you import the GIST, these embedded variables are already in the Survey Flow: you only have to assign values to them. If you are setting up a survey from scratch, you have to add the embedded data to the Survey Flow manually.

Required parameters

There are 9 required parameters that must be defined before the MATCH block in every study:

researcherID Insert your researcher ID here.
studyID Insert the name of the study here.
groupID Leave this blank — this will be set by the matching algorithm.
participantID Use the following: ${e://Field/ResponseID} — although you can use any other random string, it is strongly recommended to use the default Qualtrics Response ID as the participant ID. This guarantees that each respondent is assigned a unique participant ID which makes identifying and tracking participants much easier.
groupSize The size of the group. Use one of these values: 2,3,4,5,6,7,8.
numStages The number of ‘stages’ (the number of decisions to be transmitted). You can use any positive integer.
roles The array of roles. EXAMPLE

For more information, see roles section in Overview.

participantRole You have two options here:
1) Set this to random. EXAMPLE2) Use one of the elements of the roles array. EXAMPLEFor more information, see Randomization.
timeOutLog Leave this blank — SMARTRIQS will save warnings and errors to this variable.

If you set up all the above embedded data parameters correctly, you should already be able to match participants. You can test this by opening the link to your survey in multiple tabs and see if you can complete the MATCH block without issues.

 

Optional parameters

The following are optional parameters that you can set before the MATCH block:

conditions The array of conditions. EXAMPLE

If this embedded data is not set, SMARTRIQS will treat the study as if there was only one condition.
For more information, see the conditions section in Overview

participantCondition You have two options here:
1) Set this to randomEXAMPLE2) Use one of the elements of the conditions array. EXAMPLE

For more information, see Randomization.

maxWaitTime Use an integer between 30 and 600. This is the maximum waiting time before the participant is matched with BOTs or before the experiment is terminated, in seconds. If this embedded data is not set or invalid, the default waiting time is used (180 seconds).
freezeTime Use an integer between 1 and 30. This is the minimum time that participants have to spend on the matching screen, in seconds. This guarantees that participants have sufficient time to read the notifications, even if they are matched immediately. If this embedded data is not set or invalid, the default freeze time is used (3 seconds).
botMatch If set to yes, participants are matched with BOTs when they reach the maximum waiting time during matching. If this embedded data is not set, the experiment is terminated if the participant reaches the maximum waiting time.
terminateText Specify a string: this message is displayed if they survey is terminated when there are not enough other participants available (only if botMatch is not set to yes). EXAMPLE

Please avoid using double quote characters (“) in this string. Double quotes break the script. Use single quotes ( ‘ ) instead. Default message is: “The survey has been terminated. Please contact the researcher to receive partial compensation for your participation.”

timeZone Use an integer between -12 and 12. This parameter sets the time zone (relative to UTC+0). For example, if this is set to -5, the time zone will be UTC-5 (Eastern Time Zone). If this embedded data is missing or invalid, the default time zone is used (UTC+0).
dropInactivePlayers Use an integer between 10 and 60. This is the time limit for dropping inactive players, in seconds. If this embedded is not set or invalid, the default drop limit is used (30 seconds).
loadAnimationURL Specify the URL of the image or animation to be displayed while matching. If this embedded data is not set, the default loading animation will be used.
preMatchText Specify a string: this message is displayed on the screen before participants enter the matching stage. EXAMPLE

Please avoid using double quote characters (“) in this string.
If not set, one of the following default texts is used (depending on the group size): “On the next screen, you will be matched with another participant.” OR: “On the next screen, you will be matched with … other participants.”

matchWaitText Specify a string: this message is displayed while participants are waiting for being matched. EXAMPLE

Please avoid using double quote characters (“) in this string.
If not set, the following default text is used: “We are currently trying to match you with other participant(s). Please wait patiently to be matched. This may take a few minutes. Please do NOT minimize this window or navigate to another page.”

matchAlert Set this to yes to display a notification in the notification bar of the browser (see here) that informs participants if they are successfully matched. This can be very useful because participants will see this notification even if they navigate to a different tab while waiting to be matched. If this parameter is not set, there is no notification by default.

 

Tip for testing your surveys
Make sure to turn off (un-check) “Save and Continue” and “Prevent Ballot Box Stuffing” in Survey Options. Otherwise, Qualtrics won’t let you open the survey multiple times on the same computer. Of course, you should turn these back on before you start collecting actual data to prevent duplicate responses.