Sending and retrieving data

Matching is just the first step in any interactive online experiment — the more interesting parts are when participants can actually send their choice and messages to others. The way sending and retrieving is implemented in SMARTRIQS is very similar to how matching is implemented. In a nutshell, both sending and retrieving involves setting up a few embedded data variables in the Survey Flow, then adding the SEND or GET block after them.
To send and receive data via SMARTRIQS, first you should add the SEND and GET blocks to your survey. The recommended method is to import the Generic Interactive Survey Template (the GIST) to your Qualtrics account (see STEP II. in Getting started for importing surveys). In this template (as well as in the other demo experiments) all blocks are ‘referenced’, which means that they are automatically synchronized with the newest version in the SMARTRIQS library. This guarantees that blocks are always up to date, which is extremely helpful if a new feature is added, or in the rare case if a new bug is discovered. Also, referenced blocks and JavaScript within them cannot be edited, which prevents you from accidentally removing or editing any of the essential scripts. The other method is to directly copy+paste the JavaScript to a Qualtrics question (the source codes are available here). Since the latter method is rather error-prone, it is not recommended, unless you are setting up SMARTRIQS on your own server, or if you want to edit the JavaScript.

Tip for setting up experiments with multiple stages

Do not import the SEND or GET blocks multiple times to your survey. One cool feature of Qualtrics is that you can insert the same block several times in the Survey Flow, without having to duplicate the actual block in the Survey Editor. And since the SEND and GET blocks are fully generic, there is no need to duplicate them. Simply add them as a new element in the Survey Flow as many times as you need.

Once you have the SEND and GET blocks in your survey, the next step is to set up the embedded data variables in the Survey Flow. If you import the GIST, these embedded variables are already in the Survey Flow: you only have to assign values to them. If you are setting up a survey from scratch, you have to add the embedded data to the Survey Flow manually.

The SEND block

To keep the SEND block as simple as possible, there are only two variables that you have to specify: 1) what is the response that you want to send, and 2) what stage does it correspond to. For example, in an ultimatum game you would send the allocator’s offer in Stage 1 and the recipient’s response in Stage 2.

Required variables

sendData	This is the the name of the embedded data that you want to send. Important: you should save the participant’s response to another embedded data first, and then refer to the name of that embedded data here.
sendStage	Use a positive integer between 1 and numStages — The current stage.

SEND block. Note that we first save the participant’s response to the variable called “transfer”, and use the name of this variable when defining “sendData”. Also note that the “transfer” embedded data must be defined in a separate “Set embedded data” element before “sendData” and “sendStage” are defined.

See example SEND block

Hide example

Importantly, in SMARTRIQS each participant can send only their own responses and participants can send responses from one stage at a time. Since stages in SMARTRIQS are defined as sequential parts of the experiment, this limitation is by design. If your experiment has multiple stages, you have to add the SEND block to the Survey Flow and define embedded data variables before the SEND block multiple times.

The GET block

The GET block has more input variables than the SEND since it has to be able to retrieve multiple responses from other group members in a stage. The GET block has 3 required variables and 9 optional variables.

Required variables

getData	This is the name of the role(s) whose response(s) you want to retrieve. For example, if you want to retrieve the decision of the participant in role “A”, you should put `A` here. You can also retrieve multiple responses in one GET block. To do so, enter the names of all roles whose responses you want to retrieve, separated by commas (no space before or after commas!). For example, if you want to retrieve the responses of four players (roles “A”, “B”, “C”, and “D”), use `A,B,C,D`.
getStage	Use a positive integer between 1 and numStages — The stage from which you want to retrieve the responses.
saveData	This is the name(s) of the embedded data that the retrieved value(s) will be saved to. You have to create embedded data before this block, and use the name(s) of that variable. If you are retrieving multiple responses, you have to define multiple names here, separated by commas (no space before or after commas!). The number of retrieved values in getData must be the same as the number of embedded data names in saveData. If multiple values are retrieved, the first value will be saved to the first embedded data variable defined in saveData, the second retrieved value will be saved to the second embedded data variable defined in saveData, etc.

GET block, sample setup #1: single response. Note that we first create the embedded data variable to which the retrieved value will be saved (“recipient_payoff”). Then we use the name of this variable when defining “saveData”. Also note that we have to create “recipient_payoff” in a separate “Set embedded data” element before “saveData” is defined.

See example GET block #1: retrieving a single response

Hide example #1

GET block, sample setup #2: multiple responses. Note that we first create the four embedded data variables to which the retrieved values will be saved (“contribution_A”, “contribution_B”, etc.). Then we use the names of these variable when defining “saveData”. Also note that we have to create “contribution_A”, “contribution_B”, etc. in a separate “Set embedded data” element before “saveData” is defined.

See example GET block #2: retrieving multiple responses

Hide example #2

Optional variables

defaultData	This is the value(s) that will be used 1) if the retrieved response is made by a BOT, or 2) if the participant whose response is retrieved failed to submit their response before the time limit expired (timed out). If you are retrieving multiple responses, you have to define multiple default values, separated by commas (no space before of after commas!). The number of retrieved values in getData must be the same as the number of embedded data names in defaultData. Default data must be defined when using bots. If default data is not defined, the survey is terminated upon timeout.
maxWaitTime	Use an integer between 30 and 600. This is the maximum waiting time when retrieving responses, in seconds. If this expires, the default responses are be used. If this embedded data is not set or invalid, the value defined in the MATCH block is used. If that value is missing as well, the default waiting time is used (180 seconds).
getWaitText	Specify a string: this message is displayed while participants are waiting for others. EXAMPLE Please avoid using double quote characters (“) in this string. If not set, there is no text displayed, only the loading animation.
operation	Use one or more of the following strings: `min`, `max`, `secondMax`, `average`, `sum`, `rank`, `accuracyRank`, or `p-BeautyRank`. Note that the names of the operations are case-sensitive. If multiple operations are set, they must be separated by commas (no space before or after commas!). If this variable is set, it executes one or more mathematical operations on the retrieved data. Can be set only 1) if at least 2 responses are retrieved, and 2) if all retrieved responses are numeric. To learn more about operations, see Operations.
saveOperation	This is the name(s) of the embedded data that the result(s) of operation(s) will be saved to. You have to create embedded data before this block, and use the name(s) of that variable. If you are executing multiple operations, you have to define multiple names here, separated by commas (no space before or after commas!). The number of operations in operation must be the same as the number of embedded data names in saveOperation.
decimalPlaces	Use 0 or a positive integer: the number of decimals displayed in the results of operations. If this embedded data is missing or invalid, the default value is used (2 decimals).
breakTie	If this is set to `yes`, ties in ranks will be broken randomly. This setting applies only if `rank`, `accuracyRank`, or `p-BeautyRank` is defined in operation. Otherwise, it will be ignored. If this embedded data is missing or invalid, ties will not be broken (e.g., if four participants score 10, 10, 9, and 9 points, their ranks will be 1, 1, 3, and 3, respectively).
targetValue	This is a numeric value to which participants’ responses are compared to when calculating their accuracy-based rank. This variable is required if `accuracyRank` is defined in operation.
p-BeautyFraction	a real number between 0 and 1, the fraction used in the p-Beauty Contest Game. This variable is required if `p-BeautyRank` is defined in operation.

In the sample above we add the sum operation to a public goods game. First, we have to create the embedded data “totalContribution” in a separate “Set Embedded Data” element before the “operation” is defined. Then, we use the name of this variable in “saveOperation”. The sum of the retrieved data (the four contributions) will be saved in the variable named “totalContribution”.

See example GET block #3: executing the sum operation

Hide example #3

In this example we add three operations: minimum, maximum, and accuracy rank. First, we have to create the embedded data “minGuess”, “maxGuess”, and “myRank” in a separate “Set Embedded Data” element before the “operation” is defined. Then, we use the name of these variables in “saveOperation”. Since we use the “accuracyRank” operation, we have to define a “targetValue”. In this example, we set “breakTie” to “yes”, which means that all ties will be broken randomly. The lowest retrieved value (lowest of four guesses) will be saved in the variable named “minGuess”. The highest retrieved value (highest of four guesses) will be saved in the variable named “maxGuess”. Each participant’s own rank will be saved in the “myRank” variable. The rank will be based on the distance between guesses and the target value (42). The participant whose guess will be the closest to 42, will be ranked #1. The participant whose guess will be the second-closest to 42, will be ranked #2, etc.

See example GET block #4: executing multiple operations: minimum, maximum, and accuracy rank

Hide example #4

Operations in the GET block

The built-in mathematical operations in Qualtrics are rather limited and error-prone, however, most interactive experiments require some basic operations. SMARTRIQS features 8 mathematical operations that can be executed in the GET block.

min: returns the lowest value among retrieved values.
max: returns the highest value among retrieved values.
secondMax: returns the second-highest value among retrieved values. This operation is useful in auctions where the price is determined by the second-highest bid (e.g., Vickrey auction).
average: returns the average of retrieved values. By default, averages are rounded to 2 decimals. To change decimals, set the decimalPlaces embedded data variable in the Survey Flow.
sum: returns the sum of retrieved values.
rank: ranks participants according to their retrieved values (e.g., score on test, number of tasks completed, etc.), where rank = 1 corresponds to the highest retrieved value, rank = 2 corresponds to the second highest retrieved value, etc. Then returns the rank of the participant.
accuracyRank: ranks participants according to the absolute linear distance between their retrieved values and a target value, where rank = 1 corresponds to the smallest distance, rank = 2 corresponds to the second-smallest distance, etc. Then returns the rank of the participant. This operation is useful if you want to rank participants based on their accuracy (e.g., prediction market, guessing game). If this operation is set, the targetValue must be also defined.
p-BeautyRank: ranks participatns according to the absolute linear distance between their retrieved values and the winning number in a beauty contest game, where rank = 1 corresponds to the smallest distance, rank = 2 corresponds to the second-smallest distance, etc. Then returns the rank of the participant. If this operation is set, the p-BeautyFraction must be set.

For the last three operations, you can either allow ties in ranks (default) or force breaking ties. If you want to break ties randomly, set the breakTie embedded data variable to yes.

The COMPLETE block

This is an optional block that can be inserted at the end of the survey. It has two functions:

It informs the server that the participant has completed the experiment. This is useful if you use the SMARTRIQS progress monitor and want to see if participants have completed the experiment.
It checks the “time out log” in Qualtrics, and if there are no issues, it writes “OK — no issues” in the time out log.

However, neither of the above functions are essential, and the COMPLETE block does not affect the interaction between participants in any way. This block simply provides more information to the researcher. In more detail: participants’ progress is automatically saved in Qualtrics, so changing participants’ status to “completed” will only matter when using the SMARTRIQS progress monitor. Similarly, the time out log automatically logs issues, which means that it will remain blank if there are no issues. Using the COMPLETE block just makes it more explicit if there are no issues, by replacing the blank entry by an “OK — no issues” message.
The COMPLETE block uses the same input variables as the MATCH block: you do not have to define any new embedded data, simply add the COMPLETE block to the end of the Survey Flow.