Experiment Design: Considerations

🖥️ Using Terracotta 

When conducting an experiment within Terracotta, a teacher differentiates Canvas activities for different students. In doing so, some students will receive different versions of assignments and learning resources than their peers. To minimize concerns about fairness, please consider the following recommendations: 

1. Utilize Informed Consent in Terracotta, so that only students who volunteer to participate will experience the experimental manipulation. 

2. Utilize the “All Conditions” design option in Terracotta, so that all students receive all versions of manipulated assignments, but in different orders. 

3. Ensure that all experimental versions reflect pedagogically sound instructional strategies. Do not expose students to variations that are known to be less effective. 

4. Do not manipulate high-stakes assessments or pivotally-important learning experiences. 

5. Provide additional opportunities for students to learn the material after outside the scope of the experiment. 

6. Emphasize to students that you are available to answer any questions, and that your availability and your responses will not be manipulated as part of your research.  

Terracotta walks a user through a sequence of design decisions, such as determining the number of treatment conditions, the experiment design, the informed consent procedures, and so on. The experiment builder has three sections: design, participation, and assignments. 

🖌️ Design 

The Design section walks researchers through the basics of experiment design. Researchers give experiments a title, describe the experiment (this includes recording a research question and a hypothesis), name conditions to label experimental versions of assignments, decide how students will be exposed to these conditions (either students will receive all conditions or only one condition), and select a default condition (which is what students will experience if they do not consent to participate, or if they join the class after the experiment has already commenced). 

🙋‍♀️ Participation 

In this section, researchers determine how students will become participants in the experiment. Researchers can invite students to consent to participate, decide who will participate and enroll them manually, or include all students automatically.  

✍️ Assignments 

In the Assignments section, researchers create different versions their class assignments according to their experiment conditions. Terracotta will populate Canvas assignments with learning activities and materials that change depending on who's looking at them, automatically managing experimental variation within the buckets. Once the assignments are published within Canvas, Terracotta will determine which students see which versions of the assignments.