Williams, J. J., Li, N, Kim, J., Whitehill, J., Maldonado, S., Pechenizkiy, M., Chu, L., & Heffernan, N. (Working Paper). The MOOClet Framework: Improving Online Education through Experimentation and Personalization of Modules. [PDF from SSRN] [Google Doc]
Williams, J.J. & Williams, B. A. (2013). Using Randomized Experiments as a Methodological and Conceptual Tool for improving the Design of Online Learning Environments. Paper presented at the Data Driven Education Workshop at the Conference on Neural Information Processing Systems.
Williams, J. J., Maldonado, S., Williams, B. A., Rutherford-Quach, S., & Heffernan, N. (2015). How can digital online educational resources be used to bridge experimental research and practical applications? Embedding In Vivo Experiments in “MOOClets”. Paper presented at the Spring 2015 Conference of the Society for Research on Educational Effectiveness, Washington, D. C.
Technology for In Vivo Educational Experiments. (in progress draft manuscript).
Williams, J. J. (2014). How online educational resources provide novel affordances for conducting practical interventions and doing psychology experiments. Stanford Psychological Interventions in Educational Settings (PIES) group, Stanford, CA. [SlideShare]
Williams, J. J. (2013). Enhancing Educational Research & Practice using Experiments on Online Educational Resources. Pittsburgh Science of Learning Center LearnLab Summer School, Pittsburgh, PA. [SlideShare]
Williams, J.J., Teachman, B.A., Richland, L., Brady, S.T, & Aleahmad, T. (2014). Leveraging the Internet to do Laboratory Research in the Real World. Symposium conducted at the annual convention of the Association for Psychological Science. San Francisco, CA. [Abstract & Summary] [Youtube video of Symposium]
Williams, J. J., Saxberg, B., Means, B., Mitros, P. (2013). Online Learning and Psychological Science: Opportunities to integrate research and practice. Symposium conducted at the annual convention of the Association for Psychological Science. [description]
Williams, J.J. & Williams, B.A. (2014). Online A/B Tests & Experiments: A Practical But Scientifically Informed Introduction. Course presented at ACM CHI Conference on Human Factors in Computing Systems. Toronto, Canada. [Materials]
Technology for In Vivo Educational Experiments. (in progress draft manuscript).
Williams, J.J. & Williams, B. A. (2013). Using Randomized Experiments as a Methodological and Conceptual Tool for improving the Design of Online Learning Environments. Paper presented at the Data Driven Education Workshop at the Conference on Neural Information Processing Systems.
Updated Version:
Technology for In Vivo Educational Experiments. (in progress draft manuscript).
Doing experiments by embedding Qualtrics into EdX
The method outlined here can ensure control over which condition a user receives at different points in the course (from different Qualtrics surveys) because it assumes the final digit of the anonymous user ID is random, pulls that into Qualtrics, and assigns to experimental condition based on that.