MOOClet/AdapComp Formalism


The term MOOClet is used because this technology was first developed for MOOCs – the framework can be used to redesign any digital resource – this webpage, emails, components of smartphone apps.

Two Sentence Explanation

A MOOClet is a digital component – like an explanation in an online course or problem – designed using the MOOClet technology. This technology enables instructors and researchers to engage in a wide range of A/B experimentation, crowdsourcing, real-time data analysis, and personalization. 

Brief Explanation of MOOClet/AdapComp

The MOOClet/AdapComp Framework is a specific process for building the technology underlying components of digital resources – like explanations and other text components of webpages – to enable a range of A/B experimentation, crowdsourcing, real-time data analytics, and personalization. 


For example, we redesigned the text component that provided students with explanations for correct answers on math problems as a MOOClet. This enabled new explanations to be added dynamically while students were solving problems (crowdsourced from other students). The MOOClet enabled the use of a machine learning algorithm to compare explanations in a randomized A/B experiment, and to automatically present the best (most highly rated explanations) more frequently as data was collected.


To give a very different example, we resdesigned the emails instructors sent out in a MOOC to be MOOClets, designed to maximize response rate to emails. This allowed us to start out conducting A/B Experiments that compared three different subject lines, analyze data in real-time to discover which subject lines increased response rates for people who spent different numbers of days in the course, and modify the A/B experiment so that it transformed into active personalization of subject lines to student profiles.


Educational components we have redesigned as MOOClets include text explanations in EdX, hints in math problems in ASSISTments, study tips in Canvas problems, and emails instructors send on campus or in MOOCs.

Explanation of MOOClet

A MOOClet refers to an online component – a sentence of text on a webpage, an email, a feedback message on a problem – implemented using a particular kind of technology. The formalized process for implementing MOOClet technology is described below. But once a digital component is implemented as a MOOClet (like a text/image/video explanation, a discussion answer, a webpage), this guarantees that components of these online resources can be easily modified by instructors and researchers to:
(1) Add new versions at any point in time.

(2) Compare these multiple versions using randomized experiments (A/B testing)

(3) Personalize by providing multiple versions to different student profiles.

(4) Receive data in real-time about students' interactions with these components.

(5) Link anonymized data from other non-MOOClet sources like existing learning management systems, and use it for analyzing the impact of experiments and conducting adaptive personalization.

(6) Dynamically change the rules for how versions are presented, and to who, using hand-written rules or algorithms. This allows:

  1. A/B Experiments to be changed automatically so that the best (or better) version is presented more often as data is collected.

  2. A/B Experiments to discover how to personalize and these new rules for personalization to be implemented.

  3. Any machine learning algorithm for dynamically modifying experiments and personalizing to be deployed in real-time. Specifically, a MOOClet is an abstraction for a reinforcement learning agent (including multi-armed bandits).


The formal definition: A digital resource component (e.g. lesson/problem) is a a MOOClet IF and ONLY IF: 1. There is a collection of multiple versions of the MOOClet content that can be modified and added to at any time via API. 
2. Which version is delivered to a student is determined by a policy or function that takes as input variables from the User Variable Store, and the policy/function can be modified via API.
3. There is a User Variable Store that can receive data from the MOOClet and is modifiable by outside sources via API. 


Definition of the MOOClet/AdapComp Formalism

Click through the slideshow to see the definition of a MOOClet/AdapComp (the terms are interchangeable).

Relevant Publications/Presentations

Enabling Real-Time Machine Learning for Automated Experimentation and Personalization to Optimize Online Technologies: The MOOClets Abstraction & API (Slides) (Talk Summary)

Reza, M., Kim, J., Bhattacharjee, A., Rafferty, A.N. and Williams, J. J. (2021). The MOOClet Framework: Unifying Experimentation, Dynamic Improvement, and Personalization in Online Courses. In Proceedings of the Eighth ACM Conference on Learning @ Scale (L@S '21). Association for Computing Machinery, New York, NY, USA, 15–26. DOI:https://doi.org/10.1145/3430895.3460128

Williams, J. J., & Heffernan, N. T. (2015). A Methodology for Discovering how to Adaptively Personalize to Users using Experimental Comparisons. Late breaking results, User Modeling and Personalization. [Paper] [Poster]

Williams, J., J., Li, N., Kim, J., Whitehill, J., Maldonado, S., Pechenizkiy, M., Chu, L., & Heffernan, N. (2014). MOOClets: A Framework for Improving Online Education through Experimental Comparison and Personalization of Modules (Working Paper No. 2523265). Retrieved from the Social Science Research Network: http://ssrn.com/abstract=2523265 [PDF from SSRN] [Google Doc]

Williams, J. J., Li, N, Kim, J., Whitehill, J., Maldonado, S., Pechenizkiy, M., Chu, L., & Heffernan, N. (2015). Using the MOOClet Framework as a Problem Formulation to apply Machine Learning to automatically improve modular online educational resources through Experimentation and Personalization. Paper presented at the Human-Propelled Machine Learning Workshop at the Conference on Neural Information Processing Systems. [Overview Poster PDF]

Williams, J. J., Kim, J., Keegan, B. (2015). Supporting Collaborations between Instructors and Researchers using MOOCletsProceedings of the Second ACM Conference on Learning@Scale[Extended Abstract] [Poster]

Williams, J. J., Maldonado, S., Williams, B. A., Rutherford-Quach, S., & Heffernan, N. (2015). How can digital online educational resources be used to bridge experimental research and practical applications? Embedding In Vivo Experiments in “MOOClets”. Paper presented at the Spring 2015 Conference of the Society for Research on Educational Effectiveness, Washington, D. C.

Overview of the MOOClet/AdapComp Framework (Goals)

ADAPCOMP/MOOClet Formalism for designing User Interfaces so alternatives compared in A/B Experiments to be subsequently Personalized


Machine Learning: The MOOClet/AdapComp Formalism motivates an Interface & Specification for APIs to collect data and apply machine learning algorithms to modify large-scale online platforms in real time


Relevance of MOOClets to Machine Learning


Early (November 2014) Working Paper on MOOClet Framework


Email Joseph Jay Williams for a more updated version of this document, or other relevant papers. The MOOClet Formalism is relatively simple to define abstractly, but I have been pursuing the implications for research in online education, human-computer interaction, guiding instructional design, and machine learning and artificial intelligence.

A short memorable URL to a version of this is tiny.cc/moocletframework.

Please cite as:

Williams, J., J., Li, N., Kim, J., Whitehill, J., Maldonado, S., Pechenizkiy, M., Chu, L., & Heffernan, N. (2014). MOOClets: A Framework for Improving Online Education through Experimental Comparison and Personalization of Modules (Working Paper No. 2523265). Retrieved from the Social Science Research Network: http://ssrn.com/abstract=2523265 

The MOOClet Framework: Improving Online Education through Experimentation and Personalization of Modules