The MOOClet Framework

Brief Explanation of MOOClet/AdapComp

The MOOClet/AdapComp (Adaptive Component) Framework is a specific process for building the technology underlying components of digital resources – like explanations and other text components of webpages – to enable a range of A/B experimentation, crowdsourcing, real-time data analytics, and personalization. 

For example, we redesigned the text component that provided students with explanations for correct answers on math problems as a MOOClet. This enabled new explanations to be added dynamically while students were solving problems (crowdsourced from other students). The MOOClet enabled the use of a machine learning algorithm to compare explanations in a randomized A/B experiment, and to automatically present the best (most highly rated explanations) more frequently as data was collected.

To give a very different example, we resdesigned the emails instructors sent out in a MOOC to be MOOClets, designed to maximize response rate to emails. This allowed us to start out conducting A/B Experiments that compared three different subject lines, analyze data in real-time to discover which subject lines increased response rates for people who spent different numbers of days in the course, and modify the A/B experiment so that it transformed into active personalization of subject lines to student profiles.

Educational components we have redesigned as MOOClets include text explanations in EdX, hints in math problems in ASSISTments, study tips in Canvas problems, and emails instructors send on campus or in MOOCs.

Explanation of MOOClet

A MOOClet refers to an online component – a sentence of text on a webpage, an email, a feedback message on a problem – implemented using a particular kind of technology. The formalized process for implementing MOOClet technology is described below. But once a digital component is implemented as a MOOClet (like a text/image/video explanation, a discussion answer, a webpage), this guarantees that components of these online resources can be easily modified by instructors and researchers to:

(1) Add new versions at any point in time.

(2) Compare these multiple versions using randomized experiments (A/B testing)

(3) Personalize by providing multiple versions to different student profiles.

(4) Receive data in real-time about students' interactions with these components.

(5) Link anonymized data from other non-MOOClet sources like existing learning management systems, and use it for analyzing the impact of experiments and conducting adaptive personalization.

(6) Dynamically change the rules for how versions are presented, and to who, using hand-written rules or algorithms. This allows:

• 1. A/B Experiments to be changed automatically so that the best (or better) version is presented more often as data is collected.

  2. A/B Experiments to discover how to personalize and these new rules for personalization to be implemented.

  3. Any machine learning algorithm for dynamically modifying experiments and personalizing to be deployed in real-time. Specifically, a MOOClet is an abstraction for a reinforcement learning agent (including multi-armed bandits).

The formal definition: A digital resource component (e.g. lesson/problem) is a MOOClet IF and ONLY IF: 1. There is a collection of multiple versions of the MOOClet content that can be modified and added to at any time via API. 

2. Which version is delivered to a student is determined by a policy or function that takes as input variables from the User Variable Store, and the policy/function can be modified via API.

3. There is a User Variable Store that can receive data from the MOOClet and is modifiable by outside sources via API. 

Relevant Publication

Reza, M., Kim, J., Bhattacharjee, A., Rafferty, A.N. and Williams, J. J. (2021). The MOOClet Framework: Unifying Experimentation, Dynamic Improvement, and Personalization in Online Courses. In Proceedings of the Eighth ACM Conference on Learning @ Scale (L@S '21). Association for Computing Machinery, New York, NY, USA, 15–26. DOI:https://doi.org/10.1145/3430895.3460128