Grad Course: Designing Intelligent Self-Improving Systems Through Human Computation, Experimentation and Reinforcement Learning

Contact the instructor for more updated details, and to obtain permission to register in the course.

The course will give students an introduction to research on how to design and deploy software systems that can be deployed to real users and use data to automatically improve (see http://www.josephjaywilliams.com/research-overview). For example, building lessons that continually improve and personalise which explanations are provided to students, or building apps that motivate people to change behaviour by crowdsourcing motivational messages and use machine learning to experiment with which messages change people's decisions. 

Designing these systems draws on human-computer interaction research on crowdsourcing and human computation to generate new system actions, theories from cognitive science in designing ways to measure what helps users, and algorithms from statistical machine learning & artificial intelligence to conduct experiments and analyse data in real time. 

Examples of weekly topics are: 

> Human-Computer Interaction research on crowdsourcing and human computation, getting users to help improve a system by suggesting new system components (e.g. explanations, questions to ask people to reflect) which machine learning algorithms can then test out, to discover what is working. 
AXIS: Generating Explanations at Scale with Learnersourcing and Machine Learning
https://drive.google.com/file/d/0B-hY3HRV_efybEI3TTg3MWk5VVU/view

>Applications of algorithms from reinforcement learning, Bayesian optimisation and Recommender Systems, to do real-time data analysis of which messages motivate people to respond to emails, discovering what works for different users, and rolling these out to future people. 
Combining Dynamic A/B Experimentation and Recommender Systems in MOOCs
https://drive.google.com/file/d/0BxZ6-8qiAeAua2I4MFIzVHdfR1k/view

>Understanding an existing architecture that provide data structures and APIs for machine learning algorithms to do dynamic experiments and personalise which lessons/problems are provided to users. 
MOOClets: A Framework for Dynamic Experimentation and Personalization 
http://tiny.cc/mooclets4page
Code base in python/Django: https://github.com/kunanit/adaptive-mooclet-lti

>Cognitive Science research in designing user studies and A/B comparisons to determine which lessons or motivational messages help people learn.
Connecting Instructors and Learning Scientists via Collaborative Dynamic Experimentation
https://drive.google.com/file/d/0B1JkztQlPN2ra3c3YXJCS0Nwdk0/view