Joseph Jay Williams'
Intelligent Adaptive Interventions Lab
Joseph Jay Williams is an Assistant Professor of Computer Science at the University of Toronto (by courtesy, Psychology, Statistics and Information), and a faculty affiliate at the Vector Institute for AI
How can you turn any user interface into an intelligent, perpetually improving system?
Our answer is to use adaptive experiments that quantify the impact of alternative interventions on people, combining human intelligence (collaborations between designers, scientists, users) with artificial intelligence - machine learning algorithms and statistical tests that automatically use data from experiments to enhance the interventions future users receive.
Our research agenda is to transform components of any real-world user interfaces into intelligent, adaptive systems that are perpetually enhancing and personalizing interventions to help people. For example, we've published papers on technology for education, learning, mental health, by testing competing ideas about how to design components of online homework, apps, text messaging interventions, and other interface components. A distinctive focus is micro-experimentation 'in the wild', where we conduct and build tools for randomized A/B comparisons that can be used by practitioners and scientists to test design decisions and hypotheses about how to help people. Another focus is adaptive experimentation – applying and modifying machine learning algorithms and statistical tests for more rapidly yet reliably using data from experiments to change which interventions future participants receive.
Slides and recordings of overview talks are at Talks Illustrating Examples of Lab Research, or TEDxPortofSpain talk.
We use cognitive science theories in deciding the target actions for experimentation (e.g. explanations, motivational messages) and the metrics to optimize (e.g. student ratings, response rates). To generate new actions we design crowdsourcing workflows, leveraging our human-computer interaction research. Data from experiments is analyzed using methods from Bayesian statistics, and algorithms from machine learning are used to turn data into dynamic enhancement and personalization of users' experiences. Our self-improving systems are powered by combining human intelligence – in generating hypotheses that can be tested with data – with statistical machine learning – to automate rapid iteration and improvement.
One example of a system Prof. Williams created for automatically experimenting with explanations, which enhanced learning from math problems as much as an expert instructor [LAS 2016]. Another system boosted people's responses to an email campaign, by dynamically discovering how to personalize motivational messages to a user's activity level [EDM 2015].
In blending scientific research and applications we collaborate and consult with domain/topic experts, instructors, designers, and researchers from diverse disciplines like education, psychology, and computer science. We draw on theories and methodology from the lab research, as well as synthesizing findings from other scientists, work on behavior change, reviews of evidence-based best practices for teaching and learning, Prof. Williams practical experience as a statistics tutor, ed-tech consultant and science & technology advisor, and evaluations of educational technology products & authoring tools for e-learning.
My TEDxPortofSpain talk explains how I use this approach in education, using MOOClets to intelligently adapt explanations for how to solve math problems.
Slides and recordings of overview talks are: Slides for broad HCI audience, Recording for broad HCI audience (45 min), Slides for education audience, Recording of TEDxPortofSpain talk.
You can contact me at: williams@cs.toronto.edu. If you are interested in joining my group, you can email iaiinterest@googlegroups.com. Office directions: tiny.cc/joffice.
More about Joseph
I am currently an Assistant Professor in Computer Science at the University of Toronto (with a Graduate appointment in Psychology and Statistics). Previously I was an Assistant Professor in the School of Computing (Information Systems & Analytics, and NUS HCI Lab) at National University of Singapore (NUS). Prior to that I was a Research Fellow at Harvard's VPAL (Vice Provost for Advances in Learning) Research Group, and a member of the Intelligent Interactive Systems group led by Krzysztof Gajos in Computer Science. I have a courtesy appointment as a Research Scientist in Computer Science at Worcester Polytechnic Institute, where I am a co-PI with Neil Heffernan on an NSF Cyberinfrastructure grant. We use the ASSISTments K12 online math platform to crowdsource randomized controlled trials from the broader scientific community.
I was previously a postdoc at Stanford University in the Graduate School of Education and Lytics Lab, working with the Office of the Vice Provost for Online Learning and Candace Thille's Open Learning Initiative. I received my PhD in Computational Cognitive Science from UC Berkeley's Psychology Department. I worked with Tania Lombrozo to investigate why prompting people to explain "why?" helps learning, and with Tom Griffiths on using Bayesian statistics and methods from machine learning to characterize learning, reasoning, and judgment.
News
We are the finalist in the XPrize Digital Learning Challenge
Our QuickTA project combining AI (RL algorithms & #LLMs) with @TutorGen is one of 16 #ToolsCompetition finalists for the DARPA AI Tools for Adult Learning opportunity!
We gratefully acknowledge support from our sponsors:
We gratefully acknowledge support from our collaborators:
