Prospective Graduate Students

I invite applications for PhD/Master's research with me on topics that excite you and intersect with my research agenda

Students can apply through Computer Science, or through Psychology or Statistics (where I have cross-appointments and can supervise students). Our lab is highly interdisciplinary and currently includes students from HCI, Machine Learning, Psychology, Mental Health, Education, Economics, and Statistics – no one student is expected to have all these skills, they all teach other!

You can get a sense of our lab culture by looking at these videos which some students kindly recorded for me on my birthday! Full 9 minute version and shorter prototype A and shorter prototype B some students created.

Most of my past publications focus on education and learning, but I have a substantive set of work on health behaviour and habit change (e.g. encouraging exercise, mental health), and on projects in areas as wide-ranging as behavioural economics (encouraging charitable donations) and workforce development.

As a graduate student, you would set the agenda for research questions in collaboration with me. Just to give examples, I'm particularly interested in creating systems that combine rigorous randomized experiments with crowdsourcing and human computation, applications of statistical machine learning (e.g. bandits & reinforcement learning, NLP, recommender systems), and theories from cognitive, clinical and social psychology (e.g. self-explanation, analogical comparison, growth mindset, teaching cognitive behaviour therapy).

To find out more about what I do, you can read my Research Statement, read one or two relevant Papers and email me with a summary or reflections & questions, or look at these four talks I've given to HCI, Psychology, Machine Learning, Statistics. You can choose whichever are most relevant to the area you want to work in: Talks Illustrating Examples of Lab Research.


Based on alignment of interests and time, students may have opportunities to collaborate with people in U of T's Computer Science Education research group (e.g. Andrew Petersen), the Vector Institute/Artificial Intelligence/Machine Learning group (e.g. Amir Massoud Farahmand, Marzyeh Ghassemi), HCI people at DGP (e.g. Tovi Grossman, Fanny Chevalier), Psychology Department (e.g. Cendri Hutcherson, Mickey Inzlicht), the Education School OISE, and many other areas like Computational Social Science (e.g. Ashton Anderson).

Graduate students will play a key role in deciding which projects are pursued, but illustrative examples of potential research directions are:
  • Developing new systems for crowdsourcing the design of online problems and lessons, using multi-stage workflows that incorporate input from students, crowd workers, instructors, and learning scientists.

  • Creating and evaluating tools that enable collaboration between instructors and researchers, such as co-design of interventions and personalized lessons, and coordinated analysis of data about learning outcomes for students with different characteristics.

  • Investigating why and when prompting students to explain text/video lectures promotes learning, and understanding the effect of multi-modal interfaces that incorporate writing, speaking, and video creation. Teaching metacognitive skills and self-regulated learning of study behaviours, taking a user-centred approach to designing social-psychological interventions for enhancing motivation such as Growth Mindset and Wise Feedback.

  • Enhancing student wellness and mental health by testing interventions for encouraging people to exercise, monitor stress, apply principles from Cognitive Behaviour Therapy to managing emotions. Investigating how to support online peer-to-peer interactions for having discussions around issues like managing anxiety or developing socio-emotional skills.

  • Interpretable and Interactive Machine Learning Systems for dynamically enhancing and personalizing instruction, especially from the perspective of combining human computation with techniques from multi-armed bandits/reinforcement learning, Bayesian optimization, applications of deep learning to natural language processing.
If you're interested, please apply for the Ph.D program in Computer Science, or Psychology, or Statistics, and list me as a potential advisor. Note that the Master's is a research program available to Canadian Citizens or Permanent Residents (the MScAC is a professional master's). 

You can also send an email to iaiinterest@googlegroups.com with information about yourself, what relevant research experience you have, what parts of my website you've looked at and what you found interesting about them, what topics you're interested in and why, and why you want to pursue a PhD program.

Talks Illustrating Examples of Lab Research

HCI (Human Computer Interaction) targeted talk (UWashington DUB/HCI series)

Slides: tiny.cc/iaislides Recording: tiny.cc/iairecording  


Google slides with otter transcription: 

https://docs.google.com/presentation/d/1m8XaqRXqX6DfZItmyqJbUcW1bPSv3Bspl2nJKTS1_wQ/edit#slide=id.g9685929bb8_2_1294 


Short Title: Enhancing and Personalizing Technology Through Dynamic Experimentation


Long Title: Perpetually Enhancing and Personalizing Technology for Learning & Health Behavior Change: Using Randomized A/B Experiments to integrate Human-Computer Interaction, Psychology, Crowdsourcing & Statistical Machine Learning


How can we transform the everyday technology people use into intelligent, self-improving systems? Our group applies statistical machine learning algorithms to analyze randomized A/B experiments and give the most effective conditions to future users. Ongoing work includes comparing different explanations for concepts in digital lessons/problems, getting people to exercise by testing motivational text messages, and discovering how to personalize micro-interventions to reduce stress and improve mental health. One example system crowdsourced explanations for how to solve math problems from students and teachers, and conducted an A/B experiment to identify which explanations other students rated as being helpful. We used algorithms for multi-armed bandits that analyze data in order to estimate the probability that each explanation is the best, and adaptively weight randomization to present better explanations to future learners (LAS 2016, CHI 2018). This generated explanations that helped learning as much as those of a real instructor. Ongoing work aims to personalize, by discovering which conditions are effective for subgroups of users. We use randomized A/B experiments in technology as an engine for practical improvement, in tandem with advancing research in HCI, psychological theory, statistics, and machine learning.

Bio

Joseph Jay Williams is an Assistant Professor in Computer Science (& Psychology & Statistics) at the University of Toronto, leading the Intelligent Adaptive Interventions research group. He was previously an Assistant Professor at the National University of Singapore's School of Computing in the department of Information Systems & Analytics, a Research Fellow at Harvard's Office of the Vice Provost for Advances in Learning, and a member of the Intelligent Interactive Systems Group in Computer Science. He completed a postdoc at Stanford University in Summer 2014, working with the Office of the Vice Provost for Online Learning and the Open Learning Initiative. He received his PhD from UC Berkeley in Computational Cognitive Science, where he applied Bayesian statistics and machine learning to model how people learn and reason. He received his B.Sc. from University of Toronto in Cognitive Science, Artificial Intelligence and Mathematics, and is originally from Trinidad and Tobago. More information about his research and papers is at www.josephjaywilliams.com.

Machine Learning targeted talk (MILA-McGill/Stanford)


List of MILA (Montreal Machine Learning & Artificial Intelligence);  Slide: MILA Joseph Jay Williams Combining Reinforcement Learning & Human Computation for A-B Experimentation- Perpetually Enhancing and Personalizing User Interfaces.pptx 


Recording:

https://bluejeans.com/playback/s/WffXkZQ5VLWAXi15vrsIR8C4ym0rUgIHuftE7jBvjVfdIBRpepJDqJ9vaJfDfIQd 

(Here are all the MILA talks! https://sites.google.com/lisa.iro.umontreal.ca/tea-talks/fall-2019?authuser=0 )


Edmon Reinforcement Learning Tea Talk: Edmonton Tea Talk RLAI group.pptx 


Combining Reinforcement Learning & Human Computation for A/B Experimentation: Perpetually Enhancing and Personalizing User Interfaces


How can we transform the everyday technology people use into intelligent, self-improving systems? I consider how to dynamically enhance user interfaces by using randomized A/B experiments to integrate Active Learning algorithms with Human Computation. Multiple components of a user interface (e.g. explanations, messages) can be crowdsourced from users, and then compared in real-world A/B experiments, bringing human intelligence into the loop of system improvement. Active Learning algorithms (e.g. multi-armed bandits) can then analyze data from A/B experiments in order to dynamically provide more effective A or B conditions to future users. Active Learning can also lead to personalization, by facing the more substantive exploration-exploitation tradeoff of discovering whether some conditions work better for certain subgroups of user profiles (in addition to discovering what works well on average). 

I present an example system, which crowdsourced explanations for how to solve math problems from students and teachers, simultaneously conducting an A/B experiment to identify which explanations other students rated as being helpful. Modeling this as a multi-armed bandit where the arms were constantly increasing (every time a new explanation was crowdsourced) we used Thompson Sampling to do real-time analysis data from the experiment, providing higher rated explanations to future students (LAS 2016, CHI 2018). This generated explanations that helped learning as much as those of a real instructor. Future work aims to discover how to personalize explanations in real-time, by discovering which conditions work for different subgroups of user profiles (such as whether simple vs complex explanations are better for students with different levels of prior knowledge or verbal fluency).

Future collaborative work with statistics and machine learning researchers provides a testbed for a wide range of active learning algorithms to do real-time adaptation of A/B experiments, and integrate with different crowdsourcing workflows. Dynamic A/B experiments can be used to enhance and personalize a broad range of user-facing systems. Examples include modifying websites, tailoring email campaigns, enhancing lessons in online courses, getting people to exercise by personalizing motivational messages in mobile apps, and discovering which interventions reduce stress and improve mental health.

Psychology targeted talk (Social-Personality & Cognitive Psych at U of T)


Indiana University Psychology Talk Dec 2019 Slides: IU CogSci v2.pptx Recording: Indiana Psychology Talk Dec 2019.mp4

U of T Social Psychology Research Group talk Sep 2019: Slides: SPRG Talk v5.pptx and Recording: SPRG Talk (rough, unedited).mp4.


Conducting Adaptive Field Experiments that Enhance and Personalize Education and Health Technology


Understanding people’s complex real-world thinking is a challenge for psychology, while human-computer interaction aims to build computational systems that can behave intelligently in the real-world. This talk presents a framework for redesigning the everyday websites people interact with to function as: (1) Micro-laboratories for psychological experimentation and data collection, (2) Intelligent adaptive agents that implement machine learning algorithms to dynamically discover how to optimize and personalize people’s learning and reasoning. I present an example of how this framework is used to embed randomized experiments into-real world online educational contexts – like learning to solve math problems– and machine learning used for automated experimentation. Explanations (and experimental conditions) are crowdsourced from teachers and scientists, and reinforcement learning algorithms for multi-armed bandits used in real-time to discover the best explanations for optimizing learning. Ongoing research examines tools for managing stress by applying approaches like cognitive behavior therapy, helping university students self-regulate and plan, and encouraging health behavior change through a collaboration with Goodlife to get people to go to the gym.


Statistics targeted talk (UMichigan, Columbia)

Columbia Feb 2020

Slides: Columbia Statistics v2.pptx 

Recording: Columbia Statistics Feb 2020 Adaptive Experimentation Joseph Jay Williams.mp4 



Short Title (Statistics): Statistical Challenges in Reinforcement Learning for Dynamic Field Experimentation


Long Title (Statistics): Challenges & Opportunities in using Reinforcement Learning for Dynamic Field Experimentation in User Interfaces: Tradeoffs between Statistical Inference and Enhancing User Experience


With the goal of surfacing statistical and machine learning challenges, this talk presents applications of reinforcement learning algorithms to conduct dynamically randomized A/B experiments, automatically using data to enhance digital environments for education and health. One study used an algorithm for dynamic experimentation (Thompson Sampling for multi-armed bandit problems) to enable instructors to randomize alternative explanations to learners, and automatically use the data to reweight randomization so higher rated explanations were presented more frequently to future learners. Another study continually added new arms/conditions/explanations over time, using dynamic experimentation to continually optimize. We also present results on how using dynamic experiments can complicate inference and hypothesis testing (by impacting power and type I error), and our first steps towards adapting inference techniques to dynamic experiments. 


This talk raises the opportunities and challenges in using reinforcement learning for dynamic experiments. How do we adapt algorithms for dynamic experiments to appropriately trade-off enhancements to user interfaces against scientific discovery through valid inference? How do we explore which inference techniques are best suited to dynamic experiments? 


Using dynamic experiments has the potential to perpetually enhance and personalize the technology people receive, while changing how social and behavioral sciences collect data in the field. But this requires solving problems and integrating techniques from a range of different areas, including both Bayesian and frequentist statistics, machine learning, a range of social & behavioral sciences, causal inference, and adaptive clinical trials.


11 min TEDx Talk for broad audience

bit.ly/tedxwilliams Is the internet replacing teachers?

"Is the internet replacing teachers?" Transcript of TEDxPortofSpain talk, video at bit.ly/tedxteachers