Teaching & Mentoring



Teaching and Mentorship

The goals of my teaching, public outreach, and mentorship are to support others in gaining the skills, resources, and knowledge needed to pursue projects that they find personally valuable. I aim to model an enthusiastically curious and collaborative approach to supporting others’ work, such as by guiding the development of “living” knowledge management resources like web resources for organizing research literature (www.josephjaywilliams.com/education or Wikis for learning software (tiny.cc/qualtricswiki, tiny.cc/mechanicalturk, tiny.cc/siteshelp), which allow students to both learn and teach core skills.


Teaching Experience

I greatly enjoy teaching and enjoy reorganizing my knowledge through explaining and discussing concepts with others. This past summer I was an Instructor for the “In Vivo Experimentation” track at the Pittsburgh Science of Learning Center’s Summer School on Learning & Technology research. This involved preparing and delivering a lecture on “Using Online Resources to conduct Educational Experiments” for the 40+ attendees. It also involved co-mentoring a smaller group of 7 attendees (ranging from PhD students at Columbia Teacher’s College to a Director of Program Measurement at Pearson, who I continue to work with) in the “In Vivo” track for 8-10 one hour meeting times, where we supported their development of project proposals for an in vivo study embedding an experimental comparison in a real-world resource. I also created an online web resource (tiny.cc/learnlab2013) which provided an easily changeable yet permanently available link to web resources with the schedule, lectures, files, relevant resourceseasy posting and access to files by students bothduring the summer school

At Berkeley I was a Graduate Student Instructor for the introductory Cognitive Psychology class (Basic Issues in Cognition) and the introductory Statistics class (Research and Data Analysis in Psychology), receiving ratings of 6 to 7 out of 7. This involved multiple sections per week, in which I provided brief lectures to provide context, followed by discussion in large and small groups, and periods for students to work separately and pose questions when needed. In order to gain more exposure to the K-12 classroom, I have also volunteered as an in-class mathematics tutor at Berkeley High School.

I have given a range of course lectures to classes of 100+ on statistics, computational modeling, and applying cognitive science to improve one’s learning, as well as talks to a general audience about learning research and educational applications in cognitive science and online software at EdX, Khan Academy, the Bloomsburg Corporate Advisory Council, a Google Tech Talk, iNACOL, Educause, and Stanford’s HCI brown bag talk series.

Although I ultimately took my current position at Stanford, I was offered a position as the Science & Mathematics Education Lecturer at Berkeley’s Graduate School of Education. This is a yearly renewable position (for up to three years) that involves teaching the introductory courses to Master’s level students who want to receive their Teaching Credential, and the first year PhD students in Science & Math Education, and Education & Technology. The courses that I was prepared to teach included “Introduction to Science & Mathematics Education”, “Scientific Thinking & Learning”, and “Technology for Science Education”, and organizing the Cognition & Development colloquium series.


Teaching Interests

Given the diversity of disciplines and methodologies I have been exposed to (Experimental Psychology, Cognitive Science, Education, Computer Science, & Statistics) and efficiencies gained through a focus on knowledge aggregation and management, I am excited about and open to teaching a range of courses, especially where co-teaching is possible and online or blended resources can be incorporated from related courses.

My core strength would be in teaching Master’s and PhD level courses at the intersection of learning, education & behavior change with cloud-based Internet software & technology (desktop/tablet/mobile/other). Course topics could include “Using Technology to foster Motivation, Engagement & Behavior Change”, “Designing Online Environments with Interactions & Features that promote Knowledge Building & Reflection”, “Tools & Principles for Knowledge Management”, “Rapid Authoring Tools & Educational Technology for Online Learning”.

I am also interested in teaching or co-teaching courses that cover Instructional Design, Educational Data Mining, Intelligent Tutoring Systems, and Computer Supported Collaborative Learning. I would also lead advanced seminars for PhD students on topics like explanation, metacognition, conceptual change, online education, habit & behavior change.

Given the methodology I focus on in my research and recently teaching a course at CHI on online experiments, courses on methodology could include “Principles of Experimental Design & Analysis”, “Novel affordances of Internet resources for Data Collection & A/B Testing”, “Leveraging the Internet to conduct iterative design experiments, large-scale randomized controlled trials, and collect longitudinal data”, “Integrating Machine Learning & Experimental Design in Data Collection”, “Computational Models of Behavior & Thinking” (which I could focus on Bayesian/Probabilistic Models of Cognition and Connectionist/Neural Network models as well as Bayes Nets).

My disciplinary knowledge & teaching experience would make me comfortable teaching Statistics of the kind used in behavioral social science to undergraduate, master’s level, and PhD students, both basics (t-tests, ANOVA, Regression) and more advanced topics like Multilevel or Structural Equation Modeling, and aspects of educational assessment like Item Response Theory. Although my focus is quantitative, the education and learning sciences research groups I’ve participated in rely on qualitative methods and design approaches, which I would incorporate myself into introductory courses on methodology, and/or involve guest experts/co-teachers.


Mentoring Experience

Since 2007 I have advised and mentored over 20 undergraduate research assistants independently of my PhD advisor at Berkeley, and was very fortunate to receive the Psychology Department’s Outstanding Mentor Award this year. I have therefore been directly responsible for guiding these students in exploring research literature, evaluating and developing experiments, learning to use software to implement experiments and do statistical analysis, and helping prepare them for talk and poster presentations, as well as writing letters of recommendation and discussing graduate school applications and career directions.

In addition, I have mentored two post-degree/master’s level students in gaining research experience and acted as a mentor to 5 graduate students at Berkeley in surveying literature, designing experiments, critiquing theory, analyzing data, and writing papers. I am a consultant and mentor on Samantha Beckerman’s successful doctoral grant from NIH, having provided guidance and edits on the design of the proposed studies on online cognitive behavioral therapy, with the commitment to future help in identifying useful online technology, designing studies, and publishing papers.

In Spring 2013 I served as a Science & Technology mentor for an entrepreneurship class taught by the founding instructors of NSF’s Innovation Corps programs (Steve Blank & Jerry Engels, using “Lean Launch Pad” for startups), which teaches faculty to convert scientific work into marketable products. This involved giving general advice and feedback about relevant psychological research and online software to a 60+ class of 10 startup teams during weekly classes, with more frequent out-of-class meetings with a team developing a product to provide reviews on educational software, and a team developing video recording software for qualitative research.

Over the past few years I led a series of four reading & discussion groups involving graduate students, convening, collaboratively setting the agenda, and presenting on: Educational Applications of Cognitive Science, Bridging Psychology & Education, Behavior Change, Online Education. In my final semester at Berkeley, I secured funding to found and direct the Learning, Education And Research Network (LEARN, www.learnnetwork.net), an organization that holds cross-disciplinary research and practice meetings on online education, again drawing on input to convene and hold a series of meetings (http://www.learnnetwork.net/learn-meet).

Since I have been at Stanford, I have been playing a role in supporting graduate students in the Lytics Lab (lytics.stanford.edu) in their research, and a course seminar on online education research. I have met one-on-one with over ten existing graduate students (in psychology, education, computer science, communications) to provide feedback on current work, and I have also collaborated with several of them on research projects.

Additional general support for the Lytics Lab has taken the form of giving talks and lectures on different topics (like past research in cognitive science, existing educational technologies, principles of experimental design), being a major contributor on a university-wide mailing list for research on online education (lyticslab@lists.stanford.edu), putting together what is now called the Lytics Lab “Intranet” (tiny.cc/lyticslab, not public but I can selectively approve access requests), an extensive online Wiki to support the lab’s online education research, and creating a website for the Lytics Lab seminar (tiny.cc/lyticsseminar) that allows any seminar member to engage in collaborative note-taking, real-time annotation of presenter’s slides, and forum or email discussion.


Mentoring Approach

Providing & Involving Students and Collaborators in “living” Knowledge Management Resources to support research and practice

Resources like what is now called the Lytics Lab “Intranet” represent a core way in which I hope to support others’ research, by identifying, adapting, and documenting online software (see tiny.cc/siteshelp) that allows the co-creation of user-friendly and widely available online resources for several purposes: Wiki-Style Knowledge Management (e.g., cognitivescience.co/qualtrics, tiny.cc/mechanicalturk, www.josephjaywilliams.com/useful-software, cognitivescience.co/ behaviorchange), Dynamic Course Websites (tiny.cc/lyticsseminar), Conversion of in-person seminars and training into online digital resources while minimizing user friction (tiny.cc/learnlab2013), and Online Resources for lab research groups and collaboratives (cognitivescience.co/coglab is my focal resource) or cross-university communities (www.learnnetwork.net). 

These can all include digital resources like reading & qualifying exam lists (tiny.cc/assessmentineducation, tiny.cc/causalitycomputation), information and advice for submitting and presenting at interdisciplinary conferences like AERA, CHI, ICLS, ACM Learning at Scale (tiny.cc/chi2014), and resources on applying for grants.

The resource developed and used most extensively is sites.cognitivescience.co/coglab (access can be granted), which is used and improved by undergraduate and graduate students who work closely with me at Berkeley and Stanford. It is targeted at avoiding duplicated effort by people doing and learning similar things, helping mentees in their rapid learning of research skills, as well as to ensure efficient organization and running of a smaller scale ‘lab-within-a-lab’.

It contains step-by-step introductions for new people – on setting up Google Custom Search Engines, using software to speed up research like accessing and organize academic papers (www.josephjaywilliams.com/useful-software), screencasts and tutorials on data processing and analysis, pages explaining ongoing projects that contain background literature, collective ideas and discussions, and specific details of project-relevant files and organization.

With respect to intellectual resources, my model is to provide structure and potential directions without exerting constraints. I have a choice of introductory reading lists for lab newcomers, with choice of alternative paths (tiny.cc/introductoryreading, related public resources at www.josephjaywilliams.com/explanation-and-learning and www.josephjaywilliams.com /education). The latter represents my creation of an online list of “papers I wish I’d read when I first started research on learning”, with direct hyperlinks to PDFs to nudge people to read and revisit. My workshop paper at AIED’s MOOCshop and NIPS’s Data Driven Education Improving Learning in MOOCs by Applying Cognitive Science consolidates some of this content with additional explanation and actual online prototypes that I created (Williams, 2013).