Practicum 2.0
Advisor: Aaron Bauer
Background
Colleges and universities are experiencing record enrollments in introductory computer science courses, presenting new challenges in keeping up with demand while also offering a quality educational experience. In particular, larger classes make it harder to ensure that every student gets the practice and detailed feedback they need to learn difficult new concepts. Scalable, online tools could play an important role in meeting this challenge.
Current tools available to intro CS students, however, often provide very limited feedback, such as whether their solution to a practice problem is correct. A large body of research supports using worked examples as an effective pedagogical technique. A worked example consists of an example problem and step-by-step instructions leading to the solution. They have been shown to improve learning in a variety of domains, including LISP programming and statistics. Worked examples have been shown to be even more effective when the transition between viewing demonstrations and independent problem solving is faded. This means that the step-by-step help is gradually removed over the course of several worked examples (it fades away), transitioning the student to solving problems on their own.
Hand-authoring these faded worked examples is challenging and time-consuming on the part of educators, which in turn limits students' access to them. Fortunately, a CS comps team has already created an online system, called Practicum, that can automatically generate this kind of interactive exercises. They adapted a system originally created at University of Washington to target important CS 111 concepts including conditionals, while loops, for loops, and nested loops. You can read all about that project here.
There is tremendous opportunity to expand Practicum in all sorts of exciting directions. Here are some possibilities:
- Extending the system to cover other challenging CS 111 concepts such as recursion, object-oriented programming, and debugging. Each of these examples would be a major extension of the system and involve work across the back-end, front-end, and pedagogical content.
- There is a rich literature on techniques for visualizing program execution. Practicum currently takes a minimalist approach, visualizing current variable values and the current line of execution. A comps project could focus on exploring and evaluating new visualizations.
- Adding a new kind of problem to Practicum (such as problems about the difference between returning a value and printing a value) involves authoring the step-by-step procedure the system will teach to users. This procedure is written in a custom programming language, which the system interprets to generate the faded worked examples. This custom language is currently limited and cumbersome, and poses a significant barrier to adding new problem types, especially for non-experts. A comps project could focus on creating a new custom programming language to streamline the complex process of adding new educational content.
The project
In this project you will expand upon the existing Practicum system. This will involve the following steps:
- Study existing systems. You will begin by studying existing systems, both Practicum and others.This will consist of reading academic literature and using those systems that are publically available.
- Familiarize yourself with Practicum. Using the extensive documentation produced by a previous comps team, you will familiarize yourself with the existing codebase and implement minor extensions to solidify your understanding.
- Prototype possible features. In parallel with getting up to speed on Practicum, you will create mockups and/or paper prototypes of features you are considering adding to Practicum. You will use these prototypes to gather preliminary user feedback and inform your choice for step 4.
- Implement chosen features. Based on steps 1–3, you will decide on a direction to expand Practicum and implement it. You will also develop thorough documentation for your additions to the system.
- Evaluate your contribution. Finally, you will evaluate your new features (or the system as a whole) through user testing or a formal user study.
Recommended experience
All seniors are likely to have the background to succeed with this project. Practicum is an online browser-based tutoring system, so web programming experience (e.g., JavaScript/TypeScript, HTML) will be helpful, as will experience from Software Design (CS 257) and/or HCI (CS 344). Since you will be designing new educational content for novice programmers, experience as a prefect or lab assistant for CS 111 will be highly relevant.
References/inspiration
- Existing systems for practicing/visualizing introductory programming concepts:
- Bauer, A., O'Rourke, E., Thayer, K., Butler, E., Brand, W., & Reges, S. (2018). Practicum: a scalable online system for faded worked examples in CS1. [pdf]
- O'Rourke, E., Andersen, E., Gulwani, S., & Popović, Z. (2015, April). A framework for automatically generating interactive instructional scaffolding. In Proceedings of the 33rd annual ACM conference on human factors in computing systems (pp. 1545-1554). [pdf]
- Sorva, J., Karavirta, V., & Malmi, L. (2013). A review of generic program visualization systems for introductory programming education. ACM Transactions on Computing Education (TOCE), 13(4), 1-64. [link]