While there are many programs that can play games as well as humans, there are considerably fewer that can explain their actions. Studies, as well as intuition, suggest that humans are more satisfied with AI decision-making when they understand why the AI made the decisions it did. In our project, we aim to write a program to generate convincing explanations for decisions made by an AI playing Pac-Man.