Recommender system for choosing courses

Advisor: Dave Musicant

Meeting time: TTh 3:10-4:55

Final Results

I. Background

Recommender systems have been around for a while now. Commercial websites such as Amazon.com will make recommendations for what it thinks you should buy, based on your purchasing history. Internet radio stations such as Yahoo! Launchcast will select music for you to hear that it thinks you will like, based on your past ratings of songs. All of these systems use an idea called collaborative filtering, where positive ratings by other users are used to influence the recommendations made to you.

We will use collaborative filtering to build a system to help students decide which courses they should take at Carleton. We will use historical data from the registrar to build such a system.

There are significant privacy issues with using historical registration data on campus. It is essential that the anonymity of past students be preserved. This is actually a more complex problem than it may seem on the surface: even if you remove all personal identifiers about a particular student, you could still re-identify some students just based on knowing what courses they took. For example, an unidentified 2006 graduate who takes enough courses in computer science and in music to complete a major is clearly Jon Sulman. A significant portion of this project will be ensuring that anonymity is preserved in a fair way.

Finally, we will need to obtain approval to do this project from Carleton's Institutional Review Board. Obtaining approval is easy, but requires writing up a short application. This will need to be done this spring to ensure that we obtain approval by the time we're ready to go in the fall.

II. The Project

Here is a list of the concepts and technologies that will be necessary.

Web and database programming. PHP and MySQL will be used to create a dynamic web interface that is functional and attractive.
Java. There is a considerable amount of preprocessing that must be done on the registration data before it can be used. By using Java to do this, our tools will be cross-platform.
Collaborative filtering. A number of techniques for collaborative filtering have been published. We will also possibly invent our own.
Anonymity and Privacy. There has been considerable research in how to quantify and ensure anonymity in data. We will review this work and apply it as necessary.
Ethics. There may be things that we can do with this data that we shouldn't. This will be discussed as it comes up as part of the project.

III. References

J. Herlocker, J. Konstan, L. Terveen and J. Riedl. Evaluating Collaborative Filtering Recommender Systems. ACM Transactions on Information Systems 22(1), pp. 5-53, January 2004.

B. Miller, J. Konstan, L. Terveen and J. Riedl. PocketLens: Towards a Personal Recommender System. ACM Transactions on Information Systems 22(3), July 2004, pp. 437-476.

Herlocker, J., Konstan, J., and Riedl, J., Explaining Collaborative Filtering Recommendations. In proceedings of ACM 2000 Conference on Computer Supported Cooperative Work , December 2-6, 2000, pp. 241-250.

J. Breese, D. Heckerman, C. Kadie. Empirical Analysis of Predictive Algorithms for Collaborative Filtering. Proceedings of the 14th Conference on Uncertainty in Artificial Intelligence.

L. Sweeney. k-anonymity: a model for protecting privacy. International Journal on Uncertainty, Fuzziness and Knowledge-based Systems, 10 (5), 2002; 557-570.

L. Sweeney. Achieving k-anonymity privacy protection using generalization and suppression. International Journal on Uncertainty, Fuzziness and Knowledge-based Systems, 10 (5), 2002; 571-588.

Sheng Zhong, Zhiqiang Yang, Rebecca N. Wright: Privacy-Enhancing k-Anonymization of Customer Data. PODS 2005.