In April of 1995, Lycos had the largest index of the web with 3.6 million web pages. Today, all of the major search engines index several billion pages, as well as images, video, real-time blogs, etc. The proliferation of online data in the past ten years has increased the visibility and importance of data mining, and has also caused some fundamental changes in methods for data mining.

This project course will focus on methods for mining of large-scale unstructured data sets. The format is seminar-style, and students will read recent research papers in data mining and present them in class. Students should have basic knowledge of machine learning and statistics. A large portion of this course is the quarter-long project, in which students work in groups of 2-3 on a self-defined data mining project.

We will provide some basic datasets for these projects, including the Enron e-mail corpus and a portion of Google's crawl. All projects will be presented to a panel of VCs and thought leaders in industry and academia.

Many of the most successful web applications are social, from personalized homepages to social networks. You will learn the fundamental interface, systems, and algorithms concepts in designing social software. The case-based syllabus will cover insights from both research and industry. As a student, you will contribute to this burgeoning field through a quarter-long, team-based project. Students are required to enter the class with an initial project idea.

CS108 or equivalent programming experiences is required. CS147 is recommended, as is a working knowledge of Java, PHP, or Ruby on Rails.