Streaming First Story Detection with application to Twitter | Awesome Learning to Hash Add your paper to Learning2Hash

Streaming First Story Detection with application to Twitter

S. Petrovic, M. Osborne, V. Lavrenko. NAACL 2010

Streaming Data Text Retrieval LSH NAACL

With the recent rise in popularity and size of social media, there is a growing need for systems that can extract useful information from this amount of data. We address the problem of detecting new events from a stream of Twitter posts. To make event detection feasible on web-scale corpora, we present an algorithm based on locality-sensitive hashing which is able overcome the limitations of traditional approaches, while maintaining competitive results. In particular, a comparison with a stateof-the-art system on the first story detection task shows that we achieve over an order of magnitude speedup in processing time, while retaining comparable performance. Event detection experiments on a collection of 160 million Twitter posts show that celebrity deaths are the fastest spreading news on Twitter.

Similar Work