Parallel Index-based Structural Graph Clustering And Its Approximation

Tseng Tom, Dhulipala Laxman, Shun Julian. Arxiv 2020

[Paper]    
ARXIV Graph LSH Unsupervised

SCAN (Structural Clustering Algorithm for Networks) is a well-studied, widely used graph clustering algorithm. For large graphs, however, sequential SCAN variants are prohibitively slow, and parallel SCAN variants do not effectively share work among queries with different SCAN parameter settings. Since users of SCAN often explore many parameter settings to find good clusterings, it is worthwhile to precompute an index that speeds up queries. This paper presents a practical and provably efficient parallel index-based SCAN algorithm based on GS-Index, a recent sequential algorithm. Our parallel algorithm improves upon the asymptotic work of the sequential algorithm by using integer sorting. It is also highly parallel, achieving logarithmic span (parallel time) for both index construction and clustering queries. Furthermore, we apply locality-sensitive hashing (LSH) to design a novel approximate SCAN algorithm and prove guarantees for its clustering behavior. We present an experimental evaluation of our algorithms on large real-world graphs. On a 48-core machine with two-way hyper-threading, our parallel index construction achieves 50–151$\times$ speedup over the construction of GS-Index. In fact, even on a single thread, our index construction algorithm is faster than GS-Index. Our parallel index query implementation achieves 5–32$\times$ speedup over GS-Index queries across a range of SCAN parameter values, and our implementation is always faster than ppSCAN, a state-of-the-art parallel SCAN algorithm. Moreover, our experiments show that applying LSH results in faster index construction while maintaining good clustering quality.

Similar Work

• Avscan2vec Feature Learning On Antivirus Scan Data For Production-scale Malware Corpora
• High-quality Shared-memory Graph Partitioning
• Relative Nn-descent A Fast Index Construction For Graph-based Approximate Nearest Neighbor Search
• Streaming Similarity Search Over One Billion Tweets Using Parallel Locality-sensitive Hashing