IT 17 067

Examensarbete 30 hpSeptember 2017

A comparative evaluation of state-of-the-art community detection algorithms for multiplex networks

Mikael Dubik

Institutionen för informationsteknologiDepartment of Information Technology

Teknisk- naturvetenskaplig fakultet UTH-enheten Besöksadress: Ångströmlaboratoriet Lägerhyddsvägen 1 Hus 4, Plan 0 Postadress: Box 536 751 21 Uppsala Telefon: 018 – 471 30 03 Telefax: 018 – 471 30 00 Hemsida: http://www.teknat.uu.se/student

Abstract

A comparative evaluation of state-of-the-artcommunity detection algorithms for multiplexnetworksMikael Dubik

Community detection is the study of discovering groups of nodes more connected toeach other than to other nodes inside a network. These groups give insight into theinner workings of a network and their discovery has practical applications in manyfields of science.

A multiplex network is a special type of multilayer network where multiple graphsco-exist on different layers that are stacked on top of each other into one structurewhere each node in a single layer has representation on every other layer in themultiplex. These networks are sophisticated attempts to model real world systemsand the research in this area has contributed to increased insight and knowledge toother fields of science such as social network analysis, medicine, finance and physics.

In this thesis, we will provide an overview of three different multiplex communitydetection algorithms and a fair evaluation in regards to their resource usage andaccuracy in a unified computing environment. The algorithms are tested againstsynthetically generated datasets and real life networks.

Tryckt av: Reprocentralen ITCIT 17 067Examinator: Mats DanielsÄmnesgranskare: Matteo MagnaniHandledare: Roberto Interdonato

Contents

1 Introduction 1

2 Background 12.1 Community detection and multiplex networks . . . . . . . . . . . . . . 12.2 Modularity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

3 Related Work 53.1 Flattening . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53.2 Layer-by-layer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63.3 Maximizing modularity . . . . . . . . . . . . . . . . . . . . . . . . . . 63.4 Random walks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

4 Methods included in evaluation 74.1 PMM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74.2 Generalized Louvain . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84.3 LART . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

5 Experimental Settings 135.1 Synthetic data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

5.1.1 t1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135.1.2 t2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135.1.3 t3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145.1.4 t4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

5.2 Real life data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145.2.1 AUCS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145.2.2 DK . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

5.3 Normalized Mutual Information . . . . . . . . . . . . . . . . . . . . . . 155.4 Methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

6 Results 166.1 Resource usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 186.2 Accuracy on Synthetic datasets . . . . . . . . . . . . . . . . . . . . . . 206.3 Accuracy on Real life datasets . . . . . . . . . . . . . . . . . . . . . . . 21

7 Implementation notes 23

8 Future Work 25

9 Conclusions 26

10 References 27

1 Introduction

A topic that has attracted considerable attention in network science is the detectionof metanetworks existing inside networks, known as communities. A community isdefined as a collection of nodes that are more connected to each other than to theother nodes in the network[1] and their detection is a form of clustering on networks.The detection of communities is an important objective in understanding how a net-work functions, for example discovering cancer development in cell protein-proteininteraction networks or discovering underlying social ties in an online social network[1]. The importance of discovering communities reaches further than this and hasmany practical applications.

A network is a natural and intuitive representation of interactions between entities.Examples of networks are computer networks, paths between waypoints, online socialnetworks and biological systems. When interactions between the same entities becomediverse, it becomes difficult to represent these interactions on a single network. Forsuch interactions we present a special datastructure that is capable of handling thesedifferent types of interactions, called a multiplex network.

A great number of research has gone into discovering communities that reside in amultiplex network [2, 3]. The methods developed for community detection are basedon approximations and heuristics. In particular, the methods presented in this thesisare highly dependent on approximating and making use of a special network metriccalled modularity that measures the quality of network partitions.

In this thesis we present and evaluate, known community detection algorithms thathave been extended for multiplex networks. We present a fair evaluation of thesealgorithms using the programming and computing environment. Running time andmemory usage of the algorithms is measured using synthetic and real life datasets. Inaddition we compare the output of the methods to the ground truth to analyze theiraccuracy.

2 Background

2.1 Community detection and multiplex networks

A common type of data mining task on networks is the detection of clusters insidea network. This partitioning of entities inside a network is known as community de-tection which has gained increasing attention in the last years thanks to the spreadof Online Social Networks and availability of advanced data collecting techniques.Discovering hidden structures inside a network has become incredibly important inunderstanding the internal functioning of a network where examples of practical appli-cations of community detection includes unfolding true friendship groups in massiveonline social networks [4], identifying proteins that interact with each other that formcancerous malignant cells [3], discovering hidden cyber communities on the internetor uncovering missing interactions between entities [5]. Various different fields of sci-ences benefit from community detection such as social sciences, biology and computerscience [6, 5]

The formal definition of a community is not agreed on since there is a plethora ofalgorithms that define it different ways. Some algorithms require nodes to be fullyconnected to each other to be defined as a community [7], while others define it asa subset of some connected nodes κ. Other algorithms allow nodes to be assign intomore then one community making the communities overlap. For this thesis we definea community as a set of nodes that have more connections between each other thento other nodes in the network, as is shown in Figure 1.

1

Figure 1: An example result of community detection

The advances in information gathering reached a point where normal, flat networkscan no longer represent complex interactions between entities effectively. Wealth ofinformation is lost in such networks and community detection on flat networks canfail to reveal interesting structures. For example, in a large online social networksuch as Facebook, a user can have thousands of connections but have almost nointeraction with a majority of them. The size and density of such networks makesit difficult to analyze and yields no apparent community structure resulting in alarge monocommunity. It’s also possible to make use of another, sparser interactionnetwork such as ’liking’ or commenting on other users activity but in some cases theinteraction is too sparse to convey any meaningful information.

Figure 2: A 3 layer Multiplex network

To alleviate these problems it’s possible to represent these interactions in a multilevelnetwork structure where the different levels of the datastructure represent differenttypes of interactions. The core elements of such a multilevel structure is composed ofnodes, edges, actors and layers.

2

An actor is an entity that can have relationships with other actors. A layer is a typeof relationship an actor can have between other actors (ex. friendship in a network,following someone on Twitter). A node is a specific actor on a specific layer. An edgebetween actors represents a relationship between them [8]. Having all the elementsof the datastructure defined we can formally define the multilayer network.

Definition 1 : Given a set of actors A and a set of layers L, a multilayer network isdefined as a quadruple M = (A,L, V,E), where (V,E) is a graph and V ⊆ A ∗ L [8]

In multilayer networks, actors are connected to each other with intra-layer and inter-layer edges. A multiplex network is a special type of multilayer network in whichthe only possible inter-layer connections are between its counterpart nodes in otherlayers. Two different actors cannot be connected cross layer in a multiplex. Withthis datastructure we can preserve valuable connections of entities across differentinteractions. Multiplex networks inherit all the attributes of a regular single layernetwork. The multiplex can have weighted or unweighted edges and directed orundirected edges.

Having defined an additional dimension to standard networks, communities in mul-tiplex networks also have different structures. Communities in multiplexes can becomposed of actors across layers or just nodes on a specific layers. In addtition, mul-tiplexes can have communities that contain actors that exist in more than one commu-nity (actor overlapping) and contain nodes on a single layer in different communities,like regular networks (node overlapping). For this thesis the evaluated algorithmscan detect communities which are only composed of actors or actor overlapping andcommunities that exist only on a single layer.

2.2 Modularity

Recall our informal definition of a community earlier, the objective is to discoveractors that are connected densely inside the group and sparsely outside of it. Givensome partition, we are interested in finding out how well it’s partitioned from the restof the network or how much more frequently does a particular set of nodes interactwith each other versus interacting with nodes outside of it than expected. This metricis known as modularity [1] and is formally defined as:

Q =1

2m

∑ij

(Aij − Pij) δ(υi, υj) (1)

The first term of the formula, 12m , makes sure that Q is a normalized multiplicative

constant on the interval of [−1, 1], where positive values indicate a detected commu-nity structure. Negative values infer that no community structure was found usingthis particular partition. This is an important feature of the metric because algo-rithms that maximize this function get an indication that the observed collection ofnodes does not contain any community features.

The second term is composed of two parts: The connections of the nodes through anadjacency matrix, A, where i, j represent two different nodes. Aij refers to an edgebetween the nodes in the adjacency matrix with values of 1 if a connection existsbetween them and 0 if not.

A small number of edges is not enough to ascertain the existance of a community.The objective is to detect a smaller number of edges than expected. This surprisefactor is defined as Pij and is the random network to compare against and is called a”null model”. This is used to make a value judgment about when a particular densityof edges is significant enough to define a community [1]. There are many choicesfor a null model, for example the standard (Bernoulli) random graph or Barber’s

3

modularity for bipartite networks [9]. A null model should be chosen such that itresembles the original network. In particular, a null model that is closely relatedto the configuration model has proven to be effective for unweighted and undirectednetworks[1].

Pij = γkikj2m

(2)

This model is equivalent to a network where nodes keep their degrees and the edgesare rewired at random. The degree of nodes i, j is ki, kj and m is the total number ofedges in the entire network. This term is the statistical expectation of an edge betweenthese nodes. The idea is that if the node degree is high and they are connected, itis expected that they are together in a community, reducing the contribution to thescore. Likewise if the nodes are connected and they have a low degree count, this isan indication of a community and the score increases by larger margin.

The last term in the formula makes sure that only nodes inside the given communityare considered. The kronecker delta is defined as

δ(υi, υj) =

{1, if node i and j are part of the same community0, otherwise

}making the contribution between nodes not in the same community have no effect onthe overall score.

Detecting communities using this metric has become popular in community detection,however it has some problems. Discovering the communities that maximize the globalmodularity of a network is unfortunately an NP-hard problem. For large networks,tractable solutions to this problem involve greedy algorithms, were communities arediscovered that maximize Q.

An issue of maximizing modularity is the absorbtion of smaller communities intolarger ones. This issue is formally known as the resolution limit and a proposedsolution is to add the parameter γ that is in the inner term of equation 2 to tunethe size of the communities. This solution carries its own set of problems becausecommunities can also co-exist in different shapes and sizes [10].

3 Related Work

A great number of research has been conducted in data science on community de-tection. Various different approaches have been developed for detecting communitieswith various efficiency. On single layer, regular networks the works of Santo Fortunatoand his survey on different algorithms serves as a good presentation to the variousmethods [11].

The advancements of information gathering have led to the creation of advanceddatastructures that are capable of representing more complex relations between enti-ties. For such networks, community detection methods are not as well explored sincethese types of networks are relatively new compared to the regular ones. Researchhas already been done to evaluate a subset of these algorithms [12, 13] and continuesto increase as these multidimensional datastructures become applied more often.

Most community detection methods on multiplex networks are extensions from regularsingle layer network community detection methods. Multiplex community detectionmethods can be roughly classified into four categories.

4

3.1 Flattening

The first class of methods is one where layers inside the multiplex are combined intoa regular network. In these methods, communities are measured as the frequency ofconnections between nodes on different layers. The theory is, that the more connectedthey are between the various layers the higher the chance that these nodes form acommunity. This measure is known as redundancy [14, 8] and the methods in thisclass attempt to classify nodes with highest redundancy between each other intocommunities. The strenghts of these methods lie in their simplicity while also makinguse of single layer community detection methods which have been studied in moredetail.

3.2 Layer-by-layer

An alternative class of methods has been proposed in which communities are dis-covered independetly on different layers then aggregated to form a solution. Thereasoning behind this class of methods stems from the fact that the same entity hassimilar relations with other entities, making the layers correlated. This class of meth-ods have been developed for generic multiplex networks [8] without focusing on anyspecific type of multiplex networks.

Some methods worth mentioning that are of this class are Clique percolation [15],ABACUS [16] and Principal modularity maximization (PMM) [17]. From this classwe decided to include PMM in the thesis.

3.3 Maximizing modularity

Another way of detecting communities is by maximizing a benefit function that mea-sures a quality of a partition. Recall our definition of modularity earlier, that givensome partition we can calculate a score on how ’good’ the partition is. In theory,discovering the highest modularity of a network should result in good partitions.

The methods in this class consider the whole network at the time of computation andattempt to globally maximize the value of modularity. Some examples of methodsthat maximize modularity are simulated annealing and the Louvain method [18] whichbecame popular for its speed and minimal resource usage. Because of this we decidedto include the latter in the thesis.

3.4 Random walks

Instead of maximizing a benefit function, an alternative method to discover communi-ties is to explore the topology of the network and infer some useful information beforepartitioning the network. This is known as random exploration or random walk andthe general idea is to define a walker that walks t steps from all nodes to discover thelocal neighbourhood. Two random walk algorithms exist for this class of methods,Locally Adaptive Random Transition (LART) [19] and InfoMap [20]. We decided toinclude LART in the thesis.

5

4 Methods included in evaluation

4.1 PMM

The modularity equation defined in equation 1 can be represented in matrix form,which is crucial to this method. We now define a real and symmetric matrix B, calledthe modularity matrix.

Bij = Aij − γkikj2m

(3)

This formulation is reminiscent of the graph Laplacian, which is the basis of one thebest known graph partitioning methods, spectral partitioning [1]. Assuming that Srepresents the kronecker delta from last section where

Sij =

{1, if node i belongs to community j0, otherwise

}then modularity can be rewritten as

Q =1

2mTr(STBS

)(4)

The discreetness of S poses the modularity maximization problem as NP-hard [17]but allowing S to be continuous, it’s possible to compute the optimal S as the topl eigenvectors of B. The idea is that the leading eigenvectors of B are indicative ofcommunity structure in a network[1].

The method described so far discovers community structures by maximizing modu-larity for single layer networks. Since a multilayer network is a combination of manysingle layer networks, the top l eigenvectors of each layer are computed individually.This step is called Structural Feature Extraction, where the feature of each layer isaggregated into a feature matrix. The dimensions of this matrix are N ∗ (L ∗ l) whereN is the amount of actors in the network, L the number of layers and l is the amountof structural features extracted. The number of features is a user supplied parameter,where its usually much lower then the number of actors, l << n. The authors of themethod have benchmarked PMM for a broad range of l and recommend setting it to≥ 140 [17].

The authors of the method assume that these latent structures are shared across thelayers. The features are correlated but only after applying a Canonical CorrelationAnalysis which in this case is equivalent to applying Principal Component Analy-sis (PCA), hence the name of the method, Principal Modularity Maximization[17].Applying CCA on the feature matrix creates a low dimensional embedding, U , thatcan be clustered into discrete community assignments using the k-means clusteringalgorithm. We refer to the works of Tang et al [17] for proof and further clarification.

The method has a solid mathematical basis, but some shortcomings are visible. Incommunity detection, the objective is to find the number of communities withoutknowing before how many communities one is searching for. This makes PMM some-what unfit for problems where the exact communities is unknown, forcing users torun the same algorithm with different values of k and l to find optimal results. PMMalso clusters the actors into non-shared communities, meaning actors can only belongto one group that spans across layers. This discards vital information and may resultin structures not being uncovered that could otherwise exist.

PMM is reported to have a runtime complexity of O(n3) [1] because of the featureextraction, however the authors mention multithreading as a solution to the large

6

Figure 3: Overview of Principal Modularity Maximization

runtime complexity of the feature extraction. By extracting the features in parallelon different threads, it’s possible to achieve faster results.

4.2 Generalized Louvain

The modularity metric defined earlier was originally developed for single layer net-works. Since the inception of multilayer networks, a generalization was introducedby Mucha et al. for multilayer and temporal networks. The generalized modularityformula considers nodes from different layers [21] when computing modularity. Al-though the focus of this thesis is on multiplex networks the generalization works forall kinds of multislice networks. Formal definition of the generalized modularity is

Qm =1

2µ

∑ijsr

[(Aijs − γ

kiskjs2ms

)δ(s, r) + Csjrδ(i, j)

]δ(υi,s, υj,r) (5)

Here, the variables i, j, s, r refer to nodes i and j, on layers s and r. Just like in theoriginal version, the inner term with the adjacency matrix A, resolution parameterγ and node degree kiskjs remains unchanged and means the same, as in the originalequation. Likewise, in the original equation the µ parameter normalizes the value Qm

to be on the range of [−1, 1]. Kronecker delta δ(s, r) has the value of 1 when s = r, 0otherwise, which makes sure only nodes in the same cluster have an effect on the finalscore. This marks the end of the first term of the equation which only contributes tothe score if the actors considered are on the same layer.

The second term in the formula is the extension of the original equation, where thedifferent layers are included. δ(i, j) is 1, when the same actor is compared and itsimpact on the final result is scaled by Csjr. Csjr is a user supplied parameter on theunit interval [0, 1], which indicates how much the different layers contribute to themodularity. If Csjr is set to 0 the formula will ignore links between edges and assignnodes to communities only on a single layer. On the other hand, setting Csjr to 1

7

assigns the nodes into cross-layer groups and putting more emphasis on inter-layercommunities.

Having a way to measure the quality of a multiplex partition it’s possible to continuewith the objective of maximizing modularity. A method proposed by V.Blondel et. al[18] has the name of Generalized Louvain and is named after the university where theoriginal authors of the method were when it was conceived. The method is a greedynode grouping method where it attempts to maximize the global modularity Qm.

The Louvain method was originally developed for maximizing single layer modularitybut works also with the extended version. The extended version makes it possibleto maximize modularity in multiplex networks and thus is a ’generalization’ of theoriginal method [21]. The algorithm can be summarized in 2 steps with additionalsubsteps. The main steps are performed until no more community reassignments canbe done:

Step 1)

a. Initially, place each node of the network into a different community, so thatthere are as many communities as there are nodes

b. For each node i on the layer s the gain of Qm is computed by moving the node iinto the same community of its neighbour j. This includes the inter-layer nodesthat node i is connected to.

c. Node i is placed into the community that has the highest and positive gain toQm.

d. Perform substep b iteratively until the maximum Qm is calculated.

It should be mentioned that its possible to place a node in a community with dif-ferent gain in modularity than the highest. We can choose to place it to a randomcommunity, or place it to a random community in proportion to the modularity gain.This makes the algorithm undeterministic and allows the possibility to find a highermodularity gain, however can also find a lower one.

At step 2, a new multislice network is constructed from the communities that arediscovered from step 1. The idea is that the groups on each layer are collapsed intoa single node that represents the cluster found in step 1. The edges between thesenodes are aggregated and a weighted self loop is formed.

Figure 4: Generalized Louvain collapsing mechanism

Step 2)

a. Each layer of the network has as many nodes as communities found during Step1.

b. The inter-layer edge between the new nodes are assigned the weight of the totalamount of edges between the communities. A self-loop for each edge between

8

the nodes inside the community is added with the weight of total amount ofedges.

c. The intra-layer edge is the sum of all the intra-layer edges of the nodes thatconnect the communities. The sum is then multiplied by the user suppliedparameter Csjr

Step 2 reduces the amount of nodes needed in further iterations of the algorithm,speeding it up drastically[22]. Because of this the algorithm is fast, memory effi-cient and relatively easy to implement. The algorithm is reported to have a runtimecomplexity of O(n lg(n)) [18], which is quite fast.

4.3 LART

So far we have explored two methods that depend on maximizing modularity, albeitdifferently. The last method does not maximize modularity directly but still makesuse of it.

Suppose that we want to gather information about the network that we want toanalyze before attempting community detection. One way of gathering informationis to run experiments on a network. The experiment is done by starting at somenode i and by random chance walk between neighbouring nodes for some steps t.Recall that our definition of community is that the nodes we are searching for havedense connections between other nodes inside the community and sparse connectionsto nodes outside of it. This definition implies that nodes that are well connected havea high chance that by walking t times the walker will walk between such nodes andget ’stuck’ in the same set of nodes. By recording where the walker becomes trapped,we can infer the existence of community structure.

Random walk was introduced by Pons and Latapy for single layer networks and isformally called walktrap clustering [23]. The key idea about the method is to define anexplorer that will start from every node in the network and walk randomly for t steps.By computing the distance after the walks its possible to gain information about thetopology of the network. The idea is intuitive because there’s a high chance that theneighbours of the actor that is being examined are well connected to each other andwalking for some time will eventually lead the walker to get trapped between actors,inferring community structure.

An extension to Pons and Latapy’s method has been proposed by Zhana et. al andhas the name of Locally Adaptive Random Transition, from now on referred to asLART [19]. LART allows the walker to transition to other layers of the multiplexnetwork but when dealing with single layer networks, LART degenerates to walktrap.

LART begins by creating adjacency matrices of the different layers and concatenatingthem into a single matrix called the supra-adjacency matrix A, which has the dimen-sions N ∗L ∗N ∗L. The diagonal blocks of A contain the intra-layer edges where themain diagonal is populated with the value of the ε parameter. ε is provided by theuser and must contain values on the unit interval. The purpose of ε is to add a self-loop to every node and to make sure that the network is connected and non-bipartite1. The off-diagonal blocks of A contain the weighted inter-layer edges. The weightof the inter-layer edges is also set by ε and values closer to 1 encourage cross layercommunities while values closer to 0 tend to produce per layer communities[19].

There are three possible moves the walker can make. The walker is allowed to walkto a different actor on the same layer, to its own representation on a different layer

1A bipartite network is defined as a network whose nodes can be divided into two disjoint andindependet sets [24]. Adding a self loop guarantees non-bipartiteness

9

and stay in place. The probability of movement is defined as

P(i,k),(i,k) :=A(i,k),(i,k)

κi,k

P(i,k),(j,k) :=A(i,k),(j,k)

κi,k

P(i,k),(i,l) :=A(i,k),(i,l)

κi,k

P(i,k),(j,l) := 0

(6)

This algorithm is specific to multiplex networks in the sense that it disallows transi-tions between actors across different layers. No teleportation exists and walks acrossdisconnected components are not allowed. The transition probabilities 6 are repre-sented in the matrix P .

P := D−1A (7)

Where D is the diagonal block matrix of A with the multiplex node degrees, κi,k.Since P contains the probabilities of reaching node l from neighbouring node k wecan simulate the random walk by raising P to the power of t, which is the amountsteps taken [19]. Previous unreachable nodes i become reachable as t increases.

As t approaches infinity, P converges to stationary distribution. This has the conse-quence that the chance for the walker to reach node i becomes just a matter of thenodes degree. This implies the walker will fail to ”get stuck” and the whole multiplexbecomes a single community. This has important implications because if t is too highwe can miss important commmunity structures, but also if t is too small because thewalker didn’t get a chance to explore. The authors of the method recommend to keept in the range of t = 3L [19] for sparse networks.

Figure 5: Single layer LART example. a) Blue color is the walker movement and thedarker hue indicates trapping frequency. b) Grouping with the highest modualrity is the

optimal partition of the network

The computed distance between all the nodes is then used to create a dissimilaritymatrix, S(t). The idea is that nodes closer to each other form communities. Whencomputing the dissimilarities there are two possible cases. The first case is when thenodes are on the same layer, just their distance is compared. The second case is when

10

nodes are compared cross layer. A cross layer community is formed only when thesame structure is shared across the layers[19]. We refer to original LART paper forfurther clarification.

A natural choice to group the nodes by distance is to use agglomerative clustering.Once the clustering is finished, the optimal partition of nodes is selected based on themultiplex modularity Qm of matrix A.

5 Experimental Settings

For single layer networks, there is a lot of referenced datasets like the well knownZachary’s Karate Club [25] and the LFR Benchmark [26]. Since multilayer networksare relatively new there aren’t as many datasets as well known like they are forsingle layer networks. We generated synthetic multiplex networks using the mLFRbenchmark [13], which is an extension to the LFR benchmark. Our motivation wasto simulate small and medium sized social networks with increasing amounts of noise.

5.1 Synthetic data

The benchmark tool has many parameters to choose from. It’s possible to configurethe number of actors, layers, the average and maximal node degree and noise. Thenoise coefficient, called the mixing parameter, adds additional edges between theplanted clusters and the rest of the network to hide the planted communities, makingit more difficult for the community detection algorithms to recover them. The mixingparameter makes each node share a fraction of its links with its community and therest of the network[26] and is defined as the symbol µ in which its value is on therange of [0, 1]. When mixing is close to 0, community structures are evident in thenetwork but when µ becomes larger then 0.5, communities start to dissapear in thetraditional sense.

The planted communities in these datasets can be either discreet or actor overlapping,however the networks had no node overlapping communities. 2

When generating the synthetic networks, the benchmark saves the planted communi-ties in the network and writes them into seperate file. We used the saved communitiesto compare the output of the algorithms to measure their accuracy.

Four different sets of synthetic networks were generated. The resource benchmarkshad a fixed noise parameter set to 0.3. 3

5.1.1 t1

First synthetic set had an increase of actors from 1000 to 4000 with an average nodedegree < k >= 20 to benchmark the cpu and memory footprint of the algorithms. Therelative small size of the networks was chosen to discover the limits of the algorithms.

5.1.2 t2

The second set, which is composed of two subsets, had an increase of layers with fixedactors of 1000 and 4000 to compare performance when layers were increased.

2In actor overlapping communities nodes on different layers can be assigned to different commu-nities while in node overlapping communities, a single node inside a single layer can be assigned todifferent communities.

3For brevity, the generated synthetic networks were all undirected and unweighted

11

Parameter t1 t2 t3 t4Number of actors (N) [1000, 4000] 1000, 4000 [7000, 10.000] 1000, 4000Maximal actor degree 50 50 100 50Average actor degree (< k >) 20 20 50 20Maximal size of community 100 100 1000 100Minimal size of community 10 10 100 10Number of layers (L) 3 [2, 5] 3 3Mixing parameter (µ) 0.3 0.3 0.3 [0.1, 0.6]

Table 1: Parameters of synthetic dataset benchmarks. The brackets around values are amatematical notion and indicate range.

5.1.3 t3

The third synthetic set was similar to the first set but scaled: More actors, morecommunities and higher average degree. This benchmarks simulates a medium sizedsocial network.

5.1.4 t4

The fourth synthetic set consisted of networks with 1000 and 4000 actors with in-creasing value of the mixing parameter. This benchmark revealed how accurate thealgorithms are when increased amounts of noise was added into the network.

5.2 Real life data

Two real life datasets with ground truth were available to us. Both of the networksthat were chosen were social networks for ease of interpretation.

5.2.1 AUCS

AUCS is a collection of online and offline relationships between employees of thedepartment of Computer science at Aarhus [27]. This dataset has 64 actors withrelationships encoded on 5 different layers:

• Current working relationships

• Repeated leisure activities

• Regular lunch

• Co-authorship of publications

• Facebook friendship

The average degree of the actors range from 1.68 - 7.75 with around 200 edges.

5.2.2 DK

The second real life dataset was public Twitter activity by Danish politicians duringthe Danish general election of 2015. The data was collected and aggregated by theCommunications and Culture research group of the IT University of Copenhagen.

The Danish political system is composed of 10 different political parties in which theyare grouped into the idealogically polticial spectrum of left and right wing.

12

Left wing Right wingDansk Folkeparti Socialistisk FolkepartiVenstre Radikale VenstreLiberal Alliance SocialdemokratietDet Konservative Folkeparti AlternativetKristen Demokraterne Enhedslisten

Table 2: DK 2015 Dataset

The data contains three layers: following/followers network, replies network andretweeting network between the politicians with a Twitter account. The assignmentof the politicians into correct parties is then based on their Twitter activity. 4

5.3 Normalized Mutual Information

To test for method accuracy, we compared how much the output of the algorithmsmatch the true solution. For the datasets provided we had the correct communityassignment which allowed us to analyze the accuracy. We adopted a common usedmetric called Normalized Mutual Information. Let λa, λb denote two different parti-tions of communities [28]. Then NMI is

NMI(λa, λb) =

∑k(a)h=1

∑k(b)l=1 nh,llog

(n∗nh,l

n(a)h ∗n(b)

l

)√(∑k(a)

h=1 n(a)h log

nah

n

)(∑k(b)l=1 n

(b)l log

nbl

n

) (8)

n is the total number of nodes in the network, k(a) and k(b) is the number of commu-

nities in partitions λa, λb. n(a)h , n

(b)l and nh,l are, respectively the number of actors in

the hth community of partition λa, in the lth community partition λb and in both thehth community of λa and the lth community of λb and in both the hth communityof λa and lth community of λb. NMI is a measure between 0 and 1. When λa and λb

are equivalent, NMI is equal to 1 [17].

5.4 Methodology

In this thesis we compare the running time and memory usage of the algorithms withthree different networks. In these networks the µ is fixed to 0.3 to have a balance ofedge density.

The first experiment is conducted with multiplex networks that are increasing innumber of actors from 1000 up to 4000 actors with a fixed average node degree of 20.This gives insight how well the algorithm scale with larger input.

The second experiment is on multiplex networks with increasing number of layers,from 2 up to 5 layers. Two sets of layer increasing sets are used in the experiment,one containing 1000 actors and the other 4000.

The third experiment is a scaled version of the first experiment. We increase thenumber of actors from 7000 up to 10000 and the average degree of nodes to 50.

The accuracy experiment is composed of three parts, where they reveal how accuratethe methods are by measuring the NMI of the result of the methods compared tothe ground truth. The first part is a set of multiplex networks that have 1000 actors

4The DK dataset is directed and weighted but for this thesis it is treated as undirected andunweighted.

13

and range in µ from 0.1 to 0.6. The second part has the same range of µ but has anincrease of actors of up to 4000.

The third part of the accuracy experiment is conducted with real life datasets wherewe compare the algorithms default settings against the best possible combination ofparameters that produced the best result.

6 Results

We ran the benchmark on a Quad core Intel i7-6700 machine with 8 logical coresthat have 4000 MHz clockspeed and 16 GB of available random access memory usingUbuntu 16.04, Linux 4.4.0-79-generic operating system. We turned on all optimiza-tions that were possible to give each algorithm a fair benchmark. This means we ranusing architecture specific optimization (march=native) while also enabling the high-est level of optimization, -O4. In addition, we enabled multithreading via OpenMPand made sure that the algorithms that benefited from multithreading had the op-portunity to do so. This was done to give the algorithms full access to all resourcesavailable to them, which would make the comparison as fair as possible.

To correctly measure algorithm running time and memory usage we used a tracingframework called SystemTap. SystemTap provides the infrastructure to montior andanalyze applications without the need to recompile them, while also having the abilityto attach to them while they are running [29]. The biggest advantage of using Sys-temTap is the almost non-existant overhead when benchmarking and also the abilityto have accurate timings and measure of peak memory usage.

Figure 6: SystemTap probe example, named DTRACE0, with description of location as aparameter. In this case, markers are set before and after the Agglomerative clustering of

LART

The general idea of the functioning of SystemTap, is that C macros are implemented(called probes) inside the performance traced source code and function call is madefor each algorithm. The probes emit information, such as values of variables when itis accessed by the instruction pointer. A SystemTap script then captures the emittedvalues and the timestamp of probe access with the possibility to write the results tostandard output or a file. A probe example is displayed in Figure 6. SystemTap cando more than this, however the indepth explanation of its full capabilities is outsidethe scope of this thesis.

All of the benchmarks and experiments are reproducible and are available for down-load. 5 6

5https://github.com/mkdubik/multinet-evaluation6https://github.com/mkdubik/multinet

14

6.1 Resource usage

When analyzing the networks we supplied default values for the various parametersneeded by the algorithms. This was done in spirit of true community detection sincewhen analyzing networks we are looking for a natural partitioning into communitieswithout having to know anything about the networks in advance. These default valuesare recommended by the algorithms authors, that according to them provide the bestperformance:

Algorithm Default paramtersGeneralized Louvain method γ = 1, Csjr = 1Principal Modularity Maximization (PMM) k = 40, l = 140Locally Adaptive Random Transition (LART) t = L ∗ 3, ε = 1, γ = 1

Table 3: Algorithms and the algorithm authors default recommended parameters.

The γ parameter is recommended to be set to its default value when not knowing thecomposition of the network. If there’s knowledge beforehand about many localizedcommunities, its recommended to increase the value of γ parameter, producing smallercommunities.

The Csjr parameter is recommended to be set at 1 for Generalized Louvain whenanalyzing network without knowledge of its composition. When the amount of layersincreases, the impact of cross layer connections increases and becomes more relevant.

The parameters for PMM require prior knowledge of the network. Optimal l is deter-mined by the amount of communities the current network has and the only recom-mendation of the authors is to set it to ≥ 140 when prior knowledge of the networkis not available.

The first benchmark revealed that LART was not going to be able to process networksof real size due to the prohibitive amount of time spent when analyzing. LART alsoused an excessive amount of memory it as shown in Figure 7. Matrix multiplicationand agglomerative clustering are both time consuming operations that LART spendsthe most time in. Since LART requires the network to be non-bipartite and connectedan analysis of the network must be conducted before the algorithm is run, adding tothe computational time.

(a) L = 3, µ = 0.3, < k >= 20 (b) L = 3, µ = 0.3, < k >= 20

Figure 7: Small network resource benchmark. a) shows algorithm cpu execution timewhile b) shows algorithm peak memory usage.

The excessive memory usage of LART can be attributed to the dissimilarity matrix,D. This matrix contains the distances of every node from each other and as t increases,

15

the matrix becomes even denser. Because of this as the network grows in size themethod becomes quickly unfeasable.

Generalized Louvain had almost no additional memory overhead meaning if the sparserepresentation of the network in matrix form could fit in memory a solution wouldeventually be computed. One particular feature of Generalized Louvain allows tocompute the full modularity matrix in memory to speed up the computation, where auser supplied parameter controls how much of the matrix is allowed to be in memory.For our benchmarks we set the maximum size of the modularity matrix to less then4000 actors which explains the reduced memory usage on the larger network.

We also benchmarked when layers were increased from the traditional 3 layer setupup to maximum of 5. For tiny networks, the methods had no issues analyzing themas shown in Figure 8.

(a) N = 1000, µ = 0.3, < k >= 20 (b) N = 1000, µ = 0.3, < k >= 20

Figure 8: Layer resource benchmark with 1000 actors. a) shows algorithm cpu time whileb) shows algorithm peak memory usage.

The limitations of LART were revealed when the network and layer size was increased.The core reason for this is the amount of steps needing to explore the local topologyof each layer, which is t = L ∗ 3 for sparse networks. Besides the unreasonablecomputational time, LART had exhausted all available memory before it could finishwhile the other methods did not encounter such issues. This makes LART fit onlyin scenarios when the network size is tiny with few layers. Generalized Louvain andPMM both had no performance issues when analyzing these networks as displayed inFigure 9.

(a) N = 4000, µ = 0.3, < k >= 20 (b) N = 4000, µ = 0.3, < k >= 20

Figure 9: Layer resource benchmark with 4000 actors. a) shows algorithm cpu time whileb) shows algorithm peak memory usage.

16

We generated a second dataset were actors and maximum possible community sizeswere increased significantly. Unfortunately for this benchmark it was the end ofthe line for LART because the computational time became unreasonable and it wasuncertain if enough memory was available. Therefore LART was not included in thisbenchmark.

(a) L = 3, µ = 0.3, < k >= 50 (b) L = 3, µ = 0.3, < k >= 50

Figure 10: Medium sized resource benchmark. a) shows algorithm cpu execution timewhile b) shows algorithm peak memory usage.

Again, Generalized Louvain had no problems with the larger datasets and easily dis-covered the communities relatively fast while using almost no memory. PMM becamedrastically slower and more memory intensive as the network grew, however it didnot exceed available memory limit. It’s possible that PMM could have handled largernetworks but in the performance benchmark, Generalized Louvain reigned supremeas is shown in Figure 10. The vertice collapsing mechanism of Generalized Louvainis clearly very effecient.

6.2 Accuracy on Synthetic datasets

We continued to benchmark the algorithms in the spirit of true community detectionand used the same default parameters like in the previous section. Since the syntheticnetwoks generated a ground truth file during creation we used that to compare theoutput of the algorithms. This comparison revealed how accurate the algorithms arein discovering the correct communities according to the definition. We used Normal-ized Mutual Information (NMI) to score the accuracy, where 1.0 indicated that thediscovered community was identical to the ground truth while 0.0 being completelydifferent.

For low values of µ the algorithms had no problems finding the correct communities asindicated by the high NMI. As µ increased LART and PMM started to struggle, as isshown in 11. Lets recall that when µ is higher than 0.5 communities start to dissapearin the traditional sense. For a multiplex with 1000 actors Generalized Louvain hadno issues and was very close to 1.0

We scaled the accuracy benchmark to 4000 actors and it displayed similar results,however Generalized Louvain started to display signs of detection failure when µ washigh but much less earlier than the other two algorithms 12.

For synthetic datasets, Generalized Louvain had the best accuracy.

17

Figure 11: Tiny multiplex accuracy benchmark

Figure 12: Small multiplex accuracy benchmark

18

6.3 Accuracy on Real life datasets

For the real life multiplex networks we used the same parameters like in the syntheticdatasets but also tried various different values for them to determine which parametersgave the best results.

Method NMILART, t = 15, ε = 1.0, γ = 1.0 0.819964PMM, k = 30, l = 140 0.50854GLouvain, γ = 1.0, Csjr = 1.0 0.758757LART, t = 12, ε = 1.5, γ = 1.0 0.847523PMM, k = 30, l = 140 0.50854GLouvain, γ = 2.0, Csjr = 1.0 0.829699

Table 4: AUCS results. Highlighted rows show the best performance parameters and theNMI score

In the default and parameter exploring scenario LART had the best result. An inter-esting observation is that since AUCS has 5 layers the default step recommendationwas to set it to 15, however the results clearly show that the default value makesthe walker walk too far which obfuscates the true community structure and results inlower NMI.

No matter the choice of parameters, PMM did not perform any better. This couldbe due to the layers not being as correlated.

In this benchmark an interesting anomaly occured, where multiple smaller commu-nities were closer to the truth then fewer and larger ones. Namely in GeneralizedLouvain when the resolution parameter γ was set to a higher value then default ahigher NMI was observed. This is an indication of one of Generalized Louvain’s, andgenerally for all modularity maximization methods, shortcomings.

Method NMILART, t = 12, ε = 1.0, γ = 1.0 0.446741PMM, k = 30, l = 140 0.242595GLouvain, γ = 1.0, Csjr = 1.0 0.854493LART, t = 12, ε = 0.5, γ = 2.0 0.49728PMM, k = 30, l = 140 0.242595,GLouvain, γ = 1.0, Csjr = 1.0 0.854493

Table 5: DK results. Highlighted rows show the best performance parameters and NMIscore

In the DK dataset Generalized Louvain had superior performance by a large margin.The fact that we treated this dataset as undirected and unweighted might have hada bad effect on the other two algorithms and possibly confuse them 7.

We managed to visualize the result of Generalized Louvain and a clear seperationof communities is visible. Figure 13 shows the 8 different communities and theircomposition, the percentage of each party in a community on the reply network ofthe DK multiplex. There are two communities where its structure is entirely composedof a single party but there are also multiple instances of mixed party membership ina single community.

In all cases, the communities were dominated by politicians whose parties alignedthemselves on the left wing or the right wing, as shown in Figure 14.

7Undirected networks represent bidirectional relationships while a directed network represents aunidirectional relationships. By representing a directed networks as undirected, high amounts of

19

Figure 13: Generalized Louvain result, reply network

Figure 14: Generalized Louvain result, followers network

information is discarded

20

7 Implementation notes

All of the algorithms require sparse matrix representation of the data. For this weused Eigen, which is a C++ template library for linear algebra: matrices, vectors,numerical solvers, and related algorithms [30]. Eigen is multithreaded and allows formultithreaded matrix and vector multiplication that LART is heavily dependent on.To reduce matrix multiplication computational time we made sure that an efficientmatrix multiplication routine was implemented and also that the routine made useof multiple threads. In addition, Eigen provides a Singular Value Decompositionalgorithm implementation that is used by PMM.

Some of the algorithms require efficient additional machine learning algorithms. LARTneeds a bottom-up agglomerative clustering algorithm and PMM requires the K-means for assigning actors to communities. For this purpose we used and extendeddlib. Dlib is a modern C++ toolkit containing machine learning algorithms whichare designed to be highly modular, quick to execute, and simple to use via a cleanand modern C++ API [31].

Lastly, for PMM we included an eigenvector computational library named Spectra.Spectra is designed to calculate a specified number of eigenvalues l of a large squarematrix A. Usually l is much less than the size of matrix N , so that only a feweigenvalues and eigenvectors are computed, which in general is more efficient thancalculating the whole spectral decomposition[32]. Spectra is a header only C++reimplementation of the Fortran ARPACK package library and is dependent on Eigen.Since Eigen was already used by the other algorithms we decided to work with thislibrary.

We collected the results in python and graphed the results using matplotlib [33] andvisualized them with multinetx package [34]

8 Future Work

There is plenty of future work to be done. It would be interesting to consider directedand weighted networks since they contain more data about the relationship of entities,however they were outside of the scope of the thesis.

The actors in the synthetic and real life dataset belonged to only one community butit is known that in social networks, actors can belong to multiple communities on thesame layer. In this thesis we did not test for actor overlapping communities on thesame layer.

We believe that LART can be further optimized and made more efficient. Instead ofkeeping a dense dissimilarity matrix we would like to see a similarity matrix, meaninglarger values would be indicative of community structure. In this way it would bepossible to prune very low values and keep the matrix sparse.

Due to time constraints we could not benchmark more algorithms, using the samecomputing environment. Infomap has been shown to outperform [35] the Louvainmethod and it would be interesting to compare these two methods. Other algorithmsthat would be interesting to benchmark are Clique percolation, MiMag and layersimplification methods based on redundancy.

21

9 Conclusions

In this thesis we presented the multiplex network, which is an extension to the tra-ditional single layer network, that is capable of representing multiple different inter-actions of entities in a single datastructure. Multiplex networks have been shown tocontain communities, which is defined as a collection of entities that interact morewith each other then the rest of the network. Discovering these communities is a formof data mining and different algorithms exist to discover communities in networks. Wepresented three different algorithms that accomplish this task and evaluted their effi-ciency by comparing their execution time, memory usage and accuracy. To comparethe accuracy we used Normalized Mutual Information (NMI) which compares howmuch two sets differ from each other. The algorithms that were evaluated are namedGeneralized Louvain, Principal Modularity Maximization (PMM) and Locally Adap-tive Random Transition (LART). In spirit of community detection, the parameterswere set to default values that were recommended by the authors of each algorithm.

We generated multiple synthetic datasets and also used real life data with groundtruth. The real life networks were interactions of professors of Aarhus UniversityComputer science department (AUCS) and Twitter activity of Danish politiciansduring the Danish general election of 2015. The multiplex networks were undirectedand unweighted. In addition to the default parameter benchmark, we used a range ofdifferent values to discover optimal values.

From the results we have shown that Generalized Louvain method was superior inevery benchmark, where it was faster by a large margin and consumed less memorythen the other two algorithms. In addition, Generalized Louvain had a higher accu-racy score then the other methods in every case except for the AUCS dataset in whichLART scored a higher NMI. When searching for better values for parameters we haveshown that the default parameters performed well enough and the optimal values ofthe parameters for each algorithm did not have a large effect on the final result.

22

10 References

[1] M. E. J. Newman. Finding community structure in networks using the eigenvec-tors of matrices. may 2006.

[2] Guangyao Zhu and Kan Li. A Unified Model for Community Detection of Mul-tiplex Networks, pages 31–46. Springer International Publishing, Cham, 2014.

[3] Laura Cantini, Enzo Medico, Santo Fortunato, and Michele Caselle. Detectionof gene communities in multi-networks reveals cancer drivers. jul 2015.

[4] Bradley S. Rees and Keith B. Gallagher. EgoClustering: Overlapping CommunityDetection via Merged Friendship-Groups, pages 1–20. Springer Vienna, Vienna,2013.

[5] S. N. Dorogovtsev and J. F. F. Mendes. Evolution of networks. jun 2001.

[6] Mark Newman, Albert-Laszlo Barabasi, and Duncan J. Watts. The Structure andDynamics of Networks: (Princeton Studies in Complexity). Princeton UniversityPress, Princeton, NJ, USA, 2006.

[7] Bharath Pattabiraman, Md. Mostofa Ali Patwary, Assefaw H. Gebremedhin,Wei-keng Liao, and Alok Choudhary. Fast Algorithms for the Maximum CliqueProblem on Massive Graphs with Applications to Overlapping Community De-tection. nov 2014.

[8] Mark E. Dickison, Matteo Magnani, and Luca Rossi. Multilayer Social Networks.Cambridge University Press, New York, NY, USA, 1st edition, 2016.

[9] Michael J. Barber. Modularity and community detection in bipartite networks.jul 2007.

[10] Santo Fortunato and Marc Barthelemy. Resolution limit in community detection.jul 2006.

[11] Santo Fortunato and Darko Hric. Community detection in networks: A userguide. jul 2016.

[12] Chuan Wen Loe and Henrik Jeldtoft Jensen. Comparison of Communities De-tection Algorithms for Multiplex. jun 2014.

[13] Piotr Brodka. A Method for Group Extraction and Analysis in Multilayer SocialNetworks. dec 2016.

[14] M. Berlingerio, M. Coscia, and F. Giannotti. Finding and characterizing com-munities in multidimensional networks. In 2011 International Conference onAdvances in Social Networks Analysis and Mining, pages 490–494, July 2011.

[15] Nazanin Afsarmanesh and Matteo Magnani. Finding overlapping communitiesin multiplex networks. feb 2016.

[16] Michele Berlingerio, Fabio Pinelli, and Francesco Calabrese. ABACUS: fre-quent pAttern mining-BAsed Community discovery in mUltidimensional net-workS. Data Mining and Knowledge Discovery, 27(3):294–320, 2013.

[17] Lei Tang, Xufei Wang, and Huan Liu. Community detection via heterogeneousinteraction analysis. Data Min. Knowl. Discov., 25(1):1–33, July 2012.

[18] Vincent D. Blondel, Jean-Loup Guillaume, Renaud Lambiotte, and EtienneLefebvre. Fast unfolding of communities in large networks. mar 2008.

[19] Zhana Kuncheva and Giovanni Montana. Community detection in multiplexnetworks using locally adaptive random walks. jul 2015.

23

[20] M. Rosvall and C. T. Bergstrom. Maps of random walks on complex networksreveal community structure. jul 2007.

[21] Peter J. Mucha, Thomas Richardson, Kevin Macon, Mason A. Porter, and Jukka-Pekka Onnela. Community Structure in Time-Dependent, Multiscale, and Mul-tiplex Networks. nov 2009.

[22] A. Arenas, J. Duch, A. Fernandez, and S. Gomez. Size reduction of complexnetworks preserving modularity. feb 2007.

[23] Pascal Pons and Matthieu Latapy. Computing communities in large networksusing random walks (long version). dec 2005.

[24] R. Diestel. Graph Theory. Electronic library of mathematics. Springer, 2006.

[25] Wayne W. Zachary. An information flow model for conflict and fission in smallgroups. Journal of Anthropological Research, 33(4):452–473, 1977.

[26] Andrea Lancichinetti, Santo Fortunato, and Filippo Radicchi. Benchmark graphsfor testing community detection algorithms. may 2008.

[27] Matteo Magnani, Barbora Micenkova, and Luca Rossi. Combinatorial Analysisof Multiple Networks. mar 2013.

[28] Alexander Strehl and Joydeep Ghosh. Cluster ensembles — a knowledge reuseframework for combining multiple partitions. J. Mach. Learn. Res., 3:583–617,March 2003.

[29] IBM Red Hat, many contributions by corporations, and the open source com-munity. SystemTap, 2017.

[30] Gal Guennebaud Benot Jacob. Eigen is a C++ template library for linear algebra:matrices, vectors, numerical solvers, and related algorithms, 2017.

[31] Davis E. King. Dlib-ml: A machine learning toolkit. Journal of Machine LearningResearch, 10:1755–1758, 2009.

[32] Yixuan Qiu. A header-only C++ library for large scale eigenvalue problems,2017.

[33] J. D. Hunter. Matplotlib: A 2d graphics environment. Computing In Science &Engineering, 9(3):90–95, 2007.

[34] multiNetX is a python package for the manipulation and visualization of multi-layer networks. http://nikos.techprolet.com/multinetx-v1-0/. Accessed:2017-07-01.

[35] MapEquation. Infomap Benchmark, 2017.

[36] Michele Coscia, Fosca Giannotti, and Dino Pedreschi. A Classification for Com-munity Discovery Methods in Complex Networks. 2012.

[37] Saray Shai, Natalie Stanley, Clara Granell, Dane Taylor, and Peter J. Mucha.Case studies in network community detection. may 2017.

[38] Aric A. Hagberg, Daniel A. Schult, and Pieter J. Swart. Exploring networkstructure, dynamics, and function using NetworkX. In Proceedings of the 7thPython in Science Conference (SciPy2008), pages 11–15, Pasadena, CA USA,August 2008.

[39] Ulrik Brandes, Daniel Delling, Marco Gaertler, Robert Grke, Martin Hoefer,Zoran Nikoloski, and Dorothea Wagner. On finding graph clusterings withmaximum modularity. In PROCEEDINGS OF THE 33RD INTERNATIONAL

24

WORKSHOP ON GRAPH-THEORETIC CONCEPTS IN COMPUTER SCI-ENCE (WG07), VOLUME 4769 OF LECTURE NOTES IN COMPUTER SCI-ENCE, pages 121–132. Springer, 2007.

[40] Email Ltangasuedu and Xufei Wang. Uncovering Groups via HeterogeneousInteraction Analysis.

[41] Peter J Mucha, Thomas Richardson, Kevin Macon, Mason A Porter, and Jukka-Pekka Onnela. Community structure in time-dependent, multiscale, and multi-plex networks. Science (New York, N.Y.), 328(5980):876–8, may 2010.

[42] Brigitte Boden, Stephan Gunnemann, Holger Hoffmann, and Thomas Seidl. Min-ing coherent subgraphs in multi-layer graphs with edge labels. Proceedings of the18th ACM SIGKDD international conference on Knowledge discovery and datamining, 50(2):1258–1266, 2012.

[43] Michele Berlingerio, Michele Coscia, and Fosca Giannotti. Finding and character-izing communities in multidimensional networks. In International Conference onAdvances in Social Networks Analysis and Mining (ASONAM), pages 490–494,2011.

[44] Matthew Rocklin and Ali Pinar. On Clustering on Graphs with Multiple EdgeTypes. sep 2011.

[45] Michele Berlingerio, Fabio Pinelli, and Francesco Calabrese. ABACUS: fre-quent pAttern mining-BAsed Community discovery in mUltidimensional net-workS. mar 2013.

[46] Michele Berlingerio, Michele Coscia, Fosca Giannotti, Anna Monreale, and DinoPedreschi. Multidimensional networks: Foundations of structural analysis. WorldWide Web, 16(5-6):567–593, 2013.

25