Change-point detectors for dynamic networks aim to find changes in how a network is connected. They are evaluated on real networks with labelled events, but many of these events also change how active the network is: how many vertices take part and how many edges are present. A detector can therefore score well by responding to activity alone, without detecting any change in structure. We test whether detectors from the literature beat a simple baseline that tracks only the number of active vertices and the number of edges in each snapshot, a baseline that we motivate with a standard statistical test for a change in activity. On ten datasets, four of them from the papers of published detectors, none of thirteen detectors ranks the labelled events significantly better than the baseline when the results are pooled (AUC $0.75$, or $0.73$ with scales taken from early snapshots only, against $0.66$ for the best detector). The baseline is best on only two datasets, however, and on the four datasets with the most events three detectors that track individual vertices score higher. When we rewire networks while keeping the degree of every vertex fixed, most detectors that ignore vertex identities barely respond, but several that track individual vertices do. New detectors should be compared with this baseline and tested at fixed degree.

翻译：暂无翻译