When labels lie, the graph spreads it.
Survey & Benchmark · Graph Learning under Label Noise
When labels lie, the graph spreads it.
A unified, reproducible benchmark for graph neural networks under noisy supervision. We benchmark 12 robustness methods plus a Standard baseline, then stress-test them where it counts — strong backbones, heterophily, scale, and cost — not just clean citation graphs.
Build an experiment View on GitHub What's inside
13
method configurations
24
graph datasets
9
label-noise models
7
GNN backbones
The problem
Noise doesn't stay where you put it.
Unlike i.i.d. models, a GNN passes messages along edges, so supervision at one node can influence learned representations throughout its neighbourhood. Existing benchmarks answer this on small homophilic citation graphs with one backbone. We ask the harder questions: does robustness survive a strong, wide backbone? does it hold under heterophily and at scale? and what does it cost?
One run, end to end
Configure → run → measure, in a single pipeline.
Every configuration flows through the same path: apply the selected label-noise process, assemble a backbone with a robustness method, train with early stopping while probing oversmoothing, then evaluate on the clean test split.
pipeline overview · config.yaml → Cartesian sweep → mean ± std
What's covered
The axes we vary.
13 configurations · 12 robustness methods + baseline
Standard baseline NRGNN densify PI-GNN pairwise CR-GNN contrastive RTGNN peer + governance UnionNET support-set GNN-Cleaner label correction GraphCleaner post-hoc detect ERASE coding-rate GNNGuard edge defense CE+W2 spectral GCOD centroid reliability CommDefense self-supervised
7 backbones
GCNGATGATv2 GINGPS GCN\ tunedGAT\ tuned
9 label-noise models
UniformCoin-flipPair RandomRandom PairFlip Uniform MixDeterministicInstance
24 datasets — homophilic to heterophilic, citation to industrial
Planetoid citation
cora · citeseer · pubmed
Heterophilous platonov et al.
amazon-ratings · tolokers · roman-empire · minesweeper · questions
Co-purchase amazon
amazon-computers · amazon-photo · dblp
Social attributed
blogcatalog · flickr
GraphLAND industrial · to 1.4M nodes
hm-categories · pokec-regions · web-topics · tolokers-2 · city-reviews · artnet-exp · web-fraud
Structured gnn-benchmark · lrgb
pattern · cluster · pascalvoc-sp · coco-sp
Beyond accuracy
We measure three things at once.
Accuracy alone hides why a method wins or fails. Each run reports how it classifies, how its representations behave, and what it costs.
Classification
- Accuracy · macro-F1
- Precision · Recall
- ROC-AUC
- clean / mislabelled / recovered splits
- NumRank · Erank
- EDir · EProj
- MAD
- rank-collapse as a failure signal
- FLOPs — train & inference
- wall-clock time
- CO₂ via CodeCarbon
- the price of robustness
Oversmoothing
Compute & carbon
What we found
Robustness is conditional — not a property you inherit.
Reported gains do not automatically transfer. The benchmark surfaces failure modes that only appear off the usual small citation graphs.
Finding 01
It depends on the backbone.
On the standard GCN/GAT backbones most methods beat Standard under noise (ERASE +14.3, RTGNN +9.0, GraphCleaner +8.4 on Cora/uniform). On a strong, wide GCN\* the gap compresses toward the baseline — or the method fails outright.
Finding 02 — collapse
Geometry-based methods collapse at width. rank ≈ 1
ERASE and UnionNET fall to chance train accuracy on the 512-wide backbone — clean and noisy alike — as their embedding rank collapses to one. Healthy at h64, they degenerate at h512. Documented, not patched.
Finding 03
Structure amplifies label noise.
Message-passing-induced rank collapse is a measurable mechanism through which topology degrades representations under noise — visible directly in the oversmoothing metrics, not just the accuracy.
Finding 04 — cost
The price varies by orders of magnitude.
GNNGuard recomputes cosine attention every epoch (≈170× Standard per epoch); NRGNN's edge predictor runs out of memory on large graphs. Six of thirteen methods do not scale past mid-sized graphs.
Reproducible by design
Run the whole thing in three steps.
Pinned dependencies, a Cartesian sweep syntax in one YAML, and incremental execution that resumes where it stopped. Prefer a guided start? The Experiment Builder creates and validates the configuration for you.
1 · install (pinned: torch 2.2.1, PyG 2.5.3)
python3 -m pip install -r requirements.txt2 · point config.yaml at a dataset / noise / method
£[cora, roman-empire] expands to a sweep grid
3 · run — resumes automatically, aggregates seeds
python3 main.py -c config.yamlresults → results/<experiment>/experiment.json
1. One config, many runs. The £\[…\] sweep syntax turns a single YAML into a Cartesian grid over datasets, noise types, rates, and methods. 2. Incremental & resumable. Only missing runs execute; bump \--num-runs from 5 to 10 and only runs 6–10 fire. 3. Faithful by default. Each method mirrors its official source; the no-oracle adaptations we make are documented, not silent.
Citation
If it helps your work, please cite us.
bibtexcopy
@misc{gln-benchmark, title = {Survey and Benchmark of Graph Learning under Label Noise}, author = {Wani, Farooq Ahmad and Purificato, Antonio and Bucarelli, Maria Sofia and Di Francesco, Andrea Giuseppe and Corelli, Michael and Pryymak, Oleksandr and Silvestri, Fabrizio}, year = {2026}, note = {Code: github.com/GLNBench/GLNBench} }
Authors
Farooq Ahmad Wani Antonio Purificato Maria Sofia Bucarelli Andrea Giuseppe Di Francesco Michael Corelli Oleksandr Pryymak Fabrizio Silvestri