Comparison with Centralized MaxFuse¶
Does federating MaxFuse cost accuracy? Across 2,600 runs: not on matching geometry, yes on label accuracy, and the gap has an identifiable cause.
Experimental design¶
| Dataset | Centralized | Federated | CCA sweep | Shared params | Batching | Total |
|---|---|---|---|---|---|---|
| CITE-seq & PBMC (Antibodies) | 1 × 250 | 1 × 250 | 11 × 25 | 11 × 25 | 10 × 25 | 1,300 |
| scRNA-seq & CODEX (Tonsils) | 1 × 250 | 1 × 250 | 11 × 25 | 11 × 25 | 10 × 25 | 1,300 |
| Total | 500 | 500 | 550 | 550 | 500 | 2,600 |
Both settings use identical hyperparameters. Significance is assessed with the non-parametric Mann–Whitney U test at \(p < 0.05\), a threshold chosen to reflect the exploratory nature of the work and MaxFuse's intrinsic stochasticity. The repetition counts (25–250) exist precisely to average over that stochasticity.
Results¶
Where federation is indistinguishable¶
| Metric | p-value | Significant? |
|---|---|---|
| FOSCTTM (Antibodies) | 3.53e-01 | No ✅ |
| FOSKNN (Antibodies) | 8.76e-01 | No ✅ |
| Steadiness, non-meta | 5.32e-01 | No ✅ |
| Steadiness, meta | 3.06e-01 | No ✅ |
| Cohesiveness, non-meta | 4.66e-01 | No ✅ |
| Cohesiveness, meta | 8.52e-01 | No ✅ |
| L2/L3 Silhouette F1 | 5.59e-01 / 3.68e-01 | No ✅ |
| L2/L3 ARI F1 | 1.60e-01 / 7.10e-01 | No ✅ |
FOSCTTM is near-identical between settings: local neighbourhood structure survives federation.
Steadiness & Cohesiveness. Non-meta results cluster in the favourable upper-right quadrant.
Where centralized wins¶
| Metric | Dataset | p-value | Significant? |
|---|---|---|---|
| L1 / L2 / L3 Accuracy | Antibodies | 8.94e-32 / 3.52e-32 / 2.51e-09 | Yes |
| Cluster-Term Accuracy | Tonsils | 4.57e-59 | Yes |
| Cluster-Term Silhouette F1 | Tonsils | 8.34e-71 | Yes |
| Cluster-Term ARI F1 | Tonsils | 1.74e-45 | Yes |
| Stress (both modalities) | Tonsils | 8.70e-26 / 4.94e-34 | Yes |
| MRRE missing, meta | Antibodies | 4.93e-03 | Yes |
Accuracy per cell label. Federated tracks centralized closely but sits consistently below it.
Note the pattern across label granularity: differences become less pronounced at finer label levels (L3, with 58 labels, gives p=2.5e-09 versus ~1e-32 for L1/L2). Federated MaxFuse retains most of its classification capability at detailed annotation levels.
Diagnosing the gap¶
The interesting question is why accuracy drops while geometry does not.
Evidence 1: the geometry metrics disagree. If federation were damaging the algorithm, FOSCTTM and FOSKNN would degrade too. They do not, and neither do any of the four Steadiness/Cohesiveness tests. The embedding geometry survives intact.
Evidence 2: a confound is identifiable. Centralized MaxFuse runs batching by default on Tonsils; the federated implementation does not implement batching at all, owing to implementation complexity.
Evidence 3: the mechanism is plausible. Tonsils is extremely asymmetric: 12,977 scRNA cells against 178,919 CODEX cells. Without batching, the linear sum assignment produces at most \(\min(12977, 178919) = 12{,}977\) matches, covering just 7% of the CODEX cells. With batching, matching runs pairwise between batch pairs and retains the best correspondences, yielding far more candidate cell-to-cell matches.
Evidence 4: batching demonstrably helps. The batching experiments show moderate batching improving F1 and lowering FOSCTTM. Its absence should therefore hurt.
Evidence 5: the same confound explains the stress result. Stress is significantly worse for federated on Tonsils but not on Antibodies, which is symmetric (10,000 × 10,000) and not batched by default. Same confound, same pattern, independent metric.
A second, uncontrolled difference
Batching is the leading explanation, but it may not be the only one. The reference
implementation's propagate() changed its nearest-neighbour metric from correlation
(manuscript version) to euclidean (released package) without a version change, and its
documentation still describes the old behaviour. Propagation assigns partners to every
non-pivot cell, the majority of cells here.
If the centralized baseline used the released package while the federated implementation propagates with correlation distance, that is a second uncontrolled difference alongside batching. Verify which metric each side used before treating batching as the sole cause.
The variance trade-off¶
Even where medians match, federated results are consistently less stable. The metrics vary noticeably more run-to-run. This is a real cost of the federated setting and should be reported alongside central tendency; a median-only comparison overstates equivalence.
Meta versus non-meta¶
Independently of federation, the meta-modality consistently underperforms:
- Very low cohesiveness: MaxFuse introduces artificial cluster structures in the embedding that are absent from the original data.
- Worse MRRE-missing scores.
Both trace to the summarisation effect of the Leiden algorithm: aggregating cells into centroids removes structural information. This is the price of meta-cells, and it is paid in both settings.
Verdict¶
- Federated MaxFuse preserves matching geometry. FOSCTTM, FOSKNN, Steadiness and Cohesiveness are all statistically indistinguishable from centralized.
- It loses on label accuracy, most severely on the heterogeneous Tonsils dataset.
- The gap is attributable to a missing implementation feature (batching), not to federation itself, though a second candidate confound exists and should be checked.
- Federated results are higher-variance throughout.
The practical conclusion: federated diagonal integration is viable, and implementing federated batching is the highest-value next step.