dorsal/arxiv
View SchemaDP-FEDSOFIM: Differentially Private Federated Stochastic Optimization using Regularized Fisher Information Matrix
| Authors | Sidhant R. Nair, Tanmay Sen, Mrinmay Sen |
|---|---|
| Categories | |
| ArXiv ID | 2601.09166vv1 |
| URL | https://arxiv.org/abs/2601.09166 |
| License | http://arxiv.org/licenses/nonexclusive-distrib/1.0/ |
Abstract
Differentially private federated learning (DP-FL) suffers from slow convergence under tight privacy budgets due to the overwhelming noise introduced to preserve privacy. While adaptive optimizers can accelerate convergence, existing second-order methods such as DP-FedNew require O(d^2) memory at each client to maintain local feature covariance matrices, making them impractical for high-dimensional models. We propose DP-FedSOFIM, a server-side second-order optimization framework that leverages the Fisher Information Matrix (FIM) as a natural gradient preconditioner while requiring only O(d) memory per client. By employing the Sherman-Morrison formula for efficient matrix inversion, DP-FedSOFIM achieves O(d) computational complexity per round while maintaining the convergence benefits of second-order methods. Our analysis proves that the server-side preconditioning preserves (epsilon, delta)-differential privacy through the post-processing theorem. Empirical evaluation on CIFAR-10 demonstrates that DP-FedSOFIM achieves superior test accuracy compared to first-order baselines across multiple privacy regimes.
{
"annotation_id": "f5c86f37-dc8b-44ae-89b3-b27820bf0588",
"date_created": "2026-02-17T05:53:20.073000Z",
"date_modified": "2026-02-17T05:53:20.073000Z",
"file_hash": "4e37d9ad86afcd266e8468fc3628ff6d474718339dfc0071de61563f9926fcb1",
"private": false,
"record": {
"abstract": "Differentially private federated learning (DP-FL) suffers from slow convergence under tight privacy budgets due to the overwhelming noise introduced to preserve privacy. While adaptive optimizers can accelerate convergence, existing second-order methods such as DP-FedNew require O(d^2) memory at each client to maintain local feature covariance matrices, making them impractical for high-dimensional models. We propose DP-FedSOFIM, a server-side second-order optimization framework that leverages the Fisher Information Matrix (FIM) as a natural gradient preconditioner while requiring only O(d) memory per client. By employing the Sherman-Morrison formula for efficient matrix inversion, DP-FedSOFIM achieves O(d) computational complexity per round while maintaining the convergence benefits of second-order methods. Our analysis proves that the server-side preconditioning preserves (epsilon, delta)-differential privacy through the post-processing theorem. Empirical evaluation on CIFAR-10 demonstrates that DP-FedSOFIM achieves superior test accuracy compared to first-order baselines across multiple privacy regimes.",
"arxiv_id": "2601.09166",
"authors": [
"Sidhant R. Nair",
"Tanmay Sen",
"Mrinmay Sen"
],
"categories": [
"cs.LG",
"cs.CR",
"cs.DC"
],
"license": "http://arxiv.org/licenses/nonexclusive-distrib/1.0/",
"title": "DP-FEDSOFIM: Differentially Private Federated Stochastic Optimization using Regularized Fisher Information Matrix",
"url": "https://arxiv.org/abs/2601.09166",
"version": "v1"
},
"schema_id": "dorsal/arxiv",
"source": {
"execution_id": "a82e0e18-6df7-45ab-bc34-e7ac28240d9b",
"id": "arXiv Dataset",
"type": "Model",
"variant": "snapshot-2026-01-17",
"version": "0.1.0"
},
"user_id": 1000002
}