dorsal/arxiv
View SchemaLate Breaking Results: Quamba-SE: Soft-edge Quantizer for Activations in State Space Models
| Authors | Yizhi Chen, Ahmed Hemani |
|---|---|
| Categories | |
| ArXiv ID | 2601.09451vv1 |
| URL | https://arxiv.org/abs/2601.09451 |
| License | http://arxiv.org/licenses/nonexclusive-distrib/1.0/ |
Abstract
We propose Quamba-SE, a soft-edge quantizer for State Space Model (SSM) activation quantization. Unlike existing methods, using standard INT8 operation, Quamba-SE employs three adaptive scales: high-precision for small values, standard scale for normal values, and low-precision for outliers. This preserves outlier information instead of hard clipping, while maintaining precision for other values. We evaluate on Mamba- 130M across 6 zero-shot benchmarks. Results show that Quamba- SE consistently outperforms Quamba, achieving up to +2.68% on individual benchmarks and up to +0.83% improvement in the average accuracy of 6 datasets.
{
"annotation_id": "f1eb78aa-30db-467c-b2a2-4131b927dbee",
"date_created": "2026-02-17T05:53:20.117000Z",
"date_modified": "2026-02-17T05:53:20.117000Z",
"file_hash": "be8b81b330bc4c9772e208870f639dcf3a4838636b3b325e7aac819f9448ef61",
"private": false,
"record": {
"abstract": "We propose Quamba-SE, a soft-edge quantizer for State Space Model (SSM) activation quantization. Unlike existing methods, using standard INT8 operation, Quamba-SE employs three adaptive scales: high-precision for small values, standard scale for normal values, and low-precision for outliers. This preserves outlier information instead of hard clipping, while maintaining precision for other values. We evaluate on Mamba- 130M across 6 zero-shot benchmarks. Results show that Quamba- SE consistently outperforms Quamba, achieving up to +2.68% on individual benchmarks and up to +0.83% improvement in the average accuracy of 6 datasets.",
"arxiv_id": "2601.09451",
"authors": [
"Yizhi Chen",
"Ahmed Hemani"
],
"categories": [
"cs.LG",
"cs.AI",
"cs.AR"
],
"license": "http://arxiv.org/licenses/nonexclusive-distrib/1.0/",
"title": "Late Breaking Results: Quamba-SE: Soft-edge Quantizer for Activations in State Space Models",
"url": "https://arxiv.org/abs/2601.09451",
"version": "v1"
},
"schema_id": "dorsal/arxiv",
"source": {
"execution_id": "7ec69c77-213b-4328-b93b-49f71865834d",
"id": "arXiv Dataset",
"type": "Model",
"variant": "snapshot-2026-01-17",
"version": "0.1.0"
},
"user_id": 1000002
}