DSD-LLM: Enabling High-Fidelity 1-Bit Large Language Models via Differential Sparsity and Spectral-Decoupled Optimization

1-bitpost-training compression constitutes an extreme yet compelling regime for Large Language Models (LLMs), but existing methods suffer severe accuracy degradation in this aggressive setting. We identify that a key limitation lies in the reliance on oversimplified structural priors, such as sparsity uniformity, column-wise independence, and coupling rigidity. In this paper, we present DSD-LLM, a high-fidelity 1-bit compression framework for LLMs driven by Differential sparsity, Spectral salience, and Decoupled optimization. Specifically, DSD-LLM introduces three key innovations: 1) Hessian-Aware Differential Sparsity (HDS) redistributes a fixed sparsity budget within each projection matrix via Hessian-informed chunk sensitivity ranking, moving beyond homogeneous N:M allocation; 2) Spectral Salience Selection (S3) reformulates structural identification as a Cardinality-Constrained Quadratic Binary Optimization problem, employing spectral relaxation to capture coupled inter-column correlations; and 3) Decoupled Adaptive Search (DAS) circumvents rigid coupling rules, enabling independent boundary optimization for irregular weight distributions. Extensive experiments across diverse pretrained and instruction-tuned LLMs demonstrate that DSD-LLM achieves significant gains over strong baselines under most compression settings, while extremely aggressive sub-1-bit compression remains challenging for sensitive models.