Efficient Maximum-Likelihood Detection for the MIMO System in Hybrid Mode

In wireless communications, the multiple-input multiple-output (MIMO) system efficiently can make use of the spectrum and enhance the transmission throughput. The sphere decoding (SD) is an efficient algorithm that enables the optimal maximum-likelihood (ML) detection for the MIMO system. However, the SD algorithm has the complexity that increases rapidly with decreasing signal-to-noise ratio (SNR). Another MIMO detection algorithm that is based on differential metrics (DMs) can also attain the exact ML detection without the need of QR decomposition and matrix inversion. The complexity of the DM-based algorithm does not increase with decreasing SNR as the SD algorithm. On the other hand, the SD algorithm has lower complexity at high SNR, especially for large modulation constellation like the quadrature amplitude modulation (QAM). In this paper, we propose a new ML detection algorithm for the MIMO system based on the hybrid operation of both the SD and DM-based algorithms. We first modify both of them such that they are based on the same signal model. Then we apply both the two modified algorithms in the tree search process, with the bit-level and symbol-level operations, respectively. Simulation shows that the proposed hybrid algorithm attains the ML detection, with the same bit-error rates (BER) as the SD algorithm. It also maintains the advantages of both algorithms at high and low ranges of SNR with lower average complexity.