Joint transceiver optimization for multiuser MIMO relay communication systems

In this paper, we address the optimal source, relay, and receive matrices design for linear non-regenerative uplink multiuser multiple-input multiple-output (MIMO) relay communication systems. The minimum mean-squared error (MMSE) of the signal waveform estimation at the destination node is adopted as our design criterion. We develop two iterative methods to solve the highly nonconvex joint source, relay, and receiver optimization problem. In particular, we show that for given source precoding matrices, the optimal relay amplifying matrix diagonalizes the source-relay-destination channel. While for fixed relay matrix and source matrices of all other users, the source matrix of each user has a general beamforming structure. Simulation results demonstrate that the proposed iterative source and relay optimization algorithms perform much better than existing techniques in terms of both MSE and bit-error-rate.