The recent concept of beamspace multiple-input-multiple-output (MIMO) enables the millimeter-wave (mmWave) MIMO system to reduce the radio-frequency chains by utilising beam selection and yet achieve near-optimal sum-rate performance. In this work, we study the lens antenna array as the transmit antenna at the base station for mmWave multiuser MIMO (MU-MIMO) system. Due to the direction-based energy focusing property of the array, the beam selection is able to reduce to antenna selection without significant performance degradation as that using traditional antenna selection. For this mmWave MU-MIMO system, direction-based antenna selection method is proposed, where the inter-user interference, resulting from the case that the same antenna is selected for different users with high probability, is considered and solved by a re-selection based on minimization of sum-rate loss. Simulation results show that spectral and energy effectiveness of the proposed antenna selection method for the mmWave MU-MIMO using lens antenna array.