Spatial modulation (SM) has been shown to be a promising low-complexity alternative to the state-of-art multiple-input multiple-output (MIMO) schemes due to its novel transmission approach. This paper investigates the performance of SM-MIMO systems in the presence of two practical undesirable effects, namely in-phase (I) and quadrature-phase (Q) imbalance (IQI) and imperfect channel state information (ICSI). An optimum maximum likelihood detection (MLD) method is proposed to tackle the effects of self-interference and signal distortion caused by IQI impairment by adapting the traditional MLD technique in accordance with the asymmetric characteristics of the IQI. More particularly, upper-bounds of the closed-form average pairwise error probability (APEP) and the average bit error rate (ABER) are derived for generalized Beckmann fading channels. As erroneously interpreted channel coefficients at the receiver (Rx) cause the error rate to increase and the detection to fall short, Cramer-Rao bound, which is a lower bound on the variance of the channel estimator, is utilized to assess the estimation accuracy. The system performance is evaluated by analytical derivations that are corroborated with computer simulations. The obtained results show that ICSI and IQI should be seriously considered while designing the future SM-based wireless communication systems.