We studied the electrical properties and device performance of the Cu(In,Ga)Se2 solar cell when i-ZnO is replaced by i-ZnMgO. Admittance spectroscopy reveals a deep level attributed to the i-ZnMgO/CdS interface. The capacitance voltage experiment indicates a pronounced reduction of carrier concentration in the p-CIGS absorber. Inserting a thin i-ZnO layer between i-ZnMgO and CdS completely removes the detrimental effect of the interface states and partially reverts the carrier concentration reduction effect. Device simulations indicate that the open circuit voltage is lowered due to the carrier concentration reduction while the fill factor is sensitive to the defects at the interface.