Identification of modal parameters from response data only is studied for structural systems under nonstationary ambient vibration. By assuming the ambient excitation to be nonstationary white noise in the form of a product model, the modal parameters of a system could be identified through the correlation method in conjunction with a technique of curve-fitting. However, the error involved in the approximate free-decay response would generally lead to a distortion in the modal parameters of identification. It is shown that, under appropriate conditions, the ambient response corresponding to nonstationary input of various types can be approximately expressed as a sum of exponential functions, so that we can use the Ibrahim time-domain method in conjunction with a channel-expansion technique to directly identify the major modes of a structural system without any additional treatment of converting the original data into the form of free vibration. To further distinguish the structural modes from non-structural modes, the concept of mode-shape coherence and confidence factor is employed. Numerical simulations, including one example of using practical excitation data, confirm the validity and robustness of the proposed method for identification of modal parameters from general nonstationary ambient response.