In order to adapt the quality of videos streamed over a timevarying bandwidth channel, a network-aware bandwidth management and rate control are required. This paper proposes the well-designed bandwidth estimation and active buffer control for streaming H.264 FGS videos over heterogeneous wired/ wireless/3G networks. According to the information of measured packet round-trip-time, loss-rate and delay jitter, an improved Kalman filter is proposed to predict the throughput recursively, and determine the transmission rate in consideration of buffer fullness of a decoder. The optimal parameters of the Kalman filter, e.g., transition matrix and error covariance, can be initialized, converged and adapted to the current network, even when the session handoff occurs. In our real experiments, distinct network traffic models are simulated, and corresponding estimation results w.r.t. network information are also exhibited.